Importance Sampling – A Tutorial

Problem: Compute an expectation over a probability density function f, given only samples generated from a different probability density function g. We call f and g the target distribution and proposal distribution respectively.


  1. Draw N samples from g
  2. Calculate the probability of each sample
  3. Evaluate f over the N samples
  4. Calculate the importance weights w=f/g
  5. Draw N samples from g with new weights w


Code: Let’s say we want to estimate the Beta distribution (our target distribution), we don’t know the formula but we can evaluate (i.e. a black-box). We use the uniform distribution as our proposal distribution, meaning that we have no idea about the target distribution.

target_pdf = @(x)betapdf(x, 2, 11); % target distribution
N = 10000; % number of samples
samples = rand(N, 1); % sample from a known pdf (belief)
weights = 1/N; % weight values for each sample (probability of each sample)
eval_pdf = target_pdf(samples); % evaluate the samples with the black-box
w = eval_pdf ./ weights; % calculate importance weight
w = w / sum(w);
samples_ = randsample(samples, N, true, w); % resamle with replacement according to importance weights


The figure on the left shows the desired target distribution and our initial samples on the proposal distribution (uniform distribution). The figure on the right illustrates the re-sampled particles according to the calculated importance weights.

Posted in Machine Learning, MATLAB, Optimization, Reinforcement Learning, Robotics, Software, Statistics | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment 18-10-2017

Robotics on

Submissions on 18-10-2017

[1]  arXiv:1710.06425 [pdf, other] link

Title: Domain Randomization and Generative Models for Robotic Grasping

Joshua Tobin,
Wojciech Zaremba,
Pieter Abbeel

Subjects: Robotics (cs.RO); Learning (cs.LG)

[2]  arXiv:1710.06280 [pdf, other] link

Title: Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions

Jun Hatori,
Yuta Kikuchi,
Sosuke Kobayashi,
Kuniyuki Takahashi,
Yuta Tsuboi,
Yuya Unno,
Wilson Ko,
Jethro Tan

Comments: 9 pages. Submitted to International Conference on Robotics and Automation (ICRA) 2018. An accompanied video is available at this https URL

Subjects: Robotics (cs.RO); Computation and Language (cs.CL)

[3]  arXiv:1710.06117 [pdf, other] link

Title: Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning

Ayaka Kume,
Eiichi Matsumoto,
Kuniyuki Takahashi,
Wilson Ko,
Jethro Tan

Comments: 8 pages, 8 figures

Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Learning (cs.LG)

[4]  arXiv:1710.06092 [pdf, other] link

Title: Generalizing Informed Sampling for Asymptotically Optimal Sampling-based Kinodynamic Planning via Markov Chain Monte Carlo

Daqing Yi,
Rohan Thakker,
Cole Gulino,
Oren Salzman,
Siddhartha Srinivasa

Subjects: Robotics (cs.RO)

[5]  arXiv:1710.05947 [pdf, other] link

Title: Learning Data-Efficient Rigid-Body Contact Models: Case Study of Planar Impact

Nima Fazeli,
Samuel Zapolsky,
Evan Drumwright,
Alberto Rodriguez

Comments: 10 pages

Subjects: Robotics (cs.RO)

[6]  arXiv:1710.06422 (cross-list from cs.LG) [pdf, other] link

Title: Multi-task Domain Adaptation for Deep Learning of Instance Grasping from Simulation

Kuan Fang,
Yunfei Bai,
Stefan Hinterstoisser,
Mrinal Kalakrishnan

Comments: 9 pages, 8 figures, 1 table

Subjects: Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[7]  arXiv:1710.06406 (cross-list from cs.CL) [pdf, other] link

Title: Laying Down the Yellow Brick Road: Development of a Wizard-of-Oz Interface for Collecting Human-Robot Dialogue

Claire Bonial,
Matthew Marge,
Ron artstein,
Ashley Foots,
Felix Gervits,
Cory J. Hayes,
Cassidy Henry,
Susan G. Hill,
Anton Leuski,
Stephanie M. Lukin,
Pooja Moolchandani,
Kimberly A. Pollard,
David Traum,
Clare R. Voss

Comments: 7 pages, 2 figures, accepted for oral presentation at the Symposium on Natural Communication for Human-Robot Collaboration, AAAI Fall Symposium Series, November 9-11, 2017, this https URL

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Robotics (cs.RO)

[8]  arXiv:1710.06235 (cross-list from cs.CV) [pdf, other] link

Title: Real-time marker-less multi-person 3D pose estimation in RGB-Depth camera networks

Marco Carraro,
Matteo Munaro,
Jeff Burke,
Emanuele Menegatti

Comments: Submitted to the 2018 IEEE International Conference on Robotics and Automation

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[9]  arXiv:1710.06231 (cross-list from cs.CV) [pdf, other] link

Title: 3D Object Discovery and Modeling Using Single RGB-D Images Containing Multiple Object Instances

Wim Abbeloos,
Esra Ataer-Cansizoglu,
Sergio Caccamo,
Yuichi Taguchi,
Yukiyasu Domae

Journal-ref: Proceedings International Conference on 3D Vision 2017 (pp.

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[10]  arXiv:1710.06230 (cross-list from cs.CV) [pdf] link

Title: Fusion of LiDAR and Camera Sensor Data for Environment Sensing in Driverless Vehicles

Varuna De Silva,
Jamie Roche,
Ahmet Kondoz

Comments: Currently under review in IEEE sensors Journal

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[11]  arXiv:1710.05818 [pdf, other] link

Title: What Can Spatiotemporal Characteristics of Movements in RAMIS Tell Us?

Yarden Sharon,
Ilana Nisky

Comments: Preprint of an article submitted for consideration in Journal of Medical Robotics Research, \c{opyright} 2017 copyright World Scientific Publishing Company, this http URL

Subjects: Robotics (cs.RO)

[12]  arXiv:1710.05772 [pdf, other] link

Title: Data-Efficient Decentralized Visual SLAM

Titus Cieslewski,
Siddharth Choudhary,
Davide Scaramuzza

Comments: 8 pages, submitted to ICRA 2018

Subjects: Robotics (cs.RO)

[13]  arXiv:1710.05724 [pdf, other] link

Title: Reactive Planar Manipulation with Convex Hybrid MPC

Francois Robert Hogan,
Eudald Romo Grau,
Alberto Rodriguez

Subjects: Robotics (cs.RO)

[14]  arXiv:1710.05512 [pdf, other] link

Title: The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?

Roberto Calandra,
Andrew Owens,
Manu Upadhyaya,
Wenzhen Yuan,
Justin Lin,
Edward H. Adelson,
Sergey Levine

Comments: 10 pages, accepted at the 1st Annual Conference on Robot Learning (CoRL)

Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)

[15]  arXiv:1710.05502 [pdf, other] link

Title: Non-Iterative Localization and Fast Mapping

Chen Wang,
Lihua Xie,
Junsong Yuan

Subjects: Robotics (cs.RO)

[16]  arXiv:1710.05421 [pdf, other] link

Title: DDCO: Discovery of Deep Continuous Options forRobot Learning from Demonstrations

Sanjay Krishnan,
Roy Fox,
Ion Stoica,
Ken Goldberg

Comments: Published at CoRL 2017

Subjects: Robotics (cs.RO)

[17]  arXiv:1710.05419 [pdf, other] link

Title: Bodily aware soft robots: integration of proprioceptive and exteroceptive sensors

Gabor Soter,
Andrew Conn,
Helmut Hauser,
Jonathan Rossiter

Subjects: Robotics (cs.RO)

[18]  arXiv:1710.05268 [pdf, other] link

Title: Self-Supervised Visual Planning with Temporal Skip Connections

Frederik Ebert,
Chelsea Finn,
Alex X. Lee,
Sergey Levine

Comments: accepted at the Conference on Robot Learning (CoRL) 2017

Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)

[19]  arXiv:1710.05231 [pdf, other] link

Title: Hybrid DDP in Clutter (CHDDP): Trajectory Optimization for Hybrid Dynamical System in Cluttered Environments

Shushman Choudhury,
Yifan Hou,
Gilwoo Lee,
Siddhartha S. Srinivasa

Subjects: Robotics (cs.RO)

[20]  arXiv:1710.05048 [pdf, other] link

Title: Long-Term Inertial Navigation Aided by Dynamics of Flow Field Features

Zhuoyuan Song,
Kamran Mohseni

Comments: Accepted for publication in IEEE Journal of Oceanic Engineering

Subjects: Robotics (cs.RO)

Posted in Robotics | Tagged , , , , , , , , , , , , | Leave a comment

Generating a trajectory of quaternions between two given quaternions

First, check my previous post on slerp to learn how to interpolate between two quaternions.

In this post, we make a function that uses the previously developed interpolation to generate n points (quaternions) between and initial and a final quaternion namely, q_0, q_1.

Our previous function uses a ratio as an input that defines the ration (distance) from the first quaternion to the interpolated one. The main idea here is to start from the initial quaternion and increase the ration gradually from 0 to 1 by the given number of steps. At each level we interpolate and generate a new point and we add that to our trajectory.

Let say we want two point between the initial and the final given quaternions. The whole process is as follows:

n = 2
Q(1,:) = q_0
Q(2,:) = slerp(q_0,q_1,0.33)
Q(3,:) = slerp(q_0,q_1,0.66)
Q(4,:) = q_1

This process can be written in a function as follows:

function Q = quatinterpn(q0,q1,n)
q0 = quatnormalize(q0);
q1 = quatnormalize(q1);
Q = zeros(n,4);
Q(1,:) = q0;
Q(end,:) = q1;
h = linspace(0,1,n);
for ii=2:n-1
Q(ii,:) = quatinterp(q0,q1,h(ii),'slerp');

We can test the function as follows:

q0 = [1.0 0 1.0 0];
q1 = [-1.0 0 1.0 0];
n = 20;
Q = quatinterpn(q0,q1,n);

it will result in a trajectory as illustrated in this figure:



Posted in control, Linux, Machine Learning, MATLAB, programming, Robotics, ROS, Software, Ubuntu, Uncategorized | Tagged , , , , , , , , , , , , , , , , , , | Leave a comment

Slerp: Spherical linear interpolation in quaternion space

in my previous post, I explained the simple Lerp method for calculating linear interpolation in the quaternion space. I also mentioned that while normalizing the output could help to get an optimized result, the velocity curve is not constant. To avoid this problem, instead of doing a simple linear interpolation, we can interpolate on an arc on the quaternion unit sphere. This method is called Slerp which stands for Spherical linear interpolation and it works as follows:

Given q_0, q_1 \in H_1 and h \in [0,1] the following functions express slerp equivalently:

slerp(q_0,q_1,h) = p(p^*q)^h

slerp(q_0,q_1,h) = p(q^*)^{1-h} q

slerp(q_0,q_1,h) = (q p^*)^h p

slerp(q_0,q_1,h) = q(q^*p)^{1-h}

And it can be shown that:

slerp(q_0,q_1,h) = slerp(q_0,q_1,1-h)


Using these equations, Slerp follows the great arc which is also the shortest path on the unit sphere. Slerp also has the constant angular velocity.

A Matlab example is as follows:

q0 = quatnormalize([1.0 0 1.0 0]);
q1 = quatnormalize([-1.0 0 1.0 0]);
qi = quatinterp(q0,q1,0.5,'slerp')

qi =

0 0 1 0

Check out my previous post about Lerp which is a simpler linear interpolation.

Check out my next post about interpolating more than one point between two given quaternions and making a trajectory.

Posted in Linux, Machine Learning, MATLAB, Optimization, programming, Robotics, ROS, Software, Ubuntu, Uncategorized | Tagged , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Lerp: Linear interpolation in quaternion space

Lerp stands for Linear interpolation. It is method for linearly interpolate between quaternions.

Consider q_0 and q_1 as two quaternions and h \in [0,1]. Lerp works as follows:

Lerp(q_0,q_1,h) = q_0(1-h) + q_1 h

The interpolation curve for the linear interpolation (Lerp) gives a straight line in quaternion space and the line connects two points q_0 and q_1 with a shortcut that goes below the surface of the unit sphere. Normalizing the result could resolve this issue. However, the velocity curve would not be constant and because of that it is better to use other methods.

Matlab usage is as follows:

q0 = quatnormalize([1.0 0 1.0 0]);
q1 = quatnormalize([-1.0 0 1.0 0]);
qi = quatinterp(q0,q1,0.5,'lerp')

qi =

0 0 0.7071 0

Check my next post about Slerp.

Posted in Linux, Machine Learning, MATLAB, Optimization, Robotics, ROS, Software, Ubuntu, Uncategorized | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Robotics on arXiv 7/18/2017

Robotics on

Submissions on 18-07-2017

[1]  arXiv:1707.05303 [pdf, other] link

Title: Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model

Paul Drews,
Grady Williams,
Brian Goldfain,
Evangelos A. Theodorou,
James M. Rehg

Comments: 11 pages, 7 figures

Subjects: Robotics (cs.RO)

[2]  arXiv:1707.05301 [pdf, other] link

Title: Cheap or Robust? The Practical Realization of Self-Driving Wheelchair Technology

Maya Burhanpurkar,
Mathieu Labbé,
Charlie Guan,
François Michaud,
Jonathan Kelly

Comments: To appear in Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR’17), London, UK, July 17-20, 2017

Subjects: Robotics (cs.RO)

[3]  arXiv:1707.05110 [pdf, other] link

Title: Control of a Quadrotor with Reinforcement Learning

Jemin Hwangbo,
Inkyu Sa,
Roland Siegwart,
Marco Hutter

Subjects: Robotics (cs.RO)

[4]  arXiv:1707.05077 [pdf, ps, other] link

Title: Lower Bounds for Searching Robots, some Faulty

Andrey Kupavskii,
Emo Welzl

Subjects: Robotics (cs.RO); Distributed, Parallel, and Cluster Computing (cs.DC)

[5]  arXiv:1707.04821 [pdf] link

Title: Geometric Jacobians Derivation and Kinematic Singularity Analysis for Smokie Robot Manipulator & the Barrett WAM

Reza Yazdanpanah Abdolmalaki

Subjects: Robotics (cs.RO)

[6]  arXiv:1707.04820 [pdf] link

Title: Development of Direct Kinematics and Workspace Representation for Smokie Robot Manipulator & the Barret WAM

Reza Yazdanpanah Abdolmalaki

Subjects: Robotics (cs.RO)

[7]  arXiv:1707.04674 [pdf, other] link

Title: ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems

James Harrison,
Animesh Garg,
Boris Ivanovic,
Yuke Zhu,
Silvio Savarese,
Li Fei-Fei,
Marco Pavone

Subjects: Robotics (cs.RO)

[8]  arXiv:1707.04629 [pdf, other] link

Title: Compliant Movement Primitives in a Bimanual Setting

Aleksandar Batinica,
Bojan Nemec,
Aleš Ude,
Mirko Raković,
Andrej Gams

Comments: 1st, unpolished version, 7 pages

Subjects: Robotics (cs.RO)

[9]  arXiv:1707.04617 [pdf, other] link

Title: A Real-Time Solver For Time-Optimal Control Of Omnidirectional Robots with Bounded Acceleration

David Balaban,
Joydeep Biswas

Subjects: Robotics (cs.RO)

[10]  arXiv:1707.04796 (cross-list from cs.CV) [pdf, other] link

Title: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes

Pat Marion,
Peter R. Florence,
Lucas Manuelli,
Russ Tedrake

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[11]  arXiv:1707.04578 [pdf, other] link

Title: An Efficient Approach to Communication-aware Path Planning for Long-range Surveillance Missions undertaken by UAVs

Hrishikesh Sharma,
Tom Sebastian

Comments: 46 pages. One part of this thesis, handling the turn constrained route planning, has been published at ECMR’17

Subjects: Robotics (cs.RO)

[12]  arXiv:1707.04540 [pdf, other] link

Title: Autonomous Racing with AutoRally Vehicles and Differential Games

Grady Williams,
Brian Goldfain,
Paul Drews,
James M. Rehg,
Evangelos A. Theodorou

Comments: 8 pages, 7 figures

Subjects: Robotics (cs.RO)

[13]  arXiv:1707.04294 [pdf, other] link

Title: Ergodic Coverage In Constrained Environments Using Stochastic Trajectory Optimization

Elif Ayvali,
Hadi Salman,
Howie Choset

Comments: Accepted, IROS 2017

Subjects: Robotics (cs.RO)

[14]  arXiv:1707.04489 (cross-list from cs.AI) [pdf, other] link

Title: Freeway Merging in Congested Traffic based on Multipolicy Decision Making with Passive Actor Critic

Tomoki Nishi,
Prashant Doshi,
Danil Prokhorov

Comments: 6 pages, 5 figures. ICML Workshop on Machine Learning for Autonomous Vehicles

Subjects: Artificial Intelligence (cs.AI); Robotics (cs.RO)

[15]  arXiv:1707.04444 (cross-list from cs.CV) [pdf, ps, other] link

Title: Monocular Visual Odometry for an Unmanned Sea-Surface Vehicle

George Terzakis,
Riccardo Polvara,
Sanjay Sharma,
Robert Sutton

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[16]  arXiv:1707.04173 [pdf] link

Title: Review: Modeling and Classical Controller Of Quad-rotor

Tarek N.Dief,
Shigeo Yoshida

Comments: IRACST – International Journal of Computer Science and Information Technology & Security (IJCSITS), ISSN: 2249-9555 Vol. 5, No4, August 2015

Subjects: Robotics (cs.RO)

[17]  arXiv:1707.04084 [pdf, other] link

Title: An Earthworm-Inspired Soft Crawling Robot Controlled by Friction

Joey Z. Ge,
Ariel A. Calderón,
Néstor O. Pérez-Arancibia

Comments: 8 pages, 9 figures, 1 table

Subjects: Robotics (cs.RO)

[18]  arXiv:1707.03976 [pdf, other] link

Title: Asymptotic Optimality of Rapidly Exploring Random Tree

Titas Bera,
Debasish Ghose,
Sundaram Suresh

Subjects: Robotics (cs.RO)

[19]  arXiv:1707.04151 (cross-list from cs.LO) [pdf, ps, other] link

Title: The Reach-Avoid Problem for Constant-Rate Multi-Mode Systems

Shankara Narayanan Krishna,
Aviral Kumar,
Fabio Somenzi,
Behrouz Touri,
Ashutosh Trivedi

Comments: 26 pages

Subjects: Logic in Computer Science (cs.LO); Formal Languages and Automata Theory (cs.FL); Robotics (cs.RO)

[20]  arXiv:1707.03899 (cross-list from math.AT) [pdf, other] link

Title: A Topologist’s View of Kinematic Maps and Manipulation Complexity

Petar Pavešić

Comments: 24 pages, 15 figures

Subjects: Algebraic Topology (math.AT); Robotics (cs.RO)

[21]  arXiv:1707.03734 [pdf, other] link

Title: A Decentralized Multi-Agent Unmanned Aerial System to Search, Pick Up, and Relocate Objects

Rik Bähnemann,
Dominik Schindler,
Mina Kamel,
Roland Siegwart,
Juan Nieto

Comments: Submitted to the 15th IEEE International Symposium on Safety, Security, and Rescue Robotics 2017 (SSRR 2017)

Subjects: Robotics (cs.RO)

[22]  arXiv:1707.03804 (cross-list from cs.CL) [pdf, other] link

Title: Source-Target Inference Models for Spatial Instruction Understanding

Hao Tan,
Mohit Bansal

Comments: 11 pages

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Learning (cs.LG); Robotics (cs.RO)

[23]  arXiv:1707.03469 (cross-list from cs.CV) [pdf, ps, other] link

Title: Mobile Robot Localization via Machine Learning

Alexander Kuleshov,
Alexander Bernstein,
Evgeny Burnaev

Comments: 15 pages

Subjects: Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Robotics (cs.RO); Applications (stat.AP)

[24]  arXiv:1707.03435 (cross-list from cs.CV) [pdf] link

Title: Obstacle detection test in real-word traffic contexts for the purposes of motorcycle autonomous emergency braking (MAEB)

Giovanni Savino,
Simone Piantini,
Gustavo Gil,
Marco Pierini

Comments: 25th International Technical Conference on the Enhanced Safety of Vehicles (2017)

Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

[25]  arXiv:1707.03080 [pdf, other] link

Title: Robot Autonomy for Surgery

Michael Yip,
Nikhil Das

Subjects: Robotics (cs.RO)

Posted in Robotics | Tagged , , , , , , , , , , , , , | Leave a comment

Robotics on arXiv – 6/July/2017

Robotics on

Submissions on 06-07-2017

[1]  arXiv:1707.01152 [pdf, other] link

Title: Improving Foot-Mounted Inertial Navigation Through Real-Time Motion Classification

Brandon Wagstaff,
Valentin Peretroukhin,
Jonathan Kelly

Comments: To appear in proceedings of 2017 International Conference on Indoor Positioning an Indoor Navigation (IPIN), 18-21 September 17, Sapporo, Japan

Subjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)

[2]  arXiv:1707.01495 (cross-list from cs.LG) [pdf, other] link

Title: Hindsight Experience Replay

Marcin Andrychowicz,
Filip Wolski,
Alex Ray,
Jonas Schneider,
Rachel Fong,
Peter Welinder,
Bob McGrew,
Josh Tobin,
Pieter Abbeel,
Wojciech Zaremba

Subjects: Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Robotics (cs.RO)

Posted in Robotics | Tagged , , , , , , , , , | Leave a comment