STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation
Authors: Mohak Bhardwaj
Balakumar Sundaralingam
Arsalan Mousavian
Nathan D. Ratliff
Dieter Fox
Fabio Ramos
Byron Boots
Venue: CoRL2021 Oral (2021)
Keywords: model-predictive control
manipulation

Abstract: Sampling-based model-predictive control (MPC) is a promising tool for feedback control of robots with complex, non-smooth dynamics, and cost functions. However, the computationally demanding nature of sampling-based MPC algorithms has been a key bottleneck in their application to high-dimensional robotic manipulation problems in the real world. Previous methods have addressed this issue by running MPC in the task space while relying on a low-level operational space controller for joint control. However, by not using the joint space of the robot in the MPC formulation, existing methods cannot directly account for non-task space related constraints such as avoiding joint limits, singular configurations, and link collisions. In this paper, we develop a system for fast, joint space sampling-based MPC for manipulators that is efficiently parallelized using GPUs. Our approach can handle task and joint space constraints while taking less than 8ms~(125Hz) to compute the next control command. Further, our method can tightly integrate perception into the control problem by utilizing learned cost functions from raw sensor data. We validate our approach by deploying it on a Franka Panda robot for a variety of dynamic manipulation tasks. We study the effect of different cost formulations and MPC parameters on the synthesized behavior and provide key insights that pave the way for the application of sampling-based MPC for manipulators in a principled manner. We also provide highly optimized, open-source code to be used by the wider robot learning and control community. Videos of experiments can be found at: https://sites.google.com/view/manipulation-mpc

STORM: An Integrated Framework for Fast Joint-Space Podel-Predicttvv Control for Reactive Manipulation
Mohak Bhardwaj Balakumar Sundaralingam' / Arsalan Mousavian', Nathan Ratliff
Dieter Fox!2, Fabio Ramos Byron Boots
IVVIIIA
University of Washington
University of Sydney
A Abstract: Sampling-basee model model-predictive predictive control (MPC) is a promising tool for feedback control of robots with complex, non-smooth dynamies, and cost fune- tions. However, the computationally demanding nature of sampling-base algorithms has been a key bottleneck in their application to high-dimensional robotic manipulation problems in the real world. Previous methods have ad- dressed this issue by running MPC in the task space while relying on a low- level operational space controller for joint control. However, by not using the joint space of the robot in the MPC formulation, existing methods cannot directly account for non-task space related constraints such avoiding joint limits, sin- gular configurations, and link collisions. In this paper, we develop a system for fast, joint space campling-basee MPC for manipulators that is efficiently paral- lelized using GPUs. Our approach can handle task and joint space constraints while taking less than 8ms (125Hz) compute the next control command. Fur- ther, our method can tightly integrate perception into the control problem by uti- lizing learned cost functions from raw sensor data. We validate our approach by deploying it on Franka Panda robot for a variety of dynamic manipula- tion tasks. We study the effect of different cost formulations and MPC param- eters on the synthesized behavior and provide key insights that pave the way for the application of sampling baased based MPC for manipulators a principled man- ner. We also provide highly optimized, open-source code to be used by the wider robot learning and control community. Videos of experiments can be found at: https:/ssitesgooolle.oom www.lel...oommee//a/niaaaaaaaaaa
MPC
Introduction
Real-world robot manipulation can greatly benefit from real-time perception-diis driveen feedback con- trol [1, 2]. Consider an industrial robot tasked with stacking boxes from one pallet another robot bartender moving drinks placed on tray [3]. In both cases, the robot must ensure object stabil- ity via perception while respecting joint limits, avoiding singular configurations and collisions, and handling task constraints such as maintaining orientation during transfer. This leads to complex control problem with competing objectives that is difficult to solve in real-time.
Classic approaches solving these tasks rely on operational-space control (OSC) [4, 5, 6]. where the different task costs are formulated in their respective spaces and then projected into the joint space (i.e., the control space of the robot) via a Jacobian map. OSC methods are inherently local as they only optimize for the next time step while ignoring future actions or states.
MPC based approaches attempt find locally-optimal policy over finite horizon starting from the current state using potentially imperfect dynamics model. An action from the policy is ex- ecuted the system and the optimization is performed again from the resulting next state which can overcome the effects of model-biag bias. MPC has been successfully applied on real robotic systems that operate in the joint space of manipulator are limited to gradient-bart baseentt-bbddddd approaches [11, 12]
allowing them to rapidly react to changes in the environment [7. 8, 9, 10]. Existing MPC methods
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

Our sampling- based model-predienes control framework operates in the joint space to enable robol achieve manipulation objectives such tracking moving larget balancing ball on a plate while respecting constraints such as joint limits, singularity avoidance and collision avoidance via learned collision checking from raw sensor data.
Figure 1:
which require the cost and dynamics to be differentiable. However, manipulation tasks often involve discontinuous phenomena such as contact and complex cost terms that are hard differentiate an- alytically. Sampling based methods such as Model-Predictive Path Integral (MPPI) [13] and Cross Entropy-Method (CEM) offer a promising alternative. Here, control sequences are sampled using a simple policy followed by rolling out the dynamics model to compute sample- sample-based gradient of the objective function. These algorithms make no restrictive assumptions about the cost, dy- namics or policy class, are straightforward to parallelize and can be effectively applied on highly dynamic systems [10, 14]. These properties have also been a major factor in the increased adoption of sampling-basee based MPC by the Model-basee RI. community in recent years [15]
However, a key question still remains to be answered- how well do these control algorithms transfer to high-dimensional robots like manipulators? In this work, we describe an integrated system for sampling- based MPC that aims to answer this question. Our proposed framework, Stochastic Tensor Optimization for Robot Motion (STORM) implements highly-parallehne parallelized control architecture that can optimize complex task objectives while simultaneously ensuring desirable properties such as smoothness, constraint satisfaction, and low control latency.
A major criticism of sampling-hasee MPC algorithms for full joint space control has been their inability to produce smooth (low-jerk) trajectories [16]. To address this challenge, we study dif- ferent sub-components of sampling-based MPC and make several novel contributions such as low discrepancy action sampling, smooth interpolation and cost-function design, and demonstrate their effectiveness in scaling sampling-based methods to real robot manipulators. We also demonstrate that STORM can incorporate learned components in the control loop, by using learned self and environment collision costs. We integrate our system on a real Franka Panda robot arm where we demonstrate that feedback driven sampling-bast based MPC is able to solve complex and dynamic ma- nipulation tasks with simple models. In summary, our major contributions are'
1. A novel sampling- based MPC with the introduction of low discrepancy sampling, smooth trajectory generation and behavior- based cost functions that are key for producing smooth, reactive, and precise robot motions.
2. A feedback control framework that directly integrates learned perception components into the control loop in the form of learned self collision cost and discrete collision checker between the robot links and raw environment pointcloud from [17].
3. An open-source and highly optimized implementation of sampling-bart -based joint-space MPC, which achieves control rate of 125Hz on a single GPU, a speedup of 100x compared to existing MPPI based manipulation implementations [17].
4. Empirical evaluation in simulation and a real-world Franka Panda robot on dynamic control tasks.
2 Problem Definition
We consider the problem of generating a feedback control law (or policy) for robot manipula- with joints performing user-specified tasks in unstructured environment where is subject
For further discussion of related work please refer the supplementary material.

---

Variable
Description
Algorithm Sampling-Bane -aapld MPC
0.. 0, 0
joint position. velocity, acceleration robot state at time joint space command at time 1
21
10 d2, Be]
Input :0:
Parameter: H. N, for f == 1 do
10, H)
number of steps horizon batch of control sequences iterations of optimization change in time between steps in horizon vector of [0, H)
4- GRT STATE( me SHIFT()
n
O,
K
for 1... K
dt
5
u SAM-
COBTROOOTOSST H, N)
8, C, GENER- ATE.ROLLOUTS(s,,
Φ {
parameters of policy at time distribution over control given state sequence of H Gaussian means sequence of H Gaussian Covariances batch of N control sequences of length
Tá.
14 = Pr,h
7
de <-
E, Eeth
è, u)
Up
end
8 &
batch of N state sequences of length H batch of N cost sequences of length H
U4 NEXT SEXT_COMMAND(T.
10
EXECUTEE_COMANNDÉN
L
MPC loss function
11 end
Figure 2: We summarize the notations used on the and the sampling based MPC algorithm on the right.
to non-linear constraints and must react in real-time overcome errors due its internal dynam- ics model, state estimation, and perception. The problem can be modelled as optimal control of discrete-time stochastic dynamical system described by the equation, Jz411 P(x++ 1 ( ux), where P(I++ ( wx) defines the probability of transitioning into state 2 conditioned on 2; and control input u, E Rd. The robot chooses controls using a deterministic or stochastic closed-loop policy (- (ar)) incurs instantaneous cost c(22, We) and transitions into the next state, and the process continues for a finite horizon T.
The goal is to design a policy that optimizes a task-specific arg minE nell
objective function,
ET c(2,, u4)
where the space of all policies.
The above setup is akin to optimizing finite horizon Markov Decision Process (MDP) with con- tinuous state and action spaces as done reinforcement learning approaches [18]. Solving for complex globally optimal policy is a hard problem especially since the task objective could be sparse difficult to optimize. MPC can be viewed as a practically motivated strategy that simplifies the overall problem by focusing only on the states that the robot encounters online during execution and rapidly re-calculating a "simple" locally optimal policy. At state Tt, MPC performs look- ahead for a shorter horizon H < T using an approximate model P( 1 az), approximate cost function 0(2, ax) and a parameterized policy T to find the optimal parameters o" that minimize an objective function
Φ- arg argmin L(e, q Tay P,
where q(-) is terminal cost function that serves coarse approximation of the cost-to-go beyond the horizon. An action is sampled from Tg- and the optimization is performed again from the resulting state after applying the action to the robot. The optimization is hot-started from the solution at the previous timestep by using a shift operator, which allows MPC to produce good solutions with few iterations of optimization (usually in practice).
In the next section, we first introduce a sampling- -bsee MPC technique to solving the optimization in Eq. I and discuss the objective function, policy class, and update equations. We then present our approach for applying to manipulation problems. An overview the notation used the paper is presented in Fig. 2.
3 Sampling-Basse Model Predictive Control
Sampling-based based MPC iteratively optimizes simple policy representations such as time-independant Gaussians over open-loop controls with parameters de such that To. =II To.. Here, Φ, repre-

---

sent the sequence of means j4 [14.0... ...0...,,,.. H-1 and covariances E, ........ E1. at every step along the horizon H. standard algorithm shown in Fig. 2. At every iteration, the optimiza- tion proceeds by sampling a batch of N control sequences of length H. MnE(o.N) (..(o..))eee..8) from the current distribution (Line 5), followed by rolling out the approximate dynamies function using the sampled controls to get a batch of corresponding states Xne(o,N),Me N),,e(0,H) and costs (ne (Line 6). The policy parameters are then updated using a sample-based gradient of the objective function (Line 7). After K Z optimization iterations we can either sample action from the resulting distribution or execute the first action from the mean (Line 9). We next describe how the distribution is updated. Consider the function C(.)
H-2 Σ M ue,h) + -0
C( u,) =
(2)
where y E [o, 1] is discount factor that is used to favor immediate rewards. A widely used objective function is the exponentiated utility or the risk-seeking objective,
L= E B... exp (-) (-)((1),
mi) 60
(3)
where is a temperature parameter. For this choice of objective, the mean and covariance are updated using a sample-based gradient as,
Ei=1 ui
Mr, = - -ap)ppr + a2
(4)
Σi=1 W/
Σ 1i=1 Ws Wi(ue - pr,a) - Me,))
Σn, (1- ag
(5)
Σ wi
where Og and ag are step-sizes that regularize the current solution to be close to the previous one and,
-1 H 2
w. =xp B Σ
(( + (tt.::nn)++--11(((( +, 0H-1,,
(6)
The update equation for the mean is the same as the well-known Model-Predictive Path Integral Control (MPPI) algorithm [13]. We refer the reader to [19] for the connection and derivation of update equations. While covariance adaptation has previously been explored in the context of Path Integral Policy Improvement [20] to automatically adjust exploration, standard implementations of MPPI on real systems generally do not update the covariance [13, 19]. However, we observed that updating the covariance leads better performance with a fewer number of particles, such as stable behavior upon convergence to the goal. Once an action from the updated distribution is executed on the system, the mean and covariance are shifted forward with default values appended the end to warmstart the optimization at the next timestep (Line 3).
The above formulation of MPC offers the flexibility to extract different behaviors from our algorithm by tuning different knobs such as the choice of approximate dynamics, running cost, terminal cost, the policy class and parameters such the horizon length, number of particles, step sizes and discount factor Next, we switch our focus to the domain of robot manipulation and build our approach for sampling-basse based MPC by systematically evaluating the effects of a subset of key design choices on the overall performance of the controller.
3.1 Approximate Model
The MPC paradigm allows us to effectively leverage simple models that are both computationally cheap to query and easy to optimize, re-optimizing the control policy every timestep can help to overcome the effects of errors in the approximate model. We leverage this error correcting property of MPC and utilize the kinematic model of the manipulator our approximate transition model.
Let the robot state be defined in the joint space be the joint acceleration E Rd. At every optimization iteration, we compute the state of the robot across the horizon (& = 10n,h, ,n,,, 6,,n]. h E [0, H), E [0, N)) by integrating the sampled
[0, ô, Ø E 3d and the commanded action

---

control sequences N. Ae from the robot's initial state 2iit = [Q Amit mmi]] (i.e., current state of the real robot). This can be efficiently implemented as a tensor product followed by sum:
6 + S(1 diag(dt)
θ mni + (1) diag(dt)
where (1(1) is a lower triangular matrix filled with 1, and dt is vector of delta timesteps across the horizon". We use variable timesteps, with a smaller dt for the earlier steps along the horizon for higher resolution cost near the robot's ' current state and larger ones for later steps to get better approximation of cost-to-go, similar to [8]. We intentionally write @n,h as to highlight the fact that we compute the states across the batch and horizon with tensor operation without the need to iteratively compute the states across the horizon. This significantly speeds up the computation and is key to achieving the 8ms control latency.
Given a batch of states &, the Cartesian poses X SE(3), velocities X, and accelerations X of the end-effector are obtained by using forward kinematics F FK(O), the kinematic Jacobian J(O), and its derivative J(())a as
X x= FK(()
x= J(O)
X +()
We can compute the forward kinematics in batch as they depend only on the current state. This pro- vides an easily parallelizable formulation of the robot model that amenable to GPU acceleration.
3.2
Cost Function
The cost function č(s, a) encodes high-level robot behavior directly into the MPC optimization. This can be viewed as form of cost-shaping that allows MPC to achieve sparse task objectives while also satisfying auxillary requirements such as avoiding joint limits, ensuring smooth motions and safety. We consider cost functions that are weighted sum of simple cost terms, where each individual term encodes desired robot behavior that can be easily implemented in a batched manner. We describe a subset of the cost terms here and refer the reader to the supplementary for more details.
3.2.1 Stopping for Contingencies
The finite horizon makes MPC myopic events that can occur further in the future. Thus, it is desirable to ensure that the robot can safely stop within the horizon reaction to events that might be observed at timestep H especially in dynamic environments. We encode this behavior by computing a time varying velocity limit 0mar RR for every timestep in the horizon based user-specifed maximum acceleration 6mar and the time until H 1 1. This means the joint velocity of the robot must allow to come to a stop at the end the horizon by applying the max acceleration. Any state that exceeds this velocity is penalized by
which is expressed
[11mmar,t 10ll22
if (mman - 10] 0.0 otherwise
Ómar Sa (a))0)
Extop (0.) 110
where (1) is an upper triangular matrix filled with
3.2.2 Avoiding Cartesian Local Minima
The manipulability score describes the ability of the end-effector to achieve any arbitrary velocity from given joint configuration. measures the volume of the ellipsoid formed by the kinematic Jacobian which collapses to zero at singular configurations. Thus, to encourage the robot to optimize control policies that avoid future kinematic singularities, we employ a cost term that penalizes small manipulability scores [21, 22]
[1.0 ((O)))(())) (f.J(()))) < Km
Emanip (9t)
(10)
0..0,
otherwise
We use semi- immiiimpicit Euler integration which provides more stable solutions than explicit Euler due to it being symplectic. See https:
com/ con/pott//ttggg/tttttttttte bass
5

---

3.2.3 Self Collision Avoidance
Computing self-collision between the links of the robot for large number of configurations can be computationally expensive [23, 17]. Hence, similar to previous approaches [23, 17], we train a neural network that predicts the closest distance between the links of the robot given joint configuration (0). A difference our approach (termed jointNERF) is the use of positional encod- ing (i.e..(sn(()), co)( that improves the accuracy of the distance prediction [24]. We compute cost term as: &eett-ooll (0,) = max(0, (o.joooNNERF((.)
3.2.4 Environment Collision Avoidance
Safe operation in cluttered environments requires tight coupling between perception and control for collision avoidance. Gradient-based approaches [25] generally rely on either known object shapes or pre-computed signed distance fields that provide gradient information for optimization. However, our sampling-bast -ased approach can handle discrete costs and as such we explore collision avoidance without using signed distances. Specifically, we use learned collision checking function from Danielczuk et al. [17] that operates directly on raw pointcloud data and classifies an robot link pointcloud pc; is in collision with the environment pointcloud peane given the robot link's pose X₁-
1, if collision,
Cooll p Xi)
(11)
0, otherwise.
3.3 Sampling Strategy for Control Sequences
Random Comb
Halton Comb
The method used for sampling controls from the Gaus- sian policy can have a great impact on the convergence of the optimization and can help embed different desirable behaviors such as ensuring smoothness. Pseudorandom sequences typically used in Monte Carlo integration ex- hibit an undesirable clustering of sampled points which results in empty regions. Whereas low-discrepancy se- quences, where low discrepancy refers to the degree of deviation from perfect uniform sampling, alleviate this problem by defining deterministic samplers that correlate each point to avoid groupings [27]. Halton sequences [28] are a widely used form of low discrepancy number gen- erators that attempt to improve the rate of convergence of Monte Carlo algorithms, and are reported to achieve superior performance in high-dimensional problems [29].
Random B-Spline
Halton B-Spline
Figure 3: We compare our sampling scheme on a planar robot reacher task [26]. The holonomic robot must move from its ini- tial position (green cross) to the desired po- sition (red cross) while avoiding obstacles shown as the grey regions. The path taken by the robot is given by the black dot-da line and red circles after every 30 timesteps. The blue lines the rolled out trajectories of the top parti- cles. Our Halton B-Spline is able to find a smooth short path to the goal while Random B Spline takes longer path. Halton with comb filter is not smooth as shown by the sudden path changes.
In particular, the Halton sequence uses the first pi....,PPA Pa
prime numbers to define a sequence, W1, W2, ................................ for in-
tegers 2 0 and b 2 2 where w, = (0 (i)..... 2n (t)) and (1) = Σa=1 lab= a with i = Ex= ba for b 1} We incorporate Hal-
io, i.. E ton sequences for sampling controls that can provide better estimate of the objective function gradient. Con- trols from the Halton sequence sampled once at the beginning and then transformed using the mean (Ht) and Covariance E, of the current Gaussian policy.
positions of the robot
Furthermore, we explore two different strategies for en- forcing smoothness in sampled control sequences. The first method is a comb filter that uses user-specified co- efficients [c1, C2 ca] to filter out each sampled control trajectory along the horizon as Utch uy + C2 + C3 This method has previously been used with sampling-bast control techniques [30, 31], however, it requires extensive tuning and the filtered trajectories not guar- anteed to be smooth as neighboring samples in the horizon can have large difference in magnitude.
2 DDstancce positive when two links are penetrating and negative when not colliding. "See ht tttp::// / -quasiir equasirandom sequences/ for visualization of different discereryyyacc methods.
con. au/ wtw.asonaaaabccccco wwwassnableefefeeoee feee i Vveness-oo of

---

Figure 4: We show sequence from our collision avoidance experiment where the robot has to move between thin walls reach the orange ball held by the human. The robot tries reach the ball but moves only close as possible as any further motion would cause the elbow to hit the right book.
We propose an alternate strategy to enforce smoothness by fitting B B-splines of degree 3 to controls sampled using a Halton Sequence. The resulting curve is sub-sampled at a finer resolution to obtain smooth joint acceleration samples which are then integrated to obtain corresponding joint velocity and position trajectories using Eq 7. In Fig. 3 we show a qualitative comparison between different combinations of sampling and smoothing strategies for a planar robot trying to reach a goal while avoiding obstacles where we see that our Halton + B-Spline sampling strategy is able to better explore the action space while maintaining smoothness (see supplementary for details).
Covariance Parameterization: Conventionally, sampling- based MPC algorithms such as MPPI parameterize the covariance of the Gaussian policy to be of the form Σ Ou Laxd where ow is a scalar value and Laxd is a d x identity matrix, which forces the covariance to be the same across all control dimensions. However, in the case of manipulators it is desirable to allow the covariance of different joints to adapt independently so they can potentially optimize different cost terms such as pose reaching versus increasing manipulability. Thus also consider covariance of the form Dt = T., laxd where da = [o....., oal. Each term in du is then adapted based on the rollouts. Adapting the covariance along action dimensions has also been employed by [15] for CEM.
Our sampling strategy also offers us the flexibility of incorporating certain fixed set of action trajec- tories which could be task-dependent or even library of pre-defined desired motions. We leverage this fact by incorporating a set of zero acceleration or "null" particles which allows the robot to coast at a constant velocity once the robot accelerated sufficiently and also easily stop at the goal as demonstrated in our experiments.
4 Experimental Evaluation
Through our experiments we aim to analyze the effectiveness of STORM as framework for real- time, perception-driven feedback control in real-world manipulation scenarios. To this end, we first study the performance of STORM M in reacting to changing end-effector targets from perception data while satisfying task constraints such as maintaining user-specified orientation and avoiding obsta- cles in cluttered scenes. Second, we consider the dynamic task of balancing a ball on a tray grasped by the end effector that uses an approximate model of the ball dynamics. We also provide qualitative ablations in simulation for different components of our framework such as cost terms, sampling strat- egy and policy parameterization We additionally compare our approach to MOVELT! and OSC [32] for the standard pose reaching problem in the supplementary material.
website (https://sites, wwo.ttesssgogee....eeeee gwwgle.com//iiee.e..o com//iew/manipulatioo mmmp)).
4.1 Tracking Moving Targets while Handling Task Constraints
key strength of feedback-based MPC over "plan and execute" and OSC approaches its ability to simultaneously optimize complex cost functions over long horizon while demonstrating reactive behavior. We demonstrate this by having the robot react to changing goal poses obtained from noisy perception, while satisfying task constraints such maintaining desired orientation and avoiding obstacles. In these experiments, a ball held by a human is tracked using a depth camera and the robot tries to reach the ball as the human moves it to different locations in the workspace.
Demonstrating reactive motion and reasoning about obstacle avoidance in cluttered environments, while simultaneously coordinating large degrees of freedom leads to a hard online optimization problem. Standard planning and OSC approaches often fall short in optimizing for such behavior.
Obstacle Avoidance:
7

---

To test our method's capability to handle such scenar- ios, we setup two different table top environments, one consisting of two blocks representing common pick and place environments as shown in Fig. and an environ- ment with thin walls to represent a densely occupied space, as shown in Fig. 4. We use our perception based ball tracker to make the robot reach different positions in the environment. During the experiment, we also move the ball to some positions that are not reachable by the robot due to possibility of collision between the robot's s links and the obstacles. As seen in Fig. 4, the robot han-
0.5
Postion (m)
0.0
-0.5
-1.0
120
Time (sec)
Orvertation Error %
dles these situations very well as it prioritizes collision avoidance over reaching the pose accurately. We present our full obstacle avoidance experiments as well as several experiments in simulation in the accompanying videos.
820 100
1.22
120
Orientation Constraints: Several common manipula- tion tasks such as moving filled cup require maintaining orientation during motion which reduces the feasibility region of a controller. We test this scenario by imposing orientation constraints on the end-effector while tracking the ball. Our controller achieves a median quaternion er- ror of .288%% while tracking the ball with sufficient ac- euracy as shown in Fig. 5.
(SBC.)
Figure 5: We move the ball across the workspace while having a high weight on maintaining specific orientation of the end- effector. Our control scheme can maintain the orientation during motion as seen by the very low orientation error (< 3%).
4.2 Dynamic Object Balancing
0.0
We consider a hard dynamic manipulation task where the robot tries to balance a ball placed a tray grasped by a parallel jaw gripper as shown in Fig. 1. The location of the ball is tracked using the RGBD input from Re- alSense camera at 30Hz. We use a simplified dynamics model of the object rolling on the tray under acceleration due to gravity and do not explicitly account for friction or inertial properties of the ball and do not perform any sys- tem identification. The purpose of this task is to demon- strate the robustness of MPC under severe model-bias while optimizing complex objectives and dealing with noisy perception. Fig. shows the trajectory taken by the ball when the robot is controlled by STORM. MPC with its forward lookahead and rapid re-optimization able to anticipate the ball's motion and correct for errors due to inaccurate dynamics and perception to center the ball on the tray with an error of less than 4cm.
-1.2
-334
-335
-350
4.55
-2.05 11.00
0.5% a1s
Figure 6: The observed trajectory of the ball in the frame of the tray for the balancing task is shown able to reach an error of less than 4em us- ing simplified dynamics model and noisy perception inputs.
the plot. Our controller
Our experimental results indicate that accurate and reactive manipulation behavior can be obtained by using relatively simple models and intuitive cost functions in an MPC framework. Sampling- based MPC also provides us the flexibility to encode different desired behaviors such as smooth motions directly in the optimization and tightly couple perception with control which are key com- ponents for real-world robot control.
5 Discussion
We presented a sampling-basee MPC framework for manipulation that operates in the joint space and is able achieve smooth and reactive motions while respecting constraints, and demonstrated its performance on dynamic control tasks. The first key component of our approach a fully tensorized kinematic model that allows for GPU-based acceleration of rollouts. Second, we leverage intuitive cost terms that encourage desirable behaviors. Third, our formulation allows us leverage diverse sampling strategies to embed desirable properties directly into the optimization. However, a few key questions remain. First, performance can be made more robust by directly accounting for state un- certainty in the control loop. Second, at higher speeds, the kinematic model might induce significant model-bias. Here, learning a residual dynamics model [33] or a terminal Q-function [34] can help mitigate the effects of model-bias while still maintaining computational speed.
8

---

References
[1] Bohg, K. Hausman, B. Sankaran, O. Brock, D. Kragie, S. Schaal, and G. Sukhatme. Inter- active perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics, 33(6); 1273-1291,, 2017.
[2] D. Kappler, F. Meier, Issac, Mainprice, C. G. Cifuentes, M. Wüthrich, V. Berenz, S. Schaal, N. Ratliff, J. Bohg. Real-time perception meets reactive motion generation. IEEE Robolics and Automation Letters, 3(3):1864-187)) 2018.
[3] Luo and K. Hauser. Robust trajectory optimization under frictional contact with iterative learning. Aulonomous Robots, 1166)1144711461 2017.
[4] J. Nakanishi, R. Cory, M. Mistry, J. Peters, and S. Schaal. Operational space control: A theoretical and empirical comparison. The International Journal of Robotics Research, 27(6): 737-757, 2008. 10i: 7001177700788344000919666 URL https: /ow. org/ otg/10.11771 02783649080914
[5] C.-A. Cheng, M. Mukadam, J. Issac, Birchfield, Fox, B. Boots, and N. Ratliff, Rmpflow: A computational graph for automatic motion policy generation. In International Workshop on the Algorithmic Foundations of Robotics, pages 441 1577. Springer, 2018.
[6] A. Dietrich, T. Wimbock, A. Albu-Schaffer, and G. Hirzinger. Reactive whole-body control: Dynamic mobile manipulation using large number of actuated degrees of freedom. IEEE Robotics & Automation Magazine, 19(2):20-33, 2012.
[7] M. Bangura and R. Mahony, Real-time model predictive control for quadrotors. IFAC Pro- ceedings Volumes, 47(3):11773-1111
2014.
[8] T. Erez, K. Lowrey, Y. Tassa, V. Kumar, S. Kolev, and E. Todorov. An integrated sys- tem for real-time model predictive control of humanoid robots. In 2013 13th IEEE-RAS International Conference Humanoid Robots (Humanoids), pages 292-292222229 299, 2013. doi: 2.N.UMAANIDSS.....7000
[9] N. Scianca, D. De Simone, Lanari, and G. Oriolo. Mpc for humanoid gait generation: Stability and feasibility. IEEE Transactions on Robotics, 36(4):1 2020. doi:10. 1109/TRO.201 191199998883.
[10] G. Williams, Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou. Aggressive driving with model predictive path integral control. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 1433-1440. IEEE, 2016.
[11] Y. Tassa, N. Mansard, and E. Todorov. Control-limited differential dynamic programming. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 1168-1175. IEEE, 2014.
[12] G. Williams, A. Aldrich, and E. A. Theodorou. Model predictive path integral control: From theory to parallel computation. Journal of Guidance, Control, and Dynamics, 40(2):344-357, 2017.
[13] G. Williams, N. Wagener, B. Goldfain, P. Drews, M. Rehg, B. Boots, and E. A. Theodorou. Information theoretic mpc for model-based reinforcement learning. In 2017 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pages 1714-1721. IEEE, 2017.
[14] N. Wagener, C.-A. Cheng, J. Sacks, and B. Boots. An online learning approach to model predictive control. 2019.
[15] K. Chua, R. Calandra, R. McAllister, and Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. arXiv preprint 180..22445.11,14,, 2018.
[16] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning, pages 1101-1112, PMLR, 2020.
[17] M. Danielczuk, A. Mousavian, C. Eppner, and D. Fox. Object rearrangement using learned implicit collision functions. arXiv preprint arXiv:2011. 2020.

---

[18] M. Zhong, M. Johnson, Y. Tassa, T. Erez. and E. Todorov. Value function approximation and model predictive control. In 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL.), pages 100-107, 2013. doi: 10. 109//DDPR.2213366611
[19] N. Wagener, C.-A. Cheng, J. Sacks, and B. Boots. An online learning approach to model predictive control. arXiv preprint arXiv:1902.089667 2019.
[20] Stulp and O. Sigaud. Path integral policy improvement with covariance matrix adaptation. arXiv preprint arXiv: 206.46221 2012.
[21] C. A. Klein and E. Blaho. Dexterity measures for the design and control of kinematically redundant manipulators. The international journal of robotics research, 6(2):72-83, 1987.
[22] N. Vahrenkamp, T. Asfour, G. Metta, G. Sandini, and R. Dillmann. Manipulability analysis. In 2012 12th ieee-ras international conference on humanoid robots (humanoids 2012), pages 568-573. IEFE, 2012.
[23] D. Rakita, B. Mutlu, and M. Gleicher. RelaxedIK: Real-time Synthesis of Accurate and Fea- sible Robot Arm Motion. In Proceedings of Robotics: Science and Systems, Pittsburgh, Penn- sylvania, June 2018. fof.10.15607/RSS..188..8 XIV.0443.
[24] B. Mildenhall, P. Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, pages 405-421. Springer, 2020.
[25] M. Zucker, N. Ratliff, D. Dragan, M. Pivtoraiko, M. Klingensmith, C. M. Dellin, J. A. Bagnell, and S. S. Srinivasa. Chomp: Covariant hamiltonian optimization for motion planning. The International Journal of Robotics Research, 32(9-10); 164-114-1111 1193, 2013.
[26] M. Kobilarov. Cross-entropy randomized motion planning. In Robotics: Science and Systems, volume 7, pages 153-160, 2012.
[27] H. Niederreiter. Random number generation and quasi-Monte Carlo methods, volume 63. Society for Industrial and Applied Mathematics, Philadelphia, US, 1992.
[28] J. H. Halton and G. B. Smith. Algorithm 247: Radical-inverse quasi-random point sequence. Communications ACM, 7(12):701-702, 1964.
[29] Dick, F. Kuo, and I. H. Sloan. High-dimensiona integration: The quasi-monte carlo way. Acta Numerica, 22:133 288, 2013.
[30] Y. Yang, K. Caluwaerts, A. Iscen, T. Zhang, J. Tan, and V. Sindhwani. Data efficient rein- forcement learning for legged robots. Conference on Robot Learning, pages 1-10. PMLR, 2020.
[31] C. Summers, K. Lowrey, A. Rajeswaran, S. Srinivasa, and E. Todorov. Lyceum: An efficient and scalable ecosystem for robot learning. In Learning for Dynamics and Control, pages 793- 803. PMLR, 2020.
[32] Haviland and P. Corke. purely-reactive manipulabiltty-mxxiiia motion controller. arXiv e-prints, pages arXiv-2002, 2020.
[33] A. Zeng, S. Song, J. Lee, A. Rodriguez, and T. Funkhouser. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics, 36(4):1307-139, 2020.
[34] M. Bhardwaj. A. Handa, D. Fox, and B. Boots. Information theoretic model predictive q-
learning. In Learning for Dynamics and Control, pages 840-850. PMLR, 2020.
10


Reviews:
[{"review_id": "ZWnGtcPFuDf", "reviewer": "Reviewer_sfFw", "summary": "This paper proposes a MPC sampling-based controller that can be applied to real robotic manipulation tasks. The authors introduce several subcomponents to ensure that the tasks are solved using smooth and collision free movements. The system is tested both in simulation and on a robot, and the results look very promising.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The approximate model used for evaluating action trajectories is straightforward and efficient enough for high-frequency robotic control.\n- The idea of penalizing the trajectories that cannot stop at the end of horizon can be very effective for producing more energy-efficient motions.\n- The paper is well-written and organized.\n\nWeaknesses:\n- The paper does not explain how the learned components for collision avoidance are trained.\n- The paper claims that one advantage of their system is that it can incorporate learned components in the control loop. But to my understanding, that is the same for all MPC algorithms and I have seen a lot of works that use learned components to improve the performance/speed of their control systems.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, I like this paper. The approach is interesting and the results look very promising. I'm interested in seeing how the learned components are helping. Maybe the authors can show this in a new ablation study.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "- In Section 3.2.3, why do the authors need a neural network for predicting the closest distance between the links of the robot? Can't this be computed using forward kinematics?\n- In Figure 3, there seems to be an error in the caption where it says that initial and desired positions are shown in green and red, where it seems to be the other way around.\n- It would be nice to see the effect of the learned components in the form of another ablation study.\n\nTypos:\n- Line 107: represent -> represents"}}, {"review_id": "ZMiNH_6C4me", "reviewer": "Reviewer_U2Zx", "summary": "The presented work introduces a set of elements to improve and fasten the performance of Sampling-Based MPC methods for high-dof robots. The proposed elements are: a novel sampling approach to guarantee smoothness action trajectories and good action space cover, integrating pretrained cost function and parallelizing the computation of the joint dynamics.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n1. The presented elements allows to boost the computational time of Sampling-based MPC methods 100 times.\n2. Sampling Strategy: The paper proposes a novel sampling proceduce to guarantee smooth action trajectories and better action space cover while optimizing. The method seems interesting and they show qualitative results comparing the results sampling from normal distributions and using their proposed methods (Halton + B-Spline).\n\nWeaknesses:\n1. The paper lacks any reference to previous works trying to solve the problem of improving the computational efficiency of Samplign based MPC methods [1].\n2. Approximate Model: The paper proposes to parallelize the integration of the trajectories. Given the initial robot state, a set of control actions, and given the discretized dynamics are linear with respect to the input, they parallelize the computation of the joint position and velocity trajectories. I do have two coments with respect to the approximate model: (i) It would be more accurate if the acceleration term also influence in the position, i.e. x(t) = x(0) + v(0)t + 0.5 a(0)t^2. and you can still parallelize the computation and (ii) This parallelization only works with linear systems. In case you want to model nonlinear dynamics, for example, the robot interaction with a certain underactuated environmental element, you require to evovle the dynamics sequentially. Also, I don't see any major novelty of parallelizing the evolution of linear dynamics. \n3. Self Collision Avoidance: Computing self-collision might be computationally expensive. Thus, the paper proposes to use a pretrained neural network that predict the closest distance between the robot links. While it is a very interesting approach, this network has been previously presented in (in paper ref.[23])  and the paper focuses on applying it and adding minor modifications.\n4. An important missing point in the paper is a proper set of metrics to evaluate the benefit of each integrated element in the overall framework. The paper could follow an ablation study to evaluate the benefit of the self-collision avoidance element or a quantitative metric to compare their sampling strategy w.r.t. gaussian policy. It would also interesting to provide the mathematical equation of the ball dynamics? Is it linear? Can you parallelize it in the optimization?\n\n[1] Pinneri, Cristina, et al., \"Sample-efficient Cross-Entropy Method for Real-time Planning\", (2020)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The work is a set of individual elements to boost the computational time fro sample-based MPC methods. Evaluatiing each element individually, there is no a big contribution or novelty in any of them. The proposed approximate model  parallelization is standard and won't extend to nonlinear dynamics. Additionally, they should consider the acceleration effect in the joints position. The self-collision avoidance is just applying a previously presented work. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "The authors might focus on an ablation studies of the proposed elements, in particular it would interetsing an ablation study for the sampling methods. It is possible to define some qunatitative metrics to evaluate the performance of the Halton+B-Spline w.r.t. the others. Does it have an impact also in higher dimensional problems? Does it improve the jerkyness of the robot?"}}, {"review_id": "Xdfsa8V9nl", "reviewer": "Reviewer_xm2M", "summary": "The paper provides a new framework for sampling-based model predictive control applied to robot manipulation. Contrary to previous MPC formulations, the authors employ MPC directly in the joint space, which allows them to explicitly incorporate collision or joint limit constraints. Planning is formulated as an MDP with continuous state and actions. To efficiently find a policy over a fixed time horizon in realtime, the authors use a simplified robot model by using forward dynamics and to leverage low-discrepancy sampling to find valid control sequences (policies) over the desired time horizon. Using several manipulation-related cost function, the authors demonstrate that their framework can be used to quickly plan joint paths for a fixed-base manipulator robot to reach dynamic target locations while avoiding obstacles. They also show that their framework can be applied to dynamical balancing tasks, where a ball is balanced on a tray.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n-- The paper is well written and easily digestable. The authors did a fantastic job to motivate the problem, I really enjoyed reading the problem definition, which was easily understandable even though I am not an expert on reinforcement learning. The method section is self explanatory and I was able to follow their line of reasoning. While some details remained missing or had to be put in the supplementary material, I think the paper itself provides all the informations necessary to understand the ideas.\n\n-- The experimental evaluation is great. While the applications are challenging and useful, the authors even made the extra step of providing real-world experiments on a real robot system. The balancing skills of the ball on the tray is impressive and it is a good showcase for their method. The videos nicely supplement and demonstrate the usefulness of their approach.\n\n-- Reproducibility is often an issue with papers. The authors tackled this issue by providing code in the supplementary material, which would make it easy for other researchers to reproduce their results. \n\nWeaknesses\n\n-- It is clear to me that the authors have a deep understanding of what design choices actually work, in order to make the algorithm actually applicable. However, it would be great if you could (maybe in a later publication) try to provide more explanations of why\nyour specific design choices work. \n\n-- So far the algorithm has been demonstrated on a fixed-base manipulator. It would be interesting to see if you can scale this approach to even more complex systems, for example mobile multi-robot or other robotic systems.  \n\n-- While the results are great, I would have liked to see more reactive scenarios, i.e. where you add obstacles on-the-fly, push the robot away or remove obstacles. \n\n-- Similar to having more reactive scenarios, I would have liked to see more limitations of the approach, i.e. explicit scenarios where it fails. \nI guess one example could be where the time horizon is too small and the robot takes a wrong path. It would be nice for other researcher to see how far this idea can be pushed and at what points new methods/ideas are needed.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper provides a novel method to achieve reactive manipulation using sampling-based MPC. The experiments are convincing and I think the whole approach holds a lot of potential for a broader range of applications. While I would like to see more reactive and limiting experiments, I think the paper itself is well-written, technically sound and should be a strong accept.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "-- Please explain how you would make this method more rigorous in the future, i.e. what kind of guarantees might you be able to give if a practitioner wants to use this method.\n\n-- Consider talking more about limitations of the approach and how we can overcome them in the future. For example, you never showed that the model-based approximation performs worse at higher speeds. "}}, {"review_id": "3D35QFQHoK2", "reviewer": "Reviewer_poXv", "summary": "This paper proposes an integrated system for sampling based MPC for fast reactive manipulation control. Authors proposed different cost terms to achieve better manipulation behavior, also increase the sampling efficiency of sample-based MPC. The approach is validated both in simulation and on a real robot, and also authors plan to open source thier codebase.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- A clear description of the system description\n- A formulation to enable parallel computation of MPC on GPU, which make this MPC formulation work in real-time and on the real hardware.\n- A sample efficient approach to consider a longer horizon motion than OSC, which benefits behaviors like avoiding obstacles.\n\nWeakeness:\n- As this is a sampling-based approach, it would have been better if authors can provide a quantitative comparison by varying the number of samples needed and the length of horizon. That would give readers a better idea how the approach performs in terms of the motion quality and the computation time by varying different essential parameters.\n- For the cost term in 3.2.3 and 3.2.4, both cost terms rely on the neural network based functions. How the uncertainty introduced from neural network computation affects the algorithm? The experiments show the extrinsic evaluation of the cost terms, but it would be good to understand more intrinsically about the cost terms. Will there be any catastrophic collision due to incorrect computation from neural networks? If not, can it be guaranteed for such safety-critical terms.\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "In spite of some detailed concerns, I think this paper could have a significant contribution to the community, especially as a basic component for robot manipulation control. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- Videos are too lengthy, could be shorter with more caption to describe what happens (or increase the playback speed)\n- The last paragraph of section 3 says the sampling can also sample from a fixed set of predefined action trajectories, but it is not clear to me which experiments that explicitly demonstrate this point.\n- Does the ball dynamics consider the collision between the edge of the tray and the ball? While the result is impressive, I found the statement of ball didn’t fall in all ten episodes not quite accurate. From the supplementary video, the ball did not fall off the tray clearly due to the edge of the tray tray to stop the ball, and it doesn’t seem that  the forward model takes such a contact model into account.\n- In the supplementary, I can only see the computation time compared against OSC. Are there any performance comparisons? It would be good to show two video clips to demonstrate the difference between STORM and OSC formulation. And if you introduce potential field terms in nullspace projection, can OSC also perform a reasonable obstacle avoidance? \n"}}]

Author rebuttal:

Meta-review: 
The paper provides a new framework for sampling-based model predictive control applied to robot manipulation. To efficiently find a policy over a fixed time horizon in realtime, the authors use a simplified robot model by using forward dynamics and to leverage low-discrepancy sampling to find valid control sequences (policies) over the desired time horizon. Authors proposed different cost terms to achieve better manipulation behaviour increasing the sampling efficiency of sample-based MPC. The presented work includes a set of elements to improve the performance of Sampling-Based MPC methods for robots  with higher DOFs. The paper does not explain how the learned components for collision avoidance are trained. The authors need to clearly explain how their work is different from the prior works mentioned in the reviewers’ comments. 



**quality**: Good, but some references to the prior works are missing.

**Clarity**:  Good: Minor revision needed to improve organisation or clarity.

**Significance**:  The integration of different components and real-robot experimentations are interesting, but the theoretical significance is not clear in the current version of the paper. 

**Originality**:  The novelty and contribution of the paper are questioned by reviewers. Clarifying this may improve the paper.  

**Post reviewer discussion** the response to the reviewer's comments clarified many points. 

Decision: Accept (Oral)