Ready, Set, Plan! Planning to Goal Sets Using Generalized Bayesian Inference
Authors: Jana Pavlasek
Stanley Robert Lewis
Balakumar Sundaralingam
Fabio Ramos
Tucker Hermans
Venue: CoRL 2023 Poster (2023)
Keywords: Planning as inference
Variational inference
Nonparametric learning

Abstract: Many robotic tasks can have multiple and diverse solutions and, as such, are naturally expressed as goal sets. Examples include navigating to a room, finding a feasible placement location for an object, or opening a drawer enough to reach inside. Using a goal set as a planning objective requires that a model for the objective be explicitly given by the user. However, some goals are intractable to model, leading to uncertainty over the goal (e.g. stable grasping of an object). In this work, we propose a technique for planning directly to a set of sampled goal configurations. We formulate a planning as inference problem with a novel goal likelihood evaluated against the goal samples. To handle the intractable goal likelihood, we employ Generalized Bayesian Inference to approximate the trajectory distribution. The result is a fully differentiable cost which generalizes across a diverse range of goal set objectives for which samples can be obtained. We show that by considering all goal samples throughout the planning process, our method reliably finds plans on manipulation and navigation problems where heuristic approaches fail.

Ready, Set, Plan! Planning to Goal Sets Using Generalized Bayesian Inference
Jana Pavlasek University of Michigan pavlasek@umich edu
Stanley Robert Lewis University of Michigan stanlew@umi ch. edu
Balakumar Sundaralingam
Fabio Ramos NVIDIA
Tucker Hermans NVIDIA University Utah tucker. hermansQutah. edu
NVIDIA balakumar&@nvidke com
University of Sydney f ftozetoramos@nvidde com
Abstract: Many robotic tasks can have multiple and diverse solutions and, such, are naturally expressed as goal sets. Examples include navigating to room, finding a feasible placement location for an object, or opening a drawer enough to reach inside. Using a goal set as planning objective requires that a model for the objective be explicitly given by the user. However, some goals are intractable to model, leading to uncertainty over the goal (e.g. stable grasping of an object). In this work, we propose technique for planning directly set of sampled goal configurations. We formulate planning inference problem with a novel goal likelihood evaluated against the goal samples. To handle the intractable goal likelihood, we employ generalized Bayesian inference to approximate the trajec- tory distribution. The result is a fully differentiable cost which generalizes across a diverse range of goal set objectives for which samples can be obtained. We show that by considering all goal samples throughout the planning process, our method reliably finds plans for manipulation and navigation problems where heuristic ap- proaches fail.
Keywords: Planning inference, variational inference, nonparametrie learning.
1 Introduction
In order to accomplish diverse tasks in unstructured environments, robots need the ability to gener- ate motion plans to different user-specified goals. Such goals are often naturally expressed as goal regions in the workspace, rather than single point goals (e.g. navigate to a certain room, place an object on shelf). Traditional trajectory planning techniques require the goal regions be explicitly specified as planning objectives. This specification is challenging in cases where the goal region is uncertain. Examples of uncertain goal regions include those which are intractable model explic- itly, such as stable grasp poses across diverse objects [1, 2] or user preferences [3, 4].
We seek to solve planning problems under goal region uncertainty given only a fixed set of goal demonstrations. The key insight of our approach is model these demonstrations as samples from an underlying goal distribution. Our goal set planner plans directly these goal samples, result- ing in generalizable, data-driven planning objective which can be employed across goal regions, eliminating the need to explicitly model the goal. Planning goal distributions has been formulated as a flexible representation for number of objectives [5], but requires fully specified parametric goal distribution. Our approach instead treats the goal as an implicit distribution, from which we can obtain samples (e.g. successful grasp poses [6]) but which cannot be evaluated explicitly. We formulate this problem as inference-based planning [7, 8, 9], and seek a distribution of trajectories which terminate in high-density regions of the goal distribution (see Figure 1). Due to the complex,
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Goal Samples
Simulated Rollouts
Trajectory Execution
Figure 1: Our goal set planner considers discrete set of goal samples the planning objective. By considering multiple possible goals within the trajectory optimization process, handle challenging environments with clutter and dynamic obstacles.
multi-modal nature of the planning problem, we consider particle-based trajectory distributions and perform inference using Stein Variational Gradient Descent [10, 11j. In contrast to previous data- driven approaches to intractable planning objectives [1. 2. 3, 4, 6, 12, 13]. we require no upfront learning before planning begins.
Primary contributions. We formulate the problem of goal set planning generalized Bayesian in- ference (GBD), an emerging statistical technique which enables inference for intractable likelihoods. GBI refers to a class of problems in which the likelihood function is specified through a generic loss, typically a divergence between two distributions evaluated against observations [14, 15, 16]. We demonstrate that divergences can computed over implicit goal distributions, which allows us leverage a loss function that quantifies the divergence between the terminal state samples and the goal samples infer the posterior trajectory distribution [17]. We then propose Stein variational inference methodology to solve the problem sequence of gradient updates of set of trajectories.
Secondary contributions. We present a number of potential loss functions which can be ap- plied within this framework. We additionally compare to several previously proposed heuristic ap- proaches [5, 18]. We demonstrate the applicability our approach planar navigation and high-dimensional grasping task. By considering the full set of goal samples throughout the plan- ning process, our planner reliably finds reachable grasps in scenarios where heuristic approaches fail to reach their selected pose from the goal set. Additional demonstrations and data are available at https://janapawlaskk wtwp://www.cooo/pjpssss/osssse/a
2 Related Work
Trajectory planning to goal can be stated as an optimization problem with the objective of finding minimum cost path point in the robot workspace and can be efficiently solved using differ- entiable costs and dynamics [19, 20, 21]. Casting planning probabilistic inference has become popular technique for achieving robust control under uncertainty [22, 11, 23, 9, 24]. One popular approach is to employ a nonparametric, particle-based representation of the trajectory distribution, iteratively updating the posterior through importance sampling [23, 9]. Other methods use varia-
tional inference to efficiently infer the trajectory distribution [22, 11, 25].
Most related our work is that of Conkey and Hermans, who formulate trajectory planning goals defined as probability distribution [5]. Our proposed approach also considers the goal to be distri- bution, but we represent the goal as an implicit (sampled) distribution [17] eliminating the need for the distribution to be explicitly specified by the user. The problem of planning to a set of goals has been considered by the manipulation community for robotic grasping. A popular approach for grasp planning to goal set has been to formulate trajectory optimization and grasp selection as indepen- dent steps [26, 27, 18]. These methods first select a single goal from the set prior to planning based on scoring metric, then treat the selected sample as point goal. Similar approaches have been used to heuristically guide sampling- [28] and graph-based [29] motion planners to goal samples from sets in other contexts. Another approach is to jointly perform trajectory optimization and grasp

---

selection. Dragan et al. [30] add manually defined goal set constraint to the problem of trajectory optimization. Wang et al. [31] perform grasp selection and refinement online by estimating the grasp distribution. These methods require nomain-specinn information about the grasping problem and do not generalize across planning problems. Data-driven approaches to represent intractable planning objectives either employ high-fidelity simulation measure success (e.g of grasps) [1, 2. 6, 13], or demonstrations represent user preferences [3, 4, 12]. In these techniques, upfront computa- tional cost is required before planning begins, and must be performed again whenever environmental conditions, such observability dynamic environments, necessitate replanning.
Goal Sample Set Planning as Inference
Given a robot with state ze E X' at time t, we consider the problem of planning to goal set within the state space, C X. We consider the case of arbitrarily complex goal regions & that do not admit explicit analytic specification. Instead we can obtain a set of valid goal samples, G {>q ( h==1:N, (7) G. Examples of this type of goal include data labelled by simulation human demonstration. I.g
We seek a policy = ((x) such that the terminal state after a time horizon T lies within the goal region, ZT E G, given only goal samples G. The robot trajectory is defined as a sequence of states and actions, {(x,, ue) too.T We seek the minimum cost trajectory, 7", which solves:
T-1
arg min C (Tr: G) Σ (3t, us 2), 1=
where z denotes the environmental observation (e.g. the map the environment). The running cost, Ce(2ee U4; =), is chosen based on the domain (e.g. quadratic cost, obstacle cost, etc.). Our primary contribution is the formulation of the terminal cost, C(ær: G), when the goal region is represented only by set of samples, G.
The planning objective in Eq. can be restated in terms of probabilistic inference using the formal- ism of planning as inference [7, 8, 22, 9]. In planning as variational inference, we seek find distribution over trajectories q()) Q that minimizes the divergence with the posterior distribution of trajectories, po z), defined from the cost in Eq. (1):
(T) arg min D ((()) [l =)).
(2)
We can factorize the posterior over trajectories, T, given an environment observation, 2, and a set of goal samples, G {ag h=IIN, as:
(t)
T-1
T-1
Po(T 2, G) x Pooul ( xT)
II 1=1
p(z
24)
II 1=1
p(1++1
2t, u.)
(3)
Pgua xr)po(2 I (T)p(r')
where r' {(2,, u=)oo----1 denotes the portion of the trajectory up to time T - dr denotes the terminal state of the trajectory, and Pgea (G' (r)) is the goal likelihood given the implicit goal distribution G. We model the likelihood (o (2 (') in terms of the trajectory cost yielding:
T-1
po(= (') R0
Z exp
1
Σ
Ca Ca Me; 2)
(5)
where Z is normalization term. The likelihood Proal Pgoal( (G) FT) in Eq. (4) is intractable due to the implicit goal distribution. The following section addresses the key technical question of this work: how can we perform inference on the intractable posterior arising from an implicit goal distribution?
4 Planning to Implicit Goal Distributions as Generalized Bayesian Inference
We propose applying generalized Bayesian inference to directly approximate the solution of Eq. (4). While one could fit parametric model of Preal (G Ær) to the sampled data G. we seek generic
3

---

Envirocment,
Goal Loss
Simulated Rollouts
f gooal
Gredient
Trajectory Pa Particles
Iope
SVGD
Running Cost
Goal set, G
Gradient
Initial Stati,
Updated Trajectories, Fi++1
Figure Our SVGD-based planner takes input a set of goal samples, G. environmental observation, 2, and an initial state, Zi and iteratively updates set of trajectory particles, т. At each iteration, the particles are rolled out using simulator. Each rollout then evaluated in terms of its running cost, c. A goal loss Egaat. computes the discrepancy between the terminal points of the trajectory distribution and the goal set. SVGD updates the trajectory particles using the gradients of the combined running and goal losses.
objective for planning directly the implicit distribution that avoids task-specific pre-processing. We give brief overview of generalized Bayesian inference and Stein variational inference before explaining how we combine them to solve the goal set planning problem. Figure 2 visualizes our approach to planning as generalized Bayesian inference using SVGD.
Generalized Bayesian Inference. Generalized Bayesian inference (GBI) accounts for intractable inference problems arising from evaluating a loss function instead of a likelihood that factorizes over each observation [16]. Given posterior p(r) Y), where x are arbitrary random variables and Y )=INN is set of N observed data points, GBI approximates the posterior follows:
Pc(x Y) p(x) exp (-3C(x, Y))
(6)
where L(x, Y) defines loss between the random variables and observed data. Typically, the loss measures divergence (e.g. KL) between the data and query variables. The posterior pc(x Y) recovers the conventional Bayesian posterior when / and L(x, Y) E log ( can compute the approximate posterior by solving the following variational optimization problem:
[2r). We
Lc x Y) = arg min |3 Exzzy [Cx,, Y)) + DKL (())))p(()) qE
Stein Variational Inference. We build on work that approximately solves Eq. (2) using Stein variational inference [11, 32]. Stein variational inference represents the candidate distribution g(r) nonparametrically by a set particles, each representing a trajectory, (-(1), J=:MM. Stein variational gradient descent (SVGD) [10] employs gradient-based optimization over the particle set to mini- mize the kernelized Stein discrepancy [33] between the true and the approximate posterior. SVGD can solve high-dimensional planning problems involving complex, multi-modal posteriors [11, 32). SVGD is an iterative algorithm which applies the following update to each particle at iteration k:
T-- + (2)
M
( = M
k(7(), ( V (ogro(7() 2) (,onk((U)))
Ø=1
where k()()),7) 7) is kernel function between trajectories. We can interpret the first term inside the summation of Eq. (9) as an attractive force that moves particles according to the gradient of the log- posterior, while the second term acts as repulsive term keeping particles from collapsing point estimate. Thus SVGD leverages parallel gradient-based optimization to generate diverse set of samples more efficiently than Markov chain Monte Carlo (MCMC) samplers [9].
single
Terminal Losses for Goal Sets. We leverage results from the GBI literature which approxi- mate the optimization problem Eq. (7) using a divergence metric between two implicit distri- butions [17]. We define the implicit terminal distribution as X, (m/ Ji=ı:M and solve:
qe (xr G)
arg min Laaaal (Xr, G) q
(10)

---

100
Cross Entropy
Closest Point
Closest Point
Pass Rate (%)
Closest (Dyn.)
60
Cross Entropy
40
Energy
KL (Ratio Est.)
20
Kernel MMD
Smooth
0.0
0.2
0.4
0.6
Pinal Distance to Goal (m)
Figure Pass rate for different distance thresholds he- tween the final state and the nearest goal sample. Dashed lines represent baseline methods.
Figure Cross Entropy (left) and Closest Point (right) baselines fall into local minima failing to reach the goal. The trajectory is shown in blue, the trajec- tory distribution in purple, and the goal distribution in orange contours. The red 'x' shows the closest point.
The loss function goaal (Xr, G) measures the discrepancy between the terminal state distribution and the distribution of goal states, represented implicitly by samples X7 and G respectively.
We note that the loss defined in Eq. (10) provides measure the set of terminal states, rather than
an individual likelihood per trajectory particle required by Eq. (4). A key insight of our approach is to solve the resulting planning as inference task with SVGD, which employs the gradient of the particle likelihood, without the need to explicitly evaluate the goal loss per particle, We define the gradient in Eq. (9) as follows, combining the results from Eq. (4) and Eq. (10):
V,0 log (o 2) = V,6) log (6) + V Vuu logp(r(()) - 3 V V Co G) (II)
The terminal loss must be differentiable with respect the trajectory samples and efficient com- pute it can be used in control loop. We consider the following two sample tests which meet this criteria: The kernel Maximum Mean Discrepancy (MMD) [34], KL divergence approximation based on classification [35, 36], a smooth k-Nearest Neighbor test [37], and the energy statistic [38].
Details of the derivations of these measures can be found in Appendix A.
Experimental Results
We evaluate our method in a simulated 2D planar navigation problem and manipulation experiments both in simulation and on a physical robot. For all experiments, the goal set planner is run in MPC- style, where optimization is performed each timestep, then the first action of the lowest-cost trajectory is executed.
Baselines. Each baseline involves inference using SVGD [10, 11] with the same running costs, but with different heuristic terminal costs based on previously published work, as defined below.
Closest Point: The nearest goal sample to the start configuration is selected as the goal. The terminal cost is the squared Fuclidean distance to this point. A dynamic version of this cost is obtained by recomputing the closest point the beginning of each planning timestep [18].
True Goal Distribution: Given the true goal distribution, the objective is formulated as maximiz- ing the density of the goal distribution at the terminal point [5]. This method is only applicable to the 2D planar navigation task when the goal distribution is known.
Mixture Model: We define goal likelihood with Gaussian mixture model obtained by placing each component centered on each goal sample, with fixed covariance, akin to a kernel density estimator. The goal loss is then the cross entropy over the mixture model, which is equivalent to maximizing the likelihood [5].
Planar Navigation. We implement 2D planar navigation task to characterize the behavior of our approach. We define goal distributions over 2D position for each scene using parametric distribu- tion (Gaussian, mixture of Gaussians, or uniform). The goal set supplied to the planner consists of random set of goal samples drawn from the distribution.
5

---

05
t-0s
t-255
Figure 5: Trajectory distributions for the Goal Set Planner using the MMD terminal loss are represented as set of samples (drawn in purple). The final trajectory shown the last pane for each environment, colored with respect the velocity (yellow is fast, purple slow). The red points represent the goal samples and the orange contours represent the goal distribution.
60
15
Closest Point
Energy
Chamler Distance
Closest Pt. (Dyn.)
Chunfer Distance
(Ratio Est.)
Cross Entropy
Kernel MMD
Goal (MMD)
20
6-
Cross Entropy
Stop
Step
Figure 6: Chamfer Distance at each iteration for different terminal costs. (left) Comparison between the base- line methods and our goal set planner with MMD terminal loss. (right) Comparison between different terminal costs. The best performing baseline (Cross Entropy) is shown for reference.
To evaluate the goal planner, execute 10 runs per method per across scenes. We mea- sure the Euclidean distance between the terminal state in the trajectory and the nearest goal sample for each run. We present the pass rate for each method in Figure 3, where pass rate is defined as the percentage of runs for which the error is less than given error threshold. All methods achieve sim- ilar path lengths (see results in Appendix B.I). The cross entropy baseline performs close to the set- based methods with respect to pass rate, but is prone falling into local minima. The closest point baselines have high failure rate on challenging environments, particularly those with collisions near the goal (see Figure 4), those with the closest point in collision (see bottom-left in Figure 5). While it would be trivial to check for collisions with the closest point in the planar environment, we avoid this additional Nomminn-depentee which collisions are expensive or intractable to compute.
pre-processing step to better replicate environments in
Figure shows example trajectory distributions for the goal set planner using the MMD cost. Both the Smooth k-NN and the Kernel MMD) terminal losses display moment-matching behaviour, which posit allows them be more robust challenging environments. We posit that the kernel em- bedding in the MMD loss enables more flexible representation of the goal region compared to the other losses. We select MMD for all manipulation experiments.
Distribution matching: We hypothesize that our goal set planning achieves higher success rate in challenging environments due to its ability better approximate the underlying goal distribution. To verify this, measure the Chamfer distance between the terminal states in the trajectory dis- tribution each timestep and the goal samples. The results shown in Fig. 6. The cross entropy baseline, using the true goal distribution, achieves lower Chamfer distance than the other baselines, but performs worse than the goal set planner due to its mode-seeking behavior. Kernel MMD and smooth k-NN terminal set losses perform better on this metric than the energy statistic and the KI. approximation.
Robotic Grasping. Stable grasping of arbitrary objects is challenging problem since the goal distribution consisting of stable grasp poses is intractable to model. We evaluate our goal set planner

---

1.0
Passs Rate (%)
Success (%)
0.5
Goal Sets (MMD)
Closest Pt.
50
Closest Point
Mixture Model
Mixture Model
40
5
10
15
0.0
0.5
1.0
1.5
Goal Set (MMD)
54
Final Distance to Goal (cm)
Final Rotation Error Goal
Figure 7: Pass rate data for the grasping experiments: the Euclidean distance (left) and the L2-norm of the rotation matrix (right) between the final end-effector orientation and the goal.
Table 1:
Grasp success rate
on a grasping task using grasp samples generated in simulation [6]. We compare our method to the Closest Point and Mixture Model baselines.
We assume that the environment and target object are fully observed. Experiments are performed using the Franka Panda manipulator in the Isaac Gym simulator [39]. We use the STORM library for GPU-accelerated, fully differentiable dynamics and costs [40]. The goal set is represented by N = 100 samples of valid grasp configurations drawn from the Acronym dataset [6]. The goal set planner is used to plan to pre-grasp pose to offset grasp locations in an MPC framework over fixed horizon. The goal set planner optimizes over all goal samples at each planning iteration, and does not require pre-processing the set prior to planning. After convergence, we use inverse kinematics (IK) to move to the nearest grasp sample, and then lift the object.
We evaluate over 5 runs across 10 scenes containing 8 different objects and 5 environments. Grasp success results are shown in Table 1. We observe that primary source of failure is due to failure in the IK stage, when moving from the pre-grasp pose to the grasp pose. IK errors can arise from collisions or singularities encountered during the grasp. To better understand the performance of the goal set planner independent from the grasp stage, we measure the planning success rate of our planner the pre-grasp pose, computed as the distance from the terminal end-effector pose to the nearest goal sample. We present the pass rate for each method in Figure 7. The goal set planner achieves a higher pass rate at thresholds over approximately 4 cm. The closest point method may select goal points which are in collision in some of the example scenes, increasing the distance error, which also accounts for the discrepancy in grasp success. The rotation error is lower for closest point because it matches the orientation of the goal consistently, including for unreachable poses. We posit that the mixture model poorly approximates the true goal distribution explaining its failure.
Robot Demonstration. We demonstrate that our simulated manipulation experiments are transfer- able to the real robot, under the assumptions of known object and obstacle locations and geometries, through a robotic grasping demonstration. Figure 8 shows two successful grasping runs of the plan- ner, one in an environment without obstacles and one with obstacles. Goal samples are drawn from positive stable grasp examples from the Acronym dataset [6].
Placement. We posit that our planner is effective than point selection heuristics at dealing with multi-modal goal regions and changing environments due to its ability to consider multiple possible goal regions within the trajectory distribution. To examine this claim, we design table setting experiment where mugs are placed sequentially on a surface using demonstrations of valid configurations, inspired by demonstration based arrangement tasks [4]. The goal region consists of
Figure 8: Execution runs of the goal set planner on the real robot platform in an environment with no collisions (left three) and with collisions (right three). The robot grasps mug by planning set of successful sample grasp poses obtained from simulation [6]. Trajectories are planned offline and executed on the robot.
7

---

Mug Positions
X
Goal Samples
Tabletop
40
2.
3.
80 cm
Figure 9: Given example demonstrations of mug placement as the goal set (green), the robot plans a path to place a mug. The trajectory distribution (red) considers both free goal distribution modes, and avoids the mode which has collision due to the first mug (I), ultimately selecting free mode (2). The final positions of the mugs show relative the goal samples over 6 runs (3). The mug colors correspond to runs.
three uniform distributions representing valid terminal gripper 3D positions (see Figure 9, left). We sample 60 points to form the goal set at the beginning the task, and keep the goal set constant until all mugs are placed. Each subsequent placement operation requires the consideration of new obstacle in the scene. We employ the MMD terminal loss over the goal samples compared to the 3D terminal positions of the end effector.
We perform six runs for this task, each involving placing three mugs. Figure 9 shows an example trajectory distribution for the goal set planner when placing the second of mugs, which shows the multi-modality of the trajectory distribution. The final mug positions (Figure 9, right) are clustered corresponding to the modes of the demonstrations. We observe that the final mug placements may lie outside the sampled region. This can be caused by early termination, in some cases, trajectory distributions which converge outside the sampled region. The latter could be caused by singularities in the robot kinematics or local minima.
Discussion & Conclusion
In this work, we present a formulation for a goal set planner, which considers the planning problem in which an uncertain goal region represented as a set of samples. The planner is generalizable across goals which can be represented by implicit distributions, without the need for domain-speciinn user- defined heuristics. We find that independent of the specific loss used, the GBI formulation out- performs previously proposed approaches for planning goal sets and distributions. Our method outperforms all baselines on a 2D navigation problem and yields improved plan-to-grasp success on robotic manipulation platform.
In order to remain true to the general planning problem, our method does not consider domain- specific techniques commonly used for the experiments considered. Our grasping results would likely be improved by integration of pre-planning steps for checking reachability collisions [41], pre-grasping refinement of the final pose [31]. We hypothesize that combining these approaches with goal set planning for the application of grasping would improve robustness.
Limitations: Maintaining modes in the trajectory distribution can lend added robustness chal- lenging environments, but it also raises challenges in selecting a single trajectory to execute. We attribute the larger errors in the final pre-grasp pose of our method to the challenge of detecting con- vergence given the multi-modality of the goal planner. The terminal set losses considered work best as local measures, meaning gradient information is noisy if trajectories terminate too far from the goal. We mitigate this by applying smooth box priors around the goal samples. Furthermore, the terminal point is not guaranteed to be close to one of the points in the set. This could be mitigated by switching to point-base based cost once close to the goal set. To deploy our planner in real-world applications, more investigation is needed to explicitly handle perceptual uncertainty.

---

Acknowledgments
This work was completed in part while the first author was an intern the NVIDIA Robotics Lab. We thank the reviewers for their insightful comments. We thank Dieter Fox and Chad Jenkins for helpful discussions and feedback.
References
[1] J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg. Dex- Net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In Robotics: Science and Systems (RSS), 2017.
[2] A. Mousavian, C. Eppner, and D. Fox. 6-DoF GraspNet: Variational grasp generation for object manipulation. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2901-22 12100, 2019.
[3] D. Sadigh, A. D. Dragan, S. Sastry, and S. A. Seshia. Active preference-based learning of reward functions. In Robotics: Science and Systems (RSS), 2017.
[4] Z. Zeng, Z. Zhou, Z. Sui, and O. C. Jenkins. Semantic Robot Programming for goal-directed manipulation in cluttered scenes. In International Conference Robotics and Automation (ICRA), pages 7462-7469. IEEE, 2018. doi:10.1 101/111....1....../1... 888605588.
[5] A. Conkey and T. Hermans. Planning under uncertainty to goal distributions. Arxiv Preprint, 2020. URL http://arxiv. org/ abs/2011. 04782.
[6] C. Eppner, A. Mousavian, and D. Fox. ACRONYM: A large-scale grasp dataset based on simulation. In International Conference on Robotics and Automation (ICRA), 2020.
[7] H. Attias. Planning by probabilistic inference. In International Workshop on Artificial Intelli- gence and Statistics, volume R4 of Proceedings of Machine Learning Research, pages 9-16. PMLR, 2003.
[8] M. Toussaint. Robot trajectory optimization using approximate inference. In International Conference Machine Learning (ICML), pages 1049-1056, 2009.
[9] G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou. Information-theoreii model predictive control: Theory and applications to autonomous driving. Transactions on Robotics (I-RO), 3406):1603-1622, 2018.
[10] Q. Liu and Wang. Stein variational gradient descent: general purpose Bayesian inference algorithm. Advances in neural information processing systems, 29, 2016.
[11] A. Lambert, A. Fishman, D. Fox, B. Boots, and F. Ramos. Stein variational model predictive control. In Conference on Robot Learning (CoRL), 2020.
[12] A. Conkey and T. Hermans. Active learning of probabilistic movement primitives. In IEEE- RAS International Conference on Humanoid Robotics (Humanoids), IEEE, 10 2019.
[13] Q. Lu, M. V. der Merwe, B. Sundaralingam, and T. Hermans. Multi- dulti-fingered grasp planning inference in deep neural networks. Robotics Automation Magazine (Special Issue on Deep Learning and Machine Learning in Robotics), 27(2):55-65. 2020.
[14] P. G. Bissiri, C. C. Holmes, and S. G. Walker. A general framework for updating belief dis- tributions. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 78(5); 1103-1130, 2016.
[15] J. Jewson, J. Q. Smith, and C. Holmes. Principles of Bayesian inference using general diver- gence criteria. Entropy, 20(6):442, 2018.

---

[16] T. Matsubara, J. Knoblauch, F.-X. Briol, and C. J. Oates. Robust generalised Bayesian infer- ence for intractable likelihoods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(3):997-1022, 2022.
[17] C. Dellaporta, J. Knoblauch, Damoulas, and F.-X. Briol. Robust Bayesian inference for simulator-based models via the MMD posterior bootstrap. In International Conference on Artificial Intelligence and Statistics, pages 943-970. PMLR, 2022.
[18) W. Yang, B. Sundaralingam, C. Paxton, I. Akinola, Y.W. Chao, M. Cakmak, and D. Fox. Model predictive control for fluid human-to-robot handovers. In International Conference on Robotics and Automation (ICRA). IEEE, May 2022.
[19] N. Ratliff, M. Zucker, J. Bagnell, and Srinivasa. CHOMP: Gradient optimization tech- niques for efficient motion planning. International Conference on Robotics and Automation (ICRA), pages 494. IEEE, 2009.
[20] M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal. STOMP: Stochastic trajec- tory optimization for motion planning. In International Conference on Robotics and Automa- tion (ICRA), pages 4569 IEFE, 2011.
[21] Schulman, Y. Duan, J. Ho, A. Lee, Awwal, H. Bradlow, Pan, S. Patil, K. Goldberg, and P. Abbeel. Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research, 33(9): 1211)1251112700 2014.
[22] K. Rawlik, M. Toussaint, and S. Vijayakumar. On Stochastic Optimal Control and Reinforce- ment Learning by Approximate Inference. In Robotics: Science and Systems, 2012.
[23] M. Kobilarov. Cross-entropy motion planning. The International Journal of Robotics Re- search, 31(7):855-871, 2012.
[24] L. Blackmore, M. Ono, A. Bektassov, and B. C. Williams. A Probabilistic Particle-Control Approximation of Chante-Constrained Stochastic Predictive Control. IEEE Transactions Robotics, 2010.
[25] T. Power and D. Berenson. Variational inference MPC using normalizing flows and out-of- distribution projection. In Robotics: Science and Systems (RSS), 2022.
[26] D. Berenson, Srinivasa, D. Ferguson, A. Collet, and Kuffner. Manipulation planning with workspace goal regions. International Conference on Robotics and Automation (ICRA), pages 618 624.. IEEE, 2009.
[27] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact-GraspNee Efficient 6-DoF grasp generation in cluttered scenes. In International Conference on Robotics and Automation (ICRA). IEFE, June 2021.
[28] D. Hernández, M. Moll, and L. F., Kavraki. Lazy evaluation of goal specifications guided by motion planning. In International Conference on Robotics and Automation (ICRA), page 944-950. IEEE, 2019. 1oi: 1099//CAA.219..8995577777
[29] S. Aine, S. Swaminathan, V. Narayanan, V. Hwang, and M. Likhachev. Multi-heuristic A The International Journal of Robotics Research, 35(1-3):224-243. 2016.
[30] A. D. Dragan, N. D. Ratliff, and S. S. Srinivasa. Manipulation planning with goal sets using constrained trajectory optimization. In International Conference on Robotics and Automation (ICRA), pages 4582-4588 IEEE, 2011.
[31] L. Wang, Y. Xiang, and D. Fox. Manipulation trajectory optimization with online grasp syn- thesis and selection. In Robotics: Science and Systems (RSS), 2020.
10

---

[32] L. Barcelos, A. Lambert, R. Oliveira, P. Borges, B. Boots, and F. Ramos. Dual online Stein variational inference for control and dynamics. In Robotics: Science and Systems (RSS), 2021.
[33] Q. Liu, D. Lee, and M. Jordan. A kernelized stein discrepancy for goodness-of-fit tests and model evaluation. In International Conference on Machine Learning (ICML), 2016.
[34] A. Gretton,
Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
[35] M. Sugiyama, T. Suzuki, and T. Kanamori. Density Ratio Estimation in Machine Learning. Cambridge University Press, 2012.
[36] A. Menon and C. S. Ong. Linking losses for density ratio and class-probability estimation. In International Conference Machine Learning, pages 304-313. PMLR, 2016.
[37] M. Shilling. Multivariate two-sample tests based on nearest neighbours. Journal of the American Statistical Association, (395))7799-80, 1986.
[38] G. J. Székely and M. L. Rizzo. Energy statistics: A class of statistics based on distances. Journal of statistical planning and inference, 1248))1149-12777 2013.
[39] V. Makoviychuk, I.. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac Gym: High performance GPU-based physics simulation for robot learning. arXiv preprint arXiv:2108. 10470, 2021.
[40] M. Bhardwaj, B. Sundaralingam, A. Mousavian, N. D. Ratliff, D. Fox, F. Ramos, and B. Boots. STORM: An integrated framework for fast joint-space model-predictive control for reactive manipulation. In Proceedings of the Conference on Robot Learning (CoRL), November 2021.
[41] A. Murali, A. Mousavian, C. Eppner, C. Paxton, and D. Fox. 6-DoF grasping for target- driven object manipulation in clutter. In International Conference Robotics and Automation (ICRA), pages 6232-6238. IEEE, 2020.
[42] N. Henze. multivariate two-sample test based on the number of nearest neighbor type coin- cidences. Annals of Statistics, pages 772-783, 1988.
[43] J. Djolonga and A. Krause. Learning implicit generative models using differentiable graph tests. arXiv preprint 1rXiv:1709001066 2017.
[44] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference Learning Representations (ICLR), 2014.
[45] D. Garreau, W. Jitkrittum, and M. Kanagawa. Large sample analysis of the median heuristic. arXiv preprint arXiv: 170.00726, 2017..

---

Appendices
A Terminal Losses
This section provides detailed derivations of the different terminal loss functions, Laaal (XT: G),
compared in our generalized Bayesian inference planner. Recall, the loss function Ggeut Nr; G) measures the discrepancy between the terminal state distribution and the distribution of goal states, represented implicitly by samples Xy and G respectively.
Kernel Maximum Mean Discrepancy (MMD). The squared MMD [34] measures discrepancy between two distributions p(x) and (v)), and is defined as:
MMD"(p.) g) = Elk(,,x''), 2E|k(z, + ([)(y.2/))
(12)
where -)::)) is kernel function. In the case where distributions p and implicit, unbiased two-sample approximation of the squared MMD is given by:
2
MM. =
kx 1T))
k v,) n(n 1)
k(yi. U3)
i= jzH
jipts
(13)
where H; ~ p, Sis m and y, q. S í S n. When applied to our setting we simply define Cgoa(( (Xr, G) = MMD G) where Xy = {-/ (2) k correspond to samples from the terminal state associated with the corresponding trajectory particle
KL Divergence. A differentiable two-sample KL. divergence approximation can be obtained
through density ratio estimation via classification [35, 36). The KL divergence between distribu- tions p and q is defined as:
p(x) g(z)
DKL g) = -Ezn(()) log
(14)
This quantity can be approximated over samples x (1) p(z) if the ratio r(x) p(x)/q(x) can be computed analytically:
M
Ex-p(a) [log r(x) ~0
Σ logr logr(z!!))
(15)
M
In the case where the goal distribution and posterior distribution are implicit, the ratio r(x) cannot be evaluated. Instead, the ratio can be approximated through classification, where binary label y
indicates that sample x: drawn from p and y indicates sample drawn from g. The
ratio can then be approximated as:
P(xly 1) r" (x) P(xly 0)
(16)
which can be further simplified to:
r (Ply
=
(17)
This approximation reduces the density ratio estimation to a classification problem. Thus, the ratio computed by evaluating a binary classifier for each sample which can be performed efficiently.
Intuitively, this approach assumes that the KL divergence should be highest when the classifier cannot distinguish between the samples from each distribution. The classifier must be trained in- dividually for each estimated trajectory distribution in order to use this divergence. To improve computational efficiency in practice, the classifier can be initialized with the result from the pre- vious iteration. Finally, we can define Lyaut (Xr,r) G) = (KL(X G) where the KI. divergence is approximated following Eq. (15).
Smooth K-Nearest Neighbor. It is possible to define a differentiable two-sample test based on the well known k-NN algorithm as demonstrated in [37]. The Smooth K-Nearest Neighbor test
12

---

possesses important statistical properties such as consistency and convergence of its statistics to f- divergence. This is despite the complexity of having to solve a combinatorial optimization problem (nearest neighbor match) required by the k-NN method. Let nı be the number of samples of p, and ng be the number of samples of q. The k-NN divergence can defined as
(²)2(x) (1 =x)2 dr, ap(x) (1 a)q(a)
DNN
(18)
for mi /mm + 112) + a E (0, 1). As proved in [42], the statistic
T(X1, (n1 + n2)
(19)
where I(X,, X2) refers to the number of edges connecting samples in set X1 from a h-neighborhood graph created with points in X1 and X2, converges in probability to the DNN divergence, and can be used as an efficient approximation. To make the computation differentiable, the authors of [43] define
{ to
a set X2 {ra 182
n a
T(X1, X2) = ΣΣ
si((amm (a)+)2))
(20)
i=
where (1m +2) denotes the smax function computed the Fuclidean distances between all points, except point i. As the sof tmax function is differentiable, the statistic in Eq. (19) becomes differentiable and can be used directly as our loss function Egaat (XT, ATr.GG G). To avoid specifying a particular value for a, our implementation computes the statistic for several values and averages them as the final result.
Energy Statistic. This two-sample test is based Newton's gravitational potential energy which relates two entities by the Fuclidean distance between them [38]. Given two distributions p(x) and q the energy distance is defined as:
(p, 4) 2B[ x - y11²] B|[l - 2'11 - Billy y v/113),
(21)
where x and y independent random variables. The corresponding two-sample statistic given two sets of samples X { and Y 1 can then be written as:
(X,Y) = 2
||= Vmll 11
23
11 ymll (22)
12
n2
m
1=1
This provides a computationally efficient statistic which can be directly used as our loss Lgaal (X2, G) D
A.1 Practical Considerations
The statistics considered in this work good local approximations of distribution divergences. In the case of trajectory optimization, when the terminal states in early planning iterations are far from the goal set, the goal loss gradients can be uninformative. We therefore include a prior in our set planning method consisting of smooth uniform distribution constructed by placing a bounding box around the goal samples. This can be included in our framework by multiplying a prior over the terminal state p(ær) with the goal likelihood Eq. (6). This mitigates the poor divergence ap- proximation in early iterations. Note that the uniform prior insufficiently informative on its own, particularly in cases where the goal set is multi- modta.. Furthermore, the prior needs to be differ- entiable, which is not the case for a standard uniform distribution. Therefore we define smooth uniform prior in the region R =
S b as
((rT) x exp (dd(aT, R)² (202)
(23)
where d R2 min Ix I'), x' E R. is distance function, and controls the sharpness of the approximation.
13

---

Once inference over the trajectory distributions converges, we must select a single trajectory esti- mate to execute. common approach to accomplish this is by taking the mean, or weighted mean, of the particle set. This method is ineffective when the trajectory distribution multi-modal. An alternative approach is to pick the maximum weighted particle. Our proposed set-based terminal losses yield a single score over the whole distribution, which does not enable weighing individual particles based terminal loss. To select our final sample, instead select the lowest cost tra- jectory. In practice, we also include the prior in the weight computation to avoid local minima with very low-cost trajectories.
B
Experiment Details
B.1
Planar Navigation
The agent state 2; is composed of a 2D position and velocity, and the control signal uy is 2D acceleration. We use known, linear dynamics in fully observed environment for the controller rollouts.
Losses. For each method, the running cost is summed over each timestep. where the cost for one timestep is:
Ca u4, 2) x Qx: + uh Ru, + (SD)(
(24)
where Q and R are quadratic cost parameters for the state and action, and (SD,(ate =) is the obstacle avoidance term, computed using the Signed Distance Function (SDF) over the environment The goal loss Lyaal (X,, G) is set goal loss which is differentiable with respect to the trajectory т.
Implementation Details. For each of our goal set planner ablations, = 50 samples are ran- domly selected from goal distribution, except for KL (Ratio Estimation), which uses N = 100 samples. This method involves training a learned classifier is aided by higher sample size. For the closest point methods, the goal sample with the smallest Euclidean distance from the start state is selected. For all the methods, M 50 particles are used to represent the trajectory distribution. The particles represent the discrete control signals, u4, which are 2D accelerations at each 0.1 sec- ond timestep over a horizon of seconds. All planners are initialized with the distribution from the previous timestep, shifted to the current timestep, and run for 50 iterations. We use the Adam optimizer [44] to select the step size in the SVGD update rule.
The KL divergence uses 3 layer cully-connected network the classifier, retrainined at each timestep. To mitigate computational complexity, we warm start the training with the weights from the previous timestep. The Kernel MMD uses an RBF kernel, with bandwidth selected by applying the median heuristic over the goal samples [44]. The Smooth k-NN loss uses a value of k: = 1.
We use the RBF kernel for SVGD, and set the bandwidth using the median heuristic, popular technique for choosing the kemel bandwidth which yields good estimate in many cases [45]. Without access to data consisting of trajectory samples, assume that they normally distributed with covariance a = 1. Under these assumptions, it follows that the expected distance between samples drawn from the distribution is 2D, where D is the trajectory dimension.
Extended Results. We visualize the total Euclidean path length for each method in Figure 10. The baselines are shown with hatched bars. We exclude paths for which the terminal state is not within 40 em of any goal sample. All methods achieve similar path lengths, with the closest point baselines consistently resulting in slightly shorter paths. This is unsurprising given the distance-based goal selection bias, but comes at the cost of being less robust to different environments.
B.2 Grasping
Figure/ shows case where the goal set planner finds a reachable point from the set, but the closest point method may select goal points which are in collision in some of the example scenes. This phenomenon helps explain the discrepancy in grasp success at the cost of increasing distance error.
14

---

Path Longth (m)
N
X
21
X 38
-- 38
N- 38
N-444
--444
Closest
Cross Pt. (Dya.) Entropy
Closest
Energy KL (Ratio Est.)
Kernel MMD
Smooth
Point
K-NN
Figure 10: Average path length for each terminal cost for the planar navigation task. The points indicate results for individual runs. Only successful trajectories are included, where success is defined as getting to within 40 em from any sample within the goal set. The numbers on the bars indicate the number of successful runs out of 50 in each category.
Example failure execution for the closest point planner (top) compared the the goal set planner (bottom). The closest point selected is shown in blue. The point is not reachable in the environment, causing the robot to fail to reach it (top right). The set planner considers all the grasp samples, shown in green, It finds reachable grasp point and grasps successfully (bottom right).
Figure 11:
15


Reviews:
[{"review_id": "qMqa_RPrWu8", "reviewer": "Reviewer_Jat5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a goal set planner for planning problems in which goal regions are difficult to express explicity. At its core, the method frames the planning problem as a Bayesian inference problem and combines methods from generalized Bayesian inference and Stein variational inference to construct a differentiable loss function. Optimizing this loss function with respect to a set of sampled trajectories results in trajectories that reach an arbitrary goal set. Crucially, instead of expressing the goal set explicitly, the proposed method only requires samples from the goal set. The method is evaluated on set of experiments involving simulated 2D navigation and manipulation task, and a grasping task using a physical robot.", "summary_of_recommendation": "In summary, the paper proposes a trajectory-optimization based planner that is designed to handle arbitrary goal sets that are only represented via samples. The method includes novel and interesting ideas that I think are of interest to the wider robotics community.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- It is not 100% clear to me how the gradient in eq.(11) is derived. In eq.(4) you have the goal likelihood $p_{goal}(\\tilde{G} | x_T)$ that seems to be expressed in terms of the variational term $q_{\\mathcal{L}}(x_T | \\tilde{G})$ in eq.(10). It would be great if some additional details were provided (perhaps a via step-by-step derivation in the Appendix) how exactly eq.(11) is derived.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "First, the paper is well introduced and nicely written. The general ideas and the technical details of the method are presented well. The approach is sound and the property that only samples from a goal set are required are very appealing. The experiments are generally carried out well (although see my comments below for possible improvements) and demonstrate the effectiveness of the approach. The video in the supplementary material gives a good overview of the method and its core ideas.\n\nThe only two points that could be improved are the following:\n\n- The structure of Section 4 could be improved. In particular, the paragraphs Generalized Bayesion Inference and Stein Variational Inference feel like they should be moved to the Related Work Section. The current structure disconnects the beginning of Section 4 and the paragraph \"Terminal Losses for Goal Sets\"\n\n- While the experiments are carried out well, I would have liked to see a more diverse set of experiments. For instance, what happens when the problem consists of disconnected goal sets (e.g. a robot having to place a cup on one of two tables)? Does the method still work? My intuition is that it should still work, but this should be demonstrated", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "pkeltIbD6b", "reviewer": "Reviewer_aNNG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "A method for plan to achieve goals that are specified only in terms of a set of samples of desired trajectory end-points.   The objective is articulated as a probability distribution and Stein variational gradient descent is used to optimize a set of particles to have low divergence with respect to the target distribution.  The method is illustrated in a 2D pedagogical domain and for a grasping robot, simulated and real.", "summary_of_recommendation": "The method is very nice and well described.  However:\n- It would be good to have a real use-case for this method described:   for example, it seems more likely that I would have available an expensive (possibly noisy) grasp evaluation function (either something analytic based on a mesh or a neural net) than a set of demonstrations.\n- The experiments are kind of worrisome because it wasn't clear why we only had about a 50% success rate.  I can think of lots of reasons why that might be, but some actual error analysis seems important in order to really evaluate the underlying strengths and weaknesses of the method.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "See above", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis is a very good problem formulation.  Learning a set of good solutions or a scoring functions makes the results of this module composable with other modules downstream, for example.  It uses a very nice overall objective that optimizes a property of the ensemble\n\nThe basic experiments are well designed and illuminating.   But I'm confused about the robot grasping experiments.  I would have\nthought you would be planning the whole trajectory.  But if that's true, you wouldn't need IK, would you?\n\nSmall points:\n- Sefine the C is the terminal cost function\n- What is z (the environmental observation)?  how is that different from x?   I guess it's the static part of the state?\n- Do you assume deterministic dynamics?  I assume so if you're talking about a minimum cost trajectory.  But then we don't need p(x_t+1 | x_t, y_t)  --- it can just be a function \n-  I have never understood the attraction of writing a collision constraint in the form of equation (5), but okay\n- In figure 4, in the second frame of each batch, why does the starting point seem different from the first frame?\n- Why don't we have 100% pass rate ever?  Are some instances  infeasible?  Are we not running the optimization long enough?\n\nI suspect that this might all be clearer just written as one straight-up optimization problem without going via the probabilistic\nformulation (since really there is nothing here that is inherently probabilistic). I guess this formulation makes a useful set of\npre-existing tools (like SVGD) available, so that's good.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "NGgsmb0vcl", "reviewer": "Reviewer_bCZS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a new method that plans towards a set of (sampled) goals instead of a specific goal chosen with heuristics. The main idea is to formulate the problem as a generalized bayesian inference, which allows for making an inference without assuming the explicit distribution for goals. Specifically, the divergence between the goal distribution and terminal state distribution can be evaluated using the samples from each distributions. Then SVGD is used for inference that enables us to avoid evaluating the loss for each particle corresponding to each terminal state. Experiments are conducted on simple 2D planar environments, also on simulated grasping environments and real-world environments.", "summary_of_recommendation": "I'd like to recommend the paper to be 'weakly rejected'. In its current status, the experimental results are not sufficient to support the benefit over baseline heuristics and the main message of several experiments are not crystal clear. Moreover, the paper have a room for improvement in terms of clarity, by making it be more self-contained and be more intuitive about explaining the main method.\n\n---\n\nI have updated my score to 'weak accept' per the discussion during the rebuttal period.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "- Please address my points in Weaknesses\n- What is the exact definition of pass rate in experiments? Also, should it be 0\\~100% instead of 0.0\\~1.0?\n- Please add more labels and legends to figures, especially Figure 4 is very difficult to parse without reading the main text very carefully.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths\n- The problem setup is quite interesting and practical\n- Clear illustration of the problem setup\n- Real robot experiments\n\nWeaknesses\n- Experiments are extensive but the trends are a bit underwhelming. Not much of a significance difference compared to a bit simple baseline in 2D environments, and almost no significant difference in success rates for grasping experiments. \n- Robot experiments are nice but it's difficult to understand the main message of those experiments. What is the main challenge for these experiments and what is the main point of demonstrating that robots can work with the proposed planner? How do other baselines work for this and how would they fail?\n- This is a bit minor but also important point. I tried my best to understand the paper, but it feels like the paper is difficult to parse as it's not self-contained. The paper assumes a lot of background on multiple components and provide background in a little depth, e.g., planning as inference, stein variational inference, kernel MMD, Champer distance, etc. Formulations in Section 3 are also a bit dense to parse. Maybe adding a more intuitive explanation and also the detailed formulation (at least in the supplementary material) could help readers understand the main idea.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "Gre0Grl2Ee", "reviewer": "Reviewer_CHoH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposed an planning algorithm to a goal set with bayesian inferences. The proposed approach was evaluated in simulation first and then on a robot arm in a offline manner.", "summary_of_recommendation": "I recommend weak reject, and since I don't have a lot of the experiences on robot manipulation I could be naive. The paper seems more likely to be looking at manipulation applications so I am happy to update my review if there are more experienced review working on manipulation.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "Please refer to my questions in the \"review\" section.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is well written and the proposed method was evaluated in both simulation and real world setup. \n\nHere are a few of my concerns:\n\n1) The evaluation metrics covered a couple of things, including \"final distance to goal\" shown in Figure 7, as well as grasp success rate shown in Table 1. While the former metrics could cover both navigation of mobile robot and robot arm manipulation, the latter on is more focused on manipulation related applications. Is the navigation to the goal set (or grasp pose) the major contributing factor of success rate on manipulation? It could be better if the author could clarify on it a bit.\n\n2) The proposed approach assumed \"goal set\" however in Figure 4 (top right) the planning seems stopped at the bounding box of the all goals. Is this cutting off a bit too early or expected behavior? I was wondering if the planner would return multiple trajectories with some topological variations or it will pick the best of out of several candidates.\n\n3) In the real world experiments (Figure 8) seems only one cup is the target, not sure how the \"goal set\" in this case looks like. Could be really interesting to visualize it (like Figure 9 does, though Figure 9 does not have the obstacle setup in Figure 8), and I was wondering if some of the target pose in the \"goal set\" would require the robot arm to reach the cup from the other side of the obstacle.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a strategy for goal-set planning, for problems in which the goal region is difficult to specify using Bayesian inference. Before the rebuttal, the reviewers were somewhat confused about the robotics experiment and its significance, although they found the motivation and method convincing. The ambiguities were resolved and all the reviewers have agreed to accept the paper.
Decision: Accept (Poster)