Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work uses bi-level optimization and a normalizing flow parameterization of learned sampling distributions. This allows for learned sampling distributions for MPC without requiring differentiability of dynamics and cost functions. This approach allows for box constraints on the controls (ex joint limits).

The proposed method is compared to MPPI, FlowMPPI on a 2d navigation task, and a 7dof reaching task with a Frank robot.


strengths_and_weaknesses: Strengths:
The proposed approach appears to scale more gracefully to smaller numbers of samples, particularly for the harder Franka & Franka Obstacles tasks. 

Weakness:
I still don't have a good intuition for the average trajectory cost in terms of either wall clock time or required number of operations for inference. This work demonstrates that their approach can operate with significantly fewer samples than MPPI, but is unclear what the advantage of less samples actually implies. How much faster or slower is it to run this NF based approach vs MPPI? Does the 16-sample NFMPC run any faster or require less compute than a 1024 sample MPPI? 

There maybe clear benefits to the proposed approach here, that this manuscript would benefit from if articulated. Alternatively, if the proposed approach is actually slower or more computationally expensive even when using less samples, this work would be less compelling.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper is well written, figures are clean and the ideas are well presented.  It would be great if authors could provide some insight into the issues section to help understand the impact of this work.


issues: 
My main concern is that number of samples is an insufficient metric to compare this method to existing baselines. If your approach only requires 16 samples, while MPPI requires 1024, how much faster is your method or how much less computation is required?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes improves previous works on model-predictive control based on normalizing flow. The previous works learn the sampling distribution in the original control space. In contrast, this paper proposes to perform the learning of the sampling distribution in the latent space. As a result, the entire learning algorithm can be cast as a bi-level optimization problem which alternately optimize the sampling distribution and the normalizing flow which is responsible for mapping a sample in the latent space to the control space.
strengths_and_weaknesses: Strengths:
- The proposed improvement of updating the latent sampling distribution is reasonable and solved in an intuitive manner.
- The paper is in general well written, although I have trouble understanding some of the technical details such as Section 3.4, but it may be due to my lack of knowledge in the relevant fields such as normalizing flow.

Weaknesses:
- The experiments indeed show the improvements of the proposed method over the two baselines under comparison, however, I feel that some of the experimental results need more explanations. For example, in all experiments, it seems that the method of FlowMPPI usually performs worse than the simple baseline of MPPI especially with small sample sizes (e.g., see lines 246-248). Why is this the case? Next, as said in lines 251-252, the proposed NFMPC is able to discover more diverse paths, so what's the intuition behind this improved behavior of NFMPC? Is it because updating the sampling distribution in the latent space, as done by the proposed method, helps learn a better sampling distribution?
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper makes a reasonable and useful contribution of sampling-based model predictive control.
issues: Please see the question under "Weaknesses" above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents an approach to learning sampling distributions for MPPI. The authors leverage normalizing flows, and backpropagate through the MPC optimization procedure to train the flow model. 
strengths_and_weaknesses: ## Strengths

Overall, I think the paper is a good one. The method is straightforward, easy to understand, and clearly yields strong practical benefits. 

## Weaknesses

There are a handful of small issues which, if addressed, would substantially strengthen the paper. 

- The biggest issue is lack of investigation of OoD tasks. The authors explicitly discuss this in their limitations section, but it really is extremely important. The paper needs to investigate performance on OoD tasks and should include more complex environments (possibly featuring multiple obstacles). In general, the authors should investigate leveraging more information in the normalizing flow. For example, scaling to alternative, more complex environments would likely be substantially improved by conditioning the NF on environmental information (which as far as I can tell is not currently done). This approach has been widely investigated in the literature on learning sampling distributions for sampling-based motion planning, and I would point the authors toward that area for inspiration (the authors should probably also discuss that literature in their related work section, as it is highly related). 
- The authors should report success/cost versus wall clock time in addition to versus number of samples. The performance for a small number of samples is obviously much better than standard MPPI, but understanding the time cost of the sampling (especially for NF models) is important. 
- It is unclear if the action sequence samples are decoded/passed through the NF as iid samples or jointly. If each set of action sequences is decoded independently, investigating a joint distribution over all samples is potentially very interesting as the model would be able to learn variance reduction strategies (eg antithetic sampling). 
- The authors propose to use NF models primarily due to the fact that they enable exact evaluation of likelihoods. Generally, with the world of generative models evolving as quickly as it currently is, I would try to minimize the explicit dependence on the NF models. In particular, the authors currently state a set of necessary features for the learning model in several spots across the paper. The paper would be improved by formally stating, in one spot, the necessary features for a generative models to be used in this work, and then discuss how NF models are one example of a model that meets the desiderata whereas eg VAEs are not. This hopefully will improve the applicability of this paper as generative models continue to progress. 
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The authors should extend their method to OoD tasks, as well as improve some elements of the writing. I am willing to improve my impact score from 3 to 4 if there is (substantial) progress made in the OoD task area.
issues: As discussed above, the main issue to address is OoD tasks, as well as several more minor issues such as changing the framing around the generative model.
review_title: null

Meta-review: metareview: Phase 1:

Strengths:
The submission is overall seen as providing intuitive technical contributions and and clear presentation. The experiments support the introduction of NFs into MPC and the method works well will reduced number of samples.

Weaknesses:
The evaluation is missing out on some important details such as a comparison with respect to compute or wall time in addition to pure sampling numbers (NFs introduce additional computational cost which cannot be ignored). Some experimental results could be analyzed in further detail (e.g. low performance of FlowMPPI). Importantly, additional experiments are requested to evaluate how well the learned distribution generalizes to OOD tasks.

Phase 2:

The feedback has been generally positive (3 weak accepts) with only minor criticism. In particular, the reviews appreciate intuitive technical contributions and clarity of presentation but criticize some missing aspects of the evaluation. I agree with the reviewers and recommend acceptance. Please take the remaining points from the review process seriously and follow up with improvements on open points and promised changes.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)