Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper focuses on exploiting LLM's code-writing capabilities to infer affordances and constraints from free-form language instructions. Additionally, for contact-rich tasks, the paper demonstrates how their framework can be utilized to learn a dynamics model using online experiences. The contributions of this work are threefold: 

1) Proposing a method for extracting affordances and constraints using the foundation model in a zero-shot manner.

2) Presenting results from 30 different real-world tasks to showcase the effectiveness of their approach.

3) Demonstrating the ability to learn a dynamics model from a limited amount of online interactions.
review: This work proposes a highly promising framework for robotic manipulation that utilizes foundation models for zero-shot reasoning, planning, and motion primitives generation. I was pleasantly surprised by how the LLM can achieve motion primitives with 70-80% accuracy in the real world (Tab 1) without requiring further training or data collection. While this appears very promising and even somewhat magical, it is challenging to assess the novelty of this paper, as it seems to integrate existing studies (e.g. GPT-4, OWL-ViT, XMEM). 

Furthermore, the paper addresses the optimization problem for parameter identification (line 215), which is conditionally required, potentially raising doubts about whether the method can genuinely be labeled as zero-shot. To avoid any misinterpretations, it is crucial to highlight the heuristics and assumptions, such as the necessity of pre-defined pushing primitives for the "entity of interest" when introducing the paper (e.g., in the abstract or introduction).

Additionally, the paper attempts to handle contact-rich tasks through an environment dynamics model. However, as noted by the author (line 293), VoxPoser still lacks the capability to deal with the intricacies and complexities of contact between tools and the environment. Notably, the model does not reason over contact constraints (e.g., maintaining contact), and some examples of their contact-rich tasks, such as opening a door, do not actually involve significant contact.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I found the results this paper presented were quite surprising. However, there are some limitations, particularly concerning contact-rich manipulation and the reliance on pushing motion primitives for tool use. Additionally, I believe that more details should be provided in the main text to enhance clarity and understanding.

Therefore, based on these considerations, I would recommend a weak acceptance of the paper.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The paper proposes a model that learns to create a 3D value map to achieve language-grounded manipulation tasks by leveraging large language and vision-language models. Instead of directly learning a policy, the authors formulate the task into optimization of trajectory generation for sub-tasks given by a high-level planner. LLM is used to write a program that uses VLM to identify objects and creates affordance and avoidance maps. The maps are used to create a trajectory and motion planning is done in closed loop. The approach outperformed baselines in experiments with both seen and unseen instructions and attributes. It also showcased the ability to learn dynamics model by leveraging the trajectory in a contact-rich task.
review: # Strength
* The paper is clearly written.
* Easy to interpret the behavior as it is modular and produces value maps.
* Showed strong generalization in unseen attributes.
* High potential in practical use due to its modularity; free from primitives, applicable to different robots, close-loop control.

# Weakness
* No comparison on public dataset such as RLBench and no comparison with competitive language-grounded manipulation methods such as PerAct.
* As the model relies on pre-trained vision model, it might not be able to apply to visually different/diverse datasets or adapt to a (visually) challenging task.
* When the control dimension gets higher, the template-based code might not be sufficient to handle the robot constraints.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: (Updated) I recommended a strong accept. It is well-written, the idea is easy to understand, and the result seems impactful. It cleverly leverages large models and decomposes scene understanding from policy learning which often combined to make the task harder. Maps and trajectories are interpretable and control is (partly) isolated which can make this approach adoptable. After the rebuttal, a question on extension to complex motion was resolved and the comparison was also promised.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper’s main contribution is VoxPoser, a technique that uses GPT-4 to convert high-level natural language commands into a sequential of subtasks. Each sub-task is then translated into several density maps of the workspace, where each voxel contains a value. Two “value maps”, an affordance map and avoidance map, are used for trajectory planning. A simple random shooting motion planner is then used to greedily maximize these two maps to produce a trajectory for a robotic end effector. Other maps directly control end effector rotation and the gripper state. The authors demonstrate the capability of the VoxPoser technique on 30 manipulator tasks and explores its performance on a zero-shot set-up. The proposed method appears to outperform existing baselines. The authors also demonstrate VoxPoser’s strengths for online training. LLM-generated value maps can guide RL exploration for faster convergence online training.
review: **Originality & Significance:** As far as I know, this is the first paper to use LLMs to generate value maps in real time for MPC. The results are promising and would be interesting to the community.

**Strengths:**
1. I found the authors’ approach to grounding abstract language commands as innovative and novel. The proper array of experiments was conducted for a thorough investigation of serial manipulator tasks, and the authors were able to identify a significant improvement over existing baselines.
2. By leverage recent advancements in LLMs, the authors present new capabilities compared to past methods. The greatest advances are for zero-shot learning.

**Weaknesses:**
1. Although the authors conduct experiments on a variety of manipulator tasks, there are some limitations in the setup. First, the full occupancy of the robot arm is not considered for collisions and trajectory optimization. Second, the dynamic model for manipulated objects is simplistic. From what I understand, no consideration was given for the center of mass of the object to predict object rotation during contact. Lastly, there are better ways for trajectory optimization than random shooting.
2. The paper does not dive deeper into the sources of VoxPoser perception error. Where in the perception pipeline does the error commonly occur (OWL-ViT, Segment Anything, XMEM, etc)?
3. The paper’s scope is limited to robot manipulation, further work on mobile robot trajectory planning would validate the generalizability of this technique.
4. The paper was generally concise. However, the writing is not always clear. The appendix was needed to understand the scope of prompt engineering for each LLM call, which should an important detail of the implementation.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: While the paper would benefit from some improvements in experiment design and clarity, I believe the core concept of the paper is sufficiently interesting and presents promising results. I would recommend accepting this paper.
review_title: null

Meta-review: The paper presents a method of creating a value map for manipulation task using an LLM model. The learned valued map can be used in model-based controller to create a trajectory. Thanks to the LLM, the method can ceate a value map conditioned on complex texts such as "measure weight of apple" and "turn on lamp".

All reviewers have positive opinions on this paper. One of the reviewer questioned about the novelty of this method because it is highly dependent on the LLM, but I agree with the authors' defense that code generation scheme is new and novel. Even though the contribution might look simple, this paper will make other researchers aware how powerful LLM's can be for robotics.

Other questions that the reviewers raised are minor and can be reflected on the revision easily. I also found that the authors' comments are clear and rational. The also clearly stated the limitations such as not considering the robot arm for collisions and trajectory optimization.

Another strength of this paper is an extensive robot experiments. Compared to other papers submited in this topic, this paper presents a rich set of experiments.
Decision: Accept (Oral)