EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI

Authors: Tomoyuki Kagaya, Yuxuan Lou, Thong Jing Yuan, Lakshmi Subramanian, Jayashree Karlekar, Sugiri Pranata, Natsuki Murakami, Akira Kinose, Koki Oguri, Felix Wick, Yang You
Primary area: applications to robotics, autonomy, planning
Keywords: LLM Agent, Robotic Manipulation, Cross-Environment Knowledge Transfer

Abstract:
In recent years, Large Language Models (LLMs) have demonstrated high reasoning capabilities, drawing attention for their applications as agents in various decision-making processes. One notably promising application of LLM agents is robotic manipulation. Recent research has shown that LLMs can generate text planning or control code for robots, providing substantial flexibility and interaction capabilities.
However, these methods still face challenges in terms of flexibility and applicability across different environments, limiting their ability to adapt autonomously. Current approaches typically fall into two categories: those relying on environment-specific policy training, which restricts their transferability, and those generating code actions based on fixed prompts, which leads to diminished performance when confronted with new environments. These limitations significantly constrain the generalizability of agents in robotic manipulation.
To address these limitations, we propose a novel method called EnvBridge. This approach involves the retention and transfer of successful robot control codes from source environments to target environments. EnvBridge enhances the agent's adaptability and performance across diverse settings by leveraging insights from multiple environments. Notably, our approach alleviates environmental constraints, offering a more flexible and generalizable solution for robotic manipulation tasks.
We validated the effectiveness of our method using robotic manipulation benchmarks: RLBench, MetaWorld, and CALVIN. Our experiments demonstrate that LLM agents can successfully leverage diverse knowledge sources to solve complex tasks. Consequently, our approach significantly enhances the adaptability and robustness of robotic manipulation agents in planning across diverse environments.

Decision: reject
Reviewer scores: 6, 3, 3

Reviews and discussion:
<thread>
creation date:
1729977102901

thread_type:
official_review

<thread_object>
creator: reviewer_2mrc

<content>
summary:
The paper introduces ENVBRIDGE, a method that leverages Large Language Models (LLMs) for cross-environment knowledge transfer in robotic manipulation. The core idea is to use the control code from source environments as prompts to generate suitable control actions in new target environments. The proposed approach is tested across three common robotic manipulation benchmarks: RLBench, MetaWorld, and CALVIN.

strengths:
1. The use of LLMs for cross-environment task transfer is a creative and intriguing approach. I especially like the method of using previously successful control code as in-context prompts for generating control code in new tasks.
2. The experiments are conducted across three well-established manipulation benchmarks.

weaknesses:
1. My primary concern is the practical applicability of the method. Why should we rely on LLMs to generate control code for simulated environments, and how feasible is it to extend this approach to real-world robotic systems? As far as I understand, the method depends on ground-truth states and simulation-specific API calls, which may not easily translate to real-world scenarios where such information is not readily available. 
2. Certain aspects of the experimental setup are not clearly explained. I have raised specific questions about this below.

questions:
1. Could the authors clarify whether the method uses ground-truth states as input or camera inputs for perception?
2. The experimental protocol in Section 4.1 is somewhat ambiguous. In lines 340–341, the paper mentions “We sampled 10 tasks from it and conduct evaluations covering various tasks, instructions, and objects.” Does this imply different variations were introduced within these 10 tasks? More explanation would help.
3. In Section 4.2, why didn’t the authors use CALVIN as a source task, similar to the approach in Section 4.1?
4. In Section 4.3, what were the source tasks used for this experiment? Were they drawn from RLBench and MetaWorld?
5. The paper only compares ENVBRIDGE with VoxPoser, yet there are other related works that also utilize LLMs for generating robot control code. How does ENVBRIDGE compare to methods like Code as Policies?
6. Some minor formatting issues exist in the paper, such as mismatched quotation marks in lines 192 and 193.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730537829156

thread_type:
official_review

<thread_object>
creator: reviewer_89op

<content>
summary:
This paper proposes EnvBridge, an LLM-based method that can leverage diverse knowledge to solve complex tasks. It contains several components to endow the agents with good generalization ability.

strengths:
The proposed methods are evaluated on 3 benchmarks.

weaknesses:
- The Introduction section includes many discussions not related to LLMs for robotic manipulation, which appears redundant.

- The writing of the article is not very clear; concepts such as cross-embodiment and cross-environment need to be more distinctly defined. This significantly detracts from the overall logic of the paper.

- The quality of the figures in the paper is poor.

- I do not agree with the author's assertion that current LLM-based methods have significant generalization issues. Many LLM-based approaches have demonstrated strong generalization abilities in real-world applications. I also do not believe that the method proposed by the author addresses the existing issues effectively.

- I believe that this paper is not yet ready for submission to ICLR.

questions:
Seen above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730597064288

thread_type:
official_review

<thread_object>
creator: reviewer_d9n1

<content>
summary:
This work presents EnvBridge, a novel approach for computing robot trajectories via LLM-prompted code generation. The authors argue that transferring code across different environments, similar to human reasoning, can improve the success rate of task completion. EnvBridge extends existing methods like VoxPoser by adding a knowledge transfer mechanism to bridge across environments.

This process leverages a hierarchical code generation approach, dividing the task into subtasks (Planner) and generating 'language model programs' (LMPs) for each subtask (Composer). These LMPs produce value maps that guide the generation of an open-loop trajectory. I believe this trajectory is for the end-effector pose.

The system operates in several stages. First, an LLM generates code that outputs a trajectory, based on a language query describing the task. This is the same as VoxPoser. If unsuccessful, EnvBridge searches a repository for similar tasks where code generation failed at the same stage (Planner or Composer). Successful code from a similar task is then transferred to the target environment and used as an example in a revised prompt for the LLM, known as Knowledge Transfer and Re-planning.

strengths:
The primary strength lies in its novel approach to knowledge transfer for robot code generation via appropriate prompt engineering. The idea is well-motivated by observations of human behavior and offers a promising direction for improving LLM performance in robotics. The paper is generally well-written and presents the approach with clarity. Furthermore, the authors conduct extensive experiments across multiple tasks and environments in simulation, demonstrating the effectiveness of EnvBridge.

weaknesses:
1. EnvBridge generates open-loop trajectories, lacking the reactivity necessary for dynamic environments.
2. To my understanding, the system focuses solely on end-effector positions, neglecting possible collisions of the robot arm with the environment.
3. The user must explicitly provide information about objects to avoid in the scene, which could be automated through prompt engineering. For example, in the example memory shown in section A.3, the task is to "pick up the rubbish and leave it in the trash can.", and there are a bunch of objects given. The Planner level should create subtasks to avoid other objects (e.g. 'tomato1') while picking up the object named 'rubbish'.
4. The evaluation could be strengthened by comparing EnvBridge to other baselines like OpenVLA, which also generates end-effector poses from similar inputs.
5. The authors should provide further clarification regarding the performance discrepancies observed in Figure 2, where code from the same environment (RLBench) performs worse than transferred code from a different environment (CALVIN) - even though RLBench memory has more code examples than those in CALVIN. 
6. It would be good to have real-robot experiments, to improve the case for accepting any robotics paper.

Finally, to best understand the work, you need to see the prompts given in the appendix, and the type of prompt engineering used for each stage of EnvBridge. While this is not explicitly a weakness, these are important details worthy of being in the main paper.

### Minor Points

A possible typo on line 57: "transfer ability" should likely be "transferability"??
Throughout the paper (e.g., line 694), the authors should explicitly clarify that curly braces in the prompts `{}` denote variables to be filled in.

questions:
See above
</content>
</thread_object>

</thread>
<thread>
creation date:
1732256466809

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response to Reviews and Summary of Rebuttal Revision

comment:
Dear Reviewers,  

We thank you all for your time and effort investing in the review process. We particularly appreciate reviewer d9n1's recognition of our work's soundness (rated as excellent/4) and the positive assessment of our presentation (rated as good/3) by reviewers d9n1 and 2Mrc. We are encouraged that reviewers d9n1 and 2Mrc have recognized the novelty of our approach to cross-environment knowledge transfer for robot code generation through prompt engineering. 

We have carefully addressed each reviewer's concerns and questions in our individual responses below their respective reviews. Additionally, we have made several substantial improvements to the paper in our rebuttal revision: 

- Fixed some typos and minor formatting issues, including typos in Line 57, 694 [addressing Reviewer d9n1] and formatting issues in Line 192, 193 [addressing Reviewer 2Mrc]. 

- Incorporated more comprehensive analysis regarding the results reported in Table 2. [addressing Reviewer d9n1 Weakness 5] 

- Included new experimental results in Appendix A.4.3 investigating different knowledge sources. [addressing Reviewer 2Mrc Question 3] 

- Expanded the discussion section with more discussion on the influence of open-loop trajectories and potential collisions with environment. [addressing Reviewer d9n1 Weakness 1,2] 

 

We believe these substantial improvements address the key concerns raised in the reviews and strengthen our paper considerably. We welcome any additional questions and will provide further clarifications during the discussion period. We would be grateful if you could consider raising your ratings if you find our revisions satisfactory. 

Best regards, 

[Authors of Paper #4278]
</content>
</thread_object>

</thread>


Metareview:
metareview:
The authors propose ENVBRIDGE, an approach which uses large language models for cross-environment transfer for robotics. They transfer robotics control code between environments using the LLMs. They use three different simulators to show that the LLMs can be used to accomplish diverse tasks across these simulations. The system generates end-effector trajectories, similar to prior works like VoxPoser or PerAct, which improves generalization.

Strengths:
Using LLMs to transfer control code in this way is a novel idea.
They use three different well-established simulators: Meta World, RLBench, and Calvin.

Weaknesses:
Only these three simulators, and no real world results
No clear learning contribution
The work would be much stronger with vision experiments (noted by 2Mrc and 89oP)
How much prompt engineering is necessary for adaptation? For example, 
More baselines (the authors mention RVT in the discussion by the authors) would make results much more convincing

Overall, this work proposes some very interesting ideas, but these should be validated more strongly.