Paper: Large Language Models as Generalizable Policies for Embodied Tasks (ICLR-2024)

Reviewer comment (Theory):
The proposed method uses MLP to map LLM output to discrete actions, therefore the actions and LLM outputs are not in the same space.

Author response:
Modifying LLaRP to operate directly in the language token space rather than through the action decoder MLP is an interesting extension. We note extending LLaRP to implement this is non-trivial. Firstly, a single action in Language Rearrangement consists of at most 4 tokens (for example, “nav left counter”). This LLaRP variant requires predicting the correct sequence of 4 tokens from a vast number of tokens which decode to an invalid action. Secondly, it slows the RL process, because now the policy needs to auto-regressively predict 4 tokens per action prediction step rather than the single prediction needed in LLaRP. Despite these challenges, we agree this is an exciting direction for a follow up work and we updated the conclusion to reflect this (changes in red). 

We also note that we empirically demonstrate the policy is able to use the world knowledge from the LLM since LLaRP generalizes better than LLaRP-Scratch which does not have the pre-trained LLM weights.

Reviewer comment (Theory):
Due to the difference in spaces, it is uncertain whether the policy can fully leverage the world knowledge from the LLM.

Author response:
We also note that we empirically demonstrate the policy is able to use the world knowledge from the LLM since LLaRP generalizes better than LLaRP-Scratch which does not have the pre-trained LLM weights.

Reviewer comment (Experiments):
It would be better if the actions are directly produced by the LLM, although in that case the LLM weights may need to be unfrozen.

Author response:
Modifying LLaRP to operate directly in the language token space rather than through the action decoder MLP is an interesting extension. We note extending LLaRP to implement this is non-trivial. Firstly, a single action in Language Rearrangement consists of at most 4 tokens (for example, “nav left counter”). This LLaRP variant requires predicting the correct sequence of 4 tokens from a vast number of tokens which decode to an invalid action. Secondly, it slows the RL process, because now the policy needs to auto-regressively predict 4 tokens per action prediction step rather than the single prediction needed in LLaRP. Despite these challenges, we agree this is an exciting direction for a follow up work and we updated the conclusion to reflect this (changes in red).

Reviewer comment (Experiments):
Why not use a VLM backbone but instead use a language-only backbone?

Author response:
It is possible to change the network weight initialization in LLaRP with VLM weights instead of using VC-1 and LLaMA as in the paper. However, we started from LLaMA and VC-1 since we found the zero-shot performance of LLaMA was higher than the Flamingo VLM (12% vs. 6% success rate).

Reviewer comment (Novelty):
**incorrect claims and missing related work:** the paper states "To our knowledge, there are no prior work which demonstrate that the linguistic knowledge in LLMs can be used in online RL problems to improve generalization in embodied AI settings". There are many examples in the literature that actually have demonstrated that, some of which have been cited (such as ELLM, PALM-E, etc) and many were not discussed (e.g, SayCan, CodeAsPolicy, PercieverActor, HELM, ProgPrompt, EmbodiedGPT,...), though there are many examples of such methods. I recommend correcting the inaccuracies in such statements.

Author response:
**1. Incorrect claims and missing related work: the paper states "To our knowledge, there are no prior work which demonstrate that the linguistic knowledge in LLMs can be used in online RL problems to improve generalization in embodied AI settings". There are many examples in the literature that actually have demonstrated that.**

We corrected this claim to “To our knowledge, there is no prior work which demonstrates that LLMs can be used as vision-language policies in online RL problems to improve generalization” (changes in paper in red). This updated claim is accurate and the mentioned works do not demonstrate this. We apologize for this: this was an error that played out in the final editing of our first submitted draft (please refer to the general response). We compare to the works mentioned by the reviewer in detail below: 

- SayCan, CodeAsPolicy and ProgPrompt are zero-shot applications of LLMs that require describing the environment in text [1-3]. LLaRP adapts the LLM for _visual_ decision-making with _online RL_. By online RL, we mean learning from interactions with the environment. 
- PercieverActor, EmbodiedGPT and PaLM-E [4-6] learn from static expert collected datasets with supervised learning. LLaRP doesn’t need expert data and instead learns with RL.
- ELLM uses LLMs as a _reward to guide_ non-LLM policies from _text-based state descriptions_ [7]. We use the LLM directly as the decision-making policy from _visual observations_. 
- HELM uses LLMs (CLIP) as a “memory mechanism”, which we see as orthogonal from our contribution of using LLMs directly for policy decision-making [8]. Unlike LLaRP, HELM does not modify the LLM, use it for better generalization, or use it directly for decision-making.

Reviewer comment (Reproducibility):
**lack of details:** the main paper lacks many details such as what simulation were used, how the agent executes the skills, how the skills are defined, and what is the contribution of the defined PDDL. Although some details are provided in the supplementary material, clarifications are needed to be presented in the main paper.

Author response:
**2. Lack of task details.**

We expanded Supp. A.4 to describe the simulation in detail. We also added a description of the key simulation details in a new paragraph in the main paper (Sec 4.1). Due to limited space, we described how the agent executes the skill and skill definitions in Supp. A.3 and the PDDL definition in Supp. A.1. 

We never claim the defined PDDL is a contribution, it is only used in the implementation of the Language Rearrangement task (which is a contribution).

Reviewer comment (Novelty):
**lack of comparison to existing benchmarks:** The paper does not provide a comprehensive comparison to available benchmarks (some examples includes but not limited to Habitat rearrangement, CortexBench, AI2Thor, Behavior1k, Procthor, ALFRED). Hence, it is not clear how the new provided axes and extending instructions stand against existing work. It is quite common in the literature to dedicate a section and a table to compare various aspects of a newly proposed benchmark to existing ones from various aspects such as scale, generalization aspects, number of samples, kinds of provided data (such as language instructions) etc.

Author response:
**3. Lack of comparison to existing benchmarks.**

Thank you for the suggestion, we include a table highlighting the differences between Language Rearrangement and prior benchmarks below without citations and in full in Table 4, supplementary A.7.

Language Rearrangement has more instructions than prior benchmarks. It has almost 18x more instructions than ALFRED (151k vs 8k) and 8x more instruction templates than CALVIN (282 vs. 35). “# instructions” refers to the number of linguistically distinct instruction types, meaning “pick the apple” and “pick the pear” are counted as two separate instructions. “# Instruction Templates” refers to distinct instructions regardless of specific entities, so the same picking example is only counted as a single instruction. Also unique about Language Rearrangement is that it has dense rewards for language specified tasks. All other tasks with language instructions only define sparse success-based rewards. Since Language Rearrangement is implemented with Habitat 2.0, it also inherits the same fast simulation speeds.

| Benchmark                                         | # Instructions (Tasks)            | # Instruction Templates   | Observation Type | Reward Type     | Generalization Type                                                                              | Sim Speed steps/second | # Scenes | # Objects |
|---------------------------------------------------|----------------------------|---------------------------|------------------|-----------------|--------------------------------------------------------------------------------------------------|------------------------|----------|-----------|
| **Language Rearrangement (Ours)**                 | 151,000                    | 282                       | Visual           | Dense Reward    | Unseen Instructions [10 datasets from Tab.1], Unseen Scenes                  | 1400                   | 105      | 82        |
| ALFRED                  | 8,055                      | 7                         | Visual           | Sparse Success  | Random Split                                                                                     | NA                     | 120      | 84        |
| CALVIN                      | 400                        | 35                        | Visual           | Sparse Success  | Unseen Instructions                                                                              | NA                     | 1        | 1         |
| ARNOLD                      | 32                         | 8                         | Visual           | Sparse Success  | Random Split                                                                                     | 200-400                | 20       | 40        |
| CLIPort (Ravens) | 10 (+ Procedural)         | 10                        | Visual           | Sparse Success  | Unseen Objects [colors, shapes, types]                                                           | NA                     | 1        | 56        |
| BabyAI    | Procedural                 | Procedural        | Text             | Sparse Success  | Unseen Instructions [compositions, objects, synonyms, dialects]                                  | 3000                   | NA       | 4         |
| Habitat Rearrangement | NA                        | NA                        | Visual           | Dense Reward    | Unseen Scenes                                                                                     | 1400                   | 105      | 20        |
| Behavior1k                  | NA                        | NA                        | Visual           | Sparse Success  | NA                                                                                               | 60                     | 50       | 5215      |
| TDW                   | NA                        | NA                        | Visual           | Sparse Success  | Unseen Scenes                                                                                     | 15                     | 5-168    | 112       |
| ProcTHOR                        | NA                        | NA                        | Visual           | Dense Reward    | Unseen Scenes                                                                                     | 90-180                 | 10k      | 118       |

Reviewer comment (Experiments):
**limited comparison to existing work:** although this paper compares to relevant approaches that have proven to be effective such as zeroshot in-context text-only, as well as encoder+LSTM, there are many seminal works that are applicable to this environment and it would bring more value to comparisons if the paper actually leverages some pre-existing methods. Examples of such methods include CodeAsPolicy (Code Gen), PercieverActor (BC), and SemanticHELM (pretrained vision and LLM+LSTM). Atari is a well-established benchmark, which has many competitive baselines, which has not been provided for comparison. Providing additional evidence on other embodied AI benchmarks where previous established baselines exists provides better comparison datapoints to compare the proposed method to the existing work.

Author response:
**4. Limited comparison to existing work.**

We thank the reviewer for suggestions around additional baselines, but believe they are not suited for our problem setting.

- CodeAsPolicies [2] relies on perception modules which Language Rearrangement doesn't have. This is also similar to our ZS-ChatGPT baseline which replans based on environment feedback.
- PercieverActor [4] requires offline data for imitation learning and depth observations for constructing point clouds. Language Rearrangement assumes access to neither. 
- While Semantic HELM [8] can be combined with LLaRP and baselines to increase learning efficiency and interpretability it does not address how to generalize to unseen instructions. 

Our experiments compare LLaRP to three RL and three zero-shot baselines. We compare three different LLM/VLMs (LLaMA, ChatGPT, and Flamingo). 

**Citations**

[1] Ahn, Michael, et al. "Do as i can, not as i say: Grounding language in robotic affordances." arXiv preprint arXiv:2204.01691 (2022).

[2] Liang, Jacky, et al. "Code as policies: Language model programs for embodied control." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

[3] Singh, Ishika, et al. "Progprompt: Generating situated robot task plans using large language models." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

[4] Shridhar, Mohit, Lucas Manuelli, and Dieter Fox. "Perceiver-actor: A multi-task transformer for robotic manipulation." Conference on Robot Learning. PMLR, 2023.

[5] Mu, Yao, et al. "Embodiedgpt: Vision-language pre-training via embodied chain of thought." arXiv preprint arXiv:2305.15021(2023).

[6] Driess, Danny, et al. "Palm-e: An embodied multimodal language model." arXiv preprint arXiv:2303.03378 (2023).

[7] Du, Yuqing, et al. "Guiding pretraining in reinforcement learning with large language models." arXiv preprint arXiv:2302.06692 (2023).

[8] Paischer, Fabian, et al. "Semantic HELM: A Human-Readable Memory for Reinforcement Learning." Thirty-seventh Conference on Neural Information Processing Systems. 2023.

Reviewer comment (Novelty):
extend related work and discuss relation to existing work that has been detailed in the weaknesses section.

Author response:
**1. Incorrect claims and missing related work: the paper states "To our knowledge, there are no prior work which demonstrate that the linguistic knowledge in LLMs can be used in online RL problems to improve generalization in embodied AI settings". There are many examples in the literature that actually have demonstrated that.**

We corrected this claim to “To our knowledge, there is no prior work which demonstrates that LLMs can be used as vision-language policies in online RL problems to improve generalization” (changes in paper in red). This updated claim is accurate and the mentioned works do not demonstrate this. We apologize for this: this was an error that played out in the final editing of our first submitted draft (please refer to the general response). We compare to the works mentioned by the reviewer in detail below: 

- SayCan, CodeAsPolicy and ProgPrompt are zero-shot applications of LLMs that require describing the environment in text [1-3]. LLaRP adapts the LLM for _visual_ decision-making with _online RL_. By online RL, we mean learning from interactions with the environment. 
- PercieverActor, EmbodiedGPT and PaLM-E [4-6] learn from static expert collected datasets with supervised learning. LLaRP doesn’t need expert data and instead learns with RL.
- ELLM uses LLMs as a _reward to guide_ non-LLM policies from _text-based state descriptions_ [7]. We use the LLM directly as the decision-making policy from _visual observations_. 
- HELM uses LLMs (CLIP) as a “memory mechanism”, which we see as orthogonal from our contribution of using LLMs directly for policy decision-making [8]. Unlike LLaRP, HELM does not modify the LLM, use it for better generalization, or use it directly for decision-making.

Reviewer comment (Experiments):
provide comparison to existing benchmarks that has been pointed out in the weaknesses section.

Author response:
**3. Lack of comparison to existing benchmarks.**

Thank you for the suggestion, we include a table highlighting the differences between Language Rearrangement and prior benchmarks below without citations and in full in Table 4, supplementary A.7.

Language Rearrangement has more instructions than prior benchmarks. It has almost 18x more instructions than ALFRED (151k vs 8k) and 8x more instruction templates than CALVIN (282 vs. 35). “# instructions” refers to the number of linguistically distinct instruction types, meaning “pick the apple” and “pick the pear” are counted as two separate instructions. “# Instruction Templates” refers to distinct instructions regardless of specific entities, so the same picking example is only counted as a single instruction. Also unique about Language Rearrangement is that it has dense rewards for language specified tasks. All other tasks with language instructions only define sparse success-based rewards. Since Language Rearrangement is implemented with Habitat 2.0, it also inherits the same fast simulation speeds.

| Benchmark                                         | # Instructions (Tasks)            | # Instruction Templates   | Observation Type | Reward Type     | Generalization Type                                                                              | Sim Speed steps/second | # Scenes | # Objects |
|---------------------------------------------------|----------------------------|---------------------------|------------------|-----------------|--------------------------------------------------------------------------------------------------|------------------------|----------|-----------|
| **Language Rearrangement (Ours)**                 | 151,000                    | 282                       | Visual           | Dense Reward    | Unseen Instructions [10 datasets from Tab.1], Unseen Scenes                  | 1400                   | 105      | 82        |
| ALFRED                  | 8,055                      | 7                         | Visual           | Sparse Success  | Random Split                                                                                     | NA                     | 120      | 84        |
| CALVIN                      | 400                        | 35                        | Visual           | Sparse Success  | Unseen Instructions                                                                              | NA                     | 1        | 1         |
| ARNOLD                      | 32                         | 8                         | Visual           | Sparse Success  | Random Split                                                                                     | 200-400                | 20       | 40        |
| CLIPort (Ravens) | 10 (+ Procedural)         | 10                        | Visual           | Sparse Success  | Unseen Objects [colors, shapes, types]                                                           | NA                     | 1        | 56        |
| BabyAI    | Procedural                 | Procedural        | Text             | Sparse Success  | Unseen Instructions [compositions, objects, synonyms, dialects]                                  | 3000                   | NA       | 4         |
| Habitat Rearrangement | NA                        | NA                        | Visual           | Dense Reward    | Unseen Scenes                                                                                     | 1400                   | 105      | 20        |
| Behavior1k                  | NA                        | NA                        | Visual           | Sparse Success  | NA                                                                                               | 60                     | 50       | 5215      |
| TDW                   | NA                        | NA                        | Visual           | Sparse Success  | Unseen Scenes                                                                                     | 15                     | 5-168    | 112       |
| ProcTHOR                        | NA                        | NA                        | Visual           | Dense Reward    | Unseen Scenes                                                                                     | 90-180                 | 10k      | 118       |

Reviewer comment (Experiments):
extend empirical evaluations and comparison to existing work that has been described in the weaknesses section.

Author response:
**4. Limited comparison to existing work.**

We thank the reviewer for suggestions around additional baselines, but believe they are not suited for our problem setting.

- CodeAsPolicies [2] relies on perception modules which Language Rearrangement doesn't have. This is also similar to our ZS-ChatGPT baseline which replans based on environment feedback.
- PercieverActor [4] requires offline data for imitation learning and depth observations for constructing point clouds. Language Rearrangement assumes access to neither. 
- While Semantic HELM [8] can be combined with LLaRP and baselines to increase learning efficiency and interpretability it does not address how to generalize to unseen instructions. 

Our experiments compare LLaRP to three RL and three zero-shot baselines. We compare three different LLM/VLMs (LLaMA, ChatGPT, and Flamingo). 

**Citations**

[1] Ahn, Michael, et al. "Do as i can, not as i say: Grounding language in robotic affordances." arXiv preprint arXiv:2204.01691 (2022).

[2] Liang, Jacky, et al. "Code as policies: Language model programs for embodied control." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

[3] Singh, Ishika, et al. "Progprompt: Generating situated robot task plans using large language models." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.

[4] Shridhar, Mohit, Lucas Manuelli, and Dieter Fox. "Perceiver-actor: A multi-task transformer for robotic manipulation." Conference on Robot Learning. PMLR, 2023.

[5] Mu, Yao, et al. "Embodiedgpt: Vision-language pre-training via embodied chain of thought." arXiv preprint arXiv:2305.15021(2023).

[6] Driess, Danny, et al. "Palm-e: An embodied multimodal language model." arXiv preprint arXiv:2303.03378 (2023).

[7] Du, Yuqing, et al. "Guiding pretraining in reinforcement learning with large language models." arXiv preprint arXiv:2302.06692 (2023).

[8] Paischer, Fabian, et al. "Semantic HELM: A Human-Readable Memory for Reinforcement Learning." Thirty-seventh Conference on Neural Information Processing Systems. 2023.

Reviewer comment (Novelty):
What distinguishes this work from [1,2, 3, 4], which also appear to emphasize the evaluation of LLMs' generalization abilities in embodied settings?

Author response:
**1. What distinguishes this work from [1,2,3,4], which also appear to emphasize the evaluation of LLMs' generalization abilities in embodied settings?**

What distinguishes our work from [1,2,3,4] is that our main contribution is to demonstrate that an LLM can be adapted with _online RL_ to serve as a generalizable _vision language_ (VLM) policy for embodied AI tasks. We apologize for the confusion here. We updated the writing in sections 1 and 2 to better reflect this (changes in red) and refer to our general response for more details. 

We appreciate the opportunity to talk about this main contribution w.r.t. the prior works mentioned. [1,3] primarily focus on finetuning a _text only_ LLM with supervised learning in _text-based_ observation and action spaces. LLaRP adapts an LLM with _online RL_ for _visual, instruction_ tasks directly in the environment action space. [2] also uses RL, but does so in text-based environments. We adapt LLMs to visual tasks and environment action spaces. [4] along with related works [6,7] are "zero-shot policies for interactive decision-making tasks, without task specific training, in settings where the states and action spaces are both text-based" (Sec. 2).

Reviewer comment (Evaluation):
The paper presents successful results but does not thoroughly explore or discuss failure cases. It would be great to have more qualitative examples that enable readers to understand the limitations of employing pretrained LLMs on embodied tasks.

Author response:
**2. More qualitative examples for understanding limitations of employing pretrained LLMs on embodied tasks.**

Thank you for the suggestion. We included failure analysis in supplementary section D to support the success qualitative examples in figure 6 and section D.

Reviewer comment (Experiments):
It would be valuable to gain insights into the performance of simpler baselines, such as embedding-based models like embCLIP, or baselines from the embodied rearrangement literature such as [5], in comparison to the proposed approach. Currently, all existing baselines rely on pretrained Language Models (LLMs), with varying prompts and inputs. While this dependence on LLMs is shown to yield strong results, it would be good to understand the necessity and efficiency of LLMs for the tasks at hand. In other words, it is difficult to ground the work in existing embodied literature, since it introduces a new task (language rearrangement) and a new model (LLM-based LLaRP model trained with online RL).

Author response:
**3. Valuable to gain insights into the performance of simpler baselines.**

The LLaRP-Scratch baseline is a simple baseline that does not rely on LLMs (Sec. 5.1). LLaRP-Scratch shows the necessity of LLMs due to its lesser generalization performance (17% vs 42% for LLaRP in Table 2) and efficiency (it takes 500M vs. 50M steps to converge, Fig. 4a).

In this rebuttal, we added a comparison to using EmbCLIP [8] in LLaRP. As with [8] we used the pre-trained CLIP-ViT-L as the visual encoder for the policy. This performed slightly worse than LLaRP, achieving 40% total success rate, vs. 42% success rate for LLaRP. VC-1 [9] also reported better performance than EmbCLIP for other Habitat tasks. We also note that [5] is a method solely for improving training wall clock time, while it could help scale all our experiments, by itself it will not improve generalization performance.

Reviewer comment (Experiments):
How are the current baselines chosen, e.g., what would be the reason for not comparing with an LSTM-Flamingo baseline or using other VLMs such as InstructBLIP, LLaVA, miniGPT?

Author response:
**4. How are the current baselines chosen, why not compare with other VLMs?**

We chose Flan for the LSTM-Flan baseline because Flan is a powerful encoder model that can embed instructions into fixed length vectors, which the LSTM then takes as input. Using a decoder-based model instead, such as Flamingo, is better suited for text generation, not encoding instructions. We show this with the LSTM-LLaMA model performing significantly worse than the LSTM-Flan model (Table 2). 

We used IDEFICS since, at the time of submission, it was the state-of-the-art VLM model.

Reviewer comment (Novelty):
Technical novelty is not high. Using VLMs to control a task is not new, e.g., PaLM-E and RT-2 as discussed in this paper. It is true these two method does not employ reinforcement learning, but architectural or technical difference is slight and is not well discussed in the paper.

Author response:
**1. Technical novelty is not high. Using VLMs to control a task is not new.**

We disagree and believe our work is novel in training VLMs _with reinforcement learning_ (RL) to control a task. While other works have trained VLMs from LLMs in offline settings, no other works have trained VLMs with online RL (i.e., without expert demonstrations). RT-2 and PaLM-E are _offline_: they are given 130,000 human teleoperated demonstrations collected over almost 2 years as training data [2,3]. LLaRP only has access to interaction data collected from its current policy (it uses PPO, which is on-policy), and we demonstrate that, by adapting LLMs in this way, LLaRP achieves generalizable decision-making.

The architectural differences from online LLaRP and prior offline VLM decision-making models are rooted in the relative differences between online vs offline learning. RT-2 and PaLM-E use expert demonstrations, so their training is similar to supervised learning: they take a single image as input along with the task specification, and fine-tune the LLM to generate the text-based action instructions from the expert demonstrations. LLaRP does not have access to this expert data, and so trains using an online RL algorithm. Exploration is a critical capability for online learning, so in order to help in this regard, LLaRP takes as input all observations from the current episode (up to 32). Since LLaRP does not have access to expert actions (e.g., as text or otherwise), we add an action decoder module and train it with RL using the reward signal from the environment. 

Finally, beyond technical novelty, a useful contribution of our paper is the empirical analysis of the generalization capabilities of using an LLM as a VLM policy trained with online RL, as we explain in the next response.

Reviewer comment (Novelty):
While the one of the main claim or implication of the paper might be linguistic knowledge in LLMs can be used in online RL, but the statement itself is already validated in prior studies, e.g. [1]. While it is true that [1] focus on the textual environment, but there are no discussion how this paper extends the prior understanding on what kind of knowledge is encoded in the LLM and what is not.

Author response:
**2. While the one of the main claim or implication of the paper might be linguistic knowledge in LLMs can be used in online RL, but the statement itself is already validated in prior studies**

This was a writing error on our part since our main claim is about adapting LLMs for _vision language_ policies with online RL. We mistakenly omitted that the novelty is specific to vision-based policies with online RL in the related work section. The reviewer is correct that [1] shows LLMs for online RL in textual environments. We updated the related work writing to address this and the connection to [1] (changes in red). Thank you for bringing this to our attention. 

We apologize for the confusion, but the corrections are minimal and do not change our main contribution of adapting LLMs for VLM policies with online RL. [1] also uses LLMs in online RL, but focuses on environments with text-inputs and text-outputs, thus this method does not apply to the visual decision-making in our setting. The LLaRP architecture is also different from [1] since we train an observation encoder and action decoder module, which are additional components to operate from visual observations and general action spaces. 

The empirical analysis in Sec. 5 also contains novel insights about LLMs for embodied AI, which itself is a contribution. We benchmark a wider variety of baselines than [1]. We include zero-shot baselines with powerful LLMs (ZS-ChatGPT, ZS-Flamingo, ZS-LLaMA), while [1] only compares to Flan-T5. We train policies for over 100x more experience (200M vs 1.5M steps). We show generalization to 1,000 new instructions across 10 distinct splits. [1] only shows generalization to new objects, compositions, synonyms, and translation (French vs. English). Unlike [1] we also show the efficiency of fine tuning to new downstream tasks (Fig 4b).

Reviewer comment (Evaluation):
The paper lacks in depth analysis of the failure case, which might be important to dig in the internal knowledge of LLM.

Author response:
**3. The paper lacks in depth analysis of the failure case.**

Thank you for the suggestion. We included failure analysis in supplementary section D to support the success qualitative examples in figure 6 and section D.