Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
Authors: Vivek Myers
Chunyuan Zheng
Oier Mees
Kuan Fang
Sergey Levine
Venue: CoRL 2024 (2024)
Keywords: Reinforcement Learning
Vision-Language Models
Manipulation
TL;DR: We show decomposing tasks with language can enable few-shot adaptation to OOD manipulation tasks.

Abstract: Learned language-conditioned robot policies often struggle to effectively adapt to new real-world tasks even when pre-trained across a diverse set of instructions. We propose a novel approach for few-shot adaptation to unseen tasks that exploits the semantic understanding of task decomposition provided by vision-language models (VLMs). Our method, Policy Adaptation via Language Optimization (PALO), combines a handful of demonstrations of a task with proposed language decompositions sampled from a VLM to quickly enable rapid nonparametric adaptation, avoiding the need for a larger fine-tuning dataset. We evaluate PALO on extensive real-world experiments consisting of challenging unseen, long-horizon robot manipulation tasks. We find that PALO is able of consistently complete long-horizon, multi-tier tasks in the real world, outperforming state of the art pre-trained generalist policies, and methods that have access to the same demonstrations.

Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
Vivek Myers, Bill Chunyuan Zheng, Oier Mees, Sergey Levine, Kuan Fang! University of California, Berkeley
Abstract: Learned language-conditioned robot policies often struggle to effec- tively adapt to new real-world tasks even when pre-trained across a diverse set of instructions. We propose novel approach for few-shot adaptation unseen tasks that exploits the semantic understanding of task decomposition provided by vision-language models (VI.ss). Our method, Policy Adaptation via Language Optimization (PALO), combines handful of demonstrations of a task with pro- posed language decompositions sampled from VLM to quickly enable rapid nonparametric adaptation, avoiding the need for a larger fine-tuning dataset. We evaluate PALOO on extensive real-world experiments consisting of challenging un- seen, long-horizon robot manipulation tasks. We find that PALO is able of con- sistently complete long-houe long-horizon, mulli-tier tasks in the real world, outperforming state of the art pre-trained generalist policies, and methods that have access to the same demonstrations.
Keywords: Few-shot Learning, Vision Language Models, Robot Manipulation
Pretraining
Adaptation
Evaluation
put the tummp
D target
draver'
#Cx c2)
VLM M
propose
D prior
(1)
'grasp
"move left"
C
(2) BB
d2
open
"pull
states
longuage
actions
(().
"open ČK
"clase g griope!"
(K) C
gripper
min Lyc = ||āq - uzll
arg min Cauer
min
[T]
partations
learn longuep-conciitoone policy with behavioral cloning
rollout with optimal language
subsack decomppsitio
polley, optimmisn over longuage using a demonstrationa
Figure I: An overview of the PALO algorithm for few-shot adaptation with language. (Left) We build off pre- trained policy that has learned follow low-level language instructions from large datasel of expert demon- strations. (Middle) Given new task and a few expert demonstrations, we use a VLM to propose candidate decompositions into subtasks. We optimize over these decompositions jointly with the partitions of trajectories into subtasks, selecting the subtask decomposition that minimizes the validation error of the learned policy. (Right) At test time, condition the pre-trained policy the selected decomposition solve the task.
'Equal contribution.
'Equal advising-
Projectt webpage: https://palo-veesiii www.eeebeeeee.iitbbbeeeeeeeeeeee github. io
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Introduction
Robot learning promises policies that can adapt and generalize new behaviors. However, in practice, today's robotic policies often struggle to effectively finetune for truly new tasks [I, 2, 3, 5J. For example, consider the task of making salad: while a person could likely follow new recipe with only few examples by remembering the key steps, robot learning approach may need many more demonstrations to achieve similar performance, and still recover a more brittle policy.
key difference that allows humans to learn tasks so quickly is their semantic understanding of the world. Human have a symbolic representation of the task, such as the names of the ingredients and the steps prepare them, rather than series of low-level actions. This representation enables them to understand the task at higher level, mapping directly into low-level behaviors they are already familiar with [6, 7). How can we enable robots to quickly learn new tasks through a semantic understanding of the world?
Language provides a potential bridge between these task semanties and low-level control [8]. Recent advances in large language models (LLMs) and vision-language models (VLMs) have shown promise in understanding and
new task:
Pretrsined Language Policy
"put the turnip the drawer"
+
PALO: Policy Adaptation
Poliey Finetuning in parameter space
grounding language from a few demonstrations [9, 10]. We propose Policy Adaptation via Lan- guage Optimization (PALO), method for ex- ploiting the semantic understanding of VLMs in combination with pre- pre-trained robot policy to enable adaptation new tasks with only few demonstrations (Fig. 1).
Language Optimization
VLM
πT
π
Grasp
"Puil nome
min LBC
"Opon grippe"
optimize over parameters
Past approaches that fine-tune directly new demonstrations are often overparameterizee and sample-ineffiiient, due to the cost inher- ent in collecting teleoperated trajectories [11]. Instead, we use few demonstrations as cal-
search over Languagge
only demos needed!
need >- demos
PALO
Finetuning
Suatres Hatte
0.8
ibration set to guide the decomposition of new language task into a sequence of subtasks that can be used by a language-conditione pol- icy. Our approach samples possible decompo- sitions of the task from VLM and chooses the one that minimizes the validation еиror of the learned policy on the calibration set.
0.2
20
40
60
# Demonst rations
Figure 2: PALOO enables pre-trained generalist policies to adapt new tasks with as few as five demonstrations by searching in language space instead of parameter space.
The key is that in the few-shot setting, few demonstrations provide better signal for adapting new tasks when used select the right sequence of language subtasks with the help of VLM, rather than directly fine-tuning the policy parameters (Fig. 2). Unlike prior work, our ap- proach can learn unseen, long-horizon behaviors with fewer than 10 demonstrations across a variety of tabletop manipulation tasks.
Related Work
Our approach lies at the intersection of few-shot learning and approaches that leverage language and large pre-trained models for robotics.
Few-shot learning. Broadly speaking, few-shot learning approaches utilize diverse data to enable rapid test-time adaptation to new task from few examples. These techniques have been applied in various domains, including vision [12, 13, 14], natural language processing [9, 15], and rein- forcement learning [16, 17]. Frameworks for few-shot learning include optimization- based meta- learning [18. 19, 20], where model is trained to quickly fine-tune tasks, nonparametric

---

methods based on particular modeling assumptions such as metric approaches and Gaussian pro- cesses [21, 22, 23, 24), and in-context learning [9, 25, 26, 27], where a large model is conditioned on a context to adapt to a new task. Unlike past approaches to few-shot learning in robotics [28, 11], we show that language can be used to enable nonparametric adaptation without fine-tuning.
robotic control. While early approaches to instructionffolloing in robotics relied on manually designed symbolic representations [29, 30, 31], recent work has fo- cused on applying deep learning techniques to understand natural language instructions [32, 33]. These approaches use learned behavioral cloning policies on top of language [2, 34, 35]. connect language representations to grounded representations of the environment [36, 37, 38, 39], the compositional structure of language to decompose tasks and plan [40, 41, 42, 43, 44]. Our ap- proach is the first to enable few-shot adaptation to new demonstrations in robotics by leveraging the structure of language.
Language-condition
Foundation models and robotics. Large-scale internet pre-training has seen recent success in the domains of vision and natural language processing [9, 45, 46, 47, 9, 48). Recent work has investigated if these models can be trained and/or fine-tuned for downstream robotics tasks [32, 49, 50, 11, 51, 52]. Other work has investigated if these models can be used to provide semantic knowledge for downstream robot learning pipelines [53, 54, 55, 56, 57, 58, 59]. Our approach falls into this latter category. but unlike the past works, we perform few-shot adaptation in language- conditioned robot control using the semantic knowledge in large pre-trained VLMMs.
3 Policy Adaptation via Language Optimization
Our goal is to enable a learned language-conditione robot policy to perform new tasks with only a few demonstrations. The key insight is that the structure of language can be exploited to enable few- shot adaptation to new demonstrations in robotics. Fundamentally, few-shot adaptation to new tasks depends on policy's ability to generalize its existing knowledge correctly fit new demonstra- tions. One approach for adapting learned policy is to directly fine-tune to new demonstrations, but in robotics settings where expert data collection is costly, this is often infeasible due to overfitting.
We propose Policy Adaptation via Language Optimization (PALO), which instead uses demonstra- tions of task that is outside the training distribution with the reasoning capabilities of pre-trained vision-languee model (VLM) determine the correct sequence of decomposed subtasks that are in-distribution for the robot policy. Given a language instruction E, we compute task decompo- sition Ci:K that is both semantically consistent with the instruction (determined by the VLM) and feasible in the environment (measured by policy validation loss on expert demonstrations).
3.1 Notation
Formally, we assume a contextual Markov Decision Process (MDP) structure. We have a state space S, continuous action space A (0, 1)da, initial state distribution po. transition probabilities P. and free-form language instruction f E L chosen from the language instruction space L. We use the notation P(X) to denote the set of probability distributions over space X.
The robot selects the action a, E A based the observed state S, S at each time step t {1...H) over a finite horizon H to achieve states in St. We denote robot policy map Tāē | s,, E), which maps the state 8, and instruction f to distribution over actions âz For convenience, we assume actions are selected under a fixed isotropic Gaussian noise model unless otherwise specified, and will denote the mode of the distribution Tia St, *) as π (st. *). A robot policy then yields a distribution over trajectories ([(s,, a ~ Tí given a task distribution p E P(C).
3.2 Problem Statement
We want to solve out-of-diistributio instruction-folloviii tasks involving unseen objects and skills given only a few demonstrations. For access to a dataset that has been generated using language tasks sampled from some distribution Pprior P(C) with expert policy (ā 8, s).). For training an instruction-followe
the instruction-followe policy we assume
policy ( Nyer for (1), l T:Paiu and additional
(s,, f), assume a prior dataset D Pprior
{(T()) CLK {
3

---

seaphiiccallydeomppsscee subtask instructions C1.K E C that are distributed according to p(C1:K 80: f) for decomposition size K < H.
A target task is sampled from a separate distribution Purget E ((C) which requires interacting with unseen objects in novel ways, so the policy trained on prior performs poorly zero-shot. To solve this new task, we assume there exists an additional dataset Durget ((TI... Jn E) for TÜ T So with So Po and E Pragett collected by human experts. While large Darges can enable directly training (s, E) to solve the target task, we interested in challenging few-shot scenarios in which Diarget only contains handful of demonstrations (e.g., 5). In this paper, we tackle this challenge by decomposing the novel target task into a sequence of subtasks that are solvable by the pre- trained (s, c) using a VLM M. Notably, we do not assume any ground truth labels for the task decomposition are given, and aim to generate the optimal language decomposition C1:K based on the unlabeled demonstration dataset Durreet collected by human operators.
Our approach makes two assumptions about the structure of the target task.
Assumption 1. The target task subtask annotations locally match those of the prior dataset, i.e,, are distributed identically for i ~ Unif(1.... H)
P0
80, E) R So
80: E).
Assumption states that even if the overall target tasks in Prarget are unseen, the low-level manipula- tion skills (e.g., "close the gripper," "move the arm right") will be represented in the policy training.
Assumption 2. The VLM M can approximate the distribution of the subtask annotations Cı:K in the larget task, ie.,
PM[[ILK] $o,2) p(cr.k | so. E).
(2)
Assumption 2 states that the VLM can propose candidate task decompositions that are consistent with the instruction l in new scenes. Qualitatively, these assumptions are consistent with recent advances in robot manipulation training data [5, 51] and embodied reasoning with VLMs [60] and empirically validated in experiments in Section 4 using the BridgeDataV2 dataset [5] and GPT-4o [46] with prompting described in Appendix F.
In Section 3.6 we show that under these assumptions, our PALO algorithm can achieve low regret on out-of-distribution tasks, and discuss how violating these assumptions affects performance.
3.3 Task Decomposition with Language
To guide the pre-trained policy â to solve the unseen target task, we decompose the high-level language instruction e of the target task into a sequence of subtask instructions C1:K = (CI...... .k) for the K subtasks instruction É, we use combination of l and the subtask instructions C the input in each subtask to produce the action as a, <- #(st, ck). In our methods, we used GPT-4o [46] as backbone to generate instruction sets. We denote by MMso. E) the support of possible task decompositions sampled from this VLM (see details in Appendix F).
set of language decomposition. Instead of commanding स with the original
Aside from the sequential order of the subtasks, the robot needs to decide when to switch to the next subtask. For this purpose, we introduce an additional variable u = ( ~ Unif((d) where 2 is the space of ordered partitions of (0...H) . so ug denotes the time steps on which the robot is executing the k-th subtask. Notably, we assume the optimal solution to the target task follows fixed structure, i.e., the same subtask sequence can be used to solve the task, regardless of the initial state 8n- Meanwhile, can be different in each episode, since the number of steps needed to complete each subtask depends on So as well as stochasticity in the environment and the policy.
...
3.4 FSotS Adaptation through Language Decomposition
We design a simple sampling-base based inference algorithm to find the best c* for guiding the policy ă to solve the target task. Since the resulting actions depend on both e and u, as discussed in Section 3.3, we jointly optimize e and u to minimize cost function J over all trajectories in D Punges::
min G (M TEDens
Σ
min Bi
((c..7))
(3)

---

(= "put the turnip in the drawer"
H 'I
move the purple thing the drawer
place the perple thing
open the drawer
pick up the purple thing
L
nam the empper bodd
ptipper hanill
won the of yeor thedeance
opre
iecm glippr right perpletiing
Sive dae erape
nee the arsper Idi doose
new the srpper dusoe:
spe the pray perpletiing
C
Figure 3: A visualization of an example execution of our method on the long- long-horizon task "put the beet toy the drawer." The VLM deconstructs into candidate high-level subtasks c and low-level subtasks of and optimizes over expert trajectories.
optimal c and efim are chosen unrolled real-world evaluations, which result in successful completion the task (trajectory shown in gray).
To mcasure how well and u enable the policy स to reproduce each т, the cost function is defined with the mean squared eтror between the predicted action â and the ground truth ae at each time step l. More specifically, we evaluate the policy म on the demonstration trajectory given e and u to compute à, f- (5;,, (min((:))).))) Then the cost function defined as:
I (c, E E
- # (st, c)]l
By minimizing this cost across demonstrations, we compute a decomposition the task that would optimally perform the task by minimizing the loss between the action of the robot and the expert.
3.5 Learning Composable CnatuuctionFoollwiing Primitives
We use language-conditionel behavior cloning [61] learn a policy ( e) based on the expert trajectories of D prior To enable conditioning fine-grained hierarchical language instructions, factorize से through Cı:K:
tā 6), - Σ p(enk (
Σ (a St, cx)p((k
(5)
CLIEE
for the subtask index at time t: k, minfk: :t uk. u1:K Unif(L()}. We learn parameters 0 for tg by minimizing the following behavioral cloning objective:
(0) ae
cx), - aoll"]- -a.l)
The training dataset Dprior an augmented version of BridgeData [5], dataset containing diverse set of manipulation tasks on common household objects. Details about how the subtask instructions are generated are discussed in Appendix D. Fach e, is further partitioned into a high-level component cf and a low-level component of Our full implementation is described in Appendix D.
3.6 Analysis PALO
Our theoretical results study the regret of this approach on out-of-distribution tasks in Purget showing that it trades off the performance of the pre-trained policy on Pprior and the performance the VLM M in accurately modeling the hierarchical language decomposition p(CL.)) in Prarget We define regret with respect to the expert policy Ty and given task distribution in terms of the MSE:
( e) To( 011
Rx (T:; p) = ET.,
(7)
H HvdA
5

---

Algorithm
Policy Adaptation via Language Optimization (PALO)
Require:
a VLM M. pre-trained
policy x (ā C),
number of candidate decompositions to sample M, optimization steps N new task described by with n expert demonstrations Duaget collected manually
Input:
Output: policy #(- 8x) adapted to the new task
1: for i 1 to M do
2:
C ~ M( E)
3:
for j to N do
4:
"1:K ~ Unif(()
(6-3)
5: C1:K arg min ( minnetule U(IK.
(eq. 4)
6:
[ā st, () t (ā] St. Ok.)
7:
return TPALO-
Theorem 3.7. The -itf-fofdistrbbttion) regret of PALO on Ptarget can be bounded as:
Rxy (TeALo: Purgei) S Rry (ft; Pprioo) + +E +E[Drv Drv (Pranea (Cggel(C,,,) (Ck,), Ppriue (Ck,)) ))]
+ (2DKL [p(cxx)))) 1/2 + VM- (os-vnil
log(d)) ldn
+ 1/M + 1/K 2/2
(8)
where TPALO is from Algorithm 1, /t ( () trained on Dprie (Section 3.5), and Uni
The proof is in Appendix A. Theorem 3.7 shows that in the limit as N, .M - oo, we can decompose the out-of-distribution regret of PALO into sum of the in-distribution regret of the pre-trained policy, and the performance of the VLM accurately decomposing language tasks:
Ra (TPALO) Purges & Bey (ft; Pprice)
E2 Dxa. (w.pppeee)))leee ([x))x))))M())xx)]
+E(Drv
Drv (Pragen (Ok, ) Ppror (ox.))
training MSE
VLM aocuracy
Jocal marginal conformity
Viewing the VLM accuracy and local marginal conformity terms as the extent to which Assump- tions and 2 are satisfied, we can see that under these conditions, Theorem 3.7 lets us directly relate the performance of the pre-trained policy î on the training data Pprice to the performance of the PALO algorithm on out-o--distribution tasks.
3.8 System Details
We use ResNet-34 [62] to model the policy T(a s, s, where c (c",c" is concatenation of high- and low-level instructions. The instruction c (c+, c") is passed through frozen MUSE model [63] to obtain embeddings before being fused into the ResNet with FiLM layers [64]. Archi- tecture details are presented in Appendix C, and the overall algorithm is shown in Algorithm 1.
Experiments
In this section, we show that PALO can better adapt to long-horizen and out-of-distribution tasks from few expert demonstrations than existing learned language-conen conditioned manipulation poli- cies (both zero-shot and when finetuned to demonstrations), as well nonparametric few-shot adaptation method. Ablation studies also show all components of PALO necessary.
4.1 Experimental Setup
We evaluate on variety of long-horizon and/or unseen tasks across four scenes from the Bridge tabletop manipulation setup [5]. These involve manipulating new combinations of objects and be- haviors unseen in the training data to accomplish long-horizon tasks, such as making a salad or pouring into a bowl. For each task, we collect set of five expert demonstrations Diaget for few-shot learning. Besides separating by scenes, we can also separate the tasks into long-horizon tasks (put in, salad, sweep mints, sweep skittles) and unseen-skills tasks (pry away, pour spoon, rotate marker, rotate spoon). Experimental details and example rollouts are presented in Appendix B.
4.2 Baselines
We compare against the following baselines trained on BridgeData:
Octo [11]: A general transformer-based robot manipulation policy with diffusion action head.

---

Comparison of Methods
Sucoess Rate
0.8
0.6
0.4
0.2
Drawer
Bowl
S Sweep
Rotation
PALO
FT-Octo
FT-LCBC
RT-2-X
Octo
GRIF
VINN
LCBC
Figure 4: Comparison of PALO with baseline methods difterent scenes with standard
Figure execution our method task "pour the contents the scoop into bowl." Full breakdown of task and instructions can be seen at Appendix G
GRIF [36]: language-conditioned robot control method that uses pre-trained CLIP [45] repre-
sentations to connect language instructions to goals for the policy to reach.
RT-2-X [32]] A language-condittoned robot control model with 55B parameters that transfers knowledge from internet-scale pre-training to manipulation zero-shot.
LCBC [61]: Language imitation with ResNet and pretrained MUSE [63] embeddings.
VINN [65]] Using k-Nearest Neighbor to select actions from the training data based on similarity
between the task representations of the observation and training data. We used GRIF's CLIP encoder for the representations used calculate similarity scores.
FT-Octo: Octo transformer finetuned on the few-shot demonstration (see Appendix C.2 for details).
FT-LCBC: Similar to FT-Octo, but fine-tuning LCBC on the few-shot demonstrations.
Across eight different tasks, our PALO method yielded success rate of 71.3%, while the best zero-shot policies only resulted in a success rate of 26.3%. While most of the zero-shot methods degrade when the task became increasingly more out-of-distribution for the pretrained policy (for example, tasks in the "salad" scene achieved a 30% overall performance across the baseline models while pouring from scoop only achieved 12% performance across the models), our method remained effective, with all tasks performing at success rate 50% better.
The FT-Octo and FT-LCBC baselines allow us to compare the nonparametric adaptation of PALO to conventional parametric finetuning. While Octo trained only on BridgeData achieved moderate zero-shot success, finetuning on only five demonstrations overfit and worsened perfor- mance. The FT-LCBC baseline did benefit from finetuning, but still failed to exceed 30% success rate across all tasks. We observe that the small size of trajectories made these datasets an unfavorable candidate for finetuning, as any variance brought by the human controller may be amplified and cause unfavorable movements during evaluation. The nonparametrie VINN base- line performed well on the rotation tasks (45% success rate), but failed to achieve greater than 5% success rate on the other tasks.
4.3 Ablations
We ablate the following components of our method in Fig. 6:
Ours: Our full PALO approach
No H No high-level eи conditioning for the learned policy via masking.
No ch, No low-level ez instruction conditioning via masking.
7

---

Ablation Study
Success Rate
0.8
0.6
0.4
0.2
Drawer
Bowl
Sweep
Rotation
PALO No cff
2 No ch
Fixed Times
Zero-shot
No VLM
Figure Ablation study of PALO different scenes, plotted with standard
Fixed Times: Use fixed u = - 2 ..... (k-1)) in each trajectory to evaluate Eq. (3),
Zero-Shot Decomposition: Generate zero-shot without expert demonstrations.
No VLM: No VLM decomposition proposals by using only ć with our policy.
While the sweeping and rotation scenes gave comparable performance with masked high level instructions (No c the performance de- teriorated in Drawer and Bowl, which involved more unfamiliar items for the pretrained pol- icy. The remaining ablations (No c², Fixed Times, Zero-Shot Decomposition, No VLM) decreased performance across all scenes. These approaches are discussed in more depth in Ap- pendix E. Overall results in Fig. 6 show that all components of the PALO method are needed.
Scaling of PALO and Finetuning Approaches
0.8
Success Rate
0.6
0.2
20
40
60
80
100
120
Number of Demonstrations
4.4 Scaling with Demonstrations
PALO (Ours)
FT-Octo FT-LCBC
We study the scaling of our nonparametric method and a parametric finetuning approach with > 5 demonstrations of the skittle sweep- ing task in Fig. We observe that while Policy Adaptation via Language Optimization achieves the best performance (80%) using any number demonstrations, the Octo finetuning baseline needs at least 80 expert demonstrations to achieve comparable performance, while LCBC needs least 120 demonstrations.
Performance of PALO with demonstra- tions compared to finetuning Octo different number of demonstrations, plotted with standard error.
Figure 7:
4.5 Qualitative Results
We show successful task executions Figs. 3 and 5. While the full method robust to logically unsound instructions generated by the VLM, failures in reasoning and execution occur when we ab- late our methods. Fig. 12 and Fig. 13 are two examples in which reasoning break down in ablations. See Appendix G for details.
Discussion
We introduced PALO., an approach for few-shot adaptation unseen tasks that exploits the semantic understanding of task decomposition provided by vision-language models. In extensive real world experiments, find that PALO is able to language to adapt to unseen long-horizon robot manipulation tasks across wide range of tabletop setups.
Limitations and Future Work. We assume the dataset has a consistent format of high-level lan- guage labels and proprioception, making it more challenging to generalize our low-level heuristic generation drastically different embodiments. The discrete optimization over subtask time steps may also scale poorly with the number of subtasks and time steps. Future work could explore more efficient optimization methods for this problem.

---

Acknowledgements
This research was partly supported by AFOSR FA9550-22-1-0277 -0023, ARO W911NF-21-1-009,, NSF IIS IIS-224611,, DARPA ANSR, and the DoD through the NDSEG Fellowship Program.
References
[1] A. Xie, I.. Lee, T. Xiao, and C. Finn. Decomposing the Generalization Gap in Imitation Learn- ing for Visual Robotic Manipulation. In International Conference on Robotics and Automation. arXiv, 2024.
[2] C. Lynch and P. Sermanet. Language conditioned imitation learning over unstructured data. In Robotics: Science and Systems XVII. Robotics: Science and Systems Foundation, July 2021.
ISBN 978-0-999334477788
[3] O. Mees, L.. Hermann, and W. Burgard. What matters in language conditioned robotic imitation learning over unstructured data. IEEE Robotics and Automation Letters (RA-L), 1(45: 1205-
11212. 2022.
[4] O. Mees and W. Burgard. Composing buck-and-place tasks by grounding language. In Inter- national Symposium on Experimental Robotics (ISER), 2021.
[5] H. R. Walke, K. Black, T. Zhao, Q. Vuong, C. Zheng, Hansen-Estruch, A. W. He, V. My- ers, J. Kim, M. Du, Lee, K. Fang, C. Finn, and S. Levine. BridgeData V2: Dataset for Robot Learning at Scale. In Conference on Robot Learning, pages 1723-1736. PMLR, Aug. 2023.
[6] B. M. Lake, T. Linzen, and M. Baroni. Human few-shot learning of compositional instructions. In CogSci. arXiv, 2019.
[7] K. Ellis. Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language. In International Conference Neural Information Processing Systems, 2023.
[8] Lin, Y. Du, O. Watkins, D. Hafner, Abbeel, D. Klein, and Dragan. Learning to model the world with language. In Forty-First International Conference Machine Learning, June 2024.
[9] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, al. Language Models are Few-Shot Learners, 2020.
[10] A. Zeng, M. Attarian, B. Ichter, K. Choromanski, A. Wong, S. Welker, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, et al. Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language, arXiv:2204.0055
May 2022.
[11] Octo Model Team, D. Ghosh. H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y. Tan, L. Y. Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy. In Proceedings of
Robotics: Science and Systems, 2024.
[12] W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang. A Closer Look at Few-shot Classification. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. ICLR, 2019.
[13] O. Vinyals, C. Blundell, T. Lillicrap, koray kavukcuoglu, and D. Wierstra. Matching Networks for One Shot Learning. In Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016.
[14] Y. Song, T. Wang, P. Cai, S. K. Mondal, and J. P. Sahoo. A Comprehensive Survey Few- shot Learning: Evolution, Applications, Challenges, and Opportunities. ACM Comput, Surv,
55(113):271:1-271: 2023.
[15] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Hasson, K. Lene, A. Mensch, Milli- can, M. Reynolds, R. Ring, E. Rutherford, Cabi, T. Han, Z. Gong, et al. Flamingo: A Visual Language Model for Few-Shot Learning. In Conference on Neural Information Processing Systems, 2022.

---

[16] A. Ghadirzadeh, X. Chen, P. Poklukar, C. Finn, M. Björkman, and D. Kragic. Bayesian Meta- Learning for Few-Shot Policy Adaptation Across Robotic Platforms. In IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, IROS 2021, Prague, Czech Republic, September 27- Oct. I, 2021, pages 1274-1280. IEEE, 2021.
[17] Y. Guo, R. Du, Y. Dong, T. Hospedales, Y.-Z. Song, and Z. Ma. Task-aware Adaptive Learning for Cross-domain Few-shot Learning. In IEEE/CVF International Conference on Computer Vision, pages 1590-1599, 2023.
[18] C. Finn, Abbeel, and Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In International Conference on Machine Learning, pages 1126-1135. PMLR, July 2017.
[19] A. Nichol, J. Achiam, and Schulman. On First-Order Meta-Leariing Algorithms, 1rXiv:1803.00 803.02999, 2018.
[20] D. Chen, L. Wu, S. Tang. X. Yun, Long, and Y. Zhuang. Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile. In International Conference on Machine Learning, pages 3662-3678. PMLR, June 2022.
[21] Snell, K. Swersky, and R. Zemel. Prototypical Networks for Few-shot Learning.. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[22] M. Sendera, Tabor, A. Nowak, A. Bedychaj, M. Patacchiola, T. Trzcinski, P. aw Spurek, and M. Zieba. Non-Gaussian Gaussian Processes for Few-Shot Regression. In Advances in Neural Information Processing Systems, volume 34, pages 10285-10298. Curran Associates, Inc., 2021.
[23] P. Tighineanu, L. Grossberger, P. Baireuther, K. Skubch, S. Falkner, J. Vinogradska, and F. Berkenkamp. Scalable Meta-Learning with Gaussian Processes. In International Conference on Artificial Intelligence and Statisties, pages 1981-1989. PMLR, Apr. 2024.
[24] Z. Wang. Z. Miao, X. Zhen, and Q. Qiu. Learning to Learn Dense Gaussian Processes for Few-Shot Learning. In Advances in Neural Information Processing Systems, volume 34, pages 13241. Curran Associates, Inc., 2021.
[25] S. M. Xie, A. Raghunathan, P. Liang, and T. Ma. An Explanation of In-context Learning as Implicit Bayesian Inference. In International Conference on Learning Representations. arXiv, 2022.
[26] Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, L. Li, and Z. Sui. A Survey for In-context Learning. CoRR, abs/2301 .00334, 2023.
[27] J. Y. Zhu, C. G. Cano, D. V. Bermudez, and M. Drozdzal. InCoRo: In-Context Learning for Robotics Control with Feedback Loops, arXiv:2402.05I Feb. 2024.
[28] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. RT-1: Robotics Transformer for Real-World Control Scale, arXiv:2212.068117. Aug. 2023.
[29] T. Winograd. Procedures as a representation for data in a computer program for understanding natural language, 1971.
[30] M. Skubic, D. Perzanowski, S. Blisard, A. C. Schultz, W. Adams, M. D. Bugajska, and D. P. Brock. Spatial language for human-robot dialogs. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 34:154-167, 2004.
[31] Tellex, T. Kollar, S. Dickerson, M. Walter, A. Banerjee, S. Teller, and Roy. Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation. Proceedings of the AAAI Conference on Artificial Intelligence, 25(1):1507-1514 Aug. 2011. ISSN 2374- 3468, 2159-5399.
[32] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, et al. RT-2: Vision- LaiionLanguage-AAoe Models Transfer Web Knowledge to Robotic Control. In Conference
10

---

on Robot Learning. arXiv, 2023.
[33] M. Shridhar, L. Manuelli, and D. Fox. CLIPort: What and Where Pathways for Robotic Manipulation, UrXiv:2109.12099 Sept. 2021.
[34] C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence. Interactive Language: Talking to Robots in Real Time. IEEE Robotics and Automation Letters, pages 1-8, 2023. ISSN 2377-3766.
[35] O. Mees, L. Hermann, F. Rosete-Beas, and W. Burgard. CALVIN: A benchmark for language- conditioned policy learning for long-horizon robot manipulation tasks. IEEE Robotics and Automation Letters (RA-L), 7(3):7327-7333 7334, 2022.
[36] V. Myers, A. W. He, K. Fang, H. R. Walke, P. Hansen-Estrucch C.-A. Cheng, M. Jalobeanu, A. Kolobov, A. Dragan, and S. Levine. Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control. In Conference on Robot Learning, pages 3894-3908. PMLR, Dec. 2023.
[37] K. Black, M. Nakamoto, P. Atreya, H. Walke, C. Finn, A. Kumar, and Levine. Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models, arXiv:2310.10639, Oct. 2023.
[38] Liu, Fang, Abbeel, and S. Levine. MOKA: Ogeen-ocabullary Robotic Manipulation through Mark-Based Visual Prompting. In Robotics: Science and Systems. arXiv, Mar. 2024.
[39] W. Huang, C. Wang, R. Zhang, Li, J. Wu, and L. Fei-Fei. VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models, arXiv:2307.0599 Nov. 2023,
[40] M. Ahn, Brohan, N. Brown, Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishnan, K. Hausman, Herzog, D. Ho, J. Hsu, J. Ibarz, B. Ichter, al. Do As Can, Not As I Say: Grounding Language in Robotic Affordances. In Conference on Robot Learning. arXiv, 2022.
[41] M. Attarian, A. Gupta, Z. Zhou, W. Yu, I. Gilitschenski, and A. Garg. See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction, arXiv:2210.03825 Oct. 2022.
[42] S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Tompson, Y. Chebotar, D. Dwibedi, and D. Sadigh. RT-H: Action Hierarchies Using Language, arXiv:2403.01823, Mar. 2024.
[43] I.. X. Shi, 7. Hu, T. 7. Zhao, A. Sharma, K. Pertsch, Luo, S. Levine, and C. Finn. Yell At Your Robot: Improving On-the-Fly from Language Corrections. CoRR, abs/2403.12910, 2024.
[44] O. Mees, J. Borja-Diaz, and W. Burgard. Grounding language with visual affordances over unstructured data. In IEEE International Conference on Robotics and Automation (ICRA), 2023.
[45] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. In International Conference on Machine Learning. arXiv, Feb. 2021.
[46] OpenAl. GPT-4 Technical Report, arXiv:2303.0877 Mar. 2023.
[47] K. Grauman, A. Westbury, F. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu, M. Martin, T. Nagarajan, I. Radosavovic, S. K. Ramakrishnan, F. Ryan, et al. Ego4D: Around the World in 3,000 Hours of Egocentric Video. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). arXiv, Mar. 2022.
[48] R. Bommasani, D. A. Hudson, F. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bemstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolisson, S. Buch, D. Card, R. Castellon, N. Chat- terji, et al. On the Opportunities and Risks of Foundation Models, arXiv:2108.07258, July 2022.
[49] Y. Jiang, A. Gupta, Z. Zhang, G. Wang, Y. Dou, Chen, L. Fei-Fei, A. Anandkumar, Y. Zhu, and L. Fan. VIMA: General Robot Manipulation with Multimodal Prompts, May 2023.
11

---

[50] R. Shah, R. Martín- Marttn, and Y. Zhu. MUTEX: Learning Unified Policies from Multimodal Task Specifications. In Conference on Robot Learning. arXiv, 2023.
[51] A. O'Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, A. Tung, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, et al. Open X- <-Embodiment: Robotic Learning Datasets and RT-X Models. In International Conference on Robotics and Automation. arXiv, May 2024.
[52] R. Doshi, H. Walke, O. Mees, S. Dasari, and S. Levine. Scaling cross-embodied learning: One policy for manipulation, navigation, locomotion and aviation, arXiv:2408. 1812, Aug. 2024.
[53] T. Yu, T. Xiao, J. Tompson, A. Stone, S. Wang, A. Brohan, J. Singh, C. Tan, D. M, Peralta, et al. Scaling Robot Learning with Semantically Imagined Experience. In Robotics: Science and Systems XIX, Daegu, Republic of Korea, July 10-14, 2023, 2023.
[54] Q. Chen, S. Kiami, A. Gupta, and V. Kumar. GenAug: Retargeting behaviors to unseen situa- tions via Generative Augmentation. In Robotics: Science and Systems XIX. Robotics: Science and Systems Foundation, July 2023. ISBN 978-099--237-4---992
[55] C. Huang, O. Mees, Zeng, and W. Burgard. Audio visual language maps for robot naviga- tion. In International Symposium on Experimental Robotics (ISER), 2023.
[56] Zhang, K. Pertsch, J. Zhang, and J. J. Lim. SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling, arXiv:2306.1 1886, June 2023.
[57] W. Chen, O. Mees, A. Kumar, and S. Levine. Vision-language models provide promptable representations for reinforcement learning. arXiv preprint arXiv:2402.02651, 2024.
[58] D. Shah, B. Osinski, Ichter, and S. Levine. LM-Nav: Robotic Navigation with Large Pre- Trained Models of Language, Vision, and Action. In Conference on Robot Learning. arXiv, 2022.
[59] C. Huang, O. Mees, Zeng, and W. Burgard. Visual language maps for robot navigation. In IEEE International Conference on Robotics and Automation (ICRA), 2023.
[60] M. Kwon, H. Hu, V. Myers, Karameheti, A. Dragan, and D. Sadigh. Toward Grounded Commonsense Reasoning. In International Conference on Robotics and Automation (ICRA). arXiv, June 2023.
[61] S. Stepputtis, J. Camphell, M. Phielipp, S. Lee, C. Baral, and H. B. Amor. Language- conditioned imitation learning for robot manipulation tasks. Advances in Neural Information Processing Systems, 111133399135500 2020.
[62] K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, June 2016.
[63] Y. Yang, D. Cer, A. Ahmad, M. Guo, J. Law, N. Constant, G. H. Abrego, S. Yuan, C. Tar, Y.- H. Sung, B. Strope, and R. Kurzweil. Multilingual Universal Sentence Encoder for Semantic Retrieval. In ACI.. Xrviv, 2020.
[64] E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville. FILM: Visual Reasoning with a General Conditioning Layer. In AAAI Conference on Artificial Intelligence. arXiv, 2018.
[65] J. Pari, N. M. M. Shafiullah, S. P. Arunachalam, and L. Pinto. The Surprising Effectiveness of Representation Learning for Visual Imitation. In Robotics: Science and Systems XVIII. Robotics: Science and Systems Foundation, June 2022. ISBN 978-0-9923747-885.
[66] O. Catoni. A PAC-Bayesian approach to adaptive classification, 2004.
[67] P. Alquier. User-friendly Introduction to PAC-Bayes Bounds. Foundations and Trends@ in Machine Learning, 17(2):174-303, 2024. ISSN 1935-8237, 1935-8245.
[68] L. G. Valiant. A theory of the learnable. Commun. ACM, (7(11): 1111 142, Nov. 1984. ISSN 0001-0782, 1557-7317
12

---

[69] I. Csiszár and J. Körnrr. Information Theory: Coding Theorems for Discrete Memoryless Systems. Cambridge University Press, June 2011. ISBN 978-1-139-49998-9
[70] W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, Mar. 1963. ISSN 0162-1459.
[71] D. P. Kingma. Adam: A method for stochastic optimization. In International Conference on Learning Reprexentations, 2014.
A Proof of Theorem 3.7
Theorem 3.7. The -uutoo-disttributio)) regret of PALO on Prorge can be bounded as:
Rxg (TeALoi Prarget R (i; Pprior) 18| DTv (Prarga ), Pprior (ck:))
+ (2DKL [p(CLL)( 1/2 VM Volog(d + 1/M + 1 N (8)
where TALL is from Algorithm ( E) trained on Drioe (Section 3.5), and Unif(1 H).
Proof. We will first consider the empirical regret of the MLE estimate of CL:K, and relate it to in- distribution regret of स using PAC techniques (see Catoni [66], Alquier [67]). We will then bound the remaining error due to the approximations made by the PALO algorithm and this empirical regret.
Recall our definition of regret:
RA (T: = ET,
Σ (ss
Ts( 011
(from eq. 7)
HvdA
We can also define empirical target regret REMP measuring the fit of some E L to the target distribution Puaget in terms of Eq. (4):
REMe (CI:K) D ~ Pruset
Σ
min [CI.K, U.K. Menk, u1K
(10)
H HVdA
TEDuga
where J is the cost function in Eq. (4). PALO selects CPALO = arg min ECC ECK (CMo(CI:))))) to minimize an approximation of this quantity for samples u())
(( ~ Unif(l)):
REMe(CI:K) = E Duager Parges
1
Σ
min J(Ik ul (7) (E(1...N)
(11)
HVVA
TED
We will also define a distributional notion conditional regret for our analysis:
Rip (# su. E, C1.K) = EE T H H
min J(c:k, U2:K,
(12)
#IKEU
We now make use of the following PAC result [68, 67], which follows from Hoeffding's inequality:
Lemma A.I (Alquier [67, Theorem 1.2)). Let H be a class of functions X - [0. with [71| = M, and let P(X) be an arbitrary data distribution. Further, suppose D is a sample of size n drawn ii.dd from Then, for any E (0, 1), we have
Pr Vf E H. [x=p[((x) S (EnD(f(<)) +
log M log
2 1--
(13)
generulization risk
emptrical risk
Taking X to be the space of trajectories and H. = M(su. E) for f(c) = min, J(c, u, T). we can apply Lemma A. to the empirical regret REMP in Eq. (10) to obtain (for any e E (0, 1))
Pr VOL.K E EMsso. f), Rry (# s0, e, c1:K) & REME (CI:K) +
log M -log
2 1 - (14)
13

---

Taking CPALO to be the output of the PALO algorithm, we can relate the true regret of PALO on the current task (left) to its empirical regret (right):
Pr Rxx (ft so, é, CeALo) S REMP (CPALO) +
log log
2 21-E.
(15)
2n
Since regret is bounded by 1, we can convert to an expectation:
Duwa [Ra (it so, so,k, cPALO)] S EDana [REMP (CeALo) +
log M-loge
+
2n
Lemma A.2. Suppose u, u' ~ Unif((f) are i.i.d. samples from a uniform distribution over the ordered K K-partitions U of {1...H. H}. For any E E [0,1/K], we have
Pr K Iumilux 7ulk u S He)) S e 2H 2-
Lemma A.3. There exists an [0,1/K] such that
s E e 2 -))² S 1/K + 2/K
(16)
Since Algorithm 1 (line 4) only samples N values for instead of the full space for the min in Eq. (10), we must separately consider the degree of suboptimality in the decomposition OPaLo relative to the optimal c* = arg min M(so.t) REMP (c) that results from our approach to determine the effect of N on the final bound. Applying Lemma A.2, we can say:
EDange [Ra (f so. CPALO)]
Dunge RRMe (CEsLO)] + log M llog
2n
S Dunga REMe (e*)] + V
log &
+ C 22 -E)
2n
- Dargn REMP (c*)] +
loy logg
+ +1/K+ + -
2n
For VM/n we get
ED
[Rag (+t so, l, cpALO)]
M n (M) + 1/K + 2/K
S E REMr (c*)]
(17)
n
So, we have related the true regret of PALO on the current task (left) to its empirical regret in the limit of infinite samples (right). All that remains is compute the empirical regret, for which make use of the following lemmas.
Lemma A.4. Denote the true (unobserved) target decomposition as Ct:K- We can relate the empir- ical regret of the optimal PALO solution c* to the empirical regret of the true decomposition. EDang
REMM (c") S EDwyy [Rea (CI:K) + Drv PM((ex)) + 1/M
Lemma A.5. The empirical regret of ã can be bounded for i ~ Unif(1.... ..) as
EDunn [RMe (C1:K)] S. Rog (fr; Ppeior) + [E[ Drv (Punger ( (Ck, ) Pprior (Ck.)
Applying Lemma A.4 and Lemma A.5 Eq. (17) yields bound of the correct form.
E RESA (c*)] I E Eunn [REsE (c1:K]] Drv (p(CI))) PM) 1/M
I EED RRi (ft; Pprior]] + E Drrv (Punged (Ck.)) Pprior (Ok, ))] + Drv (p(c):)))M)) + 1/M.
To make the DTv (p((i))))) PPM))) term more interpretable as a VLM accuracy, we convert to KL divergence with Pinsker's incquality [69]:
ED REMP (CrALO) & D, RA, (ft; Pprioe)
DTv (Prager (Ck, ) Pprior ( (Ck,))
(18)
/ 22 D2LK(p((L.. (p(CI.K);
1/M.
(19)
14

---

Since ED [Rx (f su. l, CeALo)] desired result:
= Riy (TeALO: Prarga) plugging Eq. (19) into Eq. (17) gives the
Rap (TAALO' Prarget)
& Rxx (#t; Pprior)]
EE Drv (Ckr) Porior (okk))(ckk))) + V 2D (p(cLx))PM) + 1/M
VM + (oggg(MM))
+ +1/K /
(20)
n
Proof of Lemma A.2. Define {x:) be the unique index k such that i wk, and {X; to be the unique index such that E u/z. We have
Pr C 12=1luu lu n u/l 2 He)
Pr
H Σ i= 1(x; x; 2 Hs)
H Pr E - 1(x; + x s H(1 -s)
(21)
Now, we observe
K
Pr(X, * x()
Σ (1 - px, (k))
(22)
k=1
(ki)²
(23)
k=1
Eq. (22) is concave in px., and so is maximized when for any opx, and some A,
Xopx, (k) - Σ px. (k)6px (k),
ie., when px. (A) const. = 1/K for all k. Thus, we have
13 [1 * x = Pr(X, + X)) < 1 - 1/K.
Continuing from Eq. (21) with = E [ 11x; + X
P x < H1 - -) = - Pr
L(X, X,}
(11 (1 -) -)) -ni)
2((H(1 - 11)²
exp
(Hoeffding [70))
H
-21 - (1- (1-)/)))
exp
H1
=1 - exeepp exp(-2H(() (1/K -1)K
(24)
Taking the complement of Eq. (24) yields the desired result:
Pr lu lu n u s H&) I e 2H 2 -6)²
(25)
Proof of Lemma A.3. The statement follows from the ansatz
log N NHK
E = K
Plugging in,
log N 1/2
+ 2H -8) =
2/K 2/K + K
HKN
$ 1/K + N 2/K
15

---

Proof of Lemma A.4. Recall the definition of the optimal PALO solution
arg min REMP(C). cEM(s).e)
(26)
Now, noting regrets bounded by 1 from Eq. (7), we have
Danpe RRER (c*))
Damn
min (oEM(s..e)
Ramr(e)
p(c) ppn(c)) )
= -aag
min (o
REMP(Cl(
min (ell 3it PPLLK
[EEMP (c(o0)
(PM(c) -1)
p(c)
min { Mir p(c)
REMP (cl
- EDar
min {a(0) MM Pelik
REMP (c(+)) + D.Drg
PM(c)
min Hel MP
REMP (c( + Drv +Di((olcux))px((cxx))
= Duuue
Pr Pr(Reur < (6) for 1c00 ~ P(IR)
+ ReM((CI:K) + Drv (p
Duna REMP(CI:K) + Drv PM(lx))))) + 1/M.
Proof of Lemma A.S. We consider the empirical regret using the true decomposition Ur:K, C1:K ~ Purret. for & Unif(1... ... H) and k, defined as in Eq. (5):
ED [Rex((11x)] =
E. H HVdA rEDam
Σ
min I [CI.K u1.k,7) 7)
K
EDugg
Σ
min
Σ
110e (n))
H Vda reDuast
anK
n= tEu.
K
< Eur Cn ~ Dimas
Σ TE TEDans n= tEun
Jae F(s,, cm)l
H HddA
< H Hdd
E Dapee
Σ
EElee
la #(s,, cn)l1 + Drv (Prarger (cn) Pprior Ppr)o (cm))
n= tEun
K
Eng Preae
Σ Ilae (ser c1) + EE E[Drrv (Pranger (Ck,) Pprior (Ok, ))]
HVdA n= rEun
Riz (it; Pprsor) + E Drv (Prargel (Cke) Pprior (Ck.)) ))
B
Environment Details
We evaluate our method in real-world tabletop manipulation setup. We use 6DOF WidowX-255 robot interacting with various objects both inside and outside of our training distribution at 5 Hz. We one 480 RGB camera mounted on top of the model set up in BridgeData [5]. When computing observations we downsample images to 224 224.
16

---

180
180
Instruction
Instruction
beg/prnpee
Swerp Me whiee to rgw
"ecgy deriis
Bae Luile"
weskmoes in
Trake anttdd bond
your
Figure 8: Sample rollouts using PALO on testing tasks.
We evaluate our method in the following scenes, which include:
Sweep:
This scene involves an object manipulation as well as sweeping task unseen the Bridge- Data's initial training trajectories.
mint: Placing the mushroom in the pot, then sweep the mints the right using the towel.
skittles: Instead using mints and towel for sweeping, we use swiffer and skittles instead.
Drawer:
This scene involves using a drawer and perform manipulation within the space of the drawer.
put in: Open the drawer, and then put a purple object (beet/sweet potato) inside the drawer.
pry away: A pot is stored inside the drawer space, and the robot must use ladle to pry away the pot within drawer.
Bowl: This scene involves object manipulation to a bowl and perform long-horizon or 6DOF ma- nipulation.
salad: This task requires sequential object manipulation by putting a corn cob and a mushroom in the bowl.
pouring: This task requires the robot grasp scoop and pour almonds inside the scoop into the bowl.
Rotation:
This scene involves rotating a spoon and a marker fit into a white container not aligned with the pen/marker, and naive pick-and-place will not correctly align the object into the con- tainer.
spoon: Placing the spoon in the container placed on the left side of the table. Replacing the spoon with the marker and randomize location of the container while being misaligned.
marker:
We summarize the evaluation tasks in Table 1, and show example rollouts in Fig. 8.
C Training Details
We train on an augmented version of the BridgeDataV2 V2 dataset [5]. which features over 50k tra- jectories with 72k language annotations. We algorithmically augment the dataset with low-level instructions using heuristics designed over the proprioceptive states of the robot and incorporate language context by parsing the language instruction using language model. We the Adam optimizer [71] to minimize the loss function in Eq. (27).
Instead of naively looping through Algorithm 1, we batch our implementation with the exception of the outermost for loop, thus reducing time consumption during optimization by a significant margin via vectorization. We record an empirical time consumption of 470 seconds for our language optimization module on computations ran on a V4 TPU module, in which only 200 seconds are required for sampling 20000 different partitions complete the optimization for all of the 15 sets of language instructions. We our optimal plans for future use, thus reducing overhead even more.
We encode both language instructions using frozen MUSE model [63] before passing them into the main ResNet with FiLM layers [64].
17

---

Table I: Task Breakdown
Scene
Task
Long-Horizon?
6DOF required??
Instruction
"put the beet toy/purple thing into the drawer."
put in
Yes
Yes
Drawer
"pry out the pot in the drawer using the ladle."
pry away
Yes
Yes
"make salad bowl with com and mushroom."
salad
Yes
No
Bowl
"pour the contents of the scoop into the bowl."
pour scoop
No
Yes
"sweep the mints to the right after putting the mushroom the bowl."
mints
Yes
No
Sweep
"sweep the skittles into the bin after putting the mushroom in the con- tainer."
skittles
Yes
No
"put the marker into the box while aligning it."
marker
No
Yes
Rotation
"put the spoon into the cleaner while aligning it."
spoon
No
Yes
C.1 Hyperparameter Selection
We discuss the hyperparameters used in our method and baselines.
Policy Training We set our learning rate for our Adam Optimizer [71] to 3-10" and dropout rate of 0.1 in our policy head. We employ random resizing and cropping, contrast, brightness, saturation, and hue for input images. We train our policy for 300,000 steps, in which we use the checkpoint with the lowest validation MSE. The total training time takes 12 hours when trained on 4 TPU pods.
Language Decomposition Optimization During optimization, we sample M 15 random in- struction sets from GPT4-0, and we use N 20, 000 sampling steps in order find the best subtask decomposition.
In order to batch across demonstrations, which have different trajectory lengths, we pad our trajecto- ries to a certain length H (200 for long-horizon tasks, 150 for non long-horizon tasks). We sum the squared difference between generated action and oracle action in evaluation, thus giving a consistent error metric analogous to Eq. (7).
C.2 Baseline Details
We finetune an Octo-small [11] model that is trained on BridgeData with a learning rate of 3 10-4 and finetune our model's action head for 5000 steps. We use the hyperparameters set by Octo for the rest of the settings.
In order to perform tasks in long-horizon we assign language label for each task in order to trans- plant semantic understanding from human into Octo. The same language instruction for PALO eval- uation is also used for Octo finetuning.
D Augmentation Details
We train the policy by maximizing the likelihood of actions given high- and low-level instructions in the dataset Prrior:
J (0) Ep [1la (sc, (c", (c",0))lI² ||a, xe (0,c²))112 ||a. no (s (c" 1) (27)
18

---

where $1...8H S, o1. E A, A', E (0), and & are the parameters of the policy network, is an additional point representing the absence of a high- or low-level instruction, which will be represented as an embedding vector of zero during training, and = (5o, Cu...., SH 0. is a trajectory sampled from the dataset.
This objective encourages the policy to learn to follow instructions at both levels of abstraction, marginalizing over missing instructions. We chunk actions within training data into segments of length and evaluate the low level instruction within these segments and append them into the training data.
D.1 Heuristics for Low-Level Language Augmentation
We algorithmically enhance training data by using heuristics generated by the proprioception of the robot and language context, which generates the low-level instructions. The labeled language instruction is passed into language model to obtain manipulation keywords, and we combine the keywords with the proprioceptive information within that time span including translation, rotation, and gripper movement into coherent language commands.
Proprioception. We standard deviation of each action against the metadata of BridgeData [5] and determine how to describe the proprioception of the label. We determine the largest direction in which the gripper is moving (up, down, left, right, forward, backward) and the orientation it is rotating (up, down, left, right, clockwise, counterclockwise), and determine whether the movement unambiguous enough by checking the largest z-score in translation and rotation. We then combine the movement as well as the keywords extracted form language primitive commands.
Target Object. We identify the target object using prompt heuristic to be fed into GPT3.5-Turbe [9] by taking ad advantage of the fact that BridgeData consists of mainly object manipulation data. We extract two keywords: the object to be manipulated and the destination of the object, based on the fact that much of BridgeData focused on object manipulation. The precise prompt can found Appendix F
Data Filtering. We filter low-level instruction on two occasions: when the movement itself is am- biguous and when the language model gives inconsistent results. We check the former by looking up the norm of the translation and the norm rotation, and we check the latter by using regular ex- pression to see if the result was against the desired format and manually filtering out some common keywords of inadmissible GPT query. On the former occasion, we use an empty string as the low level instruction, and on the second occasion, we use only proprioceptive information for low-level instruction.
D.2 Additional High-Level Language Augmentation
We additionally augment the high-level language annotations by generating context-free rephrasings with GPT-3.5 [9]. For each trajectory with crowdsourced language annotations in the BridgeData v2 dataset, we generate such augmented language strings following the approach of Myers et al. [36].
E Ablation Details
We ablate our experiment in progressive manners, going from full implementation to using only the barebone hierarchical policy network.
PALO w/o high level instruction: while running PALO, we derive both high and low level in- struction sets. However, during inference robot, we mask out the high level instruction and feed in zero embeddings.
PALO w/o low level instruction: mask out the low level instruction and replace them with zero embeddings during inference.
19

---

Fixed Time During Optimization: for each trajectory that has corresponding length
H1, H H, choose fixed I 2H. (k-1) during optimization. We im-
plement sampling, which reduce PALO into arg max operation.
Zero-Shot Plan Generation: instead of sampling 15 plans, sample only one plan from VLM
and examine the behavior of the robot using that specific plan.
No VLM Guidance: We use only E our high level instruction, and mask out low level instruction with zero embeddings during inference.
F Prompting Methods
We employ a keyword decomposition prompt in our augmentation method and a planning prompt to generate VLM outputs. They are listed below:
Keyword Decomposition Prompt
User: "You presented with text for high level instruction for robot, and you need to extract keywords in the task description text.
In this instruction, the first keyword is the object being moved, and the second keyword, if applicable, what is the moving taking this (either another object or location) within the instruction. Only return the first and second keyword, and they should be separated by a comma. If the instruction is in another language, write your response in English.
For example, if the text instruction says "Pick up the silver lid the left to middle two burners", return "silver lid, middle of two burners".
or if the instruction says: "Move the object to the top middle side of the table.", your response should be "object, top middle side of the table".
Or the instruction says "Move the red greenish thing on the towel to the right.", return "red greendish thing on the tovel, the right".
Try your best to find the two key phrases, but if you can't find the second keyword within the instruction sentence, write "N/A". For example, if the instruction is "Move the pot lid.", the response should be "pot lid, N/A".
There might be some other description regarding confidence at the end, you safe ignore it The specific task description for you to analyze is: (instruction)
Planning Prompt
User: Here is an image observed by the robot in a tabletop robot manipulation environment. The gripper situated the top of the center table and perpendicular to it.
Now plan for the list of subtasks and skills the robot needs to perform in order to (inatrs).
Each step in the plan be selected from the available skills below:
* *ooenment direction:
* ffovard. This skill moves the robot gripper away from the camera by small distance.
backward. This skill moves the robot gripper towards the camera by a small distance.
* *left. This skill moves the robot gripper to the left of the inage by small distance.
*right. This skill moves the robot gripper to the right of the image by a small distance.
+up. This skill moves the robot gripper upward until safe height.
20

---

*down. This skill moves the robot gripper downward to the table surface.
#rotation direction:
+left. This skill tilts the gripper to an angle to the left. *right. This skill tilts the gripper
angle to the right. angle facing up.
*down. This skill tilts the gripper sup. This skill tilts the gripper to an angle facing down *clockrise. This skill rotates the gripper and the objcet it is holding clockvise.
*counterclokwiss. This skill rotates the gripper and the object it holding counterclockvise.
*gripper movement:
ccosse the gripper. This skill controls the robot gripper to close grasp object.
*open the gripper. This skill controls the robot gripper to open and release the object in hand.
You may choose between using direction, or gripper movement.
of movement direction. rotation
you were to choose to use movement direction, you may use one or two directions and include a target object, and you should format it like this:
"move the gripper towards
"move the gripper and y towards z" where x and y are the directions and is the target object.
You also must start your command with "move the gripper". Therefore, instead of saying something like "down" or "up", you should phrase it like "move the gripper doun" and "nove the gripper up". Make sure to include at least one direction in your command since otherwise this command format won't make sense.
you were choose to use gripper movement, you should format the command as "close the gripper to pick up x" or "open the gripper to release x", where I is the target object
You may discard the target object if necessary. In that case use "open the gripper".
close the gripper" you think the gripper is close the target object, then you must choose to use gripper movement to grasp the target object to maintain efficiency.
you were choose gripper rotation, you should format the command as "rotate the gripper x", where x is the target rotation direction. You need to make sure that in pouring tasks, the opening of the container is aligned with the pot. For example, if the object is aligned vertically but you want it to align horizontally, then you should call "rotate the gripper counterclockvise, If you want to tilt the object in the gripper to pour it, you should call "rotate the gripper left"
Pay close attention to these factors:
*hiich task are you doing.
*Wethee the gripper is closed.
*Whether the gripper is holding the target object.
*How far the two target objects are. If they are across the table,
then duplicate the commands with a copy of it.
* Where the gripper is. After the end of each subtask, it is reasonable to assume that the gripper will not be at where it originally was the image, but somewhere close the last target object.
21

---

Especially pay attention to the actual direction between the gripper and the target object. Remember that the robot's angle is roughly the same as the camera's angle.
To determine whether the gripper should move forward or backward, look into the edge of the table. If the target object is closer to the edge of the table that is near the top of the image, you ahould edge that is near the botton of the image, you should move backward.
forward, and if it closer to the
At the end of each subtask, you need to use the skill "move the gripper back to neutral. This will the original position of image after completing the task.
the gripper back to
Start by looking at what objects are in the image, and then plan with the direction of the objects in mind. The tasks should be completed sequentially, therefore you need consider the the gripper after each task before planning the next task.
position
You should return json dictionary with the following fields: subtask: this should be the key of the dictionary. It should contain only the verbal description of the subtask the robot needs to perforn sequentially in order to finish the task, and they should be ordered in the same way the task is completed.
list of skills: this should be the value of the dictionary. It should be list of skills the robot needs perform order to finish the corresponding subtask.
Be concise, and do not return any other comments other than the dictionary mentioned above. Do not put "subtask: or "lsit of skills: the key and value the dictionary you generate Remember only the description and list should returned.
G Execution Breakdown
In this section, we provide additional qualitative results for PALO.
G.1 Inference Details
During inference, we chunk each low-level instruction into length 8 intervals, switching to the new of low-level (and high-level, applicable) after these steps. We chose a fixed interval instead of dynamically allocated due the policy choosing to mostly stay put after finishing the action prescribed by the low-level instruction.
G.2 Success Cases
We show the full breakdowns of success cases here. Fig. 9 and Fig. 10 gives detailed description of the robot's action primitives generated by PALO during inference.
G.2.1 Full PALO Failure
While PALO robust generating language primitives that help achieve the task, does not guar- antee a successful execution of the policy as shown Fig. 11. PALO can fail when the underlying policy fails execute low-level motion, after which the robot may not be able recover and complete the task.
G.2.2 Ablation Failures
When we ablate the components of PALO, we begin to see more critical failures. Fig. 14 demon- strates of grounding failure when ey is masked out, i..., when PALO loses half of the opti- mized task decomposition.
22

---

== "pry out the pot using the ladle"
HH 1:K
Pry the using the lodle
Piek up the ladle
Move the ladle to the drawer
C:K'
move the argus dile tooath the lede
mos the groppe dwmaa
dlese the gripperr to pock up the bolk
Mine the giper firosed and tooads Oo diseer
mon the gripper left
ason lae arigger dosca tonamble the pot
move arigges tocknand
open the srijoer
Figure 9: An execution of our method on the task "Pry out the pot using the ladle."
l = "pour the contents of the scoop into the bowl"
H ':K
T:KK
Mtoen sstppe doen cnd righa sceands the soopp
the
Clos soop
de
Mare the guiaper up and lin tonerte the boor
Retste gitiper conteriiiirtee
Ieeate the griper in
nala sang the bost
Open the grlppz to minooe the swop:
Figure 10: An execution of our method on the task "pour the contents of the scoop into the bowl.". Note that the high level instruction is itself, the best-proposed language decomposition does not create additional subtasks.
Table 2: Method Comparisons
Scene
Task
PALO
RT-2-X
FT-Octo
Octo
GRIF
VINN
FT-CCBC
LCBC
put in
0.7
0.0
0.0
0.2
0.1
0.0
0.3
0.1
Drawer
pry away
0.6
0.2
0.2
0.1
0.0
0.1
0.0
0.0
salad
0.7
0.5
0.0
0.3
0.4
0.0
0.6
0.0
Bowl
pour
0.5
0.1
0.2
0.3
0.0
0.0
0.0
0.0
mints
0.7
0.3
0.1
0.2
0.0
0.0
0.2
0.0
Sweep
skittles
0.8
0.4
0.0
0.4
0.3
0.0
0.3
0.2
marker
0.9
0.4
0.0
0.1
0.3
0.4
0.4
0.0
Rotation
spoon
0.8
0.2
0.1
0.1
0.1
0.5
0.2
0.0
Average
0.71
0.26
0.10
0.21
0.15
0.13
0.25
0.08
23

---

t= "put the turnip in the drawer"
maooe the purple thing
placee the parple thing drower
1:K
open the druwer
pick the parple thing
L T.KK
sase
plipe doowe honds
anipaar dower busile
anaped
upec
asne priper
msee sptsper duose
plippr dooee
open the srpper
Figure 11: Failure in execution: while the robot completed every subtask correctly up until the last subtask, did not achieve due to errors within the policy.
Figure 12: Spatial reasoning failure occurred when masking out low level instruction. The task was to "sweep the mints using the towel." Due to the presence of the pot and the mushroom, being both strong priors within BaiageData, the policy chose not follow the high level instruction.
Figure 13: Grounding failure when high level instruction is masked While the low level instruction "move the gripper left" correctly predicts the next reasonable action, masking out the context the subtask "put the mushroom the bowl" causes the policy to overshoot its trajectory.
Table 3: Ablations
Scene
Task
PALO
No H
No cl
Fixed Times
Zero-shot
No VLM
put in
0.7
0.2
0.4
0.4
0.3
0.0
Drawer
pry open
0.6
0.4
0.2
0.1
0.4
0.1
salad
0.7
0.4
0.5
0.4
0.2
0.0
Bowl
pour scoop
0.5
0.1
0.4
0.2
0.0
mints
0.7
0.5
0.3
0.5
0.0
0.0
Sweep
skittles
0.8
0.7
0.2
0.5
0.4
0.2
marker
0.9
0.6
0.3
0.3
0.1
0.3
Rotation
spoon
0.8
0.6
0.1
0.2
0.3
0.2
24

---

t= "make a salad with corn and mushroom"
H 1:
[MASKED]
LL
Move the pripper nghe and down incurde the auslmus
Cluen the siippr the unshromm
Moxr the gripper
Move the gripper ferwadd and left
1:K
St
Figure 14: In this instance, we mask out high level instructions, the policy is only conditioned low-level instructions. We see that the low-level instruction "move the gripper forward and left." causes the robot to overshoot its trajectory and causes failure in execution.
H Evaluation Results
We present detailed results of our method across four tasks in the studied scenes in Table 2. We also present ablation results in Table We evaluate each entry of the result for 10 trials, shifting the starting location both target and background objects randomly.
I
Code
We make our code publicly available at http:://gstaaab.coo//rrrreeerrep
25


Reviews:
[{"review_id": "MOaJP8zpOi", "reviewer": "Reviewer_15gE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents PALO, a method for few-shot adaptation of language-conditioned robot policies to unseen tasks. The key idea is to leverage a VLM to generate candidate task decompositions from a sequence of frames, and then optimize over these decompositions using a small set of demonstrations. This allows the method to adapt a pre-trained policy to new tasks without fine-tuning, by selecting an appropriate sequence of subtasks that the policy can execute.", "summary_of_recommendation": "This paper presents an interesting approach to few-shot adaptation in language conditioned robot manipulation. Making appropriate revisions to address concerns around methodological details and experimental details, could make strengthen this paper.", "technical_quality": 2, "clarity_of_presentation": 2, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "1. How many demonstrations were typically used in your few-shot experiments?\n2. How many evaluations were run per task?\n3. Can you provide more details on the optimization procedure used to select the best task decomposition? (My current understanding is that it is based on randomly sampling combinations of subtasks and finding action sequences with the lowest MSE)\n4. Have you considered comparing PALO to other non-parametric methods for few-shot learning in robotics? Including comparisons with non-parametric baselines to could better contextualize the method's performance in few-shot settings.\n5. Do you have any insights into what causes failures in tasks such as \"pry away\"? Could these be attributed to limitations in the language embeddings generated or the image encoder used?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "icopVW6aos", "reviewer": "Reviewer_MWui", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "Learning long-horizon tasks from few demonstrations by decomposing the task into subtasks using GPT4o and then learning a high and low-level conditioned policy", "summary_of_recommendation": "I think if the authors add required details pointed above and add the missing experiments, then the paper will be more complete.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "1. How exactly is ‘u’ sequence initialized? Is it annotated by the human (sec 3.2, 3.3)?\n2. Is it useful to condition the policy on s0? Why or why not?\n3. How do you get c_L for the D_prior?\n4. Please also include how D_prior is different from D_target, via both qualitative examples and visualizing in encoding space with methods like t-SNE. It is unclear how D_target is OoD. Also, in what sense is it unseen - is it an unseen object, scene, or skill? A plot visualizing and summarizing this information should be added.\n5. Pretraining data stats are missing.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MTeHdIsI18", "reviewer": "Reviewer_AzAi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes PALO, a method that performs few-shot imitation by leveraging a language conditioned policy pre-trained on BridgeData and task-specific demonstrations to find a sequence of high + low level commands that make the pre-trained policy succeed in a task for which a human provides a few demonstrations. To achieve this, given the task specific demonstrations and language instruction, PALO uses an LLM to decompose the language instruction into lower level commands. Then, by optimising over the different possible low level commands and the timing to switch between them, PALO selects the sequence of high+low level commands that minimises the validation loss of the pre-trained policy on the provided demonstrations. This sequence of language commands is then selected and the task is performed. The authors find that prompting the policy with a good sequence of low level commands outperforms other alternatives, such as fine-tuning on the demos and through several experiments and ablations the paper shows that PALO outperforms several baselines.", "summary_of_recommendation": "My recommendation relies on the fact that overall the paper presents a novel method on how to decompose a task into lower level instructions to aid few shot policy learning that will be of interest to the community. The method explanation is simple and straightforward and experimental evaluations along with their ablations are convincing.", "technical_quality": 5, "clarity_of_presentation": 5, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- Can you elaborate on how you created the training dataset? How did you select the right language decompositions of the tasks present such that the low level language commands are correct?\n- Could you provide an experiment that studies the performance of PALO vs fine-tuning under different numbers of demonstrations ? Fine tuning OCTO for example on a very large number of demonstrations could result in better performance than PALO, it would be interesting to see when that happens in terms of task specific demonstrations.\n- It would be interesting to see how would the pretrained policy perform if a random language decomposition is selected and used to fine-tune the network used for PALO on the few provided demonstrations, instead of fine-tuning Octo.\n- If a failure occurs during task execution, would PALO be able to recover given that the low level instructions are fed to the policy sequentially without checking whether the low level action was successfully performed?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Strengths:

- Interesting approach that uses GPT4o for decomposing tasks into subtasks and uses it to do few-shot adaptation 
- Well-written paper with lots of experiments, ablations and comparisons that highlight the strengths of the work

Weaknesses:
- Some details about the demonstration dataset and pre-training are missing as pointed out by the reviewers. 
- More insights and analysis of failures seen during experiments would make the paper stronger. Moreover, please provide multiple runs of experiments, along with some statistical analysis of the performance.

Post-rebuttal discussion:
In the rebuttal phase, authors provided additional details missing in the original submission, as well as baselines and ablations, including multiple real-world runs of the baselines.
Decision: Accept