Robust Fine-tuning of Vision-Language-Action Robot Policies via Parameter Merging
Authors: Yajat Yadav
Zhiyuan Zhou
Andrew Wagenmaker
Karl Pertsch
Sergey Levine
Venue: ICLR 2026 Poster (2026)
Keywords: Robust fine-tuning
generalist robot policy
model merging
Primary area: applications to robotics, autonomy, planning
TL;DR: We present a simple yet effective finetuning strategy based on parameter merging to robustly add new skills to generalist robot policies, helping them generalize to unseen variations of the finetuning task and retain prior skills

Abstract: Generalist robot policies, trained on large and diverse datasets, have demonstrated the ability to generalize across a wide spectrum of behaviors, enabling a single policy to act in varied real-world environments. However, they still fall short on new tasks not covered in the training data. When finetuned on limited demonstrations of a new task, these policies often overfit to the specific demonstrations---not only losing their prior abilities to solve a wide variety of generalist tasks but also failing to generalize within the new task itself. In this work, we aim to develop a method that preserves the generalization capabilities of the generalist policy during finetuning, allowing a single policy to robustly incorporate a new skill into its repertoire. Our goal is a single policy that both learns to generalize to variations of the new task and retains the broad competencies gained from pretraining. We show that this can be achieved through a simple yet effective strategy: interpolating the weights of a finetuned model with that of the pretrained model. We show, across extensive simulated and real-world experiments, that such model merging produces a single model that inherits the generalist abilities of the base model and learns to solve the new task robustly, outperforming both the pretrained and finetuned model on out-of-distribution variations of the new task. Moreover, we show that model merging enables continual acquisition of new skills in a lifelong learning setting, without sacrificing previously learned generalist abilities.

Published as a conference paper at ICLR 2026
ROBUST FINETUNING OF VISION-LANGUAGE-ACTIOON ROBOT POLICIES VIA PARAMETER MERGING
Yajat Yadav', Zhiyuan Zhou", Andrew Wagenmaker, Karl Pertsch, Sergey Levine UC Berkeley
Standard Finetuning
RETAIN Robust Finetuning
policy merging
Pollcy
Policy
Policy
Margad Pollcy
Small fine-turing
Dataset
(-100 demonsrationn
F Finetuned Policy
Firetuning
Finetuning
Genaraist
Finetuning
Finatuning
Tacks
Figure 1: Naive approaches for finetuning of generalist policies narrowly improve target task per- formance on settings seen the finetuning data, but fail to generalize or retain generality beyond the target task. We propose simple solution: by averaging the generalist policy before and after finetuning, in weight space, we obtain finetuned policies that (1) significantly improve generaliza- tion ability unseen variations of the target task, and (2) retain generalist capabilities non-target tasks. Our approach RETAIN is a simple solution for robust policy finetuning.
ABSTRACT
Generalist robot policies, trained on large and diverse datasets, have demonstrated the ability to generalize across a wide spectrum of behaviors, enabling single policy to act varied real-world environments. However, they still fall short on new tasks not covered in the training data. When finetuned on limited demonstra- tions a new task, these policies often overfit to the specific demonstrations-nnt only losing their prior abilities solve a wide variety of generalist tasks but also failing to generalize within the new task itself. In this work, we aim to develop a method that preserves the generalization capabilities of the generalist policy dur- ing finetuning, allowing a single policy to robustly incorporate a new skill into its repertoire. Our goal is a single policy that both learns generalize to variations of the new task and retains the broad competencies gained from pretraining, We show that this can be achieved through simple yet effective strategy: interpolat- ing the weights of a finetuned model with that of the pretrained model. We show, across extensive simulated and real-world experiments, that such model merging produces single model that inherits the generalist abilities of the base model and learns to solve the new task robustly, outperforming both the pretrained and fine- tuned model that model merging performance scales with the amount of pretraining data, and enables continual acquisition of new skills in lifelong learning setting, without sacrificing previously learned generalist abilities.
-ii-r-idiitiiitiinn variations of the new task. Moreover, show
INTRODUCTION
Generalist robot policies trained on large corpora of data have recently shown impressive generaliza- tion abilities: out of the box, they can perform a range of tasks in unseen environments, generalize across scenes, viewpoints, objects, and language instructions (Intelligence et al.) Kim et al. 2024
Core contributors. Correspondence to yajatyadav @ herkeley.edu and zhiyuan_zhone @ @herkeley.ed. com
Project page

---

Published as a conference paper at ICLR 2026
Team et al. 2025 NVIDIA al. 2025; Liu et al. 2024b Qu et al. 2025 Gao et al. 2025 Amin et al. 2025). Though impressively general, these generalist policies often need to be adapted to perform effectively
downstream tasks or a new robot system, which is most commonly achieved by finetuning them on a curated dataset of demonstrations for the target task. While prior work has shown that such finetuning can lead to robust policies with tens or hundreds of hours of finetuning data (Black et al. 2024; Intelligence et al.; [Bousmalis et al. 2023; Brohan et al. 2023), collecting such amounts of robot demonstration data is challenging. As result, in practice often less than 100 demonstrations or few hours of robot data are used for finetuning (Kim et al. 2024 Team al.) 2024; Kim al., 2025). Existing approaches for robot policy finetuning struggle to preserve the generality of the pretrained model in such low-data regimes, and fail to robustly generalize far beyond the exact viewpoints, objects, and scenarios seen the finetuning data (Gao et al. 2025]
Xiong et al. 2020; Zhang et al. 2025 Wang [225 Xiang et al.) 2025 Zhu et 2025 Kaplanis ctt al., 2019). To expand the usability of generalist policies, we need robust finetuming approaches that better preserve the generality of pretrained robot policies and allow us to generalize to a broader set of scenarios on the target task.
In this work, introduce RETAIN (Robust finE-tuning wiTh pArameter merg surprisingly simple approach for robust robot policy finetuning. By simply interpolating the weights of the pretrained generalist policy before and after finetuning checkpoints that match the performance of the finetuned policy on scenarios present in the finetuning data, while generalizing significantly better to unseen variations of the target task. such as unseen object instances, positions, or viewpoints. Additionally, we observe that RETAIN preserves the generalist capabilities of the pretrained policy also on tasks other than the target task, allowing us to use RETAIN in continual learning setup by sequentially merging new skills into pretrained generalist policies (in a literal sense). We demonstrate the effectiveness of RETAIN for robust policy finetuning and sequential skill acquisition across a range of real-world and simulated finetuning tasks, achieving state-of-the-an finetuning performance. We also show that RETAIN gets even more effective when the pretrained policy is trained more data. While previous work has investigated interpolating model weights of pretrained and fine-tuned models for vision and language (Wortsman et al. 2022b a [lharco et al. 2022), this is to our knowledge the first work to investigate and analyze parameter merging for robot policies and use it to enable continual acquisition of new robotic skills.
the target task (see Fig. D). we obtain
In summary, our contributions are threefold: (1) we introduce simple approach for robust robot policy finetuning via policy parameter merging, (2) we extensively evaluate our approach across real-world and simulated robot tasks, and analyze which factors enable successful policy merging, (3) we demonstrate that our approach enables continual merging of new robot skills into state-of art generalist policies. Policies finetuned with our method generalize to novel scenarios for the new skill on real robots with ~ 40% higher success rate on average than best prior finetuning methods.
2 RELATED WORK
Adapting generalist policies on new tasks. Fueled by large-
Action Expert
scale human teleoperated robot datasets (Collaboration et al. [023
LLM Backbone
Walke et al. 2023 Khazatsky et al. [024] Shah et al. 2023), gener-
alist robot policies have recently solved a wide range of tasks across
2025 NVIDIA et al. Liu al. 2024b Anil et al. 2023 Sridhar et al. 2024 Qu et al. (025). Yet, even state-of- the-art generalist policies typically need be adapted for any given target task to achieve high performance (Black et al.) 2024). Thus, number of approaches have been proposed for training general- ist policies on a new target task: from simply finetuning them on a dataset of target task demonstrations (Black et al., 2024 Kim et al.) Team et al. (025). or mixing the outputs of the pretrained and fine-tuned policies (Bagatella)
diverse scenes (Black et al.
2024| Team et al.
Image Enc.
Language
Instruction
2025 Kim al. 2024 Brohan et 2023
State-of-the-ar generalist policies typically consist of a vision encoder, language model backbone, and action "expert" (decoder).
Figure/ 2:
2024
ct al.
2025), to alternative approaches like online and offline reinforcement learning (Mark et al,
2024
Huang et al. 2025 Yang et al. 2023b Hu et al.] 2024 Zhou et al. 2024 Xu et al. 2024).
retrieval-basee adaptation Long et al. 2025: Di Palo & Johns| 2024) or in-context improvement (Fu
et al.
2024a; Sridhar et al. 2022). In this work, we focus the most common setting, in which policies are finetuned on a target task using a small dataset of demonstrations. Various finetuning
approaches have been proposed in the literature, from simply adapting the full network on the target
2

---

Published as a conference paper at ICLR 2026
dataset (Kim et al. 2024; Black et al. 2024). to mixing target and pretraining data (Bousmalis al.
2023 Fu et al. [024bb] Dass et al. 2025). or freezing parts of the network during finetuning (Liu et al. 2023 Liang al. 2022 Sharma et al. 2023 Xu et al. 2023). While such approaches may prove effective for learning robust target task policies in "large-data" finetuning regimes with tens to hundreds of hours of finetuning data (Black et al.] 2023 Intelligence al.; [Bousmalis et al.)
2023 Brohan et al. (202)), they often struggle to retain the generality of the pretrained policy in more common, accessible settings with 100 or less target task demonstrations. In such scenarios, finetuned policies often struggle to generalize meaningfully beyond the conditions seen in the fine- tuning dataset (Zhang et al., 2025). even if the base policy had broad generalization capabilities. In this work, we propose simple alternative for robust policy finetuning in low-data regimes. In- stead of directly using the finetuned policy, we observe that merging the pretrained and finetuned policy checkpoints in weight space leads to significantly improved generalization on target tasks at no additional training or inference cost.
Model parameter merging. Our approach is inspired by work on model weight merging in vi-
sion and language domains (Wang et al. 2024b; Yadav et al. 2023; ; 2024 Nasery et al. 2025 Lu
al., 2025 Jang et al. 2024 Matena & Raffel 2022; Yang et al. 2023a Jin et al. 2022p. These
works demonstrate that interpolating between the weights of multiple finetuned models, or between
pretrained and finetuned models, can combine their capabilities or make them more robust to dis-
tribution shifts (Wortsman et al. [022bb Ilharco et al.) 2022 Neyshabur et al.) 2020 Marincione ct al. 2025). To our knowledge, our work is the first to demonstrate the effectiveness of model merg- ing in the context of generalist robot policies, and combining it with co- co-training to further improve upon vanilla model merging. Additionally, we analyze the importance of the generality of the base model, as well the importance of different parameter groups in usion-language-actiinn (VLA) policies and find often sufficient to only merge parameters from the language model backbone.
Continual learning. The focus of our work is on improving generalization of finetuned policies on target task. However, in addition, we find that our model merging approach is also effective at retaining the generalist policy's 's performance on tasks from the pretraining distribution. As such, we demonstrate that it can be used to sequentially merge multiple skills into single pretrained policy checkpoint while retaining generality. This setting is typically referred to as continual learning and there is large body of literature, both outside (Kirkpatrick et al. 2017; Schwarz et al., 2018
Lopez-Paz & Ranzato
2017:
Wang et al.. 2024c) and within robotics (Lesort et al. 2020 AAuddy
al. [023] Wan al. 2024
Meng et al. 2025 Liu et al.) 2024a Wolezyk et al. (021). Our work
differs from this line of research in that we aim to inherit and pass on the generalization ability of
pretrained model to learn new tasks robustly, whereas continual learning methods generally focus
on not forgetting old skills seen during the agent's 's lifetime.
3 PROBLEM SETTING
The goal of our work is to develop an approach for robust policy finetuning, in which a generalist policy is finetuned to a unseen
Plates in rack
new target task and generalizes variations of that target lask, like new ob- ject instances, viewpoints, scenes, or light- ing conditions, while also preserving its generalist abilitics. Formally, let M de- note an environment, S denote observa- tions (e.g., images, proprioception). A ac- tions, and T task specifications (e.g., lan- guage prompts). A policy To Ta st, T) maps state 8t E S and task T T to dis- tribution over actions a, E A. We assume access pretrained generalist policy Topea trained on a diverse set of tasks and environments, and denote its training data as D Pree For a new target task T, (e.g.. "wipe the white-
Plates rack (OOD)
Wipe whiteboard
(I)
Wipe whiteboard
(OOD)
Figure 3: Example filmstrips of the in in-distribution (ID) and out- distributionn (OOD) tasks from DROID robot experiments in Section 6
(i) (1) T,) }. In general, assume that D, is collected in a single (or small number of) environment M,, and D,, < 1 peee
board"), we assume access to demonstration dataset D, {ss a
3

---

Published as a conference paper at ICLR 2026
Behavioral cloning & finetuning, For adapting the policy to the target task, we consider the stan- dard behavioral cloning (BC) objective. For policy parameterization To and demonstration dataset D, the training objective is:
LBO ②) =
Σ
log (e( St,
(I)
(EED
We consider two main finetuning settings: Tisk-finetuning (task-FT), which we train exclusively on the target dataset D,, (e.g., because pretraining data is proprietary); and co-finetuning (co-FT), where we finetune on a mix of D, and pre to help preserve pretraining capabilities (e.g., in case of open-source pretraining datasets).
Evaluation. In practice, when we finetune a policy, we don't 't simply want work only in the setting where we collected the finetuning demonstration, but for it to complete the demonstrated task in variety of contexts or scenes. Current methods often fail in this regime because they overfit heavily to the small finetuning dataset. Therefore, to assess overfitting and robustness of the finetuned policy, we evaluate the performance of finetuned policies in the following three settings:
Target task in-distribution (ID): measures policy performance on the target task Ty with ob- jects, initial poses/layouts, camera placements, lighting conditions, and backgrounds observed in the finetuning dataset D.y
Target task out-of-distribution (OOD): measures the performance on the target task, in sce- narios not observed in the finetuning dataset, such as changes in object instances, backgrounds, lighting conditions and camera angles. This measures the robustness of the finetuned policy.
Generalist tasks: measures policy performance on tasks other than the target task, but for which we would expect the generalist policy Toyee to perform reasonably. This measures how well the finetuned policy retains generalist capabilities from the pretrained model.
CHALLENGES OF FINETUNING GENERALIST ROBOT POLICIES
To understand the challenges of ro- bust finetuning of generalist robot
mugs-on plates
pet-cn stave
Beers basker
policies, we start by evaluating the standard finetuning approach. We evaluate finetuning the full model (Task-FT) in LIBERO (Liu al.] 2024a), robotic manipulation sim- ulator containing 130 total tasks. Concretely, given state-of-the-art vision-language-actioo policy (Black ett al.] 2024), pretrained demon- stration data from 117 tasks from the LIBERO-(90, goal, spatial, ob- ject) suites, we finetune it to a new LIBERO target task. We consider three target tasks from LIBERO-10: mugs-on, on co-ooonaee....eeaaeeeeaaaaeeeeee pet-on-stove, and stems-intooball introduced in Section 3 ID, OOD, and Generalist. For OOD evaluations, we alter object positions, add new distractors, and change backgrounds. Generalist evaluations are performed over 20 tasks from the pretraining dataset. More details on OOD evaluations in Section 6.1 and Section A.7
58
200
AP0
Gradent
GESEALUIT
Figure 4: The standard approach for policy finetuning often overfits. As the policy is trained for more gradient steps, it performs worse on tasks other than the new target task ("GENERALISTT") and may even start to degrade on scenarios seen in the finetuning data ("ID"). Most impor- tantly, is not able to transfer the generality of a base pol- icy do well under variations of the target task (new object positions, instances, viewpoints; "OOD").
We measure performance in the three scenarios
Fig. shows the performance of Task-FT on three types of evaluations. We find clear tradeoff between generalist and ID target task performance: though the model gets better in ID target task after fine-tuning, it increasingly loses its generalist capabilities. Additionally, when the model is finetuned for too long, it even starts losing performance for ID tasks. Both of these phenomena are likely because the model has severely overfitted to the small demonstration dataset, and is unable to do other tasks or recover from small mistakes unseen in the dataset. More importantly, one would hope that the finetuned model can generalize to small variations of the target task not seen in the
4

---

Published as a conference paper at ICLR 2026
finetuning dataset, since the pretrained model contains knowledge for such generalization. However, all tested checkpoints above show a large gap between ID and OOD performance, showing that the model is not able to complete the target task when there are small variations. (..4ioonA..ssoooo that even with careful tuning of learning rate and number of gradient steps, such overfitting still exists.
RETAIN: ROBUST POLICY FINETUNING VIA MODEL MERGING
The previous section illustrates that standard finetuning approaches cause the model to quickly forget generalist capabilities, and fail to transfer the pretrained policy's robustness to the target task. To address these issues, we propose RETAIN (Robust finE-tuning wiTh pArameter mergINg), simple approach for robust finetuning of robot policies. Given pretrained policy weights gre and finetuned policy weights θn. we propose linearly interpolating Ppr and 8n to obtain a final policy checkpoint, 0. So, RETAIN produces final policy Tg by setting:
-= (1 (1 a) Ppe a Gtt
for a, a tunable merging weight. Though surprisingly simple, as we will see, this weight space "merging" of pretrained and finetuned checkpoints leads to significantly improved OOD perfor- mance on the target task, while retaining generalist policy capabilities (see Section 6.5). While weight merging itself already improves the policy's 's ability to retain and pass on generalist abilities, in the following we introduce two further improvements: utilizing the pretraining data Pre (in settings where it is available) to augment our task data D, during finetuning, and merging Prr and On in a mopelity- manner. We introduce these two methods below, and show how RETAIN can also enable continual adaptation to new tasks.
5.1 CO-FINETUNING
In Eq. 2). the finetuned policy weight Or can either optimized via task-finetuning or co- finetuning, as described in Section 3 In situations where the pretraining dataset, subset of it, is available, we can finetune the policy weight on a mix of 2 pre and D,, Such co- so-finetuning usu- ally leads to better retention of generalist abilities after finetuning (Bousmalis et al., [023] Fu al. 2024b Dass et al. 2025). We find that we can use model merging together with co-finetuning, which will refer to RETAIN-cO-FT,,
enable greater generalization the target task and better preserve generalist knowledge than model merging with task fine-tuning, which we call RETAIN- task-FT (see Section (
5.2
MODALITY-
MERGING
While prior works have explored model-merging in the context of uni-modal vision or language models (Lu et al., 2025, Jang al., 2024 Matena Raffel 2022 Yang et al. 2023a Jin et al.) 2022 Wang et al., 2024a), robotics is fundamentally a multi-modee problem. Modern generalist robot policies are typically instantiated as lasson-laguaageaction (VLA) models (see Fig. 2 that consist of a vision encoder (v), large language model backbone (!), and often an "action expert" module that decodes robot action outputs (a). We find that, in such multi-modal settings, it can be advantageous to use separate merging weights for different modalities. As such, we can expand the RETAIN merging objective to:
8 0
a
]
Ppre,v ppro,1
(aee on Ci
6n.v
1
oz auaa
+
On.1 Ppre,a/
0,
fpre,a/
A
Section 6.4 shows that, surprisingly, it often suffices to only merge the language model parameters.
5.3 CONTINUAL TASK ADAPTATION
Task
Task
We observe that RETAIN enables finetuned policies to retain the generalist capabilities of the pretrained policy. As such, we can use RETAIN to sequentially add tasks into a pre- trained checkpoint by iteratively merging fine- tuned weights into the base model and continu- ing finetune from the merged checkpoint (see
0
Oa
Figure 5: RETAIN enables continual merging of new skills into generalist policy backbones.
5

---

Published as a conference paper at ICLR 2026
Fig. 5). Formally, for sequence of target tasks T, .... Tn
compute sequence of adapted
RETAIN policies that accumulate new task capabilities as:
0, a) 0 + On,, In E (1...N)
(4)
where it,n denotes the parameters finetuned on the rith task.
EXPERIMENTS
The goal of our experiments to evaluate RETAIN's ability to robustly finetune generalist poli- cies to new tasks, i.e., to broaden the finetuned policy's ability to generalize to unseen settings in the target task. Concretely, we aim to answer the following questions: (1) Can RETAIN learn a new skill robustly and generalize more broadly to variations of the skill than prior finetuning ap- proaches? (2) What factors influence whether we can effectively merge pretrained and finetuned policy? (3) Can RETAIN enable continual merging of sequence of several skills into the pre-
trained policies?
6.1 EXPERIMENTAL SETUP
ID
OOD
LIBERO fiems-into-basker
DROID whiteboard
DROID plates
Figure 6: We evaluate policy finetuning on two real-world DROID finetuning tasks (left. middle) and three simulated LIBERO finetuning tasks (right, only one visualized here). In each task, we collect modest number of demonstrations (50-100) in comparatively narrow setting (blue), but evaluate on much broader set of variations for the same task (yellow), including variations scene, object instances, initial positions, lighting conditions, distractors, and viewpoints. This tests transfer of the generalization ability of the pretrained policy to the target task. Example trajectories in Fig.
Environments and tasks. We evaluate RETAIN real-world and simulated finetuning settings (see Fig. 6 For our real-world experiments, we use the DROID robot setup (Khazatsky et al.) 2024). which consists of a 7-DoF Franka robot arm with a wrist-mounted camera and at least one external camera. We design two challenging fine-tuning tasks: wiping the whiteboard with craser (which we call wh teotabbar rd) and putting the dishes into a drying rack (which we call p pla ates). For both tasks, collected in a single environment with minor position variations (see Fig. 6 blue), to mirror the vari- ation in typical narrow-data finetuning regimes (Zhang et al. 2025). We generalization of the finetuned policies on a much broader set of target environments (Fig. yellow), which include un- seen backgrounds. object instances, and camera views. See example task trajectories in Fig. 3 For our simulated experiments, we use the LIBERO simulation environment (Liu et al.) 2024a). We finetune policies pretrained on the LIBERO- spatial, goal, 90) datasets to three new tasks from the LIBERO- 10 suite: pot pot-on-stovv
collect roughly 50 and 100 demonstrations, respectively. All demonstrations
mugs-on-p! ates, and i temss cesss-inoobbbskeee We use x 45 demonstrations per task, obtained after filtering and preprocessing the 50 demos pro- vided with the LIBERO simulator, which only contain minor variations to the initial positions of each object, and again test on a much broader distribution initial positions, backgrounds, and additional distractors (see Fig. 6 right). More setup details in Section A.ttiionn.6.aam and Section A.7
For both LIBERO and DROID, we evaluate our method and baselines on three different OOD scenes. To pick the merging coefficient a, hyperparameter a for best performance on that scene. We only use a E {0.25, 0.5, 0.75) DROID. Then, the rest of the OOD seenes as the "test" scenes without any hyperparameter tuning.
OOD scene the "validation" scene, and tune the
Pretrained policies. We state-of-the-ar pretrained robot policies for experiments. For our weal-world DROID experiments, we use To To-FAST-DROID (Pertsch et al., 2025). the best open- source DROID policy the time of our experiments. For LIBERO,
no Black et al. 2023)

---

Published as a conference paper at ICLR 2026
DROO Weilhoad
Horboranc
Genenall Furforance
COO holt Herformance
COD PHSS
With wull! CAAIIAA
Figure 7: RETAIN results on two DROID tasks, whiteboard (top) and plates (bottom). RE- TAIN significantly outperform baselines in OOD evaluation and is competitive in ID evalua- tions, showing that it is able to learn new skills robustly and can generalize to its variations using pretrained knowledge. RETAIN also does best on generalist evaluations, showing that it is best at retaining abilities to solve old tasks. We tune merging coefficient on one "val" OOD scene, and use the same value for two other "test" OOD scenes.
policy fine-tuned on the LIBERO-(object, spatial, goal, 90} datasets as our pretrained policy. mo is generalist policy with flow-based action expert, while To-FAST-DROID is an autoregressive transformer based on next-token prediction.
Comparisons. We compare against prior methods, including those that incorporate regulariza- tion techniques reduce overfitting, in the robust finetuning setting. Specifically, compare our approach, RETAIN, to: "Task-FT", which finetunes the pretrained policy only on the target task dataset using behavioral cloning (Eq. ① (Bain & Sammut 1995; Black et al.) 2024); "Co-FT", which finetunes on mix of pretraining and target task data to reduce overlitiing (Fu et al. 2024b)
Dass et al. 2025). "LoRA", which uses low-rank adaptation (Hu et al. (022) during finetuning to retain more of the pretraining capabilities (Mittenbuehler et al.] 2023] Kim et al., (024)); "Freeze- FT", which freezes the language model backbone during finetuning and only updates the vision encoder, and, in the case of To. the action expert output head, following similar approaches in prior work, e.g.. Kim et al. (2024); Zhang et al. (2025); Scratch, which learns a policy from scratch on the demonstration dataset instead of finetuning generalist policy. Section Section/A.9]deke dA.9details policy classes and implementation. Section A.8 details hyperparameters choices and tuning process.
6.2 RETAIN LEARN NEW SKILLS ROBUSTLY AND GENERALIZE MORE BROADLY
RETAIN solves the finetuning task in a broader range of variations. We compare RE- TAIN with the aforementioned baseline methods when finetuning
new task, and show the per- formance on the three types of evaluations. Fig. shows results on two DROID environments, and Fig. B shows the average of three LIBERO environments. The ID and OOD evaluations show how well a method learns to do the new skill: ID evaluations show whether the method learned to fit the demonstration dataset exactly, and OOD evaluations assess whether the method has learned to generalize to the same task exhibiting variations not seen in the finetuning dataset. The OOD (val) scenes show "the upper bound" in performance when a is allowed be tuned slightly, while the OOD (test) scenes show the performance with no tuning.
In real-world DROID environments (Fig. 7). all methods perform much better on ID evaluations after finetuning, showing that the policy has adapted to the new task in the exact same context as the demonstration dataset. On ID evaluations, methods that use regularization, such as LoRA and Co- FT, perform slightly worse whit tebaadd likely because they are too constrained to adapt well to the new task. On OOD (test) evaluations, baseline finetuning methods perform significantly worse: while they complete the new task with 70-80% success rate in the ID setting, they have 30-50% success rate on average in the OOD setting. This shows that the baseline methods are very sensitive to small variations (such object change, location change, scene change) and cannot generalize
7

---

Published as a conference paper at ICLR 2026
to perform well. In comparison, both RETAIN-tusk-FI and RETAIN-co-FT perform much better, achieving more than 60% on plates and close to 80% on whit OOD evaluation. Note that this is similar to the policy's 's ID evaluation performance RETAIN can perform a generalized new skill with the same performance regardless of variations. Comparing the performance of OOD val and test sets, we see that the performance is sometimes not impacted whhiteboard) and sometimes better with slight hyperparameter tuning (plates). This shows that RETAIN somewhat robust hyperparameter, but can get even better when tuned for the particular OOD scene. Overall, in OOD evaluations, RETAIN enables the policy to outperform both the base model and other finetuned models.
whi beibabbard, suggesting that
Averre Over Three Tasks
coorie Nefarane
Figure 8: RETAIN results averaged over the three LIBERO tasks. Similar trend as Fig.
In LIBERO environments (Fig. 8) all methods exhibit trends similar to those in DROID environ- ments. Since the LIBERO simulation is much easier task than real world robotic tasks, certain baseline fine-tuning methods achieve near-perfect performance in ID evaluations. Under OOD eval- uations, both RETAIN-task-FT and RETAIN-co- -T-FT help improve the policy's 's robustmess to scene variations. We observe that the improvement in OOD performance over baselines such as Co-FT in LIBERO is smaller than that in DROID, and we attribute this to the lack of generalist capabilities of the base model. We offer more discussion of this in zzz.
RETAIN still performs well on tasks from the pretraining distribution. As shown above, RE- TAIN allows the model to not overfit to the finetuning dataset and generalize to solve a broader distribution of the finetuning task. One natural question whether RETAIN has overfit to the fine- tuning task distribution, and whether it can still solve tasks under the pretraining distribution. To evaluate retention of generalist skills, we evaluate RETAIN and baselines under our generalist eval- uation scenes. For DROID, we evaluate on 44 different real-world tasks; for LIBERO, we evaluate on 20 random tasks in the LIBERO pretraining dataset (5 each from LIBERO- LBERO-(object, spatial, goal, 90)). Fig. 7 and Fig. 8 (2nd subfig) show that RETAIN performs just as well as the pretrained model on generalist evaluations, showing that it has not lost its ability to solve old tasks from pretraining. RETAIN works even better when combined with co finettunnning as discussed below.
Model merging performs better with co-finetuning. In Fig. 7 and Fig. 8 RETAIN-cO-FT out- performs RETAIN-task-Fi in all three evaluation settings in almost all tasks. RETAIN-CO-FT is particularly effective at improving performance in generalist and OOD evaluation. In fact, we ob- serve that co-FT almost always helps improve performance over task-FT in generalist evaluation, but not always in OOD evaluation. We hypothesize this is because co-finetuning and model merging play different roles in the regularization process: co-finetuning helps the finetuned model not overfit to the small target dataset by continuous training on pretraining data, but does not elicit pretrained knowledge to help generalization on the new task; on the other hand, model merging explicitly tries to elicit pretrained knowledge and combine it with finetuning knowledge in parameter space; how- ever, doing so with a task-FT model worse at keeping pretraining abilities because the task-FT model has overfitted to the target dataset.
6.3 RETAIN SCALES WITH THE AMOUNT OF PRETRAINING DATA
In this section we ask: how well does RETAIN do when the pretrained model contains differ- ing amounts of "generalist knowledge", as induced by different amounts of pretraining data? To study how differing amounts of pretraining data affects RETAIN's performance, we use the DROID dataset, containing a large quantity of diverse robot data. We consider three pretrained generalist VLA policies: (1) the public g FAST--DDDIDD checkpoint that is trained on all of DROID and large repertoire of robot data from Physical Intelligence |Pertsch et al. (2025), (2) only trained on all of DROID, totaling 76k episodes, and (3) only trained subset of DROID of 20k episodes.
8

---

Published as a conference paper at ICLR 2026
All three policies are pretrained by taking roughly epoch over the dataset, and finetuned on the plates task with the same hyperparameters.
We then evaluate the three RETAIN-co-FT policies, each obtained by merging the respec- tive pretrained policy with its finetuned coun- terpart, on ID and OOD scenes in Fig. / To account for the performance difference of the base policies, we report the success rate in- crease between RETAIN-co-FT and co-FT. In ID evaluations, policies trained with differing amount of data perform similarly. In OOD eval- uations, more pretraining data leads to signif- icantly better performance. This shows that RETAIN scales with the amount of pretraining data, and can better "transfer" generalist knowl- edge from the pretrained model to new scenes when the pretrained model is more general. In fact, the best performing model (DROID-ALL + PI) performs nearly as good on OOD evlauation it does on ID evaluation, shown in Fig. 15
ID Perfomance Intrease
0CO Performance Increase
Success Ratie
DROID-SUbed
CROID-M
DROID-V +
DROID Sutiwee
DROID-N
DROID-M y
Figure 9: RETAIN performs better on OOD tasks when the pretrained generalist policy is trained on more data. OOD performance is eraged across three p lates scenes.
6.4 ANALYZING THE IMPORTANCE OF MERGING DIFFERENT PARAMETERS IN RETAIN
Here we seek to build a mechanis- tie understanding of how the merging
Illustration of OOD Type
coefficient impacts the performance
ID
location
Performance
0OD INpe lOcETS
of merged models. In Fig. [O] we
000 becignound
plot OOD performance against eraged across three LIBERO tasks, each with three types of OOD scenes. a = corresponds to the pretrained model, and = 1 corresponds to the co-finetuned model. Model merging (0 < a < 1) helps improve model performance in OOD evaluation, as long as the merged model is not too deviated from the finetuned model. When the merged model is too sim- ilar to the pretrained model, it does not have enough task-specific knowl- edge, and has near 0 performance. And as we have shown in Fig. 8 the model with the a value that has the best OOD performance is also comparable baselines in ID evaluations and better at generalist evaluations.
loc distractor
background
Figure 10: Left: Illustration of three different kinds of OOD scenes we consider in LIBERO, shown in pot-on-stove, Right: Plot shows how the OOD perfor- mance on three types of OOD variations changes with the merging parameter a, averaged across three LIBERO envs.
Next, we explore whether merging different parameters with different coefficients has an impact on the merged policy's performance. Specifically, consider modality-specific merging in the text of VLA policies. As explained in Section 5.2 we use separate coefficients for merging the parameters of the vision encoder, language model, and action expert. Fig. [ (left) shows the OOD performance of the merged model as we vary ag, ag, and a., from 0 to 1 on the mugs on -aa--on---lllllll elates task: dark colors represent low OOD performance, and light colors represent high OOD perfor- mance. Observe that the cube has the largest color gradient in the ag direction: this shows that the language model parameters have the most influence on performance. The best performing models has ai = 0.8 (see the highlighted plane at oj = 0.8 in Fig. [ (left) for the brightest colored dots). Next, to understand how Gep and oa impact performance, we plot in Fig. [T (middle) the change in OOD performance with these two coefficients when averaged over aq This 2D plot essentially squashes the 3D cube plot in the language direction. Unlike the behavior of oq, higher values of aa and a, lead to better performance; the best performance is achieved at a. 0g 1.
These results suggest that during model merging, it may suffice to only merge the parameters of the language model backbone (a; < 1) and leave the parameters in the vision encoder and the action expert set to the parameter values in the finetuned model (a,, = a = 1). To validate this hypothesis, compare the OOD performance of merging all parameters with RETAIN (0 < 1) only

---

Published as a conference paper at ICLR 2026
merging language model parameters. In Fig. 11 (right), we plot the performance of the two merging schemes over three different LIBERO tasks, each averaged over three types of OOD scenes. This shows that the two merging schemes achieve very similar performance, indicating that we only need to merge parameters from the language model backbone (instead of all parameters) to inherit the model's i generalization ability and robustness variations in the target scene.
003 Performanio
OOD Performance with Language Modality Merging
0.6
RETUI
RETUN el Langagge Mocal Harghng
0.4
C3
0.2
0.1
mugg
pot-on-stove
basket
Figure 11: Language model parameters have the most influence in modality merging. Left:
Merged model's OOD performance over grid search of aas ags ag and o has the most impact. Middle: OOD performance of a₂ and O₂ averaged over different an, and higher values are better. Right: Merging only the language model parameters (a,, = Qg = 1, an < 1) performs similarly to merging all parameters.
6.5 RETAIN ENABLES ROBUST LEARNING OF MULTIPLE SKILLS SEQUENTIALLY
Finally, we test whether RETAIN can enable learning multiple skills in sequence, as de- scribed in Section 5.3] and still retain its gen- eralist abilities. We consider learning the two DROID tasks sequentially, first finetuning on plates, and then using this as an initializa- tion to finetune on whiteboard. As outlined in Section 5.3 RETAIN uses the merged model from the first stage of finetuning as initializa- tion for the second finetuning stage. During evaluation time, we test whether the final pol- icy, after it has sequentially been trained on both tasks, can solve both tasks under ID and OOD evaluations. We compare against co-FT in the sequential learning setting, since it is the strongest baseline retaining prior knowledge in single-task finetuning (see Fig. 7 Fig. 12] shows the performance of the two policies on the two tasks under both ID and OOD settings. We also plot the performance of these two methods in the single-task finetuning setting in dashed lines. These two comparisons serve as the oracle performance cciling that we expect on these tasks, and not meant baselines for the sequential setting. When evaluated on the first task, plates, RE- TAIN does much better than co-FT in the sequential setting, showing that it is better at retaining its ability to solve the first task even after second round of finetuning. When evaluated the second task, wh ihobard RETAIN is also better than Co-FT under ID evaluations. RETAIN outperforms co-FT in the sequential setting under all tasks and evaluation types.
DRDID: Contincal (Plttes Whiteboard)
Figure 12: RETAIN enables continual adapta- tion to sequence of two skills. Evaluation sults show the performance of the final policy af- ter sequentially finetuning on two tasks, evaluated on different scenes. OOD performance averaged across two test scenes.
7 CONCLUSION
We present a simple yet effective method, RETAIN, for robust finetuning of generalist robot policies. We show that by simply interpolating the weights of generalist policy before and after is finetuned on a target task, we can "merge" the generalization ability of the base policy with the task-expertise of the finetuned policy. Through comprehensive real world and simulated experiments, we show that RETAIN can help the policy generalize significantly better to variations of the target task unseen in the demonstration, and is able to retain performance general tasks. We also apply RETAIN to sequentially acquire new skills in a lifelong learning setting, and find that it can robustly "merge" skills into single policy.
10

---

Published as a conference paper at ICLR 2026
8 LIMITATIONS
While we empirically verified that RETAIN works exceptionally well in helping the finetuned model generalize to out-of-distribution variations of the task, we don't understand the full scope of the reasons why model parameter merging was able to lead to such generalization. This is an interesting area for future work. We have included some discussion in Section A..0 of some hypothesess and why previous work found model parameter merging effective for vision and language tasks. Additionally, RETAIN involves an important hyperparameter, the merginge coefficient, that can be tuned. While we find in our real world experiments that RETAIN is robust to different values of this parameter, slight tuning of this parameter is needed. One avenue future work is determining a good heuristic of how to choose this value.
ACKNOWLEDGEMENT
This research was partly supported by ONR N00014-25-1-2000 This research used the Savio com- putational cluster resource provided by the Berkeley Research Computing program at UC Berkeley. We would also like to thank the anonymous ICLR reviewers for thoughtful comments and sugges- tions.
10 REPRODUCIBILITY STATEMENT
We describe all the implementation details Section A.6 and Section A.7 and hyperparameter choices in Section A.8 which should enable researchers to reproduce our results. We also remark that our algorithm is extremely simple to implement. We will share the code during the review process and also release the code publicly.
REFERENCES
Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Jared DiCarlo, Danny Driess, et al. #0.6 a vla that learns from experience. arXiv preprint 11Xv:25511.4459. 2025.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305. 2023.
Sayantan Auddy, Jakob Hollenstein, Matteo Saveriano, Antonio Saariguez-Sánchere and Justus Pi- ater. Continual learning from demonstration of robotics skills. Robotics and Autonomous Systems, 165:10 2023.
Marco Bagatella, Jonas Hübotter, Georg Martius, and Andreas Krause. Active fnn-itnnin of multi- task policies. In Forry-second International Conference on Machine Learning. 2025.
Michael Bain and Claude Sammut. framework for behavioural cloning. In Machine intelligence 15, pp. 103-129, 1995.
Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine. Zero-shot robotic manipulation with pretrained image-editing diffusion models. arXiv preprint arXiv:2310. 10639, 2023.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian lchter, et al. no: A vision- lasiio--anguuo--actton flow model for general robot control. arXiv preprint 24..:4 2024.
Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X Lee, Maria Bauza, Todor Davehev, Yuxiang Zhou, Agrim Gupta, Akhil Raju, et al. Robocat: A self-improving foundation agent for robotic manipulation. arXiv preprint arXiv: /706, 2023.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision- sision--anguag--cttiin models transfer web knowledge robotic control. arXiv preprint arXiv:2307. /58/8, 2023.
11

---

Published as a conference paper at ICLR 2026
Open X-Embodiment Collaboration, Abhishek Padalkar, Acom Pooley, Ajinkya Jain, Alex Bew- ley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anikait Singh, Anthony Brohan, Antonin Raffin, Ayzaan Wahid, Ben Linerss, Beomjoon Kim, Bernhard Schölkopf, Brian Ichter, Cewu Lu, Charles Xu, Chelsea Finn, Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Chuer Pan, Chuyuan Fu, Coline Devin, Danny Driess, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Federico Ccola, Fei Xia, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Ge Yan, Giulio Schi- avi, Hao Su, Hao-Shu Fang, Haochen Shi, Heni Ben Amor, Henrik I Christensen, Hiroki Furuta, Homer Walke, Hongjie Fang, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jae- hyung Kim, Jan Schneider, Jasmine Hsu, Jeannette Bohg, Jeffrey Bingham, Jiajun Wu, Jialin Wu, Jianlan Luo, Jiayuan Gu, Jie Tan, Jihoon Oh, Jitendra Malik, Jonathan Tompson, Jonathan Yang. Joseph J. Lim, João Silvério, Junhyek Han, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Ken Goldberg, Kendra Bymee Kenneth Oslund, Kento Kawaharazuka, Kevin Zhang, Keyvan Majd, Krishan Rana, Krishnan Srinivasan, Lawrence Yun- liang Chen, Lerrel Pinto, Liam Tan, Lionel Ott, Lisa Lee, Masayoshi Tomizuka, Maximilian Du, Michael Ahn, Mingtong Zhang, Mingyu Ding, Mohan Kumar Srirama, Mohit Sharma, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen, Nicolas Heess, Nikhil Joshi, Niko Suenderhauf, Nor- man Di Palo, Nur Muhammad Mahi Shafiullah, Oier Mees, Oliver Kroemer, Pannag R Sanketi, Paul Wohlhart, Peng Xu, Pierre Sermanet, Priya Sundaresan, Quan Vuong, Rafael Rafailov, Ran Tian, Ria Doshi, Roberto Martín Martin-Martin, Russell Mendonca. Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante. Sean Kirmani, Sergey Levine, Sherry Moore, Shikhar Bahl, Shivin Dass, Shuran Song, Sichun Xu, Siddhant Haldar, Simeon Adebola, Simon Guist, Soroush Nasiri- any, Stefan Schaal, Stefan Welker, Stephen Tian, Sudeep Dasari, Suneel Belkhale, Takayuki Osa, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Tianhe Yu, Tianli Ding, Todor Davehev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Vidhi Jain, Vincent Vanhoucke, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiaolong Wang, Xinghao Zhu, Xuanlin Li, Yao Lu, Yev- gen Chebotar, Yifan Zhou, Yifeng Zhu, Ying Xu, Yixuan Wang, Yonatan Bisk, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yueh hua Wu, Yujin Tang. Yuke Zhu, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zhuo Xu, and Zichen Jeff Cui. Open X-Embodiment: Robotic learning datasets and RT-X models. https:: http/axxvvvvv....o org/ abs 2310 08864 2023.
Shivin Dass, Alaa Khaddaj, Logan Engstrom, Aleksander Madry, Andrew Ilyas, and Roberto Martín-Martin Datamil: Selecting data for robot imitation learning with datamodels. arXiv preprint ar arXiv:2505.0960 2025.
Norman Di Palo and Edward Johns. Dinobot: Robot manipulation via retrieval and alignment with vision foundation models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 2798 28088-2805.100 IEEE, 2024.
Letian Fu, Huang Huang, Gaurav Datta, Lawrence Yunliang Chen, William Chung-Ho -o Panitch, Fangchen Liu, Hui Li, and Ken Goldberg. In-context imitation learning via next-token prediction. arXiv preprint arXivv2408. 15980, 2024a.
Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-bode body teleoperation. arXiv preprint arXiv:2401.02//7 2024b.
Jensen Gao, Suneel Belkhale, Sudeep Dasari, Ashwin Balakrishna, Dhruv Shah, and Dorsa Sadigh. A taxonomy for evaluating generalist robot policies. arXiv preprini arX/v:2503.01238, 2025.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Jiaheng Hu, Rose Hendrix, Ali Farhadi, Aniruddha Kembhavi, Roberto Martin-Martín, Peter Stone, Kuo-Hao Zeng, and Kiana Ehsani. Flare: Achieving masterful and adaptive robot policies with large-scale reinforcement learning fine-tuning, arXiv preprint arXiv:2409.1657 2024.
Donghhi Huang, Zhirui Fang, Tianle Zhang, Yihang Li, Lin Zhao, and Chunhe Xia. Co-rft: Eff- cient fine-tuning of ssioo-language-action models through chunked offline reinforcement learn- ing. arXiv preprint arXiv:2508.0 02219,.0.2......... 2025.
Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song. Hannanch Hajishirzi, Si- mon Komblith, Ali Farhadi, and Ludwig Schmidt. Patching open- vocabulayy models by interpo- lating weights. Advances in Neural Information Processing Systems, 35:29262-29277 2022.
12

---

Published as a conference paper at ICLR 2026
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. x0. 5: a vision-langee wawoonnllngugg-aaaoc action model with open-worle generalization, 2025. URL https://arxiv. wrg/abs/2504.16054 1(2):3.
Dong-Hwan Jang. Sangdoo Yun, and Dongyoon Han. Model stock: All we need is just few fine-tuned models. In European Conference Computer Vision, pp. 207-223. Springer, 2024.
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09845 2022.
Christos Kaplanis, Murray Shanahan, and Claudia Clopath. Policy consolidation for continual rein- forcement learning. arXiv preprint arXiv:1902.00225, 2019.
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty El- lis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403. 12945, 2024.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source saiio-language-action model. arXiv preprint arXiv:2406.09244 09246,, 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision laioonlanguagg-action models: Opti- mizing speed and success. arXiv preprint arXiv:2502.19645, 2025.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- -arrkkikka, al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 3326, 2017.
Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia Díaz-Rodríguez. Continual learning for robotics: Definition, framework, learning strategies, op- portunities and challenges. Information fusion, 38:52-68, 2020.
Anthony Liang. Ishika Singh, Karl Pertsch, and Jesse Thomason. Transformer adapters for robot learning. CoRL. 2022 Workshop on Pre-training Robot Learning. 2022.
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke 7hu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024a.
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt- lb: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.0786. 078.64444 2024b.
Zuxin Liu, Jesse Zhang, Kavosh Asadi, Yao Liu, Ding Zhao, Shoham Sabach, and Rasool Fakoor. Tail: Task-specific adapters for imitation learning with large pretrained models. arXiv preprint 2023.
Yayu Long, Kewei Chen, Long Jin, and Mingsheng Shang. Drae: Dynamic retrieval- augmented expert networks for lifelong learning and task adaptation in roboties. arXiv preprint arXiv:2507.04661 2025.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Wei Lu, Rachel K Luu, and Markus J Buehler. Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling. model merging and synergistic capabilities. upj Computational Materials, 1(1):84, 2025.
Davide Marincione, Donato Crisostomi, Roberto Dessi, Emanuele Rodolà, and Emanuele Rossi. Model merging improves zero-shot generalization in bioacoustic foundation models. arXiv preprint ArXiv:25/ 1.05171, 2025.
13

---

Yee Whye Published as a conference paper at ICLR 2026
Max Sobol Mark, Tian Gao, Georgia Gabriela Sampaio, Mohan Kumar Srirama, Archit Sharma, Chelsea Finn, and Aviral Kumar. Policy agnostic rl: Offline rl and online rl fin-ttning of any class and backbone. arXiv preprint arXiv:2412.00685,, 2024.
Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 15:17703- 2022.
Yuan Meng, Zhenshan Bing, Xiangtong Yao, Kejia Chen, Kai Huang, Yang Gao, Fuchun Sun, and Alois Knoll. Preserving and combining knowledge in robotic lifelong reinforcement learning. Nature Machine Intelligence, pp. 1-14, 2025.
Marcel Mittenbuehler, Ahmed Hendawy. Carlo D'Ermmo, and Georgia Chalvatzaki. Parameter- efficient tuning of pretrained visual-language models in multitask robot learning. In CoRI. 2023 Workshop on Learning Effective Abstractions for Planning (LEAP), 2023.
Anshul Nasery, Jonathan Hayase, Pang Wei Koh, and Sewoong Oh. Pleas-merging models with permutations and least squares. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 30493-30502, 2025.
Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What being transferred in transfer learn- ing? Advances in neural information processing systems, 33:512-523, 2020.
NVIDIA, :. Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang. Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li. Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang. Zu Wang, Jing Wang, Qi Wang. Jianman Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. Gr00t nl: An open foundation model for generalist humanoid robots, 2025. URL
htttp:://rrxi... http://wwwwvrgg//aaa/oo/o als /200. 14734
Karl Pertsch, Kyle Stachowiez, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025.
Delin Qu, Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao. Xinyi Ye, Qi Lv. Modi Shi, Guanghui Ren, Cheng Ruan, Maoqing Yao, Haoran Yang, Jiacheng Bao, Bin Zhao, and Dong Wang. Embodiedonevision: Interleaved vision-text-action pretraining for general robot control, 2025.
Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka
Teh, Razvan Pascanu, and Raia Hadsell. Progress compress: A scalable framework for contin- ual learning. In International conference machine learning, pp. 4528 4537. PMLR, 2018.
Dhruv Shah, Ajay Sridhar, Arjun Bhorkar, Noriaki Hirose, and Sergey Levine. Gnm: A general navigation model drive any robot. In 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 7226-7233. IEEE, 2023.
Mohit Sharma, Claudio Fantacci, Yuxiang Zhou, Skanda Koppula, Nicolas Heess, Jon Scholz, and Yusuf Aytar. Lossless adaptation of pretrained vision models for robotic manipulation. arXiv preprint arXiv:2304.066000 2023.
Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine. Nomad: Goal masked diffusion policies for navigation and exploration. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 63-70. IEEE, 2024.
Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. Ricl: Adding in-context adaptability pre-trained lisuo--lngguage-ettin models. arXiv preprint arXiv:2508.0206) 2025.
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptinn Alayrac Montser- rat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl. et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20mm 2025.
14

---

Published as a conference paper at ICLR 2026
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXhv:2405.22213, 2024.
Homer Rich Walke, Kevin Black, Tony Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pp. 1723-1736. PMLR, 2023.
Weikang Wan, Yifeng Zhu, Rutav Shah, and Yuke Zhu. Lotus: Continual imitation learning for robot manipulation through unsupervised skill discovery. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 537-544. IEEE, 2024.
Ke Wang, Nikolaos Dimitriadis. Alessandro Favero, Guillermo Ortiz-limenez, Francois Fleuret, and Pascal Frossard. Lines: Post-training layer scaling prevents forgetting and enhances model merging. arXiv preprint arXiv:2410.171465 2024a.
Ke Wang, Nikolaos Dimitriadis, Guillermo Ortiz-Jimenez, François Fleuret, and Pascal Frossard. Localizing task information for improved model merging and compression. arXiv preprint arXiv:2405.07815 2024b.
Liyuan Wang. Xingxing Zhang, Hang Su, and Jun Zhu. comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence, 46(8):5362-5383, 2024c.
Yixiao Wang. Generalization capability for imitation learning. arXiv preprint arXiv:2504.18536 2025.
Maciej Wolezyk, Michat Zajac, Razvan Pascanu, Łukasz Kuciński, and Piotr Mitos. Continual world: A robotic benchmark for continual reinforcement learning. Advances in Neural Informa- tion Processing Systems, 34:28496-28510 2021.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in- ference time. In International conference on machine learning, pp. 23965-23998. PMLR, 2022a.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannanch Hajishirzi. Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. Proceedings of the IEEE/CVF conference on computer vision 2022b.
and pattern recognition, pp.
Tian-Ye Yu Xiang, Ao-Qun Jin, Xiao-Hu Zhou, Mei-Jiang Gui, Xiao-Liang Xie, Shi-Qi Liu, Shuang- Yi Wang, Sheng-Bin -Bin Duan, Fu-Chao Xie, Wen-Kai Wang, et al. Parallels between vla model post-training and human motor learning: Progress, challenges, and trends. arXiv preprint MrXiv:2506.20960 2025.
Fangzhou Xiong, Zhiyong Liu, Kaizhu Huang, Xu Yang, Hong Qiao, and Amir Hussain. Encod- ing primitives generation policy learning for robotic arm to overcome catastrophic forgetting in sequential multi-tasks learning. Neural Networks,
2020.
Charles Xu, Qiyang Li, Jianlan Luo, and Sergey Levine Rldg: Robotic generalist policy distillation via reinforcement learning. arXiv preprint arXiv:2412.09855. 2024.
Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, and Chuang Gan. Hyper-decision transformer for efficient online policy adaptation. arXiv preprint arXiv:2304.08487, 2023.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Re- solving interference when merging models. Advances in Neural Information Processing Systems, 36:700937715,, 2023.
Prateek Yadav, Tu Vu, Jonathan Lai. Alexandra Chronopoulou, Manaal Faruqui, Mohit Bansal, and Tsendsuren Munkhdalai. What matters for model merging at scale? arXiv preprint arXiv:2410.036/7, 2024.
15

---

Published as a conference paper at ICLR 2026
Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adaptive model merging for multi-task learning. arXiv preprint arXiv:2310.02575, 2023a.
Jingyun Yang. Max Sobol Mark, Brandon Vu, Archit Sharma, Jeannette Bohg, and Chelsea Finn. Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world rein- forcement learning. arXiv preprint arXiv:2310.15144 2023b.
Wenbo Zhang, Yang Li, Yanyuan Qiao, Siyuan Huang, Jiajun Liu, Feras Dayoub, Xiao Ma, and Lingqiao Liu, Effective tuning strategies for generalist robot manipulation policies. In 2025 IEEE International Conference Robotics and Automation (ICRA), pp. 7255-7262. IEEE, 2025.
Zhiyuan Zhou, Pranav Atreya, Abraham Lee, Homer Walke, Oier Mees, and Sergey Levine. Au- tonomous improvement of instruction following skills via foundation models. arXiv preprint arXiv:2407.206355 2024.
Ruiqi Zhu, Endong Sun, Guanhe Huang, and Oya Celiktutan. Efficient continual adaptation of pretrained robotic policy with online meta-learned adapters. arXiv preprint arXiv:2503. 18684, 2025.
A APPENDIX
A.1 DROID RESULTS DETAILS
Fig. reports performance of two DROID tasks: whiteboard and plates. Here we present the average of the two tasks. The OOD performance shown here is averaged across the val and test scenes.
DRDEX Awerage Acrow Wh Intaaadd Conrrnts: Nofarmnnc
GLAT GEAROAA
Figure 13: Average results on performance of the DROID tasks: whiteboard and plates.
A.2 LIBERO RESULTS DETAILS
Fig. 8 reports the average performance of three LIBERO tasks: pot pot-on- -tove, mugs on on-pltee and iitems-inton innoonooo Here we present the individual performance for all three tasks in Fig. 14. The OOD performance shown here are averaged only across the two test scene.
Additionally, we offer here more discussion on why we see less improvement of RETAIN over baselines LIBERO as compared to DROID. For DROID, the base model FAST was trained on 76k diverse trajectories in 564 scenes, while the LIBERO base model is only trained on 5.3k trajectories in 117 scenes with fairly limited diversity. As such, the LIBERO base model contains much less generalist ability than the DROID base model. And as will shown in Section 6.3 merging with a less general base model inherits less generalization power, giving less improvement under OOD evaluations. There is also much bigger gap between val and test OOD scenes, because the different types variations for each OOD scene has a different difficulty level for the limited generalist model in LIBERO.
16

---

Published as a conference paper at ICLR 2026
LIBERO, mougg-oo--pstes
300
Geswrin Ferforannce
UIBRR: Pot on Stove
of
Basket
263 Pedormance
Gesaraio Infurivee
Figure 14: Detailed results on performance of the three LIBERO tasks: pot- pot-on-stove, and tems items-into-box.
A.3 DATA SCALING RESULTS DETAILS
Fig. 15 highlights more details of the data scaling results, examining for which base models RETAIN provides the greatest benefit: it shows the ID/OOD performance for co-FT and RETAIN-co-FT with varying amounts of pretraining data that led to the difference results presented Fig. 9 We see that as we scale pretraining data, ID performance is similar between co-FT and RETAIN-CO-FT, while the benefit provided by RETAIN on OOD greatly increase with the generalist capabilities of the base model, and scales we pre-training data.
RETAIN Performance nce with Differing Pretraining Data
Useat Suioo
SASS0
UNOIO Sulel
UADI A
CPCD- USCSR, M.
UNCI Schalt
Figure 15: Detailed results on the benefits of RETAIN with increasing pretraining data.
A.4 ABLATION ON LEARNING RATE AND GRADIENT STEPS
Here, we ablate the learning rate and number of gradient steps we take in the task-FT policy in Fig. to study whether better hyperparameter choices can reduce or resolve overfitting") In Fig. 4 we use learning rate 2.5e 5. Here in Fig. [6, we ablate four different learning rates: one greater than original and two smaller. We evaluate the model at every 100 gradient steps to also ablate on the number of gradient steps we take. With a larger learning rate, it's clear that the overfitting issue is more severe, and the performance all three kinds of evaluations (ID, OOD, Generalist) go down to
Following the best practice from Black et al. 2024), we always use learning rate warmup period of 1000 steps in LIBERO and consine decay schedule over 30k steps to 1 1/10 of the peak learning rate. The learning rate we report here is the peak learning rate.
17

---

Published as a conference paper at ICLR 2026
near zero. With smaller learning rate, the model still performs well in ID evaluations, and suffers less from forgetting generalist capabilities (measured from Generalist evaluations). This to be expected because the finetuned model is closer to the pretrained model with smaller learning rate. However, note that with a smaller learning rate, the OOD evaluation performance is also worse. We plot the OOD performance achieved using the original learning rate in Fig. Aas the dotted orange line in Fig. [6, and the gap between the dotted and solid orange line shows the performance gap in OOD evaluation when we lower the learning rate, possibly due to underfitting. This experiment shows that tuning the learning rate and the number of update steps does not solve the problem of overfitting during finetuning. While lowering the learning rate can retain more generalist knowledge, it does not prevent the gradual loss of it. More importantly, the lower learning rate leads to worse OOD evaluation performance.
Days
Figure 16: Ablation on learning rate and number of gradient steps for task-FT on mugs -eas--on-e ca--son-plllee com-pplaee pplates task in LIBERO.
A.5 ANALYSIS OF FINETUNING PATH IN PARAMETER SPACE
To understand why model parameter merging helps, we first try to understand here how the param- eters change during fine-tuning. In particular, we are interested understanding how linear the finetuning path is in parameter space, since model parameter merging only moves weights in linear paths.
To start out, we check the colinearity of the vectors 0 and f where 0 is the parameter of the finetuned checkpoint at gradient steps 100 (i.e. we plot the difference vector every 100 steps during finetuning). We measure the colinearity as the cosine similarity between the two difference vectors. Fig. shoows this for the four different learning rates we ablated in Fig. [6] Since no values close to I. this shows that the changes in parameter space is highly non-linear. This is expected since the parameter trajectory of deep neural networks is often highly non-linear.
Next, attempt to more directly visualize the path/direction of the parameters during finetuning by projecting them down to 2D with Principal Component Analysis (PCA). Specifically, we again consider the difference of the weight vectors, X; 8.+1 0;, at every 100 steps during finetuning. In Fig. 18 we plot the first two principal components of X, in blue, and label the points i. Indeed, that for all learning rates, the direction of the parameters is highly non-linear. With small learning rates, the direction oscillates lot; with larger learning rates, the direction bends in a certain direction. In addition, we plot the two principal components of the parameter-merged model as well in orange. As expected, since the the merged model takes a linear path and the finetuned one does
1.0
Cocine Similarity Parameter Ciffernnce Throughout Training
rat Se.
0.6
0.4
0.0
750
1750
Zoudier Depss
Figure 17: Cosine similarity of parameter difference vectors during finetuning, showing that the finetuning path is highly non-linear.
18

---

Published as a conference paper at ICLR 2026
Figure 18: PCA projection of the parameter difference during finetuning to 2D. Each subplot corre- sponds different learning rate.
Brybe
123
33
223
Figure 19: Singular Values of the parameter differences during finetuning. Each subplot corresponds to different learning rate.
not, the two models end in very different places in the parameter space. This shows that model merging achieves different solution than any checkpoints on the finetuning path.
Finally, we analyze changes in all directions the difference vectors X,. instead of just the two prin- cipal components shown in Fig. 18 We take the difference vector matrix Y = [X1; X2j X,, and compute the singular values YYT and Fig. 19 If the parameter vectors lie in a linear path, then all difference vectors would point in the same direction and there should only be 1 non-zero singular value. However, it's clear from Fig. 19 that most singular values are non-zero, showing that the path is non-linear in many dimensions. This generalizes the intuition from Fig. 18 to more dimensions, and shows that model merging indeed achieves a different solution than the finetuning path.
All these analysis experiments go to show that the finetuning path is highly non-linear, and therefore model parameter merging actually results in different solution than the finetuned models.
19

---

Published as a conference paper at ICLR 2026
A.6 DROID SETUP DETAILS
This section outlines the details of the DROID setup that we used for our real-world experiments.
A.6.1 DATASETS: WHITEBOARD
The whiteboard task-dataset consists 50 human tele-operated trajectories collected using a Oculus Quest 2 controller. As explained in 6.1] we collect the demonstrations with fixed setup, with the only diversity being 5 different eraser initial positions and 2 orientations (vertical / horizomtal). All demonstrations use the language prompt "wipe the whiteboard". Each step in our dataset consists of a base camera image (always collected from the right external camera), a wrist camera image, 8D-state, 8D-action, and the language instruction.
A.6.2 DATASETS: PLATES
The plates mask-dttsset consists of 100 human tele-operated trajectories collected in similar manner above. Again, the demonstractions are collected with a fixed setup, with the only diversity being plate colors, 2 dish racks, and 2 dish rack orientations (vertical horizontal), 80 of these 100 demos also contain distractor objects chosen from training set of distractors. All demonstrations use the language prompt "put the plate in the dish rack".
A.6.3 TRAINING DETAILS
We utilize [7x joint angles, Ix gripper position] as our proprioceptive state and [7x joint velocity, 1x gripper position] of the DROID dataset, publicly available here, to normalize states and actions during training and inference by applying guantil-norrmalizaion The base and wrist camera images go through several transforms (random crop, resizing to 224x224, and color jitter) during training.
actions both during training and inference. We also the norm-stats
The finetuning is performed with an action horizon of 10 environment steps, thus the policy learns to output action chunks of shape (10,8).
A.6.4 EVALUATION DETAILS
We use the 7-DoF Franka robot arm for our experiments. We use the same language instruction as training during evaluation, resize our images to 224x224, use the same state/action space specifica- tion, and normalize in the same manner training.
During evaluations, we additionally binarize the policy's s gripper action 0/1 (open/close), well as clip action magnitudes. The policy's 's action horizon is 10 environment steps, and during eval- uation, we set the open loop horizon to 8: so, during evaluation, we receive 10 actions from the policy, execute the first 8, and then request a new action chunk. We execute the predicted actions at Hz.
a control frequency
The policy is served NVIDIA H200/H100 throughout our evaluations. As the specifies, inference requires at least GB of VRAM.
the openpi repository
A.6.5 EVALUATION CRITERIA: WHITEBOARD
For all whiteboard evaluations, we use the criteria specified in Table I to assign partial success. We perform 10 trials per policy evaluation, for both the ID and OOD evals.
A.6.6 EVALUATION CRITERIA: PLATES
For all plates evaluations, perform 10 trials per policy evaluation, for both the ID and OOD evals.
the criteria specified in Table 2 to assign partial success. We
A.6.7 GENERALIST EVALUATIONS DETAILS
As detailed in Tables 3 4 and 5 we measure policies' generalist capabilities by evaluating them on
44 tasks, distributed throughout 9 distinct scenes and 17 different language instructions. Importantly,
20

---

Published as a conference paper at ICLR 2026
Subtask
Cumulative Score
Pick up Eraser
0.2
Approach Whiteboard
0.4
Set Eraser on Whiteboard While Still Grasping It
0.6
Performs the Wiping Motion
0.8
Erases 90% of Text
1.0
Penalty: -0.1 if any of the above is done, but the eraser got flipped the process.
Table I: Subtasks and cumulative score for the whiteboard task.
Subtask
Cumulative Score
Picks up Plate
0.2
Moves to Dish Rack
0.4
Rotates and Aligns Over Groove
0.6
Tries to Insert Plate Into a Groove
0,8
Successfully Inserts Plate Into a Groove
1.0
Penalty: -0.1 1.11 if any of the above done but with the small grooves.
Partial: 0.5 if tries to do the inserting motion but the plate is misoriented.
Table Subtasks and cumulative scoring for the plates task.
ensure fair comparison, we ensure that the initial conditions, camera angle, lighting, and all other such factors per task are kept the same across the various policies that we evaluate.
A.7 LIBERO SETUP DETAILS
This section outlines the details of the LIBERO setup that we used for our simulated experiments.
A.7.1 LIBERO PRETRAINING
In order to obtain base-model serve the starting point for RETAIN in LIBERO. pretrain To on a mixture of LIBERO datasets. Specifically, we use 90 tasks from LIBERO-90, 9 tasks from LIBERO-object, tasks from LIBERO-spatial, and tasks from LIBERO-goal, for total of 117 tasks in our pretraining dataset. For all LIBERO datasets, pretraining and finetuning, we utilize pre-processed RLDS versions gathered from here and here These datasets consist of LIBERO demonstrations that have been preprocessed to upscale images, filter out transitions with idle actions, and remove failure trajectories. More details, such as hyperparameter choices, for the pretraining stage itself can be found in A.8 All subsequent fine-tuning and evaluation uses the normalization stats this pretraining dataset, applying mean/standardddeviattoo normalization.
A.7.2 LIBERO FINETUNING DATASETS
As mentioned in 6. we finetune on 3 tasks from LIBERO-10: pot- on stove, mugs-on on con-platess plates, and sttes-innni intto Laito-bassee basket. For each dataset, we obtain pre-processed and filtered versions as described above. 20 highlighs what these finetuning tasks, and thus also our ID evals, look like.
The language instructions for each libero fine-tuning dataset are:
pot-on-stove:
"turn on the stove and put the moka pot on it"
mugs-on plates: "put the white mug on the left plate and put the yellow and white mug on the right plate"
"put both the alphabet soup and the cream cheese box in the
basket"
21

---

Published as a conference paper at ICLR 2026
Scene Image
Task
Trials
Randomization
Rubric
I: pick up spoon; move spoon towards dish rack; 2: put spuon in dish rack (anywhere)
put the spoon in the dish rack
swap spoon and carrot position, evals each
4
I: up carrot; 1,5:
swap spoon carrot position. evals each
put carrot in bowl
4
rot towards bowl; 2: put carrot
I: pick up plate; 1.5: move plate lowards dish rack; put plate into dish rack (anywhere)
randomize initial position of the plate in front the robot
put plate dish
2
2 cloth initially on the left and right side the open
1: move towards cloth;
wipe the table
2: perform lateral ""wiping- style" motion
2 plate initially on different dish rack holders (middle and end)
I: towards red plate: 1.5: picks plate: 2: places / drops plate onto the table
put the plate on the table
1: picks up paper ball; 1.5 moves paper towards brown bin: 2: puts paper ball into bin
randomize initial position of per ball on table
clean up table
2
Table 3: DROID Generalist evaluation tasks grouped scene (Scenes 1-4). Each task contains associated trial counts, randomization details, and evaluation rubrics.
22

---

Published as a conference paper at ICLR 2026
Scene Image
Task
Trials
Randomization
Rubric
I: moves towards the drawer; 2: closes drawer
close the drawer
4
top, two bottom drawer
put the stapler the notebook
put stapler in higher and lower position on the table
I: picks stapler; 2: puts pler on notebook
put stapler higher and lower position on the table, open and bottom drawer
I: picks up the stapler; 2: puts stapler the drawer
stapler in the drawer
1: pick 2: perform wiping motions on the white- board, the smiley
initial eraser position the left and right of whiteboard
clean the whiteboard
2
swap initial position of marker and cup, two local modifications each
I: picks marker; 1.5: rotates to put marker roughly up- right position; 2: puts marker in cup
put the marker in the cup
Table 4: DROID Generalist evaluation tasks grouped by scene (Scenes 5-8). Each task contains associated trial counts, randomization details, and evaluation rubrics.
We utilize [3x end effector (EEF) Cartesian position, 3x EEF rotation (roll/pitch/yaw), 1x gripper position] both our states and actions. Similar to the DROID finetuning dataset, each step in our datasets consists of I base image, wrist image, 7D state, 7D action, and the language instruction. As described earlier, we use the norm-stats of our pretraining dataset for normalizing states and actions during both training and inference. The base and wrist camera images go through several transforms (random crop, resizing to 224x224, and color jitter) during training.
The finetuning is performed with an action horizon of 50 environment steps, and the actions are padded to be 32-dimensional, thus the policy learns to output action chunks of shape (50, 32).
A.7.3 LIBERO EVALUATION DETAILS
We use the LIBERO simulator for evaluation. Each ID eval conducted for episodes. Each OOD eval is conducted with 5 seeds, 10 episodes/seed. 21 variations we test for each the tasks.
and 23 show the 3 types of OOD
The Generalist evals consists of 20 tasks, each from LIBERO-object, LIBERO-spatial, LIBERO- goal, and LIBERO-90, and each task is tested for 10 episodes. Table 6 highlights few tasks per LIBERO eval suite that use in generalist evals.
The seed controls OOD randomization such as translation of the objects, spawning random distrac- tors, etc. We use the same language instruction training during evaluation, resize images to 224x224, and use the same statc/action spaces.
During the evaluations, dimensional policy prediction. We let the simulator step for 10 steps before starting execution. The open loop horizon is set to 5, and follows similar pattern the DROID evals.
extract the 7D actions by taking the first elements from each 32-
The evaluation criteria for all libero evals are for failure, for success, as determined by the simulator environment.
23

---

Published as a conference paper at ICLR 2026
Scene Image
Task
Trials
Randomization
Rubric
any configurations the black sponge not starting in the blue bowl
put the black sponge in the blue bowl
I: picks up object: 2: puts ob- joct location
2
two configurations where rod bottle not starting in the black bowl
put the red bottle the black bowl
I: picks up object: 2: puts ob- ject in correct location
any two configurations where the watermelon starting the purple howl
put the watermelon in the purple bowl
I: picks up object; 2: puts ob- ject location
2
move the watermelon from purple bowl to the blue bowl.
two configurations where the watermelon starts the pur- ple bowl
1: picks up object: 2: puts ob- ject in correct location
2
any two configurations where the tape is not starting in the purple bowl
put the tape in the purple bowl
I: picks up object; 2: puts ob- ject location
2
waterbottle starts differ- ent positions on left side of the table
put waterbottle the left side of the table
picks up object; 2: puts joct in correct location
2
Table 5: DROID Generalist evaluation tasks grouped by scene (Scene 9). Each task contains asso- ciated trial counts, randomization details, and evaluation rubrics.
24

---

Published as a conference paper at ICLR 2026
Figure 20: Three LIBERO tasks we use for finetuning: pot-on-stove mugs-on-plates, and -itemm-intt-baee
Figure 21: Three types of out-of-distribution variation of the LIBERO mugs task. The three dif- ferent type are: (1) small translation object positions, (2) big translation to object position and additional distractors, and (3) background change.
Figure 22: Three types of out-of out-of-distribution variation of the LIBERO items-into-bast
task. The three different type are: (1) small translation to object positions, (2) big translation to object position and additional distractors, and (3) background change and additional distractors.
25

---

Published as a conference paper at ICLR 2026
Scene Image
Task
LIBERO Eval Suite
put the bowl on the plate
LIBERO-goal
pick up the alphabet soup and place it in the basket
LIBERO-object
pick up the black bowl from table center and place it on the plate
LIBERO-spatial
put white bowl on plate
LIBERO-90
Table 6: Sample of LIBERO Generalist Evaluations for each evaluation suite.
26

---

Published as a conference paper at ICLR 2026
Figure 23: Three types of iitoo--ddittrbutionn variation of the LIBERO pot on stove task. The three different type are: (I) small translation to object positions, (2) big translation object position and additional distractors, and (3) background change.
A.8 HYPERPARAMETERS
A.8.1 CHOOSING MERGING COEFFICIENT &
For each task that we consider (e.g. mugs-on-plate in LIBERO, plates in DROID), we test policy performance on several scenes, which are different variations of the same task with different objects, distractors, and backgrounds etc. To choose what values of a we use, we use one OOD scene as the "validation" scene, and tune the hyperparameter for best perfor- mance on that validation scene. Then, we use the rest of the OOD scenes as the "test" scenes, and report the performance of all methods only the test scenes. In DROID experiments, we only tune a E {0.25, 0.5, 0.75) on validation scene, while in LIBERO experiments we tune E {0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1} since it is cheap to do We find that DROID a = 0.5 typically tends to perform well across tasks.
A.8.2 CHOOSING NUMBER OF GRADIENT STEPS
For choosing how long the task-FT and co-FT should go on for, we evaluate checkpoints ID evaluation to assess how well they fit the data. We then pick the earliest checkpoint that achieves maximal performance, to get checkpoint that learns the target task well but does not overfit to fientuning dataset, so that it is the strongest baseline.
A.8.3 REAL-WORLD EXPERIMENTS' HYPERPARAMETERS
Below are tables specifying the hyperparameters we finalized upon for each of our finetuning runs for real-world experiments.
Hyperparameter
Value
Batch Size
32
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
3e-5
End-LR
2e-6
Warmup-Steps
100
Decay Steps
1000
Gradient Steps
500
Weight Decay
le-10
Optimizer
Adam(b1=0.9, b2:00.9 eps=le-8)
Clip Gradient Norm
1.0
Table 7: Training hyperparameters for task-FT on DROID whiteboard.
27

---

Published as a conference paper at ICLR 2026
Hyperparameter
Value
Batch Size
32
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
3e-5
End-LR
2e-6
Warmup-Steps
1000
Decay Steps
10000
Gradient Steps
9999
Weight Decay
lc-10
Optimizer
Adam(bl=0.9, b2=0.95, eps=le-8)
Clip Gradient Norm
1.0
Cotraining-Me
80% task, 20% pretrain
Table 8: Training hyperparameters for co-FT DROID whiteboard.
Hyperparameter
Value
Batch Size
32
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
3e-5
End-LR
2e-6
Warmup-Step
500
Decay Steps
5000
Gradient Steps
1500
Weight Decay
le-10
Optimizer
Adam(b1=0.9, b2-00000900..660005..... eps=le-8)
Clip Gradient Norm
1.0
Table 9: Training hyperparameters for task-FT on DROID plates.
Hyperparameter
Value
Batch Size
32
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
3e-5
End-LR
2e-6
Warmup-Steps
1000
Decay Steps
10000
Gradient Steps
5000
Weight Decay
le-10
Optimizer
Adam(b1=0.9, b2=0.95, eps=le-8)
Clip Gradient Norm
1.0
Cotraining-MMi
80% task. 20% pretrain
Table 10: Training hyperparameters for co-FT on DROID plates.
Hyperparameter
Value
Merging Weight for task-FT
75% task-FT, 25% base model
Merging Weight for co-FT
50% task-FT, 50% base model
Table 11: Merging hyperparameters for DROID whiteboard.
Hyperparameter
Value
Merging Weight for task-FT Merging Weight for co-FT
50% task-FT, 50% base model
50% task-FT, 50% base model
Table 12: Merging hyperparameters for DROID plates.
28

---

Published as a conference paper at ICLR 2026
Finally, in continual learning experiment, exactly the same hyperparameters as those used for plates-co-F found in [0] just applied sequentially twice to first cotraining on the plates dataset, then on the whiteboard. In the continual learning setup, the Merging weight is also always fixed at 50% task-FT, 50% base model.
A.8.4 SIMULATION EXPERIMENTS' HYPERPARAMETERS
In order to perform our simulation experiments, had to first perform round of pretraining 117 LIBERO tasks, as described earlier. Here are the hyperparamters for this pretraining. We back- tested various checkpoints of this pretraining on the entire libero suite, and settled on step 10,000 as being a good candidate to serve as a base model, as it performed the best on both seen and unseen libero tasks.
Hyperparameter
Value
Batch Size
64
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
2.5e-5
End-LR
2.5e-6
Warmup-Steps
1000
Decay Steps
30000
Gradient Steps
10000
Weight Decay
le- 10
Optimizer
Adam(b1=0.9, b2=0.95,, eps=le-8)
Clip Gradient Norm
1.0
Table 13: Hyperparameters for LIBERO pretraining.
29

---

Published as a conference paper at ICLR 2026
Both task-FT and co-FT all 3 libero finetunting tasks the set of hyperparameters, pro- vided below. We determine the number gradient steps to choose by sweeping over all taken checkpoints, evaluating them ID and OOD, and picking the best performing ones.
Hyperparameter
Value
Batch Size
64
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
2.5e-5
End-LR
2.5e-6
Warmup-Steps
1000
Decay Steps
30000
Gradient Steps
500 (items-into-base) pot-on-stove), 1000 (mugs)
Weight Decay
le-10
Optimizer
Adam(b1=0.9, b2=0.95, eps=le-8)
Clip Gradient Norm
1.0
Table 14: Training hyperparameters for task-FT on LIBERO-i DoBERO-items-into-aallee mugs,
pot-on-stove.
Hyperparameter
Value
Batch Size
64
Learning Rate Schedule
Linear Warmup with Cosine Decay
Peak-LR
2.5e-5
End-LR
2.5e-6
Warmup-Stery
1000
Decay Steps
30000
Gradient Steps
1000 (items-into-basket, pot-on-stove, mugs)
Weight Decay
le-10
Optimizer
Adam(b1=0.9, b2=0.95, eps=le-8))
Clip Gradient Norm
1.0
Cotraining-Mix
50% task, 50% pretrain
Table 15: Training hyperparameters for co-FT on LIBERO-itemm-intt-bbbskke mugs,
pot-on-stove.
30

---

Published as a conference paper at ICLR 2026
As explained earlier and similar to our DROID procedure, after checking all checkpoints and picking the best-performing one, we then apply RETAIN on it to enhance its performance on OOD and check various merging coefficients in the range [0.0, 1.0], and
Generalist evals. In simulation, after doing so, here are our final merging parameters.
Hyperparameter
Value
Merging Weight for task-FT
90% task-FT, 10% base model
Merging Weight for co-FT
90% task-FT, 10% base model
Table 16: Merging hyperparameters for LIBERO t.ems-i into-bakee..
Hyperparameter
Value
Merging Weight for task-FT
80% task-FT, 20% base model
Merging Weight for co-FT
90% task-FT, 10% base model
Table 17: Merging hyperparameters for LIBERO mugs
Hyperparameter
Value
Merging Weight for task-FT
90% task-FT, 10% base model
Merging Weight for co-FT
70% task-FT, 30% base model
Table 18: Merging hyperparameters for LIBERO pot-on-stove.
31

---

Published as a conference paper at ICLR 2026
A.9 DETAILS ON BASELINE METHODS
Here we provide addtional details on the baseline methods we compare against.
Task-FT We fine-tune all parameters of the base policy according to the behavioral cloning loss in Eq. D.. All data are sampled from the fine-tuning dataset.
Co-FT We fine-tune parameters using the behavioral cloning loss, and each update batch sampled from both the pretraining dataset and the finetuning dataset with fixed weight. See Section A.8 for specific weight values we use for different tasks.
LoRA LoRA (low rank adaptation) freezes all the weights of the base pretrained policy, and finetunes an adapter head with a low rank bottleneck. Typically the adapter head has much fewer parameters than the base pretrained model. The resulting policy achieved by adding the weights of the frozen pretrained policy and the low rank adapter head.
Freeze-FT Similar to Task-FT, but we freeze the parameters in the language model backbone and finetune only parameters from the action expert and vision encoder.
Scratch Training policy from sratch. To make it comparable to the other VLA baseline policies, use the same лo architecture but initialize from the Paligemma VLM weights, without pretraining on any robot data.
A.10 WHY DOES RETAIN WORK SO WELL?
While we have shown empirically in this work that RETAIN works well across real and simulated tasks, we don't understand the full scope of the reasons why model parameter merging works so well empirically. However, previous work in computer vision and Largelanguage models have also shown empirical benefits of merging parameters of the pretrained and fine-tuned model (see Section 2) Similar to our work, these previous works are also largely empirical and corroborate our findings in real-world robotics setting. Specifically, Neyshabur et al. (2020) found that fine-tuning from the same pretrained model results in regions where solutions are connected by a linear path along which error remains low, a phenomenon known as "linear mode connectivity" [2]. [3] and [4] explained that SGD typically converges to a solution that is on the boundary of this low-error path, while weight merging is able find point centered in this region, which often has slightly worse train loss but substantially better test error. We attribute the performance gains we see also to this, though call for more rigorous future work to explain this more rigorously.
A.11 QUALITATIVE ANALYSIS OF SUCCESS AND FAILURE MODE OF RETAIN
Typically, we observe that RETAIN improves the robustness of the merged policy OOD evalua- tions. Compared to task-FT policies, which are brittle and will fail out-of disstribbuionn scenarios catastrophically and is unable retry, the RETAIN policies typically exhibits more robust behavior, and can recover from failure using its generalist knowledge. Typically, we observe that the task-FT policy either either does the full task successfully, or cannot do the task all. In comparison, the RETAIN policies usually at least partially complete the task. However, we do observe that the RE- TAIN policies sometimes fail due to (1) imprecise execution of the task and stuck constant retry mode and (2) produces an action that does not solve the task (though still semantically meaningful), and is unable to successfully continue afterwards. We provide two qualitative examples of this in Fig. 24 and Fig. 25
32

---

Published as a conference paper at ICLR 2026
Figure 24: Failure Example of RETAIN in DROID whit eebaaddb task: The arm picks up the eraser, but drops it on the whiteboard instead of wiping left and right with it.
Figure 25: Failure Example of RETAIN in DROID plates task: the arm is not able to precisely pick up the green plate, and so constantly retries this until the policy times out.
33


Reviews:
[{"review_id": "ZepVIoY8yd", "reviewer": "Reviewer_R6dX", "summary": "The paper proposes a very simple strategy for finetuning of policies by merging model parameters of pretrained models and finetuned models. The work presents empirical results on Droid robot setup and Libero simulations and find a surprising effectivity of this method in achieving task performance in the fine tuned task while retaining performance on the original (generalists) tasks as well as generalization to variations of the fine-tuning task.", "questions": "I know that your paper is not trying to give a deeper understanding why the simple model merging does the right thing. Nevertheless, I would like to know if you have analyzed the following:\n- we do a linear interpolation in weight space. \n- if the fine-tuning training path in weight space would be more or less linear, then simply training with a lower learning rate or less steps should have the same effect. \nHere the actual questions:\nQ1a: Have you ablated for learning rate / number of updates steps. \nQ1b: I think a non-linearity check for the gradient path would be interesting: compare the vector of $\\theta_0$ to $\\theta_{n/2}$ with $\\theta_{n/2}$ to $\\theta_{n}$ in terms of collinearity. \n\nQ2: The results in the paper appear cherry picked in comparison to all results, e.g. presented in Fig. 12.\nFor those tasks, Co-FT and RETAIN-Co-FT are very similar here (sometimes one or the other wins). \nI suggest creating a more balanced representation in the main paper and tone down your claims. \n\nQ3: Sec. 6.5: How was the $\\alpha$ parameter selected?  I understood that $\\alpha$ (I guess the same for all modalities) is selected based on the best OOD test performance. This is against the Machine Learning practice! Never select hyperparameters on the test set! Maybe I am mistaken here, please clarify.\n\nQ4: Which $\\alpha$ parameter did you use for the DROID experiments? How was it selected?\nAlso, did you use different $\\alpha$s for the modalities in the experiments, and how did you select this weighting?\n\nQ5: How does alpha depend on training parameters (how many updates) etc?\n\nQ6: Sec 6.6: What would happen if you finetune on both / all sequential tasks at once. As you do Co-FT, you anyway have all the data available, so I think that would be a good comparison / upper baseline. \n\nQ7: for the sequential learning, what would you expect if one finetunes the base model on both tasks and then merges both together, as a weighted sum. This would allow parallel improvement and sounds like an interesting use case. \n\nComments:\n- You have one or three additional hyperparameters (see Q4), and depending on their selection, I am not so surprised to see that you can find a setting that works well in your experiments, the question is: will these hp settings also work for a new environment?\n\nRelated work: also a paper that looks into finetuning from few demonstrations:\nActive Fine-Tuning of Multi-Task Policies, Bagatella et al\nThis paper does not do model merging but learned output merging and might be interesting to related to.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- simple method\n- extensive experiments", "weaknesses": "- no deeper understanding\n- same unclear results and overly strong claims\n- questionable hyperparameter selection\n\nBecause the method is so simple and not really clear, from a fundamental perspective, why it should have the presented favorable properties, extra care has to be taken in conducting the experiments and interpreting the results. See my questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FedaUDhHj4", "reviewer": "Reviewer_Jiug", "summary": "This paper addresses the overfitting of generalist robot policies during small-scale dataset finetuning, a process that compromises pre-trained knowledge and inhibits generalization to OOD tasks. The proposed method, RETAIN, mitigates this by merging the parameter spaces of the pre-trained and finetuned policies via linear interpolation. Experimental results indicate that RETAIN achieves robust generalization to novel tasks while preserving broad, general capabilities. The paper also provides a comparative analysis against prior strategies, offering specific insights into weight merging with different parameter groups.", "questions": "- For weakness 1: What is the recommended procedure for selecting $\\alpha$ in a practical? Can $\\alpha$ be estimated from the data?\n- For weakness 2: Was the modality-specific merging analysis (i.e., merging only LLM parameters) validated on the real-world DROID tasks? Does the same finding hold?\n- For weakness 3: Do the authors have a hypothesis for the anomalous co-FT result in the continual learning experiment (Fig 10)? Could this be due to a specific relationship between the $\\textit{plates}$ and $\\textit{whiteboard}$ tasks?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The paper addresses a critical and practical problem in robotics: how to adapt generalist policies to new tasks with limited data without overfitting or forgetting.\n- The proposed method, RETAIN, is simple and efficient, relying on parameter interpolation with no added inference cost.\n- The authors provide a valuable comparative analysis against several prior finetuning strategies, offering significant insights to the research community.\n- The evaluation protocol is clear and effectively highlights the core problem.", "weaknesses": "1. The selection of the $\\alpha$ merging coefficient is unclear and appears to be highly task-dependent (from Appendix A.5). The paper does not offer a methodology or heuristic for tuning $\\alpha$ based on specific task characteristics.\n2. The insightful analysis on modality-specific merging (Sec 6.5) seems limited to simulation experiments. It is unclear if the conclusion that only LLM parameters need merging holds for the more complex, real-world DROID tasks.\n3. The continual learning claims (Sec 6.6) are partially undermined by an unexplained anomaly in Figure 10, where the co-FT baseline achieves surprisingly high OOD performance on the second task, which the authors acknowledge but do not investigate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cXzb44gjcM", "reviewer": "Reviewer_mT9g", "summary": "This work focuses on VLA fine-tuning, and proposes model merging to induce robustness and mitigate forgetting. Standard fine-tuning on reasonably-sized single-task datasets (i.e. ~100 demonstrations) are known to improve performance on the specific tasks, but degrade it elsewhere (as confirmed by the authors in Figure 2); standard solutions (such as co-training or LORA) tend to only marginally alleviate this issue, or require access to the pre-training dataset. The authors propose a simple linear interpolation strategy between pre-trained an fine-tuned weights, which may optionally specialize to the three sub-networks (vision encoder, language backbone and action head). This technique can be applied in a single fine-tuning round, or over multiple rounds in a continual learning setting. Authors claim that model merging preserves the generalist performance of the VLA, while also boosting OOD performance on the fine-tuning tasks. Their claims are supported by experiments spanning real (DROID) and simulated (LIBERO) environments. The authors include a straightforward ablation of the interpolation parameter $\\alpha$.", "questions": "### Minor issues\n- Model merging is not an entirely novel idea, as the authors acknowledge in the related works. I would suggest adding a couple of references in the introduction as well, which does not point to previous work in model merging. This is not a major weakness, as the authors acknowledge existing works, and focus on empirical evaluations in a multi-modal setting, which is to the best of my knowledge under investigated.\n- Line 143: dataset tuples should be $(s_t^{(i)}, a_t^{(i)}, T^{i})$ to be consistent with the notation in Equation 1.\n- Line 196 is later contradicted: the model does not improve monotonically the longer it is trained, even ID.\n- \"mugs\" is not formatted correctly in the caption of Table 14.\n\n### Questions\n- Did the authors consider regularization to the pre-trained model (i.e. distillation) as an option to prevent model drift? This is for instance done to prevent response collapse when fine-tuning LLMs.\n- What do error bars represent in Figures 6 and 7?\n\n### Conclusion\nOverall, the paper makes an interesting contribution to an open problem, and I currently recommend acceptance. My main concerns revolve around the experimental settings, and the modest gain in performance with respect to Co-FT. These concerns are outweighed by the fact that (a) experiments are well described, and thus different training regimes can be explored in future work and (b) the strongest baselines require access to pre-training data, while model merging does not. Nonetheless, clarifying whether the training regimes considered are already designed to avoid as much overfitting as possible would improve this work's significance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- Experimental evaluation is extensive, involving both simulated and real-world experiments. Moreover the experiments are described in great detail.\n- The problem tackled by this work is timely has clear practical interest.\n- This paper is written in a clear and understandable way.\n- The evaluation of different merging coefficient for different components is interesting on its own.", "weaknesses": "- The main weakness of this work is that it is not entirely clear whether the overfitting issue arises due to sub-optimal hyperparameter choices. Figure 2 shows that performance of baselines such as Co-FT fluctuates and decays for a large number of gradient steps: at 500 gradient steps both ID and OOD performance on the new task is high, and generalist performance has not decayed. The supplementary material details that the number of gradient steps was chosen for each algorithm/environment combination, but the tuning process is not clear.\n- The performance gains from model merging are often minor, and the method does not consistently beat baselines such as Co-FT (e.g. in continual learning settings). More importantly, these performance gains might be due to precise tuning of $\\alpha$, which is set on a per-task basis. It is unclear how well the method would perform out of the box, without extensive tuning of $\\alpha$, which might be prohibitive in real-world settings.\n\nA few minor issues will be detailed in the following section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xR8g8gkMOH", "reviewer": "Reviewer_Rx6M", "summary": "This paper addresses a critical challenge in robot learning: finetuning large, generalist Vision-Language-Action (VLA) policies on new tasks with limited demonstration data often leads to catastrophic forgetting of prior skills and poor generalization on the new task itself. The authors propose a surprisingly simple yet effective method called RETAIN (Robust finE-tuning with parameter mergINg). The core idea is to linearly interpolate the weights of the pretrained generalist policy and the policy after it has been finetuned on a small dataset for the new task. The authors evaluate RETAIN across both simulated (LIBERO) and real-world (DROID) environments, using a pretrained π₀-style policy. Their evaluation framework rigorously distinguishes between three settings: 1) in-distribution (ID) performance on the new task, 2) OOD performance on variations of the new task, and 3) performance on the original \"generalist\" tasks. The results demonstrate that RETAIN significantly improves OOD generalization on the new task compared to standard finetuning methods (Task-FT, Co-FT, LoRA) while successfully retaining the policy's original generalist capabilities. The paper further analyzes modality-specific merging, finding that merging the language model parameters is often most critical , and shows initial promise for using RETAIN in a sequential, continual learning setting.", "questions": "- The paper reports success rates but lacks a qualitative analysis of failure cases. When the RETAIN policy fails, how does its failure mode differ from the base policy or the fully finetuned policy? Does it produce more conservative, \"safer\" failures by reverting to the generalist model's behavior, or does it produce novel, unpredictable failures stemming from the weight interpolation? This analysis would provide valuable insight into the method's reliability.\n- How does the method perform after 5, 10, or N tasks? Does performance on earlier tasks degrade as more skills are merged? The proposed iterative merging process ($\\tilde{\\theta}_{n}=(1-\\alpha)\\cdot\\tilde{\\theta}_{n-1}+\\alpha\\cdot\\theta_{ft,n}$) might dilute the knowledge of the original pretrained model and early tasks over time.\n- A key finding from the STAR-Gen benchmark was that VLAs struggle significantly with semantic generalization (e.g., novel instructions for the same behavior). Does RETAIN improve robustness to linguistic variations, or does it primarily bolster visual robustness? How does the method handle changes to unobservable physical properties (e.g., mass, friction), another axis in STAR-Gen?\n- Does a larger or more diverse finetuning dataset generally permit a higher $\\alpha$ (leaning more towards the specialist model), whereas a smaller, narrower dataset benefits from a lower $\\alpha$ to better preserve the generalist priors?\n- Your choice of linear interpolation is compelling due to its simplicity. I am curious whether you experimented with or considered other parameter merging techniques from the broader deep learning literature. For example, methods like task arithmetic (where task vectors are computed and added/subtracted) have shown success in NLP. Was linear interpolation the first and most obvious choice, or did you find it superior to other, perhaps more complex, merging schemes for VLA policies?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The proposed method is exceptionally simple to implement, requiring only a weighted average of two model checkpoints. This lack of complexity is a major advantage for practical application, as it incurs no additional training or inference cost and can be easily integrated into existing workflows.\n- The paper tackles a highly relevant and pressing issue. As the field moves towards large foundation models, the ability to efficiently and robustly adapt them to new, specific tasks with minimal data is a key bottleneck for real-world deployment. The paper's motivation is strong and clearly articulated with empirical evidence of overfitting in standard methods (Figure 2).\n- The experimental design is a significant strength. The use of both simulation (LIBERO) and real-world robots (DROID) provides compelling evidence. The explicit differentiation between ID, OOD, and Generalist performance allows for a nuanced analysis that directly supports the paper's core claims about improving robustness and retaining prior knowledge. This evaluation structure is a practical application of the principles advocated by more formal taxonomies like STAR-Gen.\n- The finding that the LLM backbone parameters are the most influential for merging  provides a deeper insight into the functioning of VLAs. It suggests that the generalizable, abstract knowledge is heavily encoded in the language-centric weights, which aligns with the broader understanding that VLM pretraining is a primary source of semantic reasoning.", "weaknesses": "- While the paper demonstrates that parameter merging works, it offers limited insight into why it works so effectively in the non-convex loss landscape of deep neural networks. The approach is presented as an empirical observation. A more rigorous discussion, potentially drawing from literature on model soups, loss landscapes, and mode connectivity in NLP/CV, would significantly strengthen the paper. For instance, does merging find a wider, flatter minimum between the pretrained and finetuned solutions, which is known to correspond to better generalization? This lack of theoretical grounding leaves the method feeling more like a \"trick\" than a principled technique.\n- The merging coefficient $\\alpha$ is the method's key hyperparameter, and its selection is not adequately addressed. The authors state they swept over values to find the best-performing ones for their experiments. This is not practical for a real-world user who may not have the resources to perform extensive OOD evaluation to tune $\\alpha$ for every new task. How should a user select $\\alpha$ without access to a diverse OOD validation set? The paper does not propose a heuristic or a low-cost method for estimating a good value. The optimal $\\alpha$ values differ significantly between the LIBERO (often 0.8-0.9) and DROID (0.5-0.75) experiments. The paper attributes lower OOD improvement in LIBERO to a less capable base model, but does not explain why this would lead to a different optimal merging ratio. This suggests that the optimal $\\alpha$ may depend on the pretrained model's quality, the finetuning dataset size, and the task similarity, making it difficult to set a priori.\n- The paper claims that RETAIN \"enables continual acquisition of new skills in a lifelong learning setting\". However, the supporting experiment involves only two sequential tasks. This is insufficient evidence for robust continual learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)