Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts
Authors: Zhongjian Qiao
Rui Yang
Jiafei Lyu
Xiu Li
Zhongxiang Dai
Zhuoran Yang
Siyang Gao
Shuang Qiu
Venue: ICLR 2026 Poster (2026)
Keywords: Offline Reinforcement Learning; Cross-domain Policy Adaptation
Primary area: reinforcement learning
TL;DR: This paper aims to enhance the train-time and test-time robustness against dynamics shifts for cross-domain offline RL.

Abstract: Single-domain offline reinforcement learning (RL) often suffers from limited data coverage, while cross-domain offline RL handles this issue by leveraging additional data from other domains with dynamics shifts. However, existing studies primarily focus on train-time robustness (handling dynamics shifts from training data), neglecting the test-time robustness against dynamics perturbations when deployed in practical scenarios. In this paper, we investigate dual (both train-time and test-time) robustness against dynamics shifts in cross-domain offline RL. We first empirically show that the policy trained with cross-domain offline RL exhibits fragility under dynamics perturbations during evaluation, particularly when target domain data is limited. To address this, we introduce a novel robust cross-domain Bellman (RCB) operator, which enhances test-time robustness against dynamics perturbations while staying conservative to the out-of-distribution dynamics transitions, thus guaranteeing the train-time robustness. To further counteract potential value overestimation or underestimation caused by the RCB operator, we introduce two techniques, the dynamic value penalty and the Huber loss, into our framework, resulting in the practical Dual-RObust Cross-domain Offline RL (DROCO) algorithm. Extensive empirical results across various dynamics shift scenarios show that DROCO outperforms strong baselines and exhibits enhanced robustness to dynamics perturbations.

Published as a conference paper at ICLR 2026
DUAL-ROBUST CROSS-DOMAIN OFFLINE REIN- FORCEMENT LEARNING AGAINST DYNAMICS SHIFTS
Zhongjian Qiao', Rui Yang", Jiafei Lyu", Xiu Li", Zhongxiang Dai Zhuoran Yang". Siyang Shuangg Qiu 1 UHK, PIUC, Tsinghua University, 'Yale University, (SUH Tencent zhongqiao2 -cny cityu. hk, ciinangiucitty...o edu.
ABSTRACT
Single-domain offline reinforcement learning (RL) often suffers from limited data coverage, while cross-domain offline RI. handles this issue by leveraging addi- tional data from other domains with dynamics shifts. However, existing studies primarily focus on train-time robustness (handling dynamics shifts from training data), neglecting the test-time robustness against dynamics perturbations when deployed in practical scenarios. In this paper, we investigate dual (both train- time and test-time) robustness against dynamics shifts in cross-domain offline RL. We first empirically show that the policy trained with cross-domain offline RI. exhibits fragility under dynamics perturbations during evaluation, particu- larly when target domain data is limited. To address this, we introduce a novel robust cross-domain Bellman (RCB) operator, which enhances test-time robust- ness against dynamics perturbations while staying conservative to the out-of- distribution dynamics transitions, thus guaranteeing the train-time robustness. To further counteract potential value overestimation or underestimation caused by the RCB operator, we introduce two techniques, the dynamic value penalty and the Huber loss, into our framework, resulting in the practical Dual- Cross- domain Offline RI. (DROCO) algorithm. Extensive empirical results across var- ious dynamics shift scenarios show that DROCO outperforms strong baselines and exhibits enhanced robustness to dynamics perturbations. Code is available at https: /ttt://gtttta thub. com /ttp///22////OO../// git.
INTRODUCTION
Reinforcement learning (RL) (Sutton & Barto, 1999) has been a vital tool in various fields, such as embodied manipulation (Zakka et al., 2023; Jiao et al., 2025) and natural language process- ing (Ouyang et al., 2022; Rafailov et al., 2023). The success of typical RL often relies on numerous online interactions with the environment, which can be costly or even risky in the real world. Offline RL (Levine et al., 2020), instead, trains the policy with only a pre-logged offline dataset, eliminating the need for interactions with the environment. However, offline RL often struggles with a limited offline dataset. To address this, recent studies (Wen et al., 2024; Lyu et al., 2025; Liu et al., 2022) have explored -oomain Offline RI. In this setting, data from the target domain is limited, but have access to datasets from relevant but distinct domain (the source domain), which may tain sufficient offline data. The goal cross-domain offline RL is to utilize the datasets from both the source domain and the target domain to learn an effective policy for the target environment.
Although cross-domain offline RL is promising, simply merging the source domain dataset and tar- get domain dataset for policy training induces policy divergence and suboptimal performance (Wen et al., 2024). The issue stems from the dynamics mismatch: the transition dynamics of the source domain may differ from that of the target domain. Recent advances tackle this issue by learning domain classifiers to estimate the dynamics gap (Liu et al., 2022), or by filtering source domain data based on mutual information (Wen et al., 2024) or optimal transport (Lyu et al., 2025). These works focus on enhancing the train-time robustness of the policy against dynamics shifts, that is, handling the source-target dynamics mismatch. However, they overlook the occurrence of potential dynamics
"orrrsponning Author

---

Published as a conference paper at ICLR 2026
shifts during the deployment of the learned policy in real-world environments. For example, an RL policy for robotics manipulation is trained on data collected from a real robot (target domain data) and an imperfect simulator (source domain data). When the policy is deployed on the real robot, the robot's physical components may degrade over time, causing the transition dynamics to deviate from that observed in the target domain dataset. Consequently, the policy's 's performance may dete- riorate during deployment, highlighting the need for methods that ensure test-time robustness, that is, addressing the dynamics mismatch between the target and deployment environment.
In this paper, we initiate the investigation of dual (both train-time and test-time) robustness to dy- namics shifts in cross-domain offline RL. We first empirically show that with limited target domain data, the learned policy could be highly fragile to test-time dynamics shifts. To address this is- sue, we propose Dual-RObust Cross-domain Offline RI (DROCO), bringing new perspective on robustness specifically tailored for cross-domain offline RI, going beyond single-domain ro- bust RL (lyengar, 2005; Kuang et al., 2022). The core component of DROCO novel robust cross-domain Bellman (RCB) operator, which we theoretically prove enhances test-time robust- ness against dynamics perturbations while remaining conservative to the out-of-distribution (OOD) dynamics transitions (Liu et al., 2024a), thus guaranteeing train-time robustness. However, value overestimation or underestimation may occur when using the RCB operator. To mitigate this, we introduce two techniques, the dynamic value penalty and the Huber loss (Huber, 1973), to our frame- work, resulting in our practical DROCO algorithm. Our contributions are summarized as follows.
We empirically demonstrate the fragility of cross-domail offline RL to test-time dynamics shifts and initiate the study of dual robustness in this setting, contributing new perspectives to the field.
We introduce novel RCB operator that is theoretically proven to achieve dual robustness against dynamics shifts. We further introduce dynamic value penalty and Huber loss to mitigate value overestimation underestimation, yielding our practical algorithm, DROCO
Extensive experiments across diverse dynamics shift scenarios including kinematic and morphol- ogy shifts demonstrate that DROCO outperforms strong baselines and exhibits significant robust- ness against various test-time dynamics perturbations.
PRELIMINARIES
RL. We consider a Markov Decision Process (MDP) (Puterman, 1990) which is defined by the six-
tuple M (S, A, P. 7) where S is the state space, A is the action space, P S x A + ASS) S x A + [-rmax, mmax is the reward function, p is the initial state distribution, and y is the discount factor. The objective of RL is learn a policy æ SS + ((A) that maximizes the expected discounted cumulative
is the transition dynamics, A(-) is the probability simplex, r(s,a)
return E2 Ei= >'r(st, d.)]. We define Q" a) == Ex [Er r"((st, ar][so s, ag = a] and (T(s) Ea-T(-la) (Q*(s.
Cross-Domain RL. In cross-domain RL, we have access to a source domain MDP Mac (S. A. Pae, r. P. 7) and targer domain MDP Mur = (S. Pur.r, r.).)). The only difference between the two domains is the transition dynamics, as considered by previous works (Wen et al., 2024; Lyu et al., 2025; Qiao et al., 2025b). In the offline setting, only a target domain dataset Dur and source domain dataset Dan: are available. We aim to leverage the mixed dataset Dare U Danr to learn a well-performing agent in the target domain.
Enhancing Robustness in RL. Robust RL aims to optimize the worst-case policy performance to enhance the robustness against environmental perturbations. Different from standard RL, robust RL applies the following robust Bellman operator for Bellman backup:
Trodmsst ((, a) = r(s, a) + inf Ex'PPa(-lsa max Q(s', a')
MEM,
E'EA
where M. is the dynamics uncertainty set under some distance metric. If we choose Wasserstein distance (Villani et al., 2008) as the distance metric, then M, is the Wasserstein uncertainty set: (I)
M. = +M W (PM(11s, a), PM(((s,)) a)) Se).
a)) = inf Pa)) Kg [d(s ازله is the Wasserstein dis- tance between P-r(- a) and P (- a). I(. is the joint distribution, and d(.:.) +) is an element- wise distance metric such the Fuclidean distance.
where W
2

---

Published as a conference paper at ICLR 2026
3 Is CROSS-DOMAIN OFFLINE RL SENSITIVE TO TEST- DYNAMICS PERTURBATIONS?
To motivate our approach. we conduct an empirical study on the sensitivity of cross-domain offline RL to lest-time perturbations. Our key finding is that cross-domain offline RL could be highly sensitive to test-time dynamics perturbations, especially when limited target domain data given. Therefore, enhancing test-time robustness is crucial for cross-domain offline RL.
We adopt the hopper-v2 task from Mu- JoCo (Todorov et al., 2012) as our target do- main, and the full-size hopper-expert- dataset from D4RI. (Fu et al., 2020) as the target domain dataset. To simulate dynamics shifts in the source domain, create modi- fied hopper-v2 environment with kinematic shifts (called hopper-kinematies. by constraining the robot's joint rotation range. For the source domain dataset, train an expert-level SAC (Haarnoja et al., 2018) policy and collect 1M samples in environment with it. To examine the test-time robustness of cross-domain offline RI, we first train policy using IGDF (Wen et al., 2024) on the full-size source and target domain datasets for IM steps. We then evaluate the trained policy under four conditions: (1) the original target euvironment (clean), and (2-4) kinematic perturbations with three levels (easy, medium, hard) following Lyu et al. (2024b). As shown by the blue curve in Figure 1, the policy demonstrates vulnerability to intense dynamics shifts, with performance degradation of 40.9% (medium) and 72.4% (hard) compared to the clean environment.
hopper-expert-v
100% D4RL
100
10% D4RL
Normalized Score
17.9%
4. 40.99%
1'5..2%
72.4%
4..7%%
B6.5%
cepper-kinemation
easy
medium
Shift Level
Performance comparison with different dataset sizes under dynamics perturbations.
Figure 1:
To better mimic the challenges when target domain data is limited in cross-domain offline RL, construct a reduced target domain dataset by sampling only 10% of the dataset. Our experiments reveal that the policy trained with this limited target data (while retaining full source domain data) is significantly more vulnerable to dynamics perturbations. As illustrated by the orange curve Figure 1, performance degradation intensifies across all shift levels compared to the full-data case, demonstrating substantially reduced test-time robustness.
We attribute this phenomenon to the discrepancy between the true dynamics and the observed dy- namics in the target domain dataset, whose magnitude inversely correlates with the dataset size. This discrepancy causes the policy to overfit to the dataset dynamies, thereby reducing its robustness to dynamics perturbations. These results highlight the necessity of enhancing test-time robustness against dynamics shifts for cross-domain offline RL, which we address in the following section.
4 DUAL-ROBUST
OFFLINE RL
In this section, we present dual-robust cross-domain offline RL. Inspired by Liu et al. (2024c), we first define the robust cross-domain Bellman (RCB) operator and additionally provide practical version of it. We then show that dual-robustness can be achieved by applying the RCB operator solely on the source domain data. Finally, we present our practical algorithm, DROCO.
4.1 ROBUST
BELLMAN OPERATOR
Definition 4.1 (RCB operator). The robust cross-domain Bellman (RCB) operator TRCB is defined
as
+ E. E
max d'ma(tler
Q(s',a))
Q(s', .'))
if M -Mar
TRCBQ= =
(2)
r r MEM. EainPra
max Q(s a(s),a') a'mp(t)ss)
if M = -Mrrs
3

---

Published as a conference paper at ICLR 2026
where p.-l--) is the behavior policy, and max Q a') denotes taking maximum ac- lions in the support of ju(1 (((-))8), i.e., maxa'EAst. p(a")=)>> Q(s', ,')'
In Equation 2, we assume that the source and target domain datasets share the same behavior policy ü. following Wen al. (2024). Note that this assumption is only for notational simplicity. Even does not hold, we could replace je with the respective behavior policies without affecting our analysis. The basic idea behind the RCB operator is that if (s, s') comes from the target domain dataset, we use the standard in-sample Bellman operator (Kostrikov et al., 2021; Xu et al., 2023) for backup enhance the performance; if the data are sampled from the source domain dataset, apply the in-sample robust Bellman operator (which integrates in-sample learning into the robust Bellman operator) to achieve dual robustness dynamics shifts, which we will discuss later.
We now characterize the dynamic programming property of the RCB operator and give the following proposition. All proofs are deferred Appendix B.
Proposition 4.1 (y-contraction). The RCB operator is "-connraactonn operator in the complete state-action space (IR (SxAA, -I- where || lloe denotes the lxx norm, i.e., 11 -2RCBQ211 S 7l1Q1 - Q2 for any Q-functions Q1 and Q2.
Proposition 4.1 presents that the RCB operator is a 7-contraction in the tabular MDP setting. How- ever, directly applying the RCB operator for backup is unrealistic, since we are not available the uncertainty set Me. given that the source environment is a black box. To handle this issue, we introduce the following dual reformulation of Equation 2 under the Wasserstein distance measure.
Proposition 4.2 (Dual Reformulation). Letting M, be the Wasserstein uncertainty set defined by Equation 1, then the term inf MeM. P sa maxx'~R(-1s)) Q(s',a')) in Equation 2 is equivalent
E Ey' inf max Q(5. a'). s.t. d(s',)) d'~((-18)
Proposition 4.2 provides a solution for transforming the intractable dynamics disturbance into the tractable state perturbations. Based on Proposition 4.2, we propose the practical RCB operator.
Definition 4.2 (Practical RCB operator). The practical RCB operator TRCB defined as
PPM
a(s',a)) d'mp(-lor)
i MMarr
R
r r+7BB infseu,(">)
max Q(5, a)) a ((pp)-1)))
M Marc
where - (B
S } is the state uncertainty set.
The key distinction between TRCB and TRCB lies in their Bellman target computation for source domain data. While TECB requires the dynamics uncertainty set M, that is typically unavailable, TREB solely relies on the state uncertainty set (s').. Since s' is observable in the source domain dataset, (s') can be constructed through noise perturbations of s'. This makes TRCB more feasible for Bellman backup than TECB, and the subsequent analyses are based on TRCB- Moreover, the following proposition shows that TRCB still possesses the same favorable property as TRCB, i.c., TRCB remains >ycontraction.
Proposition 4.3 (y-contraction). The practical RCB operator a y-contraction operator in the space (R |xxAl, || (lx)), iee., ||TRCaQI -TRc I <1ll1 Q2 D2 for any Qı and Q2.
4.2 DUAL ROBUSTNESS AGAINST DYNAMICS SHIFTS
In this section, we conduct a comprehensive analysis of both train-time and test-time robustness against dynamics shifts when employing the practical RCB operator. We first make the Lipschitz continuity assumption about the learned Q function, which is widely used in prior theoretical studies of RI. (Mao et al., 2024; Ran et al., 2023; Xiong et al., 2022; Liu et al., 2024b).
Assumption 4.1 (Lipschitz Q function), The learned Q function is Ko w.n.t state 8, i.e., Va E A, V81, 82 S, [Q(s1, Q(+2, a)) & KQ || 82ll.
4

---

Published as a conference paper at ICLR 2026
We then analyze the train-time robustness against dynamics shifts from source domain data. Stan- dard Bellman updates on source domain data might cause Q overestimation due to OOD dynamics issues (Liu et al., 2024a; Niu et al., 2022), necessitating conservative Q estimation for robust performance. Proposition 4.4 shows that the learned QRCB maintains bounded by applying TRCB- Proposition 4.4 (Train- time robustness against dynamics shifts), Assume that is chosen such that supp ( (: 8, (,)))
(f(se) for all (s, a, Mare) Dare. Then under Assumption 4.1, the learned Q function by applying TRCB satisfies:
Qp(s,a) 2ycKq
S Qucu(s, S a)(s,a). V(s,a) Dares
1-7
where Q7 is the Q function of optimal ji-supported policy' in the larget domain.
Proposition 4.4 suggests that, if a proper is chosen such that the uncertainty set (o(e covers the support of Piar(-ss, a), then the erroneous value overestimation will not occur, and the OOD dy- namics issue is mitigated. Thus, the train-time robustness against dynamics shifts is guaranteed. We then analyze the test-time robustness against the environmental dynamics perturbations. Let TRCB and VRCB be the policy and value function learned by applying the practical RCB operator, respec- tively. When the target environment undergoes dynamics perturbations (Pur(-) (-) - Ppe(()),, the value function of TRCB within perturbed dynamics Pper. denoted as Vppre, is bounded by Proposition 4.5. Proposition 4.5 (Test-time robustness against dynamics shifts). Assume that is chosen 5, s,))) S. (s(rer) for all (8, a, sere)
such that supp( |
Daro- As long as
W(Pper (- a). Prar(1|s, ()) S c, then for 4sg E Dare, we have
VEROB (so) 2 VRCB(SS)
(3)
where c max (c(Ua((tar) S Us(orr))) Kerr ~ P.nr(11s, a), (8, a, aac)) ~ Derc).
Proposition 4.5 gives that, for any disturbance with intensity below the threshold (measured in Wasserstein distance), the value of the learned policy TECh in Pper exceeds VRCB- This implies that for any initial state Su E Desc. TRCB achieves better performance under perturbed dynamics Pper than in the worst-case scenario, thereby improving dest-time robustness against dynamics shifts. Furthermore, Proposition 4.4 and Proposition 4.5 reveal that, (1) dual robustness can be achieved by solely applying the RCB operator the source domain data; (2) there a trade-off between the two robustness notions, which is controlled by More discussions can be found in Appendix A.
4.3 PRACTICAL ALGORITHM
In Section 4.1, formalize the practical RCB operator. Its application presents two key challenges: (1) determining the uncertainty set Us(s)) (2) computing the minimum Q value within this set. Although one can fix and adopt random sampling within U,( it lacks flexibility. In addition, if P... deviates far from Pa... then e would be too large, leading to overconservatism, compromising the performance. To address these limitations, we propose our practical algorithm, DROCO
Determining the uncertainty set via ensemble dynamics modeling, Instead of fixing c and ran- domly sampling from Us,e), DROCO first trains an ensemble dynamics model (Janner et al., 2019; Yu et al., 2020; Liu et al., 2024c) (.) {Po, (.)),, on Din via maximum likelihood estimation (MLE) simulate (-)
S.p (..... [log Po. (sl.s
i 11,,2,..,
(4)
then we use the ensemble prediction set X' = ......................... sx ~ (-l., a), (s, E Dace to approximate sampling from the uncertainty set. This replacement is motivated by two key insights: (1) dual robustness only requires the uncertainty set around support of (-r(- a) rather than & thus alleviating the unnecessary conservatism; (2) each ensemble member's prediction naturally serves as a sample from this uncertainty set. In this way, the practical RCB operator for source domain data becomes:
FRCB Q + inf
Q(5/ at) (((-)t)1))))
max
if M Mac.
(5)
(-18) is fi-supported if (als)) for any action that ((als) 0.
5

---

Published as a conference paper at ICLR 2026
However, the ensemble prediction set cannot cover the support of (-r(() required in Proposi- tion 4.4, such that the overestimation of Q value may still occur. Proposition 4.6 reveals that only limited overestimation would occur when applying Equation 5 as the Bellman target.
Proposition 4.6 (Limited overestimation). If sups,a Drv (Pa(( Pta (-1s,a) a)) se < 1o2 have
max Q(1/) ad) s Ses (11aa) (f(
a'~a(1l)) Q(5',a') +(1-(1-26) Tmax
inf {. Mep (1s,a)
max
Proposition 4.6 holds under the assumption that the prediction error of the dynamics model stays small, which is difficult to fulfill given that the target domain data is limited and the dynamics model tends to overfit. Moreover, value underestimation may also occur due to the infimum operator. Therefore, we introduce the following techniques for the underlying value estimation issue.
Tackling overestimation and underestimation. We adopt two techniques to address the value estimation issue: dynamic value penalty, and using Huber loss (Huber, 1973) for Bellman update.
Instead of directly using Equation for Bellman backup, we introduce a value penalty term:
u(s, a, s') [s' Psc(-(s,a))
max Q(s'.a') a') a'>p(-1e")
inf {x Pe, (1s,,)
max Q(5/, ot) Ld(a((1e)))
(6)
We then unify the source and target dynamics in the practical RCB operator, reformulating it as
TRCnQ(s, r(s, P
Q u(s,a, al>p(()s))
max
where M Mar or Mae and 8 serves a dynamic penalty coefficient that provides flexible control over value estimation. Specifically, we recover the practical RCB operator by setting / to 1.0, > 1.0 will increase the penalty to mitigate value overestimation, and B < 1.0 reduces the penalty to alleviate value underestimation. Although the dynamics model and value penalty are widely applied in offline RL (Yu al., 2020; Sun et al., 2023; Liu al., 2024c; Qiao et al., 2025a), our difference lies in the specific usage of the dynamics model and design of the penalty term.
Remark.
If we use IQL (Kostrikov et al., 2021) for policy optimization, then d's xe Vis'), and Equation 6 can be re-written as (aple>
u(s,a,s') (ss (s)(-)s,a)) (v(s))
[V($)]]
inf Po,
(8)
We note that Equation 8 resembles the value discrepancy term in VGDF (Xu et al., 2024), which V(s') E (1 (1s.a) [+()]- However, we extend this term by incorporating additional penalties for test-time dynamics shifts, whereas VGDF only addresses train-time dynamics shifts.
The second technique we adopt is the Huber loss (Huber, 1973), well-established technique for noise-resistant optimization (Yang et al., 2024b; Roy et al., 2021). We replace the regular &z loss in the Bellman update with the Huber loss:
Lg = ED, 1/8 (Q(s,a)) - TrenQ(s.aa) E.... [[((s.))TT((<.0)}]]
(9)
|a| with being the transition threshold and T being the standard Bellman operator for target domain data. Specifically, if severe value estimation error occurs such that (Q(s.a) T a)l > 8, the & loss would transition to &g loss to improve robustness against outliers. This technique helps mitigate value estimation error. The last step is to utilize offline RL algorithms such as IQL to optimize the policy as in other works (Lyu et al., 2025; Wen et al., 2024). We present the detailed pseudo-code of DROCO in Appendix D.2.
0..5a?
where ls(a) =
(8(|a) - 0.56), |a| 2 8

---

Published as a conference paper at ICLR 2026
Table 1: Evaluation Results with train-time kinematic shifts. half-halfchhetaah. hopp=hopper, walk=walker2d, m-medium, mc=medium-exye expert, mr=medium-replay e=expert. We report the nor- malized score evaluated in the target domain, and + captures the standard deviation across seeds.
Dataset
IQL*
CQL*
BOSA
DARA
IGDF
OTDF
DROCO (Ours)
half-m
45.2
37.7
39.6
44.1
4..
42.2++0.1
4.2
half-mr
22.1
23.6
26.3
21.6
22.9+1.4
15.6+3.1
26.9-:3.2
half-me
43.7
54.8
42.2
52.7
57.148.9
46.714.4
60.117.1
half-e
49.7
36.0
84.3
47.4
47.6+2.1
79.61:3.0
67.4
hopp-m
48.8
35.7
71.4
48.8
54.3#6.6
46.31:3.7
55.4+5.3
hopp-mr
40.2
43.2
29.5
41.6
30.0#5.2
26.2#4.44
47.3=7.00
hopp-me
12.5
49.6
17.0
11.6.:0.6
58.1+4.9
54.0+-6.4
hopp-e
62.6
47.9
94.8
59.1
70.143.2
97.0+3.3
89.3+99.6
walk-m
48.7
47.7
44.5
43.4
51.8+2,4
43.0+2.1
7.3
walk-mr
12.6
17.8
4.8
15.6
11,211111
10.7#119
27.7+3.00
walk-me
95.4
61.4
35.1
85.3
90.6:13.4
63.1
78.5
walk-e
90.1
83.8
41.9
85.5
93.7+45.8
98.9+2.1
106.0=0.8
ant-m
89.9
58.2
28.4
98.9
88.0
86.1#3.7
92.7=6.33
46.8
39.4
22.0
42.1
58.2.19.7
39.611.11
44.8
ant-me
106.1
100.6
102.5
104.8
112.814.0
105.1
119.0
ant-e
111.0
94.3
57.6
115.1
119.2:55.6
111.6
120.0
Total
925.4
789.9
774.5
923.0
964.3
969.8
1105.2
EXPERIMENTS
In this section, we conduct extensive experiments to examine our method. We aim to answer the following two questions: (1) Can DROCO outperform prior strong baselines across various train- time dynamics shifts and dataset qualities? (2) Can DROCO show enhanced robustness against test-time dynamics perturbations? We also test the parameter sensitivity of DROCO.
5.1 MAIN RESULTS
Experimental Settings. Following previous works (Lyu et al., 2025; Wen et al., 2024), We employ 4 MuJoCo (Todorov et al., 2012) tasks as source domains: ha fcheetah-v2, hopper-v2, walker2d-vv and ant-v3. For the target domain datasets, we utilize data qualities from D4RL (Fu et al., 2020) for each task: medium, medium-repa
medi -itiuum-eeptttt and expert, totaling target domain datasets. For the source domain, we introduce kinematic shifts and morphology shifts in the target domain setup. We collect source domain datasets with data qualities, resulting in a total of 32 (4[tasks] x 2[shiff ft types] x 4fdata qualis liaiiisss ties)) source domain datasets. Each pair of the source and target domain datasets shares the same task type (such as hopper-v2) and dataset quality (such as expert). More details about the tasks and datasets can be found in Appendix C.
Baselines. We consider the following baselines: IQL", CQL* (which train IQL (Kostrikov et al., 2021) and CQL (Kumar al., 2020) with the mixed dataset Duar U Pac),, BOSA (Liu et al., 2024a), DARA (Liu et al., 2022), IGDF (Wen et al., 2024) and OTDF (Lyu et al., 2025).
Results. We run each baseline and DROCO for 1M training steps over 5 random seeds, and report the results with train time kinematic shifts in Table (the results with morphology shifts are deferred to Appendix E.2). Note that our evaluation is under the clean target environment. Empirical results demonstrate that DROCO achieves superior performance in out of 16 tasks, outperforming all baselines. Furthermore, in terms of the total normalized score, DROCO achieves a remarkable 1105.2, significantly surpassing the second-best method, OTDF (969.8) 98), by 14.0%. We attribute the suboptimal performance of DROCO on the remaining datasets to its trade-off between performance and robustness, whereas other methods only consider performance. However, DROCO still achieves a competitive performance compared to other baselines on these datasets. These results indicate that DROCO exhibits superior train-time robustness against dynamics shifts.

---

Published as a conference paper at ICLR 2026
halfchhetah-kinemalieessperr
halfchhetta-kinemalieeessee
hallcheetakkinemali--sss
80
DROCO
80-
--- DIOCCO
80
DIOCO
70-
OTDF
70
OTDP
Normalizas Score
-7
OTDP
66
1GDF
1GDF
60
IEDF
Normalized
50
40
Nommalzzed
40
50
50
40
30
30
20
10-
10
casy
casy
Perturbation
(a) Kinematic shift
(b) Morphology shift
(c) min Q perturhation
Figure 2: Evaluation results under different types and levels dynamics perturbations.
hopper-kinematic-m
wallkerzo-kiemmatiomm
hopper-kinemation
wallker26-kiemmttiemm
Normalized Score
60
Noralize Soore
Normalized Score
80
6:30
8-10
60
40
6:30
0-10
6+5
80
6:00
6:50
40
Normalzze,
30
AR0.1
20
20
pro.5
20
8-0.5
B-1.0
p-1.0
10
0.0
0.5
B-1.2
1.0
0
0.5
f-12
1.0
0.0
0.5
1.0
%
0.5
1.0
166
168
Step
168
Step
1e6
(a) Effecct of
(b) Effect of
Figure 3: Parameter sensitivity experiments on / and 8.
5.2 EVALUATION UNDER DYNAMICS PERTURBATIONS
Experimental Settings. Test-time robustness can measured by the degree of performance degra- dation in the face of dynamic perturbations, compared to the clean environment. To examine the test-time robustness of DROCO, we introduce three kinds of dynamics perturbations during evalua- tion in the target environment: kinematic perturbations, morphology perturbations, min Q perturba- tions (Yang et al., 2022; 2024b). The first two perturbation types mirror the source domain dynamics shifts, each implemented at three intensity levels following (Lyu et al., 2024b); easy, medium, and hard. The third perturbation type represents an adversarial attack strategy that modifies dynamics by finding the state ş from Us(' that minimizes the value, i.e., arg min (st) Q(s, (T)), where is related to the perturbation scale. We consider DROCO to exhibit better test-time robust- ness if it demonstrates less performance degradation than the baselines under the same shift severity.
We evaluate our method against two baselines (IGDF and OTDF) under three perturbation types with varying intensity levels. Due space constraints, we only report results when the source domain dataset is wallfcheetere weth--iiemaattt www.eeaaatceeetet...eeemeem as illustrated in Figure 2. A wider range of evaluation can be found in Appendix E.3.
Results.
Our experiments demonstrate that DROCO exhibits superior robustness to dynamic perturbations compared baseline methods. Specifically, under easy-level kinematic shifts, DROCO shows only a 19.3% performance degradation (from 67.4 to 54.4). whereas both IGDF and OTDF suffer over 50% performance deterioration. We notice that DROCO displays greater sensitivity to morpho- logical perturbations than to kinematic perturbations, with a 42.1% performance decrease under easy-level morphological variations. We attribute this to the absence of morphology shifts in the source domain data, rendering the policy less adaptable to this unseen perturbation type. Neverthe- less, DROCO still outperforms both baselines: under the same conditions, OTDF and IGDF exhibit performance declines of 62.4% and 78.9% respectively. Notably, DROCO maintains consistent robustness against min Q perturbations across all scales. At the highest perturbation scale of 0.2, DROCO's performance decreases by 37.9% %, compared to 73.6% and 84.0% for OTDF and IGDE.
8

---

Published as a conference paper at ICLR 2026
5.3 PARAMETER SENSITIVITY
We examine the sensitivity of DROCO to the introduced hyperparameters. There are two main hyperparameterss in DROCO: the penalty coefficient / and the transition threshold 8.
Penalty Coefficient B. The parameter 3 controls the intensity of value penalty. A larger 3 leads to a stronger penalty and suppresses value overestimation, and vice versa. We sweep 3 across {0.1, 0.5, 1.0, 1.2] and show the experimental results with medi um datasets in Figure 3 (a). We observe that different tasks prefer distinct B. For example, setting 3 = 0.1 achieves the best perfor- mance for hopper peeer--kinmmat.. weweemattiieiiiiiieeee um, while walker2d kinemat i ceweeimeeeiieeeeeeee um prefers 1.0.
Transition Threshold 6. The parameter 5 determines when the lg loss turns to & loss for Bellman update. A larger 6 corresponds a more lenient transition condition. To test the effect of 6, we select 5 among {5, 10. 30, 50} and conduct experiments with medi dduumreppla datasets. The results in Figure (b) indicate that too small 8 (e.g., = 5) leads to inferior performance, while setting 8 = 30 achieves good performance. However, we find the optimal 8 varies across different tasks through additional experiments, and more discussions are provided in Appendix E.5.
Remark. Although different values of 3 and ô are preferred for different tasks (as shown in Ap- pendix E.5), we could still find some patterns across different tasks. We find that setting 3 < 1.0 works for most tasks, implying that value underestimation occurs more often due to the infimum operator. We also find that larger ó (30 and 50) is preferred for most tasks. We believe it is because the f2 loss is beneficial for training stability. Therefore, for new task, we could first try < 1.0 and 8 = 30 (or ā = 50). This could serve as guideline for finding the best hyperparameter.
RELATED WORK
Offline RL. In offline RL, a fixed dataset is provided, and no further interactions are allowed. As a result, conventional off-policy RL algorithms suffer from the extrapolation error due to OOD actions and exhibit poor performance (Kumar et al., 2020; Fujimoto et al., 2019). To address this challenge, various offline RL algorithms have been developed. Common solutions include incorporating policy constraints (Kumar et al., 2019; Fujimoto & Gu, 2021), learning a conservative value function (Ku- mar al., 2020; Lyu et al., 2022: Jin et al., 2021: Zhang et al., 2024), leveraging a dynamics model to facilitate policy learning (Yu et al., 2020: 2021; Qiao et al., 2025a; 2024; 2026), performing in- sample learning to avoid querying OOD actions (Kostrikov et al., 2021; Xu et al., 2023; Garg et al., 2023; Zhang et al., 2023), etc. However, these methods require that the offiine dataset contains large amount of data. In contrast, we focus on cross-domain offline RL, which relaxes the target data coverage requirement.
Cross Pross-Domain RL. Cross-domain RL (Niu et al., 2024) faces the challenge of domain mismatch, including observation mismatch (Yang et al., 2023), viewpoints mismatch (Liu et al., 2018; Sadeghi et al., 2018), and dynamics mismatch (Wen et al., 2024; Lyu et al., 2025; Xu et al., 2024; Niu al., 2022; 2023), etc. In this paper, we exclusively focus the dynamics mismatch. Previous studies handle this issue by adaptively penalizing Q value on source domain samples (Niu et al., 2022), capturing dynamics mismatch from a representation learning perspective (Lyu al., 2024a) and value discrepancy perspective (Xu et al., 2024), modifying the reward function in the source domain (Liu al., 2022: Eysenbach et al., 2020; Xue et al., 2023; Wang et al., 2024), etc. We focus on the offline setting. where the current works (Liu et al., 2022; 2024a; Wen et al., 2024; Lyu et al., 2025: Wang al., 2024: Yan et al., 2026) primarily consider the dynamics shifts from the source domain data, while we further consider the dynamics shifts from environmental perturbations.
Robust RL. Robust RL (lyengar, 2005; Xu & Mannor, 2010) aims to learn a policy resilient to environmental perturbations or data corruption. One line of research in robust RL focuses on train- time robustness against data corruption (Yang et al., 2024b; Zhang et al., 2021; 2022; Ye et al., 2023; Yang et al., 2024a; Xu et al., 2025), while another line addresses test-time robustness against environmental perturbations (Yang et al., 2022; Zhihe & Xu, 2023; Shi & Chi, 2024; Liu et al., 2024c). These works focus only single perspective of robustness terain-timeaa test-time) and the single-domain offline settings. For the cross-domaie setting, (He et al., 2025; Liu & Xu, 2024a; Van al., 2025) study robust cross-domain RI. but they are different from our work, since they still only

---

Published as a conference paper at ICLR 2026
consider aspect of robustness and focus the online setting. In contrast, work addresses the cross-domain offline setting and jointly considers both train- time and test-time robustness.
7 CONCLUSION
In this paper, we investigate the dual (train-time and test-time) robustness against dynamics shifts in cross-domain offline RI.. We propose novel RCB operator and theoretically demonstrate its ability of dual robustness. To further handle the potential value estimation error, we add a dynamic value penalty and use Huber loss for Bellman update, yielding our practical DROCO algorithm. Through extensive experiments across various dynamics shift scenarios, we show that DROCO outperforms prior strong baselines and exhibits strong robustness to dynamics perturbations.
ACKNOWLEDGEMENTS
This research was supported part by the Hong Kong Research Grants Council (GRF 11217925, GRF 16209124) and the National Science Foundation of China (Grant 72371214). The authors would also like to thank the anonymous reviewers for their valuable comments on our manuscript.
REFERENCES
Abien Fred Agarap. Deep learning using rectified arXiv: 1oXiv:/803.03355 2018.
linear units
(relu).
arXiv preprint
Benjamin Eysenbach, Swapnil Asawa, Shreyas Chaudhari, Sergey Levine, and Ruslan Salakhutdi- nov. Off-dynamics reinforcement learning: Training for transfer with domain classifiers. arXiv preprint arXiv:2006. 13916, 2020.
Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprini arXiv:2004.072/9, 2020.
Scott Fujimoto and Shixiang Shane Gu. minimalist approach to offline reinforcement learning.
Advances in neural information processing systems, 34:20132-20145, 2021.
Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, pp. 2052-2062. PMLR, 2019.
Divyansh Garg, Joey Hejna, Matthieu Geist, and Stefano Ermon. Extreme q-learning: Maxent rl
without entropy. arXiv preprint arXiv:2301.02328, 2023.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International confer- ence on machine learming, pp. 1861-1870. Pmir, 2018.
Yiting He, Zhishuai Liu, Weixin Wang, and Pan Xu. Sample complexity of distributionally bust off-dynamics reinforcement learning with online interaction. Forty-second International Conference Machine Learning, 2025.
Peter Huber. Robust regression: asymptotics, conjectures and monte carlo. The annals of statisties, pp. 799-821, 1973.
Garud N Iyengar. Robust dynamic programming. Mathematics of Operations Research, 30(2): 257-280, 2005.
Michael Janner. Justin Fu. Marvin Zhang, and Sergey Levine. When to trust your model: Model- based policy optimization. Advances neural information processing systems, 32, 2019.
Kechen Jiao, Zhirui Fang, Jiahao Liu, Bei Li, Qifan Wang, Xinyu Liu, Junhao Ruan, Zhongjian Qiao, Yifan Zhu, Yaxin Xu, et al. Tepo: Thought-centric preference optimization for effective embodied decision-making. Natural Language Processing, pp. 9585 9599, 2025.
Proceedings of the 2025 Conference on Empirical Methods in
10

---

Published as a conference paper at ICLR 2026
Ying Jin, Zhuoran Yang, and Zhaoran Wang. Is pessimism provably efficient for offline rl? In Proceedings of the 38th International Conference on Machine Learning, pp. 5084-5096, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 441266980, 2014.
Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
Lingkai Kong, Haichuan Wang, Tonghan Wang. Guojun Xiong, and Milind Tambe. Com- posite flow matching for reinforcement learning with shifted-dynamies data. arXiv preprint arXiv:2505.23062, 2025.
Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q- learning. arXiv preprint arXiv:21 10.06169, 2021.
Yufei Kuang, Míao Lu, Jie Wang, Qi Zhou, Bin Li, and Houqiang Li. Learning robust policy against disturbance in transition dynamics via ctate-conservative policy optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7247-7254, 2022.
Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off- off-policy q-learning via bootstrapping error reduction. Advances in neural information processing systems, 32, 2019.
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 13:1179-1191, 2020.
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tuto- rial, review, and perspectives on open problems. arXiv preprint arXiv:2005.0164) 2020.
Anqi Li, Dipendra Misra, Andrey Kolobov, and Ching-An Cheng. Survival instinct in offline rein- forcement learning. Advances in neural information processing systems, 36:62062-62122 2023.
Jinxin Liu, Hongyin Zhang, and Donglin Wang. Dara: Dynamics-aware reward augmentation in 2022.
offline reinforcement learning, arXiv preprini
Jinxin Liu, Ziqi Zhang, Zhenyu Wei, Zifeng Zhuang, Yachen Kang, Sibo Gai, and Donglin Wang- Beyond ood state actions: Supported cross-domales offline reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 1 13945-13953, 2024a.
Tenglong Liu, Yang Li, Yixing Lan, Hao Gao, Wei Pan, and Xin Xu. Adaptive advantage-guided policy regularization for offline reinforcement learning. In International Conference on Machine Learning, pp. 31400-31424. PMLR, 2024b.
Xiao-Yo Yin Liu, Xiao-Hu Zhou, Guotao Li, Hao Li. Mei-Jiang Gui, Tian- Yu Xiang, De- De-Xing Huang, and Gung-Guang Hou. Micro: modellbased offline reinforcement learning with a conservative bellman operator. In Proceedings of the Thirty-Thing International Joint Conference Artificial Intelligence, pp. 4587-4595, 2024c.
YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Imitation from observation: Learn- ing to imitate behaviors from raw video via context translation. In 2018 IEEE international con- ference robotics and automation (ICRA), pp. 1118-1125. IEEE, 2018.
Zhishuai Liu and Pan Xu. Distributionally robust off-dynamics reinforcement learning: Provable ef- ficiency with linear function approximation. In International Conference on Artificial Intelligence and Statistics, pp. 2719-2727. PMLR, 2024a.
Zhishuai Liu and Pan Xu. Minimax optimal and computationally efficient algorithms for distri- butionally robust offline reinforcement learning. Advances in Neural Information Processing Systems, 37:86602-86654 2024b.
Jiafei Lyu. Xiaoteng Ma, Xiu Li, and Zongqing Lu. Mildly conservative q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems, 35:1711-1724, 2022.
11

---

Published as a conference paper at ICLR 2026
Jiafei Lyu, Chenjia Bai, Jing-Wen Wen Yang, Zongqing Lu, and Xiu Li. Cross-domain policy adaptation by capturing representation mismatch. In International Conference on Machine Learning, pp- PMLR. 2024a.
Jiafei Lyu, Kang Xu, Jiacheng Xu, Mengbei Yan, Jingwen Yang, Zongzhang Zhang. Chenjia Bai, Zongqing Lu, and Xiu Li. Odrl: A benchmark for off-dynamics reinforcement learning. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024b. URL https: http:p::///p///e//eeeeee http///peeeeeeeeeeeeeeeeeee.eeee net/
Jiafei Lyu, Mengbei Yan, Zhongjian Qiao, Runze Liu, Xiaoteng Ma, Deheng Ye, Jing-Wen Yang, Zongqing Lu, and Xiu Li. Cross-domain offline policy adaptation with optimal transport and dataset constraint. In The Thirteenth International Conference on Learning Representations, 2025.
Yixiu Mao, Qi Wang. Yun Qu, Yuhang Jiang, and Xiangyang Ji. Doubly mild generalization for offline reinforcement learning, arXiv preprint arXiv:2411.079.0 2024.
Haoyi NIu, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu, Xianyuan Zhan, et al. When to trust your simulator: Dynamics- hybrid affline-and-onlin reinforcement learning. Advances in Neural Information Processing Systems, 15:36599-366122 2022.
Haoyi Niu, Tianying Ji, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu, and Xianyuan Zhan. H2o+: an improved framework for hybrid offline-and-online with dynamics gaps. arXiv preprint arXiv:2309. 12716, 2023.
Haoyi Niu, Jianming Hu, Guyue Zhou, and Xianyuan Zhan. A comprehensive survey of cross- domain policy transfer for embodied agents. arXiv preprint arXiv:2402.04588, 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Kuan-Chen Pan, MingHong Chen, Xi Liu, and Ping-Chun Hsich. Survive on planet pandora: Robust cross-domain rl under distinct state-action representations. In ICML 2024 Workshop: Aligning Reinforcement Learning Experimentalists and Theorists
Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications data science. Foundations and Trends@ in Machine Learning, 11(5-6))355-607, 2019.
Martin L. Puterman, Markov decision processes. Handbooks in operations research and management science, 2:331-434, 1990.
Zhongjian Qiao, Jiafei Lyu, and Xiu Li. Mind the model, not the agent: The primacy bias in model- based rl. In ECA/ 2024, pp. 1824-1831. IOS Press, 2024.
Zhongjian Qiao, Jiafei Lyu, Kechen Jiao, Qi Liu, and Xiu Li. Sumo: Search-based uncertainty es- timation for model-based offline reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 20033-20041, 2025a
Zhongjian Qiao, Rui Yang, Jiafei Lyu, Chenjia Bai, Xiu Li, Zhuoran Yang. Siyang Gao, and Shuang Qiu. Cross-domain offline policy adaptation with dynamics-am and value-aligned data filtering. arXiv preprint arXiv:2512.0245: 2025b.
Zhongjian Qiao, Jiafei Lyu, Boxiang Lyu, Yao Shu, Siyang Gao, and Shuang Qiu. Model-based offline rl via robust value-aware model learning with implicitly differentiable adaptive weighting. In The Fourteenth International Conference on Learning Representations, 2026.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-523 537411, 2023.
12

---

Published as a conference paper at ICLR 2026
Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, and Yang Yu. Policy regularization with dataset constraint for offline reinforcement learning. In International Conference on Machine Learning. pp. 28701-28717. PMLR, 2023.
Abhishek Roy, Krishnakumar Balasubramanian. and Murat A Erdogdu. On empirical risk minimiza- tion with dependent and heavy-tailed data. Advances in Neural Information Processing Systems, 34:8913-899 8926, 2021.
Fereshteh Sadeghi, Alexander Toshev, Eric Jang, and Sergey Levine. Sim2real viewpoint invariant visual servoing by recurrent control. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4691-4699, 2018.
Laixi Shi and Yuejie Chi. Distributionally robust model-based offline reinforcement learning with near-optimal sample complexity. Journal of Machine Learning Research, 25(200):1-91, 2024.
Yihao Sun, Jiaji Zhang, Chengxing Jia, Haoxin Lin, Junyin Ye, and Yang Yu. Model-bellman inconsistency for model-based offline reinforcement learning. In International Conference on Machine Learning, pp. 33177-33194. PMIR, 2023.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. Robotica, 17(2): 1999.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 5026-5033. IEEE, 2012.
Linh Le Pham Van, Minh Hoang Nguyen, Hung Le, Hung The Tran, and Sunil Gupta. Hybrid cross- domain robust reinforcement learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 112-128. Springer, 2025.
Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2008
Ruhan Wang. Yu Yang, Zhishuai Liu, Dongruo Zhou, and Pan Xu. Return augmented decision transformer for off-dynamics reinforcement learning. arXiv preprint arXiv:2410.23450, 2024.
Xiaoyu Wen, Chenjia Bai, Kang Xu, Xudong Yu, Yang Zhang, Xuelong Li, and Zhen Wang. Con- trastive representation for data filtering cross-domain offline reinforcement learning. arXiv preprint arXiv:2405.061 066929 2024.
Huaqing Xiong, Tengyu Xu, Lin Zhao, Yingbin Liang, and Wei Zhang. Deterministic policy gra- dient: Convergence analysis. In Uncertainty Artificial Intelligence, pp. 2169-2169. PMLR, 2022.
Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, and Xi- anyuan Zhan. Offline rl with no ood actions: In-sample learning via implicit value regularization. arXiv preprint arXiv:2303. 2tt.:22333./888 15810, 2023.
Huan Xu and Shie Mannor. Distributionally robust markov decision processes. Advances in Neural Information Processing Systems. 23, 2010.
Jiawei Xu, Rui Yang, Shuang Qiu, Feng Luo, Meng Fang, Baoxiang Wang, and Lei Han. Tack- ling data corruption in offline reinforcement learning via sequence modeling. In The Thirteenth International Conference on Learning Representations, 2025.
Kang Xu, Chenjia Bai, Xiaoteng Ma, Dong Wang, Bin Zhao, Zhen Wang, Xuelong Li, and Wei Li. Cross-domain policy adaptation via value-guided data filtering. Advances in Neural Information Processing Systems, 36, 2024.
Zhenghai Xue, Qingpeng Cai, Shuchang Liu, Dong Zheng, Peng Jiang, Kun Gai, and Bo An. State regularized policy optimization on data with dynamics shift. Advances in neural information processing systems, 012322226322933 2023.
Mengbei Yan, Jiafei Lyu, Shengjie Sun, Zhongjian Qiao, Jingwen Yang, Zichuan Lin, Deheng Ye, and Xiu Li. drosi-domain offline policy adaptation via selective transition correction. arXiv preprint arXiv:2602.057776 2026.
13

---

Published as a conference paper at ICLR 2026
Rui Yang, Chenjia Bai, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, and Lei Han. Rorl: Ro- bust offline reinforcement learning via conservative smoothing. Advances in neural information processing systems, 01223811-33866 2022.
Rui Yang, Lin Yong, Xiaoteng Ma, Hao Hu, Chongjie Zhang, and Tong Zhang. What is essential for unseen goal generalization of offline goal-conditioned ri? In International Conference on Machine Learning. pp. 39543-39571 PMLR. 2023.
Rui Yang. Jie Wang, Guoping Wu, and Bin Li. Uncertainty- offline variational bayesian reinforcement learning for robustness under diverse data corruptions. arXiv preprint
arXiv:2411.00465 2024a.
Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, and Tong Zhang. Towards robust offline reinforcement learning under diverse data corruption. In The Twelfth International Conference on Learning Representations, 2024b.
Chenlu Ye, Rui Yang, Quanquan Gu, and Tong Zhang. Corruption-rbustt offline reinforcement learning with general function approximation. Advances Neural Information Processing Sys- 2023.
Heng You, Tianpei Yang, Yan Zheng, Jianye Hao, E Taylor, et al. Cross-domain adaptive transfer reinforcement learning based on state-action correspondence. Uncertainty in Artificial Intelli- PMLR, 2022.
gence, pp.
Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, and Tengyu Ma. Mopo: Model-based offline policy optimization. Advances in Neural Information Processing Systems, 33:14129-14411 2020.
Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, and Chelsea Finn. Combo: Conservative offline model-based policy optimization. Advances in neural information processing systems, 34:28954-28967, 2021.
Kevin Zakka, Philipp Wu, Laura Smith, Nimrod Gileadi, Taylor Howell, Xue Bin Peng, Sumeet Singh, Yuval Tassa, Pete Florence, Andy Zeng, al. Robopianist: Dexterous piano playing with deep reinforcement learning. arXiv preprint arXiv:2304.041500 2023.
Dake Zhang, Boxiang Lyu, Shuang Qiu, mladen kolar, and Tong Zhang. Pessimism meets risk: Risk-sensitive offline reinforcement learning. In Forty-first International Conference on Machine Learning, 2024.
Hongchang Zhang, Yixiu Mao, Boyuan Wang, Shuncheng He, Yi Xu, and Xiangyang Ji. In-sample actor critic for offline reinforcement learning. The Eleventh International Conference on Learn ing Representations, 2023.
Qiang Zhang, Tete Xiao, Alexei Efros, Lerrel Pinto, and Xiaolong Wang. Learning cross-domain correspondence for control with dynamics cycle-consistency. arXiv preprint arXiv:2012.09811 2020.
Xuezhou Zhang, Yiding Chen, Xiaojin Zhu, and Wen Sun. Robust policy gradient against strong data corruption. In International Conference Machine Learning, pp. 12391-12401. PMLR, 2021.
Xuezhou Zhang, Yiding Chen, Xiaojin Zhu, and Wen Sun. Corruption-robust offline reinforcement learning. In International Conference Artificial Intelligence and Statistics, pp. 5757-5773. PMLR, 2022.
YANG Zhihe and Yunjian Xu. Dmbp: Diffusion model- predictor for robust offline reinforce- ment learning against state observation perturbations. In The Twelfth International Conference on Learning Representations, 2023.
14

---

Published as a conference paper at ICLR 2026
A MORE DISCUSSIONS OF DROCO
In this section, we provide further clarifications on several questions regarding DROCO that readers might be concerned about.
1. Why not test DROCO on more tasks such as Antmaze and Adroit?
In our experiments, we evaluate method DROCO and other baselines on MuJoCo-based tasks (e.g., halfcheetah-r2 and hopper-v2). This experimental setting is standard and has been adopted by recent works such as VGDF (Xu et al., 2024), IGDF (Wen et al., 2024), OTDF (Lyu et al., 2025), and CompFlow (Kong et al., 2025). By following these established settings, we believe our experiments are sufficient for evaluating DROCO's effectiveness.
On the other hand, existing literature (Lyu et al.. 2024b) indicates that Antmaze tasks with vary- ing map structures are highly challenging for cross-domain RL (Observation 3, p.7), as adapting policies across structural barriers remains difficult. Similarly, cross dross-domain RL methods often fail on dexterous hand manipulation tasks (Adroit) with kinematic or morphology shifts (Observation 4, p.8). Empirically, we find that not only DROCO but all baseline methods (e.g.. BOSA, DARA, IGDF, OTDF) struggle to achieve meaningful performance on Antmaze and Adroit tasks.
We emphasize that enabling cross-domain RL to succeed in such challenging settings (Antmaze and Adroit) remains an open problem (p.17 in (Lyu et al., 2024b)) and falls beyond the scope of this work. We believe that MuJoCo tasks with dynamics shift provide sufficient and appropriate testbed for evaluating our method.
2. Can DROCO be extended into settings where the source and target domains have distinct state-action representations?
The answer is yes. Although this work follows the setting of recent studies (e.g., BOSA, DARA, IGDE, OTDF) and assumes identical state and action spaces across source and target domains, DROCO can be generalized to domains with distinct state-action representations. This can be achieved by incorporating techniques such as inter-domain mapping via dynamics cycle consis- tency (Zhang al., 2020). Other mapping methods (You al., 2022; Pan al.) also compatible with our framework and could be seamlessly integrated. Thus, DROCO remains applicable even under varying state-action spaces.
3. Why is the Wasserstein uncertainty set chosen instead of other uncertainty sets?
Although there are other possible choices of uncertainty set like the (s, recttangggyyy (Iyengar, 2005) and d-rectangularity (Liu & Xu, 2024b), their dual reformulations often result in complex constraints regularizations and are typically limited simple linear MDP settings. In contrast, the Wasserstein uncertainty set admits an elegant closed-form dual reformulation (Proposition 4.2), which allows converting dynamics perturbations into a simple state uncertainty set-a property crit- ical for practical implementation. Moreover, the Wasserstein metric inherently couples state transi- tions, enabling a natural mapping to state perturbations and providing geometric interpretability.
4. Why not apply the RCB operator on target domain data to improve test-time robustness?
On the one hand, Propositions 4.4 and 4.5 show that dual robustness is guaranteed as long as (-as(, a)) S (srre) holds- hoens-even even when the RCB operator is applied only to source do- main data. Even when this condition is not fully satisfied, our techniques (dynamic value penalty and Huber loss) still enhance robustness. Therefore, applying the RCB operator to target domain data is unnecessary.
On the other hand, target domain data is important for achieving high performance in the clean target environment. Applying the RCB operator to would introduce conservatism and compromise performance. Moreover, since the target data is scarce, any improvement in test-time robustness from using them would be limited. Thus, the optimal strategy is to apply the standard Bellman operator target data to improve performance, and the RCB operator to source data to enhance robustness.
5. Is there trade-off between train-time and test-time robustness?
There is a trade-off between ically, when supp( Pear (-s. (srre) is satisfied, further increasing might bring excessive
and test-time robustness, and it is controlled by e. Specif-
15

---

Published as a conference paper at ICLR 2026
conservatism. While this enhances test-time robustness against dynamics shifts (since is mono- tonically increasing with respect to e), it sacrifices performance on the clean target domain, thereby reducing train-time robustness.
B PROOFS OF PROPOSITIONS
B.1 PROOF OF PROPOSITION 4.1
Proof. We recall the definition of the RCB operator below:
Q(s', .d))
B PM
max a'>vi(-le")
if M MMar
+ +-yinfMMM ES'NPM
max Q(s ((s',a') a' d'ma(-lo')
if M MMMs
Let Q1 and Q2 be two arbitrary Q functions. Then for any state-action pair (s.a), if the next state ~ Pur(-1s, a). we have
[ - TRCH Q21la0 = y max 8,a
E. max a'mp(tls")
Qi(s',d)-
max Q2(6', ,a)) d'ma(-1())
S 7 max Egr
max Qi('',d')= a'~R(11s)
max
Q2(d',a)
a'~A((0>)
S max
QQ - Q211oo
3,a
1/ Q21loo
where the second inequality holds from the fact that for any function f1, fz, any variant x ~ X,
max fi()) - max fa(x) S x
[f(()) - f2(x))
(10)
xmx
If the next state s' ( (-11 a), we have
Q TRRCQQ21.
? max
inf MEM.
Ey max
Q1(6",a))
inf MEM.
Ey max Q2(5',a')
a'mpp(1s>)
(dmmp(tls))
Qi(5'aa'= a'~p(-113)
Q2(s',a')
max
8,0,8'
S max ||Q: - Q21la0
|1Q Q211
where the first inequality comes from the fact that for any function f1, f2, any variant x: X,
min fi()) x fa(x)) I max [x) fz(x))
(11)
x-
Combining the results, conclude that the RCB operator is "yconttaccton operator in the com- plete state-action space, which naturally leads to the conclusion that any initial Q function would converge to unique fixed point by repeatedly applying TRCB. This completes the proof.
B.2 PROOF OF PROPOSITION 4.2
We first introduce the following lemma before proving Proposition 4.2.
Lemma B.1. Let S be measure space and P. be probability measure S. We further let S + IR. be any measure function and c S x S + Ryo be cost function. Then for any scalar the following equality holds
inf
(Bx) XW(P, E.ar inf (f($) Ac(s,
(12)
PNP(S)
B~S
where P(S) represents all probability measures on S, and W is the Wasserstein distance w.r.f the cost function
16

---

Published as a conference paper at ICLR 2026
Proof. We prove this lemma by showing the left-hand-side (LHS) of Equation 12 is equivalent to its right-hand-Se (RHS).
According to the definition of Wasserstein distance, the LHS could be written
LHS
inf PEP(S)
[gwplf($)1++ + inf PET(P,P)
(((,s)~~l(ss,5)
The optimization of P and the inner optimization over E T(P, P) could be combined into single optimization over all couplings y whose first marginal is P, and the second marginal, P. could be arbitrary P(S). We then have
LHS = inf (Ba) ( 8)1)
TET(P,P)
(13)
inf (s,,) [f($) + Ac(s,
(ET(P,P)
where the second equality holds by the linearity of expectation.
By the disintegration theorem for measures, any coupling Y E T(P, P) could be represented as product of its first marginal P and stochastic kernel K (dā(s) S - P(S) such that
(ds, d5) K (dd)s))
(14)
This implies that optimizing over all couplings 7 E [(P, P) is equivalent to optimizing over all possible stochastic kernels K. We substitute Equation 14 into Equation 13 to obtain
LHS = inf / [f()) + Ac(s, 8) K {(d)ss) K
inf Egn [Ba [f()) + Ac(s, 3)]
We change the position of the infimum operator and the inner expectation to get
LHS = E
inf [S- [f()) + yc(s, $)] ((()))))())
(15)
We then solve the inner minimization problem for fixed S:
inf (g (S($) + 7c(s, 8)]
(((()))(S))
Let (8) A f(8) + 70(8, 8). The problem is to find a probability measure (-|ss) that minimizes the expectation of (5). is obvious that this minimum achieved by concentrating the entire probability mass on point & where (3) attains its infimum.
Let 8* arg infses (5). The optimal measure Dirac measure 82- centered s*. Therefore, we have
inf (S [f()) + 7c(s, 8)] K(-I)))P(S)
= Eind, [S)]
(16)
S(8*) 8*)
inf (f()) + 7c(8, 8)]
#ES
Finally, substituting Equation 16 back into Equation 15, we obtain the RHS of the lemma
LHS E.>P
inf (f($) + yc(s, lEs
RHS.
This concludes the proof.
Now give formal proof for Proposition 4.2. We restate it follows.
17

---

Published as a conference paper at ICLR 2026
Proposition B.1 (Proposition 4.2). Letting M. be the Wasserstein uncertainty set defined by Equa- tion 1, then the term inf MEM. Ee' maxxa=p((1)) Q(s', d')] in Equation 2 is equivalent to
Q(5,0)) s.t. 5(
E E
inf max & d'>p(ls)
(17)
Proof. The original term (OT) is constrained optimization problem that can be solved by the La- grange multiplier method. Let (s) = m(-x- (s,, a). Then we define the Lagrange function
((M, M (s,a) (55) (W(M..M)-)) -c)
The LHS is equivalent to solving the dual problem:
OT sup inf C(M, A). A20 M
(18)
For fixed A, we solve the inner minimization problem inf 17 L(M, A):
inf C(M, = inf (Ex-)()1))) V(8)) + MM) - Ae. M
(19)
According to Lemma B.I, we have
inf Bg ((*) AW(M, (o inf (V(3) + Ac(s', x)){ M
(20)
Substituting Equation 20 into Equation 19 and Equation 18, we obtain the dual reformulation of the original term as
OT = sup E. (((10) inf + Ac(s' +x()))+ - de X
(21)
The RHS in Equation 21 is exactly the Lagrange dual reformulation of Equation 17. This implies Equation 17 holds, which concludes the proof.
B.3 PROOF OF PROPOSITION 4.3
Proof. We only discuss the case where Parc (-ss a), since for s' Pu,(-1s,a), the proof identical as Proposition 4.1. For s' ~ (- a), letting Qı and Q2 be two arbitrary Q functions,
we have
| TRCBB22las max Eg'
inf max Q1(3, (EU.((") A((13))
inf sEU,(e") a' 8) Q2 a')
max Eg 8,0
inf
Q1(5, a')
inf max Q2(3,a')
BEU. A(-18)
aEU, -(-
max
|Q
Q2 fl
=? |Q
Q2
where the second inequality holds from Equation 10 and Equation Then, we can conclude that the practical RCB operator is still y-contraction operator.
B.4 PROOF OF PROPOSITION 4.4
Proof. For any (s, a) E Dare.
TRCBQ(s, - -TQ(s,) a)
Eg Eyl-PMr
inf max Q(3, ((3,a') ('
- E, Pa
Q00,, (a)) a'm(a(-lee)
max
inf max Q(s', d')= - Egar Parua
max Q(s' Q(5(.a)) ammpptlls))
M'r a' a'mp(-lss)
50,
18

---

Published as a conference paper at ICLR 2026
where the first inequality holds by supp(Par((ss,a)) a)) S Us(sss)). In the mean time, for any (s. a)
Die. we have
QRC ())
Exg
inf max Q(3,a') (EUL(l")) dl1((-118)
-Bs'~ PMar
max (a'm((10)
Q15', ((6),a1)
=Y
Eg
Q((,aa)
max a''ma(-1e"))
((5),a)
oe w2-))(-18))
? min
max
Q(5,a))
Q(s', a'm-j((-1o"")
a' d'm(p(-1a)
8'~PMor
Letting 8* arg ming
max Q d') and 8* arg max (1m<p(113)
max a'-opit-lo")
Q(s a²) we have
s' PMie
TRCBQ(SS, a) aTQ(s,a)
27
max Q(3,d') max d'~((((())
Q(5",
d'~a((133t)
2 7 (Q(3,a") ((s",a)))
2 22eKQ,
where a* = arg max d'-"(tls*)
Qss (() and the last inequality holds by the Lipschitz continuity as-
sumption and triangle inequality.
Combining the above results, we have
TQ(s. 2ycKo S TRCBQ((s S TQ(ssa))
(22)
Let Q* denote the Q value at iteration and let the initial Q value be Q". After one iteration using the RCB operator and the oracle optimal Bellman operator, according to Equation 22,
(s 27eKo (1 (1-7) s @kcu(s a) s (s,
Suppose when h: = i, we have
27eKQ (1 -1)) s (s, (ss,a), iezt.
('(s.a) 1 --9
(23)
For A: + 1, we have
TQkeu (s, a) - 2 S (R a) = TRCB a) $ (Rkcu(s, (s) a).
On the one hand, we have
TQcca(s,a)
T ('((sa)- 2yeKo (1 7()
27eKo (1 -3))
r(s,a) a)
(5(s
a'mpp(-1s")
1-7
2yeke (1 70)
as --BaN-P
a'~p((s") ('(6',a))
max
-7 -
2geKo (1
(s)(ss,)) 7 1-
7²)
(s, a) - 27eKQ (1 7²)
Therefore, we have
2 (8,
2yeeKQ (1 (1-))) (1- -2ycKQ
2 Q 11 (8, - - -
(24)
29eKQ (1 1- x+1))
= Q (s,a)
19

---

Published as a conference paper at ICLR 2026
On the other hand, Therefore, have
TuRcs (s, = = @
@ (8, a) - @'+ as) a).
(25)
Combining the results of Equation 24 and Equation 25, we have
2 (1 + = QRCB (s, a) S Q++
(st) 7
Hence, Equation 23 still holds for & i 1. Therefore, Equation 23 holds for all & E Zt. is large enough, such that QRCB and Q(s, a) converge the fixed point, then we have
2 s QRCB(S, a) s (7(s,)),
(s) 1-7
which concludes the proof.
B.5 PROOF PROPOSITION 4.5
Proof. The learned value function (s) by repeatedly applying TECB satisfies:
VRCB($) r(ss a²) PB
inf sEti(ss)
VACa(8)
((ssa) + BB..Baa (EE
VRCB (8) |]-
where a* T-CB((1s) = arg max amp(t1ss)
inf
Let c = max (c(Ua S (e(l's)), for ~ Par(1118, (8, (8, s'rc) ~ Darc). Then we have
Pur
inf (sEUc(ot)
VRCB(S)
EginPe
inf VRCB(S) (aEU,(s")
since (s(se) has broader region than (e(s)a)) Given any dynamics P which satisfies W(P(-ss,a), ( (-1,, a)) S c, we can iteratively evaluate Trcb within P:
pl-1(s) Tp (s))
r(s, a TRCA (Rcn(:(s)) + P(
(V*(s')
2 r(8,a TRCB (-1)) + 7. inl V²(3)
sElicle)
If we initialize (s) as VRCB (s), have
(s) Tp mncs (V)
2 r(8, TRCH + 7B
inf
VRCH($)
(EEUc(ar)
r(s,a~ TRCB (-1)) + E.
inf sEUl((")
VRCB (3)
RscB(s)
(s).
According to monotonicity of the Bellman operator, we have V2 T
Junn (V+ >
Tpec (V* (s). Similarly, we can get (*) 2 1--1(s) 2 ... 2 (s))) = VRCB(s). Given that Trues is a 7-contraction, V (s) lim (*(s) (RCB(s), which proves Equation and conclude the proof.
k0000
20

---

Published as a conference paper at ICLR 2026
B.6 PROOF OF PROPOSITION 4.6
Proof. We draw the proof inspiration from (Lyu et al., 2022).
Given that
s,Ps,a Drv (s,( a), Par()ls,a)
$ <<< 1 2 we have
>> 2e
2 sup Drv (ar(1)))))
5,8
(ur(s'ss,,
Σ
Pur(s'1
Pur(o'1s,
E
Par(6'1s, Panr (ur((''(s,)))
d'Euupp(Pu
('supp(Pu(()ssa)
Σ
Note that the maximum Q value Qma S 1-y.
Thus, we have
inf (#33~~Paa((s1a))
max Q(5), at) (a(rv((-1o))
max Q(5). af)
$ E (2) (f/op(-1o())
S (6' E (upp(Pa(((ss.))))
Eg (g a'mppl118")
max Q(s', Q(s',aa) a²)
+ Uls (s),
Qmax
Es' (
max c'mpp(-(s") a(s',a))
+ (1 (P(s) E supp(Pan(-1s, (s)p)(Pal(ss.)))))))
Tmmx
1-7
< (g/pPr((saa)
max e'~p(-(s))
Q(s
+ +1 (1-(1-2))M) 1-7¹
Tmax
where the first inequality uses the law of total expectation. Thus, we conclude the proof.
C EXPERIMENTAL SETTINGS
In this section, we introduce the detailed environmental settings missing in the main text.
C.1 TASKS AND DATASETS
Target domain and datasets. We directly adopt the four locomotion tasks from MuJoCo En- gine (Todorov al., 2012) as the target domain tasks: halfcheetah-v22 hopper-v2, wa walker2d- ant v3. For the target domain datasets, we reuse the datasets in D4RI. (Fu al., 2020) for each task. Since cross-domain offline RI. only allows small quantity of target do- main data, sample 10% data from the original D4RL datascts the target domain datasets. The target domain datasets consist of four data qualities for each task: the medium datasets that contain samples collected by early-stopped SAC policy; the medium-replay datasets that represent the replay buffer of the medium-level SAC agent; the medium-expert datasets that mix the medium data and expert data at 50-50 ratio; the expert datasets that are collected by SAC policy trained to the expert level. The trained policy is evaluated in the target domain, and the evaluation metric use is Normall zed Score in D4RL:
Jr Junddo
Normalized Score
100%,
Jespent Jrandom
21

---

Published as a conference paper at ICLR 2026
Halfcheetah
Hopper
Walker2d
Ant
Target Domain
Kinematic Shifts
Morphology Shifts
Figure 4: Visualization of the target domains and source domains with kinematic shifts and mor- phology shifts, across four tasks (hal fcheetah, hopper, wa walker2d, ant).
where J, is the return acquired by the trained policy in the target domain, and Jexpert and Jrandom are the returns acquired by the expert policy and the random policy in the target domain, respectively.
Source domain and datasets. To simulate the source domain with dynamics shifts, we consider the four MuJoCo tasks (halfs weelffcheeahhhae ah2 hopper-v2, wa walker2d-v2, ant-v3) with kine- matic shifts and morphology shifts introduced as the source domain. The kinematic shifts refer to some joints of the robot being broken and unable to rotate, while the morphology shifts indicate that the robot's s morphology is modified, differing from the target domain. To illustrate this more clearly, visualize the robots in both the target and source domains for all four tasks in Figure 4. We also provide detailed code-level modifications for implementing the dynamics shifts in the following section.
For the source domain datasets, we follow data collection process similar to D4RL. Specifically, we train an SAC policy in the source domain for 1M environmental steps and log policy checkpoints at different steps for trajectory rollouts. The medium datasets are collected using a logged policy that achieves approximately half the performance of the expert policy. The medium-replay datasets consist of the logged replay buffer from the medium-level agent. The expert datasets are collected using the final policy checkpoint, while the medium-expert datasets are 50-50 mixture of medium- level and expert-level data. Note that all the source domain datasets contain about 1M samples, whereas the target domain datasets contain much fewer samples.
C.2 KINEMATIC SHIFTS REALIZATION
To simulate the kinematic shifts in the source domain, we modify the xml files of the original environments. Specifically, we change the rotation angle of some joints of the simulated robot for different tasks:
halfcheetah-Kinemmane The rotation angle of the joint on the thigh the robot's back leg is modi- fied from 0.52, 1.05] to [- 0.0002, 0.0105).
broken back thigh joint <joint axis="O 0" damping="6" name="bthigh" pos="0 0" range="
-.0052 " stiffness="! "240" sype="hinge!
22

---

Published as a conference paper at ICLR 2026
hopper-kinematic: The rotation angle of the head joint is modified from [-100, 0] to |-0.15, and the rotation angle of the foot joint is modified from |-45, 45 to [-18, 18).
broken head joint
<joint axis-"O -1 0" name-"thighjooine pos-"O 0 1.05" range
-0.15 0 type-"hinge"/>
broken foot joint
<joint axis="O -1 O" name= fame="foot_joint pos="0 0.1" range=" 1-1 18 type="hinge"//
walker2d-kinematies The rotation angle of the right foot joint is modified from [-45, 45] to 0..4, 0.45].
broken right foot joint <joint axis-"O -1 0 fame==foot_joins" pos-"0 0 0.1" range -00.4 0.45" type="hinge"/>
ant-kinematic: The rotation angles of the joints on the hip of two front legs are modified from 30, 30] to 0.33
broken hip joints of front legs <joint axis="0 1" name="hip_1* pos="O, "0. 00 0.0 0.0" range=" " -00.3 0.3 type="hinge"/>
<joint axis="O type="hinge""
name "hip 2" pos-"O, "0 0.0 0.0" range-" "-0.3 0.3
C.3 MORPHOLOGY SHIFTS REALIZATION
Akin to the kinematic shifts, we modify the xml files simulate morphology shifts:
haffcheetah-morph: The sizes of the back thigh and the forward thigh are modified.
back thigh
<geom fremto="O 0 0 -0.0001 -0. 000001" name="bthin size="0.046
type="capsule"/
<body name="bshin" pos="-0. 0001 o -0.0001">
front thigh
<geom fremto="O 0 0 0.00 0 0.0011" name="fthight size= "0 00.046" type="capsule""
<body name="fshing pos="O, 0001 0.0001">
hopper-morph: The head size of the robot is modified.
head size
<geom friction="0.9 9 fromto=" 1.45 size="0.125" sype="capsule"//
1.05" name= tane=="toro_geom
walker2d-morph: The thigh the right leg of the robot is modified.
right leg
<body name="thigh" pos="O 1.05">
<joint axis="O -1 0" name= "tam==tthighjoint: pos="O 0 1.05" range " -50
0" type="hinge"/>
<geom friction="0.9" fromto="O 0 1.05 0 0 1.045" name="thigh_ger
size="0.05* cype="capsule"//
<body name="leg" pos="0 0 0.35">
<joint axis="0 -1 0" name="leg_ pos="0 0 1.045" range=" -150 0" type="hinge"/>
<geom friction="C. fromto="0 0 1.045 0.3" name= "eeg_geom"
size="0.04 sype="capsule"/
23

---

Published as a conference paper at ICLR 2026
<body name="foot" pos="0.2 0*> <joint axis="0 -1 0 name= foane==foot_oae
pos="0 0 0.3" range="-45
45" type="hinge"/>
<geon friction="0. 9 fromto="-0.0 0 0.3 0.2 0 0.3" name="
foot_geom" size=" 1amz==00.00 066 "ype="capsule//
</body>
</body>
</body>
ant-morph: The size of the robot's two front legs is reduced.
front leg
<geom #romto="0.0 0.0 0.0 0.1 0.1 0.0"
name="left_ankle_gee size
="0.08 08" type= capeulee/s front leg
<geom fromto="0.0 0.0 0.0 0.1 0.1 0.0" name= name==rriht_ankle__eeee
size="0. 08" type="
D IMPLEMENTATION DETAILS
In this section, provide the implementation details for the baselines use in experiments and our method, DROCO.
D.1 BASELINES
IQL": IQL" is the cross-domain adaptation of IQL (Kostrikov et al., 2021). IQL" follows the same algorithmic procedure except being trained both target and source domain datasets. The state value function is trained by expectile regression:
Lv = B(s.a)-D,..DD - (s(()))
where Lz( | 1(u 0) I(-) is the indicator function, and 0' is the parameter of the target network. This expectile regression enables learning an in-sample optimal value function. Subsequently, the state-action value function is updated by:
Le [ss.sarss' [(r(s, a) + +Voo(s') Qs(s.a))
Then the advantage value is computed A(s,a) a) = Q(,,a) Vs, Based on this, the policy is obtained through exponential ddvaataage-wiihtee behavior cloning:
L - -B...))DD...D [exp(3 x A(s, a)) log (a]s))]
where 3 is the inverse temperature coefficient. We implement IQL based on the official codebase" of IQL.
CQL.: the cross-domain version of CQL (Kumar al., 2020) similar to IQL*, CQL learns conservative value function that lower bounds the true value function:
1 _ E [
Lq = alE.ED log Σ exp(Q(ss a)) Ba-ull((s
-
The policy π is then optimized with SAC (Haarnoja al., 2018). We implement CQL* based on the implementation of C
BOSA: BOSA (Liu al., 2024a) identifies key challenges in cross-domain offline RL: the state-action OOD problem and the dynamics OOD problem. To address these, BOSA proposes two support constraints. Specifically, BOSA handles the OOD state-action problem by supported policy
hhanp://gthhb.ccm///ssstroo///i learningggit https://gghhh..ccm//iisooffoooo
24

---

Published as a conference paper at ICLR 2026
optimization, and mitigates the OOD dynamics problem by supported value optimization. The critic updated through supported value optimization:
Q = B( QQ, (8, (l ( [((Pta((''(s,a) > (lao)))))(QQ(() ()) - -y) a' a'waa(s')
where II(+) is the indicator function, and (s Is, a) the estimated target domain dynamics, and is the threshold coefficient. The policy in BOSA updated by supported policy optimization to mitigate the OOD action issue:
Cx = DE ao) QQ,, (s, S.t. ES mix (s) >
where e' is the threshold coefficient, and a (+|s) is the behavior policy of the mixed datasets Dere U Purr learned with CVAE (Kingma et al., 2013). We adopt the BOSA implementation from ODRL* benchmark (Lyu al., 2024b), which provides reliable implementations for various off- dynamics RL algorithms.
DARA. DARA (Liu al., 2022) employs dynamics-aware reward modification achieve dynamics adaptation, extending DARC (Eysenbach et al., 2020) to the offline setting. Specifically, DARA trains two domain classifiers 4s (target) St, ae, st+1) and 4osa (target|s;, at) follows:
LOSAS ED, [log 4s (target st, ae; s1+1)l + ED... [log(1 as (target|ss, de, 81+1))]
Cosa = E., llog a (target| 8t, a+)] + EED,, [log(1 - 9o (target) 8t, 0x))]
The domain classifiers are used to quantify the dynamics gap log Pa (8.11)))a))) between the source domain and the target domain according to Bayes' rule. Then the estimated dynamics gap serves as a penalty to the source domain rewards:
PM (se41 s.aa))
Dosss (target)se: dei St+1 40s. (source|ss, dz) goass (source 8t, (t 814 y0,, (target) 8t, ax)
ÍDRA = - X dr, 6, Uz) lolog
(26)
where À controls the intensity of the reward penalty. We use the DARA implementation from ODRL and follow the hyperparameter setting in the original paper: À is set to 0.1, and the reward penalty clipped within 10, 10] for training stability.
IGDE. IGDF (Wen et al., 2024) quantifies the domain discrepancy between the source domain and the target domain with contrastive representation learning. To facilitate effective knowledge transfer, IGDF implements data filtering to selectively share source domain samples exhibiting smaller dy- namies gaps. Specifically, IGDF trains score function h(-) using (s, a, S(ar) ~ Durr the positive samples, and transitions (s, a, s'rre) as the negative samples, where (s, a) Denr and S Dar- fu()) optimized via the following contrastive learning objective:
h(s, 8(rr) -B Bein log EveRu M8, a,
Based on the learned score function, IGDF proposes to selectively share source domain data for training value funetions:
Lq = ED.. [(Qo - TQo) + 200 a, s') [I(h(s,a,s)) > h2x [
2
where II(.) is the indicator function, is the weighting coefficient, & is the data selection ratio. We implement IGDF based on its official codebase
OTDE OTDF (Lyu et al., 2025) estimates the discrepancy between the source domain and target domain by computing the Wasserstein distance (Peyré et al., 2019):
|
W(u,
min MEM 1=1
Σ
Σ
C(us, u/)) Pe,0'
(27)
25

---

Published as a conference paper at ICLR 2026
where Suc L Gare and M is the coupling matrices. After solving Equation 27 for the optimal coupling matrix j²,, the OTDF computes the distance between a source domain sample and the target domain dataset via
Sec u' Sur Our + Sar the concatenated vectors, C' the cost function
1Par d(u)) = Σ
C(ui, ue Wire
ue = (s SStes ($ ~ D
Then the critic is updated by
L Ep.. [[Q> - TQo)=] + ((s,a,"') [exp(a x d)I[(d > dx) d
Furthermore, OTDF incorporates a policy regularization term that forces the policy to stay close to the support of the target dataset:
- EADD.DD, log Tuu (T(-Is)[s),
where L, the original policy optimization objective and 3 is the weight coefficient, T is the behavior policy of the target domain dataset learned with a CVAE. We run the official code for OTDF in our experiments.
D.2 IMPLEMENTATION DETAILS OF DROCO
In this part, we provide more implementation details of DROCO omitted in the main text. First, we model the target domain dynamics using a neural network that outputs a Gaussian dis- tribution over the next state: (s (s,a) = N (no a), (8, a))). We learn an ensemble of N dynamics models {Po. N Es, > with each model trained independently with maxi- mum likelihood estimation (MLE) on the target domain dataset:
S ((e,,,)) [og P (d'1
(28)
When we sample from the uncertainty set, we can directly sample from each dynamics model N(uo, Eg;) as the sampling points. We can then compute the value penalty and penalize the Q value of source domain data when leveraging IQL for policy optimization. Specifically, perform expectile regression to train the function:
(n) = E(s,a))D...D,,, (Colss a) (s))),
where (z( 1 1(u 0) and (0, 1). For xe 1, V,, capture the in-sample maximal Q (Kostrikov et al., 2021): (s) x maxa<a(1s) Q(s, a). We can then practically compute the value penalty as
u(s, a 8') (s' ~ p))(-1s,a)) (v(s))
inf (V(s)))
(29)
and the practical Bellman target can be written
FRCBQ(s, a) = r(s, a) ylEg [V(5)) - u(s, a, 8')]
(30)
Then, we incorporate Huber loss and have the following Q training loss:
FROm (Q( Ex... [ss (Qo(s,a) FRCU (occQ
+ ED... [(Qo(s.a)-TQ((s.))]] TQo
(31)
where 4s the Huber loss. The final step policy learning. We follow IQL and utilize exponential wevight imitation learning to extract the policy:
(+) = Ep,,, [exp(Q(s, a) - V(s)) log (als)]
We show the detailed pscudocode of DROCO in Algorithm 1.
26

---

Published as a conference paper at ICLR 2026
Algorithm
Cross- domain Offline RL (DROCO)
1: Require: Source domain offline dataset Dae, target domain offline dataset D mixed offline
dataset Diix
2: Initialization: Policy network Tæ. value network V₁, Q network Qo. ensemble dynamics model
P& = (Pr J penalty coefficient B. transition threshold 6 for Huber loss
3: // Train the ensemble dynamics model
4: for each model gradient step do
5: for each ensemble member Ps:. do
Compute loss L.:: ()),,,,))ED [log Pe: (8'1s,))
6:
7: Update Pa:, using Cw;
8: end for
9: end for
10: // /DD Learning
11: for each gradient step do
12:
Sample bare == ((8, a, r, 8')} from Da
13:
Sample bar- = {(8, a, 8')} from Drr
14:
// Optimize the Vig function
Compute loss Lv:
16:
LV (s..)-DD..U [C3 (Qo(s,a) -Vy(s)
Update V,, using Lv
18:
I/ Compute the value penalty
compute u(s, s') = 1(s' as) (v(s) - inf Po (1,,0) [v()]]
19:
20:
I/ Optimize the Qo function
21:
Compute loss Lo:
22:
- (r + (, Mss (Qa(s, - (r - Bu(s, a, s()))
LQ = -
23:
Update Qo using Lo
25:
// Update target network
Update target network parameters: 0' - 1)) + 100'
27:
Policy Extraction (AWR)
28:
Compute advantage A(s, a) = Qs a) (s)
29:
Optimize policy network n,, using advantage-weighter regresssio (AWR):
30:
L, E(ssa))DD,,DD,, [exp(aA((s, a)) log (a(s))
31: end for
27

---

Published as a conference paper at ICLR 2026
hopper-inematii-exxprtt
hopper-kinematic-xxper
100% DARL
Normalized Score
100
10%
Normalizer Linallized Score
100
80
80
60
60
40
20
40
100% D4RL
10% D4RL
clean
easy
medium
hard
0.0
0.05
0.10
0.15
0.20
Shift Level
Perturbation Scale
(a) Morphology Perturbation
(b) min Q perturbation
Figure 5: Evaluation results of IGDF under morphology and min Q perturbations with different sizes of target domain data.
E EXTENDED EXPERIMENTAL RESULTS
E. EXTENDED RESULTS OF MOTIVATION EXAMPLE
In Section 3, we demonstrate our motivation with simple example. In this part, we provide more details and results for the motivation example.
The source and target domains are hopper peopper-iiematiii www.iineaaiiiommmmmmm c-v2 and hopper-v2 respectively, with their corresponding datasets being hopper-ke nemati nematic-expes exppert and pypper-expeee Fig- ure in Section 3 demonstrates performance across different target domain data sizes under three test-time kinematic perturbation levels (easy, medium, hard), implemented in (Lyu et al., 2024b). We further evaluate the trained IGDF under morphology perturbations and min-Q perturbations (with other settings unchanged), presenting results Figure 5.
The results clearly show that with only 10% D4RL data, IGDF's robustness to dynamics perturba- tions is significantly weaker compared to using 100% D4RI, data. Notably, under easy-level mor- phology perturbations, IGDF with 10% D4RI. data exhibits 66.9% performance drop, versus only 21.8% degradation with 100% data. These findings, combined with the results in Section 3, validate motivation that cross-domain offline RL is particularly sensitive to dynamics perturbations when limited target domain data is available, underscoring the need for enhanced test-time robustness.
E.2 EVALUATION UNDER MORPHOLOGY SHIFTS
In the main text, present DROCO's evaluation results under kinematic shifts. In this section, supplement with additional results under morphological shifts, providing comprehensive assess- ment of DROCO's train-time robustness against diverse dynamics shifts.
Experimental Settings. The target domain tasks and datasets remain consistent with Sec- tion 5.1: the target domain tasks include ha ahe hopper-v2, walker2d-v2 and ant-v3, and the target domain datasets comprise four data qualities (medi um, medi reedummarppl.. medium-expert., expert, expert) for each task. The difference lies in the dynamics shift type in the source domain. We implement morphology shifts described in Appendix C.3 and collect the corresponding source domain datasets.
Baselines. We adopt the same baselines in Section 5.1: IQL", CQL", BOSA, DARA, IGDF and OTDF.
Results. We run each baseline and DROCO for IM steps over 5 random seeds, and present the results with train-time morphology shifts in Table 2. It is clear that DROCO delivers superior per- formance to baselines. Specifically, DROCO achieves the highest performance in 9 out of 16 tasks. In terms of the total normalized score across all tasks, DROCO attains a remarkable 1166.4, significantly outperforming the second-best baseline OTDF (1025.1). Combined with the results in
28

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation results with train-time morphology shifts. half-halfcheetah. hopp=hopper, walk=walker2d, m-medium, mc=medium-exye exxpert,, mr-medium-replay e=expert. We report the nor- malized score evaluated in the target domain, and captures the standard deviation across 5 seeds. We bold the highest scores for each task.
Dataset
IQL*
CQL
BOSA
DARA
IGDF
OTDF
DROCO (Ours)
half-m
45.8
40.2
41.3
45.6
45.5+0.1
44.3++.2
0.2
half-mr
26.1
21.3
27.8
28.9
24.243..
19.7+2.5
27.914.4
half-me
63.0
54,6
44.4
59.2
6.
42.913.6
70.115..6
half-e
65.2
66.7
78.6
55.4
56.0#6.2
74.2+5.0
79.2+13.9
hopp-m
56,4
32.8
28.7
49.5
55.5+2.9
49.1#2.2
56.3+1.66
hopp-mr
51.3
37.6
40.6
53.5
54.915588
24.94:3.4
51.6++8.77
hopp-me
35.8
36.6
20.2
38.2
43.343.6
51.8++39
82.34-.1
hopp-e
87.2
67.9
64.3
77.1
51.5++29
113.2+5.9
92.5+112
walk-m
32.6
43.1
40.3
25.0
33.0+2..
40.3+7.1
60.1+33.4
walk-mr
9.0
2.0
2.9
6.9
9.5+0.4
14.1
15.5+4.7
walk-me
27.6
22.4
46.7
42.2
75.7=11.8
66.7+5.3
78.9+9944
walk-e
103.4
79.0
30.2
102.7
108.316.7
103.5+199
104.5+1.7
89.1
57.3
36.1
96.4
91.6:1..4
92.5+2.77
94.5+2..
ant-mr
59.7
39.5
24.0
64.1
58.247.11
69.618.1
66.91449
ant-me
113.1
107.3
100.5
111.9
116.843.5
107.344.4
120.3+1.5
ant-e
116.3
94.4
76.3
124.5
126,8
111.042.4
120.0
Total
981.6
802.7
702.9
981.1
1012.7
1025.1
1166.4
Section 5.1, these findings conclusively demonstrate DROCO's superiority across different types of dynamics shifts, highlighting its strong train-time robustness against dynamics shifts.
E.3 EXTENDED EVALUATION UNDER DYNAMICS PERTURBATIONS
In this section, we supplement with more experimental results for evaluating the test-time robustness of DROCO.
We first extend the results in Section 5.2 by incorporating broader range of datasets. We evalu- ate the robustness of DROCO against two baselines (IGDE, OTDF) under varying levels three perturbation types: kinematic, morphology, and min Q perturbations, following the methodol- ogy in Section 5.2. Additional experiments are conducted using hopper-nozph- wa walker2d-k nematic cetraatcceeepet....m and ant-mo-rp--xxer with results presented in Figure 6. We can see that DROCO demonstrates superior ro- the baselines. For instance, on the
source domain datasets,
bustness all three perturbation types compared dataset under min Q perturbations, DROCO exhibits only 23.4% performance degradation (from 106.0 81.2) at the highest perturbation level (0.2), sub- stantially lower than IGDF (75.3%) and OTDF (55.9%). This enhanced robustness is consistently observed across all datasets and perturbation types, confirming DROCO's improved test-time ro- bustness against dynamics perturbations.
walkerr2d-kinmmtttie
We further evaluate DROCO's test-time robustness using varying target domain dataset sizes. Ex- periments are conducted under different levels of min Q perturbations, with target domain sizes set to 100%, 50%, and 10% the original D4RL datasets. The source domain datasets comprise Popper-morpee expert,
and ant ant-morph- werph-exxprtt As shown in Figure 7, all methods demonstrate improved robustness against dynamics perturbations with increasing target domain data size, consistent with our claim in Section 3. Notably, DROCO maintains superior robustness across varying data sizes and perturbation scales compared to IGDF and OTDE, further validating its effectiveness in enhancing test-time robustness.
29

---

Published as a conference paper at ICLR 2026
Kinematic Shift
Merphology Shift
min Perturbation
hopger-morph-eper
hopper-mot popper-morphe moppeet
hogger-morph-expen
DROCO
100
1.00
NormelZeed Saore
8b
Normalzzed
Norma
80
4:-
40
20-
20
Shift Level
Shift Level
Iviturbation Scale
110
welker22 Salkerr2d-Enemmaae UC liceepeett
walker www.e//--ieeaa--eeeeeeoeee
1.0-
100
100
Nomalized Score
Nomallzed Schrea
Nomaltzed Sc
80
60
60
DROCO
40
DROCO
DROCO
GID?
aTDr
DTD
ICDF
20
1GDF
KGFF
20
clean
medium
hard
dean
medium
hard
0.50
0.30
0.15
0,20
Shift Level
Shift Level
Purturbation Scale
ant ont-morph expert
ant masmmmrphh expert
130
ant-morph-expert
120
OTDF
120
120
OTDF
Normalted Soore
100F
mordaadedddeee Score
10DF
Noz malltedd Score
GGDF
1.00
200
200
80-
60
42-
daan
030y
medmm
hard
doan
easy
modium
hard
0.0
0.05
0.10
0.15
0.20
Porturbation Scale
Figure 6: Evaluation results under different types and levels of dynamics perturbations.
30

---

Published as a conference paper at ICLR 2026
100% DARL
50% D4RL
10% D4RL
hopper-mophh expert
hopper-morph expett expert
hopper meoppermmrrphooo morph-expert
120
120
110
01DF
110
DROCO
G1DF
100
DID
Normaltzed Store
6G3P
Nemralad Soore
100
IC0F
150F
90
80
Hormalized
80
20
Aerturbatin Scale
Perturbation Seale
Perturbation Seale
walker2d htt.:r2ddkkomaaiccoooo cwcppaai----ccetttttttt
Wa Walkor22 www.lkinnnnnmiiiieee exppert
Walker di kinematic expprrt
DROCO
110
1.0
01DF
Noemalized Score
IGDF
Normalizedd Score
90
70
60
DROCO
60
0.0
0.05
0.30
0.35
0.20
3.6
0.10
0.15
0.20
3.05
0.30
0.55
020
ant-morph-exper
ant-marph-expert
ant-morph-expert
0ROCO
DROCO
DROCO
120
129
Norallzee Scone
Normallve Scoee
110
110
100
100
200
80
0.0
0.05
0.30
0:35
0.20
3.0
0.05
0.10
0.35
0.20
0.0
0.05
0.30
0.15
0.20
Perturbation Scale
Perturbation Scale
Perturbation Scale
Figure 7: Evaluation results under different perturbation levels and different data sizes.
31

---

Published as a conference paper at ICLR 2026
W/ dynamic penalty
W/o dynamic penalty
110
Normallized Score
90
70-
50
helfcheetah-tinee
halfcheetah morph-e
hopper-kine-e
hopper-morph-e
walker2d-kine-e
walkerr2dmorphhee
ant-kine-e
ant-morph-e
Datasets
Figure 8: Ablation study on value penalty
E.4 ABLATION STUDY
We provide supplementary ablation study results that are omitted from the main text. Specifically, examine the effects of replacing the adaptive value penalty with fixed value penalty and substi- tuting the Huber loss with the regular &g loss.
Fixed Value Penalty. A fixed value penalty corresponds setting 3 1.0 across all tasks. Figure compares the performance of DROCO with dynamic versus fixed penalties across eight datasets. The results demonstrate that the dynamic value penalty generally outperforms the fixed penalty dataset where the fixed penalty achieves the highest
(B = 1.0), except the ant-morph-expee performance.
We further evaluate the test-time robustness of DROCO under diverse dynamic shifts using both penalty schemes. Following the experimental setup in Appendix E.3, our results in Figure 9 reveal an interesting trade-off:: while the fixed value penalty leads to slightly degraded performance, it provides marginally improved robustness against dynamic perturbations. This suggests that setting to a larger value induces more conservative policy that is less sensitive to dynamic perturbations, albeit at the cost of policy performance.
Regular lg Loss. The standard &g loss implements conventional Bellman updates for source domain data without special outlier handling. We evaluate DROCO's performance on medi weiiimm-xpppppp expert datasets comparing the Huber loss versus the & loss and present the results in Figure 10. The results show that Huber loss generally produces superior performance, while E loss achieves marginally better results on hal -
and walker2d-kinn -me datasets.
We further examine the test-time robustness against dynamic perturbations using both loss functions. Figure 11 reveals that using Huber loss consistently provides stronger robustness across perturbation types, underscoring its critical role in enhancing robustness.
E.5 EXTENDED PARAMETER STUDY
the main text, we test the sensitivity of DROCO to the penalty coefficient and the transition threshold 6 on certain datasets. In this section, we present extended results for more comprehensive analysis.
Penalty coefficient 8. 8 controls the intensity of the value penalty. We sweep B across (0.1, 0.5, 1.0, 1.2] and further conduct experiments on walker2d-moze morpp-oxopeeeeaeeee and ant-morph-expert" datasets. We present the learning curves of the performance and the Q value in Figure 12. We find that 3 < 1.0 is generally preferred, yielding better performance and Q value convergence, while setting
1.2 would cause value underestimation and inferior performance.
Transition threshold 8 determines the transition point from & loss to l loss. We vary among (5, 10, 30, 50) and conduct experiments wa werkd-rmmaae coadd-morp-mmee and amt-morpp--m
32

---

Published as a conference paper at ICLR 2026
Kinematic Shift
Marphology Shift
min Q Perturbation
hogges-horpp--epeell
hopper-morph-eeeee
hogpe--merh--speer
wi dynamic ponalty
---
dynamic penalty WD dyranic panalty
100
wio synamic penalty
100
100-
Normalzeed Scare
GIDF
Normalized Scare
GIDr
(G3F
80
ay dynamir perwity
we dynamic penaity
20
20
eary
Shift
Level like 2d Bee--innemaii-epertttt
Perturbation
weller22-kinemali-eree
Soilker2d-innematicc
100
100
100
90
Nomallzad Score
80
40
Norralized
70
60
dynamic
40-
dynamic
cleon
e3sy
medium
hard
cean
035y
modium
hard
0.0
9.05
0.30
8.15
020
Shift
Perturbation
ant-merph expert
ant-marphexpert
ant-morph
120
Norallzed Stoare
Nomallzzd Score
110
1.00
100
100-
80
dymamic penalty
Synamic penalty
dynanicc penoity
02
penalty
60
wD dynamic peralty
70-
dynamic panalty
OTDF
OTDF
OTDF
IGDE
ICDF
60
KGD
cloon
635y
medium
hard
ciaon
mzdium
hard
0.0
0.05
0.10
0.15
020
Shift Level
Shift Level
Perturbation Scale
Figure 9: Ablation study value penalty
120
huber loss
w/o huber loss
100
Normaliized
80
70
60-
40
halffchetthh-knnemme
halfcheetah-moophh hopper-kine kine---
hopper-marph--e Datker2d-kine-ms me
walker2d walker2d-morph-mee morph-me
ant-kine-me
ant-morph-me
Datascts
Figure 10: Ablation study on Huber loss
33

---

Published as a conference paper at ICLR 2026
Kinematic Shift
Morphelegy Shift
Perturbation
hopper-norph--xxxe
hopper-morph-expert
huber
lods
weo
1.00
190
weo
Normalize Stores
Formalized Score
GTDF
IDD
IGD?
IGDr
80
20
medium
0.05
0.10
9.20
Shift Level
Shift Level
Perturbation Seale
weller2d-tinemalicessee
We liber 26-Kinemeticeepppet
rolleer2-tinematiceeppe
130
100
1.00
Normalized Strie
Normalized Score
80
60
80-
w/ huber loss
Wi Tuber loss
wt Puber loss
We huber loss
WD huber locs
huber loss
C1DF
01Dr
01DF
easy
Shift
Shift Level
Perturbation Scale
ant-merph-expert
ant-morph-expert
120
Noraaized Score
Nomalllze Scone
11.
3.00
3.00
1.00
huber loss
luber
Fuber
huber
40
cleon
e3sy
mestum
hard
cean
e05y
modium
hard
3.05
0.30
5.5
020
Figure 11: Ablation study on Huber loss
34

---

Published as a conference paper at ICLR 2026
walkerr walker22-morph-expeer
Q value
ant-mo
300
value
100
200
Nomo Score
200
100
pH411
pi0011
HH0.1
p-011
0-0.5
0-0.
am1.2
1-
B-1.2
2-1
0.0
0.5
1.0
0.0
0.6 Stap
1.0
0.0
0.5 Step
1.0
0.0
0.6
1.5
Stop
166
166
166
Stap
165
Figure 12: Effect of B
walkerrdd moradd-oooppmamaamaaam
330
Q value
ank unk-morph me
value
100
200
120
220
Ovalue
230
Nomalizat
60
40
0-5
6-5
6-3
0-5
180
20
140
0.0
0.5 Slip
1.0
-100
0.5 Slap
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Slop
Slap
Figure 13: Effect of
datasets, with Figure 13 showing the performance and value learning curves. Our results demon- strate bataset-ddpenden sensitivity to ō: while 8 = 50 exhibits a satisfying performance and 8 5 yields suboptimal performance on wa felker2dd adrrrhh -ee, DROCO not sensitive on ant ant-morph-me Since no single 8 value universally outperforms across all tasks, we specify the 8 values used for each dataset Appendix F.
Ensemble size N. We also examine the effect of the dynamics model ensemble size N on training. In DROCO, N represents
sampling number within the uncertainty set. Typically, a larger N corresponds to a smaller sampling error when computing TRCB- We conduct experiments on various datasets with N across {3, 5, 7, 7,9). The results presented in Table 3, where we find distinct difference across different N, which means the ensemble size is not sensitive hyperparameter. Thus, could use the default value of
Table 3: Effect of N
Dataset
N
N =
N
N =
half-me-kine -innemmtiii
58.4 4.4
62.2 + 8.6
60.1 7.1
57,9 9.6
half-me-morph
65.4 + 8.4
70.1 5.6
74.9 3.3
half-e-kinematic
68.7 1 6.8
67.0
6..4
66.3
half-e-morph
76.0 + 4.1
7.1
7.92
7.04
hopper-me mepger-me-kee kinematic
Sopper-me-kinematie
51.7 3.4
5.74
6..0
52.5
poppe--memmopp
85.6 6.7
5.5
82.3
83.2
hopper-e-kinematie
88.3 + 10.2
87.0 + 8.1
89.3 9.6
9.6
86.9 + 7.2
hopper-e-morph
91.1 1.0
94.9 + 2.2
92.5 1.2
90.7 + 0.8
Average
73.2
74.7
74.4
73.9
E.6 PERFORMANCE COMPARISON UNDER OBSERVATION AND REWARD SHIFTS
In this part, further examine the generality of DROCO under observation and reward shifts, in addition to dynamics shifts.
Observation shift. To simulate the observation shift, follow the observation corruption setting in (Yang et al., 2024b), and corrupt 30% source domain data by modifying the state of transitions (s, as,,r 8')
+ X std(s), (1fformm[11110p d, represents the state dimension, and
35

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison under observation shifts without observation normalization.
Dataset
IQL
IGDF
OTDF
DROCO
dalf-e-kinematie
21.7+6.2
13.4+2.0
28.613.4
34.71:..8
half-e-morph
43.349.5
33.8+5.8
46.418.3
40.8
haff-me-kinematic
38.9+44.4
40.0+4.66
37.5+6.2
46.3+-9.3
half-me-morph
37.2++4.1
45.3+55.4
33.7+:4.0
43.4+668
hopper-e-kinematie
34.616.4
43.514.88
36.247.9
48.647.4
hopper-e-morph
60.6114.5
32.31499
53.9+11.5
62.948.7
hopper-me-kinemate kinemaait
1.4:0.11
0.0.0..0
16.943.1
20.317.4
hopper-me-me sopper-me-morph
16.7+2.3
22.7=4.00
31.4+5.7
36.516.8
Average
31.8
28.9
35.6
41.7
Table 5: Performance comparison under observation shifts with observation normalization.
Dataset
IQL
IGDF
OTDF
DROCO
half-e-kinematic
26.5114.7
21.41.5.0
33.8:1..1
42.6.1..1
half-e-morph
42.7+7.0
42.5+5.7
51.914.44
44.6:17.4
half-me-kin mineematti
41.2++5.2
35.5+2.9
44.3:22.9
51.3:1..6
half-me-morph
46.413.6
49.2.16.0
45.6113.3
43.0:2211
hopper-e-kinematic
39.6+17.3
67.
49.3++5.7
54.419.3
hopper-e-morph
66.3+66.
38.5+3.7
57.7+:..0
73.2++6.2
hopper-me-kinematic
9.0+1..3
2.410.1
22.243.4
34.2:5..6
hopper-me-morph
16.412.4
29.7+3.0
26.711.1
46.6
Average
36.0
34.6
41.4
48.7
std(s) the dy-dimensional standard deviation of all states the source dataset. Our experiments consist of two parts: (1) directly employ several baselines (IQL. IGDF, OTDF) and DROCO in this observation shift setting without introducing other techniques; (2) we introduce the observation normalization technique (Yang et al., 2024b) to baselines and DROCO. Both parts of the experi- ments are conducted on multiple datasets, with results presented in Table 4 and Table 5. We find that introducing observation shifts would degrade the algorithm's performance, and the observation nor- malization technique can mitigate performance degradation. In both experimental settings, DROCO demonstrates better performance than baselines on most datasets.
Reward shift. To examine the generality of DROCO to reward shifts, we further design a reward shift setting: we randomly select 30% of source transitions (s, a, s') and modify the reward r to i p Uniform[-1, 1]. That is, we completely abandon the reward information and switch to random rewards.
Under this reward shift setting, we conduct experiments multiple datasets to compare the perfor- mance of DROCO with baseline methods (IQL, IGDF, OTDF). The experimental results are reported in Table 6. Surprisingly, we find that reward shift does not significantly affect performance. This observation may be explained by the survival instinct of offline RI. (Li et al., 2023), which suggests that offline RL naturally exhibits robustness to misspecified reward.
The results show that DROCO still outperforms other baselines under both reward shift and dynam- ics shift settings. We attribute the enhanced performance of DROCO under observation and reward shifts to the components dynamic value penalty and Huber loss which mitigate value estimation caused by observation and reward shifts. We believe this finding, along with our above sults under the observation shift setting, demonstrates the generality of DROCO across observation, reward, and dynamics shift.
36

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison under reward shifts.
Dataset
IQL
IGDF
OTDF
DROCO
Ralf-e-kinematte
47.514.2
45.8+3.0
72.213.8
66.016.3
half-e-morph
60.745.3
52.243.6
70.345.8
76.41:422
half-me-kee cifnemmiiii
41.1+3.7
55.2+4.4
43.6+-4.9
57.4+
half-me-morph
61.7++2.1
55.8++4.9
39.0++3.3
63.9+-2.4
hopper-e-kinemaati
58.816.1
67.0:15.7
95.5111..3
85.0
hopper-e-morph
84.7+5..1
46.211488
100.3
91.444.5
hopper-me-kinematte minemmitt
10.1
8.340.77
42.616.33
48.11664
Sopper-me-morpe
34.8+4.3
41.1#14.7
47.3+5.6
78.6+8.3
Average
49.9
46.5
63.9
70.9
Table Performance comparison under distinct behavior policies between source and target domain
datasets.
Source
Target
IQL
IGDF
DROCO OTDF
half-medium
medium
45.2::.11
45.2:10.1
42.2=0.1
45.3
half-medium
expert
47.5+t.11
45.4+113
58.3+2.8
52.6-1.22
half-expert
medium
47.1#1.5
46.8+22.4
51.7=0.4
58.5+0033
half--xpertt
expert
49.7::3.6
47.6+2.11
79.6#3.0
67.4+5.8
hopper-medium
medium
48.8+:2.1
54.316.6
46.3133.7
55.4
hopper-medium
expert
56.114.44
61.8+4.4
69.313.99
8..8:162
hopper-expert
medium
53.6:t2.4
61.3:4..7
51.4=2.1
62.2
hopper-expert
expert
62.616..9
70.1
97.013.3
89.3-9.6
walker2d-medium
medium
48.7+1.9
51.8+2.4
43.0+2.11
70.8+33.3
walker2d-medium
expert
71.4+3.7
82.5+5.3
76.8+4.1
94.6=
walker2d-expent
medium
55.4+33.1
58.6+55.5
57.9+2.0
83.0
walker2d-expert
expert
90.1+3.2
93.745..8
98.9+2.1
106.0
ant-medium
medium
89.915.1
88.0+4.6
86.113.7
92.7-6.3
ant-medium
expert
107.6+1.8
112.4+3.3
105.9+2.3
110.3+2.0
ant-expert
medium
93.7+3.5
90,2+2.8
98.6-4.55
100.4+2.3
ant-expert
expert
111.043.3
119.2+5.6
111.6+229
120.0+-2.1
Average
67.4
70.6
73.4
80.6
E.7 PERFORMANCE COMPARISON UNDER DISTINCT SOURCE AND TARGET BEHAVIOR POLICIES
In practice, the behavior policies between the source and target domain datasets could be different. To address this concern, We consider four tasks (hal fcheetah, hopper, malker2d, ant) with kinematic shifts. We relax the constraint of identical behavior policies, allowing the source and target datasets to have different qualities (medium or expert). For instance, medium-quality source dataset may be paired with either medium- or expert-quality target dataset. All other experimental settings follow Section 5.1, with IQL, IGDF, and OTDF as baselines. The results presented in Table 7. The results indicate that DROCO maintains its superiority over the basclines even when the source and target behavior policies differ. achieves the highest average (80.6) and best performance on 12 out of 16 datasets. These findings demonstrate the effectiveness DROCO in scenarios with differing behavior policies.
F HYPERPARAMETER SETUP
In this section, we provide the detailed hyperparameter setup for DROCO in our experiments. In Table 8, list the network architecture and the training setup of DROCO, well the main hyperparameters of IQL, since we utilize IQL for policy optimization. The distinct value 3 and for each dataset under kinematic shifts and morphology shifts are presented in Table and Table 10.
37

---

Published as a conference paper at ICLR 2026
Table 8: Hyperparameter setup for DROCO
Hyperparameter
Value
Network
Actor network
(256, 256)
Critic network
(256, 256)
Ensemble model network
(400,400,400,400)
Ensemble size
Activation function
ReLU (Agarap, 2018)
Training
Learning rate
10
Optimizer
Adam (Kingma & Ba, 2014)
Discount factor
0.99
Target update rate Source domain batch size Target domain batch size Dynamics model batch size
x 10-3
128
128
256
Dynamics model training steps Policy training steps
x 10
106
IQL
Temperature coefficient
0.2
Maximum log std
Minimum log std
-20
Inverse temperature parameter /
3.0
Expectile parameter
0.7
Table Detailed hyperparameter setup for DROCO, where the source domain datasets are under kinematic shifts.
Table 10: Detailed hyperparameter setup for DROCO, where the source domain datasets are under morphology shifts.
Dataset
Value of 3
Value of 5
Dataset
Value of 3
Value of 5
half-m
0.1
30
half-m
0.1
10
half-mr
0.5
50
half-mr
0.5
50
half-me
0.5
30
half-me
1.2
30
half-e
0.1
30
half-e
1.2
30
hopp-m
0.1
50
hopp-m
0.5
50
hopp-mr
0.5
50
hopp-mr
0.1
50
hopp-me
1.0
30
hopp-me
0.1
10
hopp-e
0.5
30
hopp-e
0.1
10
walk-m
1.0
50
walk-m
0.1
50
walk-mr
0.5
30
walk-mr
0.5
50
walk-me
0.5
50
walk-me
walk-e
0.1
10
walk-e
0.1
10
0.1
30
ant-m
0.1
30
ant-mr
1.0
30
ant-mr
0.1
30
ant-me
0.1
30
ant-me
0.1
10
ant-e
1.0
30
ant-e
1.0
30
G COMPUTE INFRASTRUCTURE
The compute infrastructure for all experiments is listed in Table 11.
38

---

Published as a conference paper at ICLR 2026
Table 11: Compute Infrastructure
CPU
GPU
Memory
AMD EPYC 7452
RTX3090x
288GB
Table 12: Training time comparison between various methods. h=hour(s), m=minute(s).
IQL'
CQL"
BOSA
DARA
IGDF
OTDF
DROCO
5h24m
10h22m
5h49m
6h13m
6h56m
9h17m
7h26m
H
TIME COST
We list the training time of DROCO and all baselines (IQL., CQL.', BOSA, DARA, IGDE, OTDF) for I M training steps in Table 12. We note that the additional time cost for DROCO mainly comes from the training of the ensemble dynamics model. However, since we can save the trained dynamics model weights, no retraining is required for subsequent experiments.
I BROADER IMPACTS
This paper presents method aimed at enhancing dual robustness against dynamic shifts in cross- domain offline RL. Our work has potential positive social impacts; for example, it could inspire the development of humanoid robots capable of robust performance in non-stationary environments. Currently, we have not identified any negative impacts our research.
DECLARATION ON LLM USE
In this work, L.MS are used solely for grammar polishing of an early draft and are excluded from core aspects of the research, such as method conception, theoretical proof, and experimental work.
39


Reviews:
[{"review_id": "NAYhSwThTL", "reviewer": "Reviewer_z4fr", "summary": "The paper studies dual robustness in cross-domain offline RL. It considers train time robustness to distribution shift between source and target, and test-time robustness to perturbations around the target dynamics. It proposes a Robust Cross-domain Bellman (RCB) operator that performs robust backups (min over uncertainty set) on source data and standard backups on target data. Experiments are performed on D4RL MuJoCo tasks to show performance gains and test time stability.", "questions": "What is the precise theoretical and/or algorithmic novelty beyond the split backup? Any coverage aware or calibration guarantees that are new? The paper needs better positioning in relation to vast literature on robust offline RL/DRO. \n\nCan your approach extend to TV/MMD balls (even approximately)? What breaks or becomes intractable?\n\nAny results for reward or state shift to demonstrate generality?\n\nHow does performance scale with the amount of target data and epsilon? Where does RCB become too conservative?\n\nThe source and target behavior policies are often different. Can you analyze this setting?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The goal of achieving both train time and test time robustness in offline RL is well motivated.\n+ The RCB operator that separates robust and standard updates is simple especially with the duality result which simplifies the uncertainty set of distributions to one over states.\n+ The paper gives good empirical results with RL benchmarks showing that the approach outperforms baselines under moderate dynamics shifts.", "weaknesses": "- The robust Bellman backups and the derived contraction properties are standard. As far as I understand, the main idea is to split robust and standard updates, which is conceptually incremental. The paper seems conceptually incremental for ICLR.\n- The setup is restrictive as only dynamics shift is modeled. Typically, there is shift in reward, observation, state/action spaces, etc. \n- The theoretical results largely follow directly from known robust RL results. For example Prop 4.1 showing the contraction is immediate for discounted robust Bellman operators. The train time conservatism and lower bound properties (e.g., Prop 4.4) are classic robust RL analyses. Even the test time guarantee of Prop 4.5 arguing that the performance is better than the worst case value when the true perturbation lies inside the set is by construction and standard in DRO. \n- The framework is restricted to Wasserstein distance without extensions to TV/MMD or other divergences.\n- It is unclear how to choose or tune the uncertainty radius. Also if the domain shift is large, the approach is likely going to be over conservative given the requirement that the target lies in the uncertainty set. \n- I did not quite understand the value penalty and the Huber loss parts as they are treated superficially without sufficient depth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CbITPMfjBN", "reviewer": "Reviewer_XtEu", "summary": "This paper enhances offline RL by introducing two types of robustness: train-time robustness and test-time robustness. Train-time robustness addresses the dynamic shift between the source-domain dataset and the target-domain dataset, while test-time robustness focuses on the shift from the target dynamics to the deployment dynamics. Considering that (1) deployment dynamics may change in real-world settings and (2) existing methods often fail to generalize well under such changes, I believe this paper makes a valuable contribution.\n\nThe proposed method centers around a novel Robust Cross-domain Bellman (RCB) operator, which integrates two types of Bellman operations: (1) the standard Bellman operator for training on the target-domain dataset and (2) a “robust” Bellman operator for policy evaluation on the source-domain data. The theoretical analysis, in my view, can be derived from classic results in robust reinforcement learning.\n\nThe theoretical justification mainly concerns the convergence (contraction) property of the proposed RCB operator. I believe that the conclusions for both the idealized and practical cases can be obtained with relatively minor modifications to existing robust RL proofs.\n\nThe empirical results (Table 1, on MuJoCo tasks) adequately demonstrate the advantages of the proposed algorithm.", "questions": "Q1. Lipschitz Q-function assumption:\nWhy do you cite recent studies to justify the Lipschitz continuity assumption? If I am not mistaken, this assumption is a standard one in Q-learning and can be found in many textbooks.\n\nQ2. Test-time robustness:\nIt is somewhat difficult for me to see the benefits of DROCO regarding test-time robustness. Could you please elaborate on this aspect explicitly in the Experiments section?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The theoretical justification is solid, covering both the idealized case (Proposition 4.1) and the practical case (Proposition 4.3).\n\n- The motivation and analysis for both train-time and test-time robustness  (Proposition 4.4 and 4.5) are meaningful and potentially impactful, although their direct relevance to practitioners might be limited.\n\n- The empirical evaluation is convincing. The chosen baselines are sota methods for offline RL and cross-domain offline RL, yet the proposed algorithm (DROCO) achieves significant improvements over them.", "weaknesses": "I do not see any major weaknesses worth highlighting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kCslrlnJvA", "reviewer": "Reviewer_k93k", "summary": "This paper investigates dual robustness in cross-domain offline RL, addressing both train-time robustness against source-target dynamics mismatch and test-time robustness against deployment-time dynamics perturbations. The authors introduce a novel Robust Cross-domain Bellman (RCB) operator with theoretical guarantees, and develop the practical DROCO algorithm using ensemble dynamics models, dynamic value penalty, and Huber loss. The experimental results across 32 scenarios demonstrate that DROCO achieves good performance.\n\nThe authors employ dynamics perturbations of large magnitude, which may diverge from realistic scenarios. More critically, to put the theory into practice, the algorithm adopts a practical scheme that deviates from its core theoretical assumptions: using an ensemble of dynamics models trained on limited target data to approximate uncertainty. To compensate for the potential value estimation errors caused by this deviation, DROCO further introduces a dynamic value penalty and the Huber loss.\n \nAlthough the method demonstrates performance surpassing baselines and enhanced robustness on specific Mujoco tasks, its effectiveness is highly dependent on a set of sensitive hyperparameters that are difficult to tune effectively in a purely offline setting. Furthermore, the experimental scope is confined to locomotion tasks and has not been validated on more challenging benchmarks, which leaves the generalizability and practical value of its claimed \"dual robustness\" open to question.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The problem formulation is both novel and practically important. While existing cross-domain offline RL methods focus exclusively on train-time robustness, this work is the first to systematically study both train-time and test-time robustness together. The motivation is compelling, with Figure 1 clearly demonstrating that policies trained with limited target domain data are highly vulnerable to test-time dynamics perturbations. This observation reveals a critical gap in current approaches that assume deployment environments will match the target domain exactly.\n\nThe theoretical contributions are well-developed and rigorous. The RCB operator elegantly handles dual robustness by applying robust Bellman updates only to source domain data while using standard updates for target data.", "weaknesses": "My primary concern is the insufficient analysis of generalization. Moreover, the experiments are confined entirely to MuJoCo tasks. Maybe authors can consider more experiment for validation. \n\nThe paper's own sensitivity analysis (Section 5.3) showed that the optimal values for β and δ vary significantly across different tasks and datasets. In a real-world offline scenario, it is nearly impossible to tune these parameters to their optimal values for a new task due to the inability to validate against the target environment. The method's claim to practicality is diminished if its strong performance relies on meticulous, per-task tuning that cannot be replicated in practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)