Semi-Supervised Cross-Domain Imitation Learning
Authors: Li-Min Chu
Kai-Siang Ma
Ming-Hong Chen
Ping-Chun Hsieh
Venue: Accepted by TMLR ()

Abstract: Cross-domain imitation learning (CDIL) accelerates policy learning by transferring expert knowledge across domains, which is valuable in applications where collection of expert data is costly. Existing methods are either supervised, relying on proxy tasks and explicit alignment, or unsupervised, aligning distributions without paired data but often unstable. We introduce the Semi-Supervised CDIL (SS-CDIL) setting and propose the first algorithm for SS-CDIL with theoretical justification. Our method uses only offline data, including a small number of target expert demonstrations and some unlabeled imperfect trajectories. To handle domain discrepancy, we propose a novel cross-domain loss function for learning inter-domain state-action mappings and design an adaptive weight function to balance the source and target knowledge. Experiments on MuJoCo and Robosuite show consistent gains over the baselines, demonstrating that our approach achieves stable and data-efficient policy learning with minimal supervision.

Published in Transactions on Machine Learning Research (02/2026)
Semi-Supervised Cross-Domain Imitation Learning
Li-Min Chu
andy27564 @gmaail com
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwen
Kai-Siang Ma
wawmama..ss10@myyaa. edu.
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwen
Ming- Ming-Hong Chen
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchn, Taiwan
Ping- Chun Hsieh
Ping-Chun
pinghsich@nycu.comm edu.
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchn, Taiwan
Reviewed on OpenReview: https: optpreveeeereeeeee.mmmee net/ forum? id= WARInb JawZ
Abstract
Cross-domain imitation learning (CDIL) accelerates policy learning by transferring expert knowledge across domains, which is valuable in applications where the collection of expert data is costly. Existing methods are either supervised, relying on proxy lasks and explicit alignment, or unsupervised, aligning distributions without paired data, but often unstable. We introduce the Semi-Supervised CDIL (SS-CDIL) setting and propose the first algorithm for SS-CDIL with theoretical justification. Our method uses only offline data, including small number of target expert demonstrations and some unlabeled imperfect trajectories. To handle domain discrepancy, we propose novel cross-domain loss function for learning inter-domain state-action mappings and design an adaptive weight function to balance the source and target knowledge. Experiments on MuJoCo and Robosuite show consistent gains over the basclines, demonstrating that our approach achieves stable and data-efficient policy learning with minimal supervision. Our code available at https:/ / thub. com/ NYCU-RL- wall--andttaaaaaa/aaaa
1
Introduction
In contrast reinforcement learning (RL), which requires extensive environment interactions and carefully designed reward functions, imitation learning (IL) offers compelling alternative by enabling efficient be- havior acquisition through direct imitation of expert demonstrations, thus avoiding the need for complex reward engincering exhaustive exploration. IL has demonstrated remarkable success in arcas such as robot control (Mandlekar et al., 2022), aulonomous driving (Le Mero et al., 2022), and visual navigation (Nguyen et al., 2019), where access to expert data facilitates the development Despite these advantages, conventional II. methods commonly roly on the assumption that the expert data and the target environment share the same domain characteristics, a condition that is rarely satisfied in prac- tice. In world scenarios, collecting expert demonstrations in target domains such physical robotics
highhperformance control policies.

---

Published in Transactions on Machine Learning Research (02/2026)
self-driving cars is often prohibitively expensive hazardous. In contrast, simulated environments, al- though capable of providing large-scale data, typically differ in their underlying dynamics and state-action representations.
Cross-domain imitation learning (CDIL) addresses this challenge by transferring expert knowledge from a source domain to a target domain with distinct dynamics or state-action spaces, thereby improving effi- ciency and reducing reliance on target-domain data. Existing CDIL approaches can be broadly categorized based on the level of supervision they require: (1) Supervised methods typically rely paired or un- paired demonstrations in various proxy tasks to explicitly establish correspondences between domains. For example, (Sermanet et al., 2018; Liu et al., 2020) leverage paired visual trajectories for time-contrastivee or stane-translation learning, while (Raychaudhuri et al., 2021) align demonstrations via state-distribution matching. (2) In contrast, unsupervised methods avoid paired data by aligning distributions or learning invariant representations, such as adversarial domain adaptation (Kim et al., 2020; Zolna et al., 2021) and optimal transport-based alignment (Fickinger et al., 2022). These methods often assume an isomorphism between the source and target domains; however, such an isomorphism can generally be satisfied in multiple ways (possibly infinitely many), resulting in ambiguity in alignment and potential instability. In general, supervised approaches are constrained by the cost of collecting paired trajectories, unsupervised approaches often suffer from unstable transfer, and Nomain-speciff solutions lack generality.
In this paper, we propose to study Semi- Supi-super CDIL, which achieves cross-domain transfer by lever- aging source-domain (imperfect) demonstrations and requires only minimal target-domain supervision by combining a small number of target-domain expert demonstrations along with imperfect target-domain tra- jectories, without the need for additional proxy tasks. This semi-supervised CDIL sctting offers promising balance between supervision cost and transfer capability. However, this setting remains largely unexplored, motivating the need for a principled framework.
To address this, we propose AdaptDICE, the first algorithmic framework for Semi-Supervised CDIL that extends the single-domain distribution correction estimation (DICE) to achieve knowledge transfer across domains with discrepancies transition dynamics, state space, and action space. AdaptDICE leverages source-domain knowledge together with limited offline target-domain data, enabling stable policy learning without roquiring paired trajectorics or assumptions isomorphism. The core of our approach consists of three components:
Cross-domain mapping loss: AdaptDICE introduces a novel mapping loss that learns cross-domain mapping functions by minimizing the Bellman error within the mapped sourco-domain space, thereby bridging domain gaps while preserving source-domain knowledge. The learned cross-domain mappings enable knowledge transfer of source-domain density ratios to the target domain.
Hybrid density ratio for cross-domain policy extraction: AICE derives the target-domain policy by combining density ratios of both domains via the cross-domain mappings. The resulting hybrid density ratio serves as the mechanism that facilitates transfer across domains in imitation learning.
Adaptive weighting: The hybrid density ratio further modulated by an adaptive weighting factor, B(t), defined as smooth function of the relative estimation errors between the source and target density ratios. This adaptive mechanism dynamically balances the contributions of both domains, enabling stable adaptation without manual hyperparameter tuning.
Through theoretical analysis, establish the convergence guarantee of AdaptDICE for the underlying density-ratio estimation, achieving a polynomially decaying error bound, with the expected error adaptively bounded by the better of the two domains. Through extensive experiments, we show that AdaptDICE achieves consistently better performance than various baseline methods in challenging oflline cross-domain scenarios (with as few as only 1 labeled target-domain expert trajectory) across multiple standard bench- mark tasks in MuJoCo and RoboSuite. We also provide an ablation study to demonstrate the benefits of hybrid density ratios for effective cross-domain transfer. These results highlight the effectiveness and wide applicability of Adap under minimal supervision.
2

---

Published in Transactions on Machine Learning Research (02/2020)
Learn Inter Comm-Daminn Mappings
Train
Train
Scurce Data For Piacy laaks
Source Expert Dherss
Train
Source Expert Demo
CDIL
Target Expert pepr Dat
Scurce Expers Importect Dota
Orline
TT
Interation
Target Domsin
Target Domain
Target Domwin
Torget
Deployment
Deploymant
Deployment
(a) CDIL with proxy tasks.
(b) Unsupervised CDIL.
(c) Semi-supervised CDIL.
Figure 1: An illustration of CDIL formulations: (a) CDIL with proxy tasks utilizes annotated target demon- strations through paired or unpaired proxy tasks, trading off accuracy and data efficiency. (b) Unsupervised CDIL relies only on source experts and unlabeled target data and typically requires assumptions on domain similarity (e.g., isomorphism) and can suffer from ineffective transfer. (c) Semi-supervised CDIL combines limited labeled target data with unlabeled trajectories, balancing supervision cost and transferability.
2 Preliminaries
2.1
Imitation Learning
In this section, we introduce the fundamental concepts and notation for the general CDIL, building the standard Markov Decision Process (MDP) framework. For a set X, let A(X) denote the set of all probability distributions over X. We model each environment as an infinite-horizon discounted MDP, defined as M == AS, P. 7.). where is the state space, A is the action space, P : S x A + A(S) represents the transition function, giving the probability P(s 18;, 0z) of a transition from state 8, to 8t-1 by taking action at at time t, f E ASS) the initial state distribution, and y E [0, 1) is the discount factor. A policy π S + (AA) maps states distributions over actions. For a given policy n, the occupancy measure is 5, = a
7P((s; defined dd (s, a) == (1 7)) 7
In the canonical CDIL setting, the learning involves two domains, referred to as source domain (src) and target domain (Lar), defined respectively as Marc == ( Pere,Mere; Msrc; 7) and Mtar == (Star, Ptar, Mtar, 7). Notably, the source and target domains can differ significantly in transition dynamics, state space, and action space. We assume that both MDPs share the same discount factor y. In CDIL, the information available the imitator can be summarized follows:
Target domain: The imitator operates in Maar and aims to mimic target-domain expert policy TE Star + ((Ara)) by learning from either online interactions in the target domain or pre-collected target-domain datasct that can consist of either expert and non-expert demonstrations. In CDIL, both the online interaction budget and the size of the largel-domain dataset. are presumed to be small.
Source domain: In CDIL, due to the limited target-domain data available, the imitator also leverages auxiliary information from the source domain (e.g., via pro-collected sourcc-domain datasct D
The goal of the imitator is to learn a policy T : Star + A(Acar) that can effectively mimic the expert policy based on the data from both domains. In the CDIL literature, there exist two commonly adopted problem settings:
CDIL with proxy tasks: This supervised setting assumes the existence of prozy or alignment tasks with expert demonstrations in both source and target domains (Kim et al., 2020: Raychaudhuri et al., 2021) as a form of supervision. For instance, in robot control, primitive tasks like walking jumping serve proxies for more complex target tasks. Cross-domain knowledge transfer is achieved via domain alignment, i.e., by learning cross-domain state-action mappings from these auxiliary expert data, either paired or umpaired across domains (Figure 1(a)). This supervised setting enables more accurate alignment and stable transfer but requires substantial data collection or annotation.
In the RL literature, the occupancy measure d is also called the discounted state-action visitation distribution.
3

---

Published in Transactions on Machine Learning Research (02/2026)
Unsupervised CDIL: This setting relies primarily source-domain expert data and uses target- domain demonstrations, hence considered "unsupervised" (Fickinger et al., 2022; Franzmeyer et al., 2022). Learning cross-domain mappings still requires online interaction with the target environment (Figure 1(b)). While avoiding reliance on target-domain experts, unsupervised CDIL algorithms often assume strong domain isomorphism and can sulfer from instability or misalignment under substantial domain discrepancies.
2.2 Regularized Distribution Matching
In an offline setup, (single-domain) imitation learning can be formulated as regularized distribution matching problem, which minimizes the discrepancy in occupancy measure between the expert and the imitator under behavior regularization to overcome extrapolation error in offline learning. Specifically, one notable offline IL formulation is introduced by DemoDICE (Kim et al., 2022) and substantiated by the following objective fanction:
== argmax, - (d" -
- oDr. dd
(1)
where a > O is the regularization coefficient and d and du denote the underlying occupancy measures of the expert and imperfect datasets, respectively. As the objective function in (1) generally non-convex in n, one can use d' instead of as the decision variables and thereby reformulate (1) as a convez constrained optimization problem that minimizes the KL divergence between the imitator's and expert's occupancy measures under set of Bellman flow conservation constraints (i.e., one equality constraint for each state). By resorting to the dual problem and strong duality and introducing a numerically-stable transformation, we can arrive at an equivalent unconstrained problem:
(A(ss,)) 1+a
min L(v;r) = (1- Esrp(((s) + (1 + a:) log E (s.a)
exp
(2)
where {v(s)] hess are the Lagrange multipliers (one for each constraint), r(s, := logarithmic density ratio, and (s(s,a) a) := r(s, a) +y (e))) - vss). We let v² denote the optimizer of (2). Notably, one can interpret r(s, a) as the pseudo reward and v(s) as the pseudo value function in offline II, and hence (s,s, a) can be viewed as the advantage function under r and v.
a)) is the
Once the dual problem in (2) is solved and the optimal dual variables v* are obtained, we can extract the optimal primal variables d= and the corresponding optimal policy x" as follows: (i) We first define density ratio w* (s, a) = d' (s, a). The duality gives us that for each (s,
A (s,
as = exp =exp
1 a
(ii) Based on the derived w w"(s, a) in (3), one can extract T² through weighted behavior cloning, which minimizes E(s,a)aac [w"(s, a) log (als)) over
3 Methodology
In this section, we formally describe the proposed formulation of Semi-Supervised CDIL. Then, we present A AdeptDICE (Adaptive Cross-Domain Imitation Learning with DIstribution Correction Fstimation), the first algorithmic framework for Semi-Supervised CDIL. We start by introducing a prototypic algorithm based on AdaptDICE designed for efficient. knowledge transfer across domains and analyze the convergence of the proposed algorithm in the tabular case. Subsequently, we propose an adaptive function that dynamically balances source- and target-domain knowledge accommodate diverse domain disparitics and then provide practical implementation beyond the tabular case.
3.1 Semi- Supervisd CDIL
As described in Scction 2.1, fully supervised CDIL enables accurate cross-domain alignment but requires substantial target-domain demonstrations, while unsupervised CDIT. avoids this cost but often fails under
4

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1 AdapíDICE
Require:
Source-domain Quee and whre pretrained with DemoDICE, weight function / N + [0, step
size 17, target-domain offline dataset Dia = ((8,a,8)) expert dataset D ((8, a)) C car
1: Initialize discriminator network Ctar Star x Atar + [0
2: Update the discriminator by (10): Ctar < arg min Jenr (D.. tart Dlar)
3: Compute Ttar(s, a) -lo (8, 1) for (s, a) E D
tur
4: Initialize ( H Plar: T())
5: for iteration t = ......T do
6: Sample Prrr of size Ntar from the ofiline dataset DU tar
7: Compute ((t) by (18)
8: Update the mapping functions by (4): G(0), (T()) + arg minG. IMAP(G, H; rtar, Qsee (1-1) Dear)
Update the pscudo value by (5): (t)
< (t-1) v Bar LDICE (ace(V/e (t-11, star x(t- Deas)
10: Update the policy by (9): x(() +- arg min, LBC G1t), H(),, 11(0),1(0 B()), Wsrer w warr Dtar)
11: return Target-domain policy T
large domain discrepancies. In practice, target domains usually provide few expert demonstrations, whereas imperfect or suboptimal data are casier to obtain.
To address this, we formalize Semi-Supervised CDIL, assuming access to a small labeled subset of expert demonstrations and the rest as unlabeled trajectories. Labeled data provides minimal supervision, while unlabeled data captures target dynamics (Figure 1(c)), reducing annotation cost and avoiding instability, The information available to the imitator in Semi-Supervised CDIL can be summarized as follows:
Target domain: In this paper, we focus on the oflline setting, where the imitator has access to a pre- collected expert dataset DE, of (s, a, s') tuples, which are generated by The and follow the occupancy measure der as well as an imperfect dataset Dian. which generated by policies with unknown degrees of optimality. We define the union dataset as DU : DE UD! and let d denote the corresponding mixture of occupancy measures. In CDIL, DE is presumed to be scarce (e-g., as few only one trajectory), and hence cross-domain transfer is especially needed.
Due to the limited target-domain data, the imitator leverages an auxiliary source- domain dataset Darei which can consist of both expert and non-expert demonstrations in the source domain. With Dare, one can also leverage an oflline IL algorithm to pre-train policies or other relevant networks for cross-domain transfer. In particular, source-domain pseudo Q-function Qaee can be obtained as pre-trained critic, which serves as a transfcrable component and will be utilized later in our framework to guide cross-domain adaptation.
Source domain:
3.2 Proposed Algorithm
In this subsection, we formally present AdaptDICE and describe its key components. The pscudo code is provided in Algorithm 1. The algorithm leverages a larget-domain offline dataset tar = [(s,( a, 8')) alongside pre-trained source-domain models, to learn an optimal policy for the target domain.
Before describing the key components of AdaptDICE, we first provide overview of its learnable com- ponents. The algorithm begins by training discriminator Caa (Lines 1-2) to induce a pseudo reward, representing a logarithmic density ratio Ptar (Line 3), which guides subsequent optimization. AdaptDICE then iteratively updates three key modules: (1) a pseudo value function f updated via DICE-based gra- dient descent; (2) mapping networks ( and H(t that align source and target domains by minimizing
a Bellman error; and (3) the target policy (1) refined through weighted behavior cloning with adaptive weight ((t). These updates occur within the loop (Lines 5-11), facilitating robust cross-domain knowledge transfer. Notably, this transfer is achieved by querying source signals (Qure and Ware) on the mapped tuples (G(s), ((, effectively utilizing them as informative priors. Meanwhile, the target density ratio waar and the mappings (G, H) are learned directly from the target-domain data to avoid degenerate mappings.
5

---

Published in Transactions on Machine Learning Research (02/2026)
We now describe the three components updated inside the training loop (Lines 8-10) in the following order: the cross-domain mapping loss, the DICE loss, and the cross-domain policy extraction. The pseudo-reward computation, which is performed once before the loop, is detailed at the end.
Gross-Doman Mapping Loss. The mapping loss IMAP (Line 8) learns cross-domain mapping functions : + Sare and H Star A Aare which aligu source- and larget-domain transitions such that target-domain state-action pairs can be evaluated under the source-domain critic. We learn G and H by minimizing the following loss, which enforces Bellman consistency in the mapped source-domain space:
IMAP(G, Htar; Qarey n, Dear) = (ssa,ss)) |tar + [Qrro (Cs) ((rc(G())
(4)
where Ttar is the target-domain pscudo reward (computed once before the training loop; detailed at the end of this subsection), and Quee :- (8, a) + (er=Pen(((ssa)) (sa)) is source-domain pre-trained optimal pseudo Q-function. By minimizing this loss, we obtain mappings G and H* that effectively bridge the domain gap, ensuring that the source-domain knowledge can be reliably adapted to the target domain without direct modification Q the tabular sctting, LMAP yiclds optimal mappings by solving the finite state-action space, minimizing Bellman error over Dear-
DICE Loss. In AdaptDICE, the loss LDICE (Line 9) employed to learn the pseudo value function for the density ratio w based on the target-domain data. Specifically, updates yaar through gradient-basod optimization on the following loss on offline samples Dears i.e.,
AAnar (8, (nm
1ta ]
(Paar Tears n, Dear) = 7e) (Maar (s)] - (1+ a) log (s
exp
(5)
where Maar is the initial state distribution induced from the dataset D
Cross- Domain Policy Extraction. By integrating the mapping function mentioned before, we define the cross-domain density ratio Wevoss that is combined with the pre-trained source-domain density ratio Ware and the learned target-domain density ratio wtar:
Weroess8, a) == Ware (G(s), + +1 3) Wear (8,
(6)
where 3 E [0 is a parameter that balances the contribution of the source and target domains. The density ratios for the source and target domains are defined
Avmno ((ss)))))))))
Wsre (G((s, H(,)) a)) = cxp
(7)
1
Amar (8,
Whar (iar == exp
(8)
1 l+a
where Mare denotes the pre-trained source-domain pscudo value function and Vtar is the learned target-domain
pseudo value function. This cross-domain density ratio is used in the behavioral cloning loss LBc (Line 10):
(B G, H, Wares Wtary Dear) - (ea))-Daa [Weross (s, a) log (als))
(9)
is minimized to learn an optimal policy in a tabular setting with a finite state-action space. In Algorithm 1, we employ an adaptive weighting factor 3(1) (Line 7) whose exact form detailed in Section 3.4, enabling the weighted behavioral cloning loss during training without manual tuning of 3()).
As mentioned carlier, before entering the training loop, we first train a discriminator to obtain the target-
domain pseudo-reward as follows:

---

Published in Transactions on Machine Learning Research (02/2026)
Pseudo-Reware Computation. A discriminator network Cuar Stae x Atar [0, 1] is trained to distin- guish expert state-action pairs from DE far against the full target-domain dataset tars minimizing the binary
cross-entropy loss Jerr (Line 2):
Jew (D tar: Dlar) == (E(,,))Dg,, [log Star (s, a)] + (e,a) [log(1 Ctar (sar(s)))))
(10)
The target-domain pseudo-reward (Line 3) is constructed as
Iear (s, a) == log(L1 - 1), which approximates the true logarithmie density ratio log(d (s, a) (s,
(11)
3.3 Convergence Analysis
In this subsection, thcorctically establish the convergence of AdaptDICE, focusing on how the cross- domain density ratio w a) approaches the optimal density ratio w a). To measure this convergence, we define the density ratio error:
wi (s. a) (ss a)]-
(12)
We then derive upper bound this error, highlighting the role of the weighting factor B(t) in balancing source and larget domain contributions to achieve efficient convergence.
Building on the formulation in Section 3.2, the cross-domain density ratio in AdaptDICE is given by:
(s, w cross (s, B(t) (G H (s, a)) + + ((t))
(13)
where Ware(G(0) (8), ( (s.a) a)) = exp (Aracc (s), H ()) is the pre-trained source-domain density ratio, Waat (s, a) exp(A, (s, a)) is the larget-domain density ratio at iteration 1, and B(t) N + [0 is weighting factor balancing the contributions of the source and target domains. The mapping functions G' and H denote the learned state and action mappings that align target-domain samples with the source-domain representation. To quantify the convergence of wiless (s,a) a) to the optimal density ratio wist (8, a), we define the pointwise density ratio errors:
(1)
Au!te (t) (8, a) = ( (G (s), (1 (s, a)) - wisr (s, a)l, (u == [w (s, a) (s,
(14)
and their expected errors over the target-domain distribution Dtar:
n (t) arc = (es,a)DDen Ware ( (8), H (8) a)) - (8, a)), Awidl = EI
(0) (x) - wis (8, (s,(a) a))-
(15)
For convergence analysis, we define the optimal solution set var as Siar = (Viar + C 1sw |CE where 1sm the all-ones vector over Star: and the orthogonal projection IIs; (Ptar) = arg minges;.. || y112-
We next provide the convergence guarantee of AdaptDICE, whose proof is deferred to Appendix C. Theorem 1. [Upper Bound of with learning rate S 1/L,, for each (8,a), the cross-domain density ratio error is bounded as follows:
Density Ratio Error Under AdaptDICE] Under AdoptDICE,
(t) (s, - (s, a)l < ((t)aw (s, a) + +(1 -1-180 (1-B(t)) Cu a) exp
Cu
viarll2
|| viorlla
V8
Vt
(16)
where L, is the smoothness constant of LDICEE, Viar = ns (v(0)), and C. is constant. Moreover, by selecting 3(t) as
0.
if (m S Am
B(t)
1,
otherwise
the expected cross-domain density ratio error satisfies
E(e.,)) /n (t) (8, a) ( a)l S min min(Am(!)) Am!l)
(17)

---

Published in Transactions on Machine Learning Research (02/2026)
Remark. The bound in Theorem 1 demonstrates the effectiveness of AdaptDICE in adaptively combining source- and larget-domain density ratios. By choosing ((t) to favor the domain with the smaller expected error, AdaptDICE achieves an error bound no worse than the better of Aw Am When the source- domain density ratio is more accurate (d.e., Award < Aw setting ((t) = 1 leverages the pre-trained source model to accelerate convergence compared to relying solely on the target-domain density ratio. This adaptability is particularly valuable in scenarios with limited target-domain data, as it enables efficient knowledge transfer while mitigating errors due to domain discrepancies.
3.4 Design Choice of Weighting Factor ((t))
Recall that Theorem establishes discrete selection rule for the weighting factor ((t) that yields an optimal expected error bound. However, such hard-switching mechanism may cause undesirable oscillations in practice when the source and target errors are comparable. To achieve smoother and more stable adaptation behavior, AdaptDICE employs continuous weighting scheme that interpolates between the source and target density ratios. Specifically, we define (t) as function of the expected density ratio errors:
A
((1)
(18)
Amirr
Am
where Aware Awly denote the expected estimation errors of the mapped source-domain and target-domain
density ratios (Equation (15)). This formulation adaptively adjusts B(t) according to the relative reliability of the two estimators: when the source ratio is more accurate (Aware < Aw(Wl) B(() > 0.5 and smoothly increases toward 1 as the source estimator becomes more reliable, emphasizing source-domain information; conversely, when the target estimator is more accurate, B(t) decreases toward 0, shifting the emphasis to the target domain. Such a smooth transition avoids abrupt switches between domains and maintains stable weighting during training.
While this adaptive weighting does not alter the theoretical convergence rate established in Theorem 1, It offers notable practical advantages. First, continuous interpolation mitigates oscillations in B(t) when the
source and target estimators exhibit similar errors, leading to smoother updates of witt) and more stable policy optimization. Moreover, enables gradual transition of reliance from the source to the target domain, preventing abrupt degradation and improving robustness when the target-domain data are limited or noisy. These properties collectively improve the empirical stability and reliability of AdaptDICE in diverse cross-domain adaptation scenarios.
3.5 Practical Implementation
In this subsection, we present the practical aspects of AdaptDICE, including optimization of the mapping functions, the implementation the adaptive function, and the gradient- based training of loss functions.
Gross-Domain Mapping Optimization. To align the source and target domains with potentially dis- tinet state and action spaces, we optimize mapping functions G Star + Sere and H Star Atar - Aero using cross-domain loss LMAP in Equation (4) that ensures Bellman consistency across the source and target domains. To ensure that G and H map target-domain states and actions to source-domain feasible region, we adopt normalizing flows, following (Brahmanage et al., 2023) in the action-constrained RL litera- ture (Hung el al., 2025), to map the outputs of G and H inlo the source-domain feasible region. Specifically, target-domain states and actions are first mapped by fully-connected layers into a latent dummy region (e.g., a hypercube M)), then transformed by a normalizing flow into the feasible source-domain region. The flow model is pre-trained entirely offline using source-domain feasible samples and kept fixed during target-domain learning, with only the preceding network layers updated.
8

---

Published in Transactions on Machine Learning Research (02/2026)
Adaptive Function Implementation. The adaptive weighting function B(t) N - [0,1] prioritizes the domain whose density ratio is closer to the optimal target-domain ratio wian (8,a). It. is computed as:
B(t)
Amhir Eey
(19)
A
where the empirical error estimates are:
Aui srC
Das (t)
Ware (G())))) H (s, - ()
(20)
A (an
Whar (8, t0 a1)
(21)
Since the true (s, a) is unavailable during training, we use ( a) as its practical approximation: Wae (s, (i (s,a). To stabilize A which can exhibit high variance due to limited target-domain data, we apply moving average follows:
AW (1)
(A (t-1) + (1 A
(22)
where 1/ = 0.9 is the smoothing factor. This ensures robust estimation of ((t,, enhancing stability in the cross-domain policy extraction.
Loss Function Optimization. While MAP and LBc mentioned in Section 3.2 yield solutions in the tabular setting, practical implementation constraints, such as scalability and computational efficiency, necessitate gradient-based updates using neural networks. In Algorithm 1, train G, H, and the policy a(!) via stochastic gradient descent IMAP and LBC, respectively, leveraging neural network architectures to handle dig-ndimensional state-action spaces effectively.
4 Experiments
In this section, we present the main experimental results from the CDIL experiments, designed to evaluate the algorithms in benchmark robot control tasks. Our goal is to assess how effectively different methods leverage offline datasets from both source and target domains policy learning. Unless stated otherwise, we report the average and the standard deviation over random scods for all the experimental results.
4.1 Setup
Evaluation Domains. We evaluate AdaptDICE on various continuous control tasks in MuJoCo and Robosuite to test the cross-domain transfer capability under varied dynamics, representations, and task complexities. Specifically:
MuJoCo: We adopt standard MuJoCo environments, including Hopper-v3, HalfChootah-v3, and Ant- v3, as the source domains. Following the procedures described in (Zhang et al., 2021), we modify these environments to construct the corresponding target domains.
Robosuite: We utilize Robosuite (Zhu et al., 2020), widely adopted framework for robot arm manip- ulation, to evaluate our algorithm on BlockLifting, DoorOpening and TableWiping tasks. For each task, the Panda robot arm serves as the source domain, while the UR5e robot arm is designated as the target domain.
The detailed specifications of the source-domain and target-domain morphologies are provided in Section D. The state and action space dimensions of each domain are provided in Table 5 in Section D.
Baselines. We compare AdaptDICE with recent CDIL algorithms, including SMODICE (Ma et al., 2022), GWIL (Fickinger et al., 2022), and the IL variant of IGDF (Wen et al., 2024). All the algorithms share the same source-domain and target-domain datasets for a fair comparison, with differences arising from the specific use of source-domain datasets, as detailed below:

---

Published in Transactions on Machine Learning Research (02/2026)
Table Configurations of the target-domain dataset various scenarios.
Environment
Dataset Set
Expert Traj.
Sub-optimal Traj.
Sub-optimal Composition
Hopper Wipiig
60
10 expert + 50 random
Default
Others
101
expert 100 random
Expert Rich
5
60
10 expert + 50 random
Hopperr/ Wiiien
Sub-Optimal Rich
50 expert + 250 random
Expert Rich
5
101
expert + 100 random
Others"
Sub-Optimal Rich
505
expert 500 random
Others includes Ant, HalfCheetah, BlockLifting, and DoorOpening.
GWIL (Fickinger et al., 2022): GWIL an unsupervised CDIL method that learns from source-domain expert demonstrations as well as the target-domain data collected online. To adapt GWTI. the offline setting, we modify the online replay buffer to be supplied by the offline dataset. Due to the original exper- imental setup in the GWIL paper, we use only one source-domain expert. trajectories in our experiments, without using source-domain imperfect demonstrations.
SMODICE (Ma et al., 2022): SMODICE was originally designed to tackle imitation from dynamics morphologically mismatched experts. To evaluate its performance in the Semi-Supervised CDIL setting, SMODICE can utilize the source-domain expert trajectories and the full target-domain dataset to enhance learning in the target domain, without using the source-domain imperfect demonstrations. This ensures fair comparison without fundamentally changing the algorithm of SMODICE.
IQLeaa (Wen et al., 2024): IGDF is cross-domain representation learning algorithm that achieves data filtering via contrastive learning and can be integrated with any off-the-shelf RL method to address offline cross-domain RL. One can adapt IGDF to IL by using an off-the-shelf offline IL method, such as IQ-Learn (Garg et al., 2021). Specifically, the training can be done in two stages: (i) At the representation learning stage, TGDF learns to encode state-action representations that enable effective data filtering across domains. (ii) At the imitation stage, it leverages the learned encoders to select high-quality source trajectories for training, thereby improving the targel-domain policy in CDIL.
Dataset Configurations. The larget-domain datasets are constructed by mixing a small number of expert trajectories with a larger number of sub-optimal ones, following the configurations summarized in Table 1. Similarly, the source-domain datasets contain 400 expert trajectories and 2000 sub-optimal trajectories, where the sub-optimal set is composed of 400 expert and 1600 random rollouts. More detailed dataset specifications are provided in Sention D.2.
4.2 Evaluation Results
In Table 2, we report the final performance values of AdaptDICE and baseline methods across various contin- nous control tasks, with the corresponding training curves provided in Figure 2 (Section D.3). AdaptDICE consistently outperforms the baseline CDIL algorithms in both MuJoCo and Robosuite, especially in high- dimensional robot arm manipulation tasks where learning from limited expert demonstrations particularly challenging. We also observe that SMODICE and (GDF+IQ-Lead suffer on most of the tasks in this chal- lenging low-data regime, and these results are consistent with those in their original papers, where hundreds of demonstrations are usually needed to achieve comparable return performance. Morcover, we observe that GWIL can barely make any learning progress. We hypothesize that this results from the fact that GWIL can only learn up to an isometry between the policies across domains due to its unsupervised nature. Recent works (Choi et al., 2023; Huang et al., 2024) also report that GWIL can struggle in practical cross-domain settings, suggesting that its need for expert supervision and inability to leverage imperfect demonstrations can limit its applicability.
10

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Experimental results of AdaptDICE and other baseline methods in the Default setting.
Environment
AdaptDIC (Ours)
SMODICE
GWIL
IGDF+IQleam
Hopper
2289.9 = 328.7
2046.6 53.3
9.8 = 0.03
286.6 = 78.6
Ant
3672.6 450.8
86.3
28891. 17.4
-358.3 1106.6
IallCheetahh
4171.1 2659.2
-15.1 42.5
-800.7 228.8
2179.5 2422.1
BlockLifting
38.5 + 13.9
0.8
0.01 = 0.001
5.4
DoorOpening
139.5 + 30.8
9.6 14,6
34.5
70.0
TableWiping
19.7 18.5
7.6
0.03 0.6
-15.5 34.9
4.3 Ablation Study
To better understand the contributions of the source-domain and larget-domain density ratios in AdaptDICE, we conduct an ablation study by using only the cource-domail density ratio Ware or the target-domain density ratio Wear during policy extraction. Specifically, we evaluate the following variants:
AdE with Ware only: The policy trained exclusively with the pre-trained source-domain density ratio, ignoring any target-domain density information. This is equivalent choosing (t) 1, for all t.
AdaptDICE with waar only: The policy relies solely the target-domain density ratio learned from the oflline larget-domain data, without leveraging source-domain knowledge. This is equivalent to choosing 3(t) = 0 for all t and hence reduces to the single-domain DemoDICE (Kim et al., 2022).
Table 3: Experimental results of two ablation variants of AdaptDICE in the Default dataset setting.
Environment
Ware only
Whar only
AdaptDICE
Hopper
2039.2 366.6
2534.2 = 200.3
2289.9 + 328.7
Ant HalfCheetah
1047.8 129.1
3921.5 247.1
3672.6 + 450.8
-2.0
-2.3
4171.1 2659.2
BlockLifting
4.3 + 3.7
11.9 13.3
38.5 - 13.9
DoorOpening
60.1 + 16.5
1..1773.
139.5 + 30.8
TapleWiping
2.9 + 4.1 13.9 13.01
19.7 + 18.5
In Table 3, we show the final performance values of AdaptDICE and its two ablation variants (i.e., with Ware only and Wtar only), with the corresponding training curves shown in Figure 3 (Section D.3). In most of the tasks (HalfCheetah, BlockLifting, and DoorOpening), the full AdaptDICE method with sourco-to- larget transfer significantly outperforms the variants using only Wsrc or Wtar; highlighting the imporlance of combining source-domain and target-domain density ratios. In the Ant environment, the only variant of AdaptDICE performs well, while the Onle-Only variant performs rather poorly, suggesting the transfer mechanism remains rather reliable even under substantial discrepancy between the two domains.
4.4 Effect of Dataset Configurations
To evaluate AdaptDICF's reliability various dataset configurations, we conduct additional experiments on the number of target-domain trajectorics. As Semi-Supervised CDIL uses both labelod expert demonstrations
DE and unlabeled or sub-optimal trajectories DL we vary the two dataset. sizes separately: (i) increasing far DE with fixed | (Expert Rich) to assess annotation cost, and (ii) increasing [DL with fixed DDE (Sub-Optimal Rich) to evaluate robustness to imperfect data. The configurations are summarized in Table 1. This is meant to corroborate AdaptDICE's adaptability under the trade-off between supervision and sub- optimal trajectories in low-data regimes.
Table shows that AdaptDICE reach higher final performance in both Expert Rich and Sub-Optimal Rich regimes than in the Default setting. In Expert Rich, additional labeled demonstrations accelerate convergence, while in Sub-Optimal Rich, abundant imperfect data improves representation and reduces
11

---

Published in Transactions on Machine Learning Research (02/2026)
overfitting
These results demonstrate AdaptDICE's flexibility in trading off costly expert annotations for cheaper sub-optimal data, often yielding superior gains with larger dataset size. We also report the training curves in Figure 4 (Section D.3).
Table 4: Experimental results of AdaptDICR under various dataset configurations.
MuJoCo Environments
Robosuite Environments
Hopper
Ant
HalfCheetah
BlockTifting
DoorOpening
TableWiping
Default
2289.9 + 328.7
3672.6 + 450.8
4171.1 + 2659.2
38.5 = 13.9
139.5 + 30.8
19.7 18.4
Expert Rich
2577.5 171.9
4494.2 1 760.1
9296.0 1 1589.5
65.1 1 17.3
161.7 1. 16.0
37.6 18.9
Sub-Optinal Rich
2381.3 299.3
5038.1 207.1
10105.9 1 1821.7
68.9 1 25.7
181.41 5.0
44.1 1 20.0
5 Related Work
Single-Domain Imitation Learning with Imperfect Demonstrations. A central challenge in IL is ensuring robustness against noisy or suboptimal demonstrations Early approaches exlended behavioral cloning by explicitly modeling demonstration quality (Wu et al., 2019; Sasaki & Yamashina, 2021; Xu et al., 2022) or by re-weighting samples to place greater emphasis on higher-quality data (Tangkaratt et al., 2020; Wang et al., 2021). Other strategies employed adversarial training or regularization to mitigate the influence of low-quality data, such as through disagreement penalties (Brantley et al., 2019) or confidence-basee filter- ing (Cao al., 2022). Additional methods leveraged distribution matching and state-occupaney measures to stabilize learning from fixed, imperfect datasets, thereby addressing issues such covariate shift and suboptimal actions (Brown et al., 2019; Kim et al., 2022; Ma et. al., 2022; Yu et al., 2023; Mao el al., 2024). While these methods effectively account for demonstration quality, they typically assume that training and deployment data originate from the same domain. This assumption limits their applicability in scenarios requiring cross-domain transfer, a gap that CDIL is designed to address.
Gross-Domain Imitation Learning under State and Action Discrepancies. There are two major sub-categories in this setting:
CDIL with proxy tasks: Paired-trajecton approaches exploit explicit correspondences between source and target domains, typically through observation state alignment. Representative strategies include context translation (Raychaudhuri et al., 2021), timo-contrastive networks (Sermanct et al., 2018), and state distribution matching (Liu et al., 2020). Extensions of these methods scale to multi-domain or long- horizon transfer, such as multi-domain behavioral cloning (Watahiki et al., 2024) and hierarchical skill aligmment (Lin et al., 2024). Beyond visual or stato-level alignment, several works incorporate skill-level decomposition to enhance generalization across tasks.
In contrast,
methods do not rely on direct correspondences between source and target trajectories. Instead, they leverage adversarial objectives to facilitate transfer under limited supervision (Kim et al., 2020; Zolna et al., 2021; Giammarino et al., 2025). More recent efforts extend this line of work to handle unpaired data across variations in representation (Zhang el. al., 2021) and dynamics (Kedia et al., 2025), thereby improving robustness in scenarios with heterogeneous domains.
Unsupervised CDIL: Unsupervised approaches remove the need for paired data by aligning distributions or learning domain-invarian, representations. Techniques such as optimal transport (Fickinger et al., 2022; Nguyen et al., 2021) and adversarial domain adaptation (Choi et al., 2023) mitigate discrepancies at the distributional or feature level, while embedding-baser methods focus extracting task-relevant representations for transfer (Franzmeyer et al., 2022; Pertsch et al., 2022). Recent advances further incorporate denoising strategies to enhance robustness against domain-induced noise (Huang et al., 2024).
Cross-Domain Imitation Learning with Aligned State and Action Dimensions. We categorize mothods in this class those where the source and target domains share identical state and action dimensions but differ in dynamics or contextual factors. These approaches can be broadly divided into two directions. First, represendalionn-leel methods aim to learn invariant features (Yin et al., 2022; Lyu et al., 2024a)
12

---

Published in Transactions on Machine Learning Research (02/2026)
contextual embeddings (Liu et al., 2023a) to enhance robustness across domains. For example, Wen et al. (2024) proposes contrastive representation approach for data filtering in cross-domain offline reinforcement learning. We adopt and extend this approach as a baseline, incorporating dynamics-aware adjustments to better accommodate CDIL in heterogencous settings. Second, dynamics- methods explicitly address discrepancies transition dynamics, either by adapting policies to varying environments (Liu el al., 2023b) or by introducing off-dynamics objectives (Kang et al., 2024).
6 Conclusion
In this work, we proposed AdaptDICE, the first semi-supervised distributionn-coorection framework for Semi- Supervised CDIL that scales to heterogeneous state and action spaces. AdaptDICE integrates sourco-domain knowledge with limited offline target data without requiring paired trajectories or explicit domain mappings. We provided theoretical analysis showing the convergence guarantee for density ratio estimation, and demonstrated through extensive experiments on MuJoCo and Robosuite that AdaptDICE consistently out- performs other baselines. Ablation studies further confirm the importance of combining source-domain and target-domain density ratios for stable transfer.
While AdaptDICE offers improved stability and flexibility in the Semi-Supervised CDIL setting, its effec- tiveness still depends on the availability of small amount of labeled target data, which may not always be accessible in real-world scenarios. Future work includes exploring the integration of stronger representation learning or better offiine adaptation mechanisms to further enhance cross-domain transferability.
Acknowledgment
This research was partially supported by the National Science and Technology Council (NSTC) of Taiwan under Grant Numbers F14-2628-F-A49-002 and 114-6334F-N9-002-BBB3 This work was also partially supported by the Center for Intelligent Team Robotics and Human- Robot Collaboration under the "Top Re- search Centers in Taiwan Key Fields Program" of the Ministry of Education (MOE), Taiwan. We also thank the National Center for High-performance Computing (NCHC) for providing computational and storage resources.
Bibliography
Janaka Chathuranga Brahmanage, Jiajing Ling, and Akshat Kumar. FlowPG: Action-constraineed policy gradient with normalizing flows. In Advances in Neural Information Processing Systems, 2023.
Kiante Brantlcy, Wen Sun, and Mikacl Henaff. Disagroement-regulaiizee imitation learning. In International Conference Leurning Representations, 2019.
Daniel Brown, Wonjoon Goo, Prabhat Nagarajan, and Scott Niekum. Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations. In International conference ma- chine learning, 2019.
Sébastien Bubeck et. al. Convex optimization: Algorithms and complexity. Foundations and Trends6 in Machine Learning, 2015.
Zhangjie Cao, Zihan Wang, and Dorsa Sadigh. Learning from imperfect demonstrations via adversarial confidence transfer. In International Conference on Robotics and Automation, 2022.
Ming-Hong Chen, Kuan-Chen Pan, You-De Huang, Xi Lüu, and Ping-Chun Hsieh. Cross-domain policy optimization via bellman consistency and hybrid critics. In International Conference on Learning Repre- sentations, 2026.
Sungho Choi, Scungyul Han, Woojun Kim, Jongscong Chac, Whiyoung Jung, and Youngchul Sung. Domain adaptive imitation learning with visual observation. Advances in Neural Information Processing Systems, 2023.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Arnaud Fickinger, Samuel Cohen, Stuart Russell, and Brandon Amos. Cross-domain imitation learning via optimal transport. In International Conference Learning Representations, 2022.
Tim Franzmeyer, Philip Torr, and João F Henriques. Learn what matters: Cross-domain imitation learning with task-relevant embeddings. Advances in Neural Information Processing Systems, 2022.
Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: Datasets for deep data-
driven reinforcement learning. arXiv preprint arXiv:2004.079.10 2020.
Divyansh Garg, Shuvam Chakraborty, Chris Cundy, Jiaming Song, and Stefano Ermon. TQ-Learn: Inverse soft-Q learning for imitation. Advances in Neural Information Processing Systems, 2021.
Vittorio Giammarino, James Queeney, and Ioannis Ch Paschalidis. Visually robust adversarial imitation learning from videos with contrastive learning. In International Conference on Robotics and Automation, 2025.
Tuomas Haarnoja, Aurick Zhou, Pieler Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International conference on machine learning, 2018.
Kaichen Huang, Hai-Hang Sun, Shenghua Wan, Minghao Shao, Shuai Feng, Le Gan, and Dc-Chuan Zhan. DIDA: Denoised imitation learning based on domain adaptation. arXiv preprint XrXiv:2404.088 03382, 2024.
Wei Hung, Shao-Hua Sun, and Ping-Chun Hsich. Efficient action-constrained reinforcement learning via acceptance-rejection method and augmented MDPs. In International Conference on Learning Represen- tations, 2025.
Yachen Kang, Jinxin Liu, and Donglin Wang. Off-dynamics inverse reinforcement learning. IEEE Access, 2024.
Kushal Kedia, Prithwish Dan, Angela Chao, Maximus Pace, and Sanjiban Choudhury. One-shot imitation under mismatched execution. In IEEE International Conference on Robotics and Automation, 2025.
Geon-Hyeong Kim, Scokin Seo, Jongmin Lee, Wonscok Jeon, HyeongJoo Hwang, Hongscok Yang, and Kee- Eung Kim. DemoDICE: Offline imitation learning with supplementary imperfect demonstrations. International Conference on Leurning Representations, 2022.
Kuno Kim, Yihong Gu, Jiaming Song, Shengjia Zhao, and Stefano Ermon. Domain adaptive imitation learning. In International Conference on Machine Learning, 2020.
Lue Le Mero, Dewei Yi, Mehrdad Dianati, and Alexandros Mouzakitis. A survey on initation learning techniques for end-to-end autonomous vehicles. IEEE Trunsactions Intelligent Transportation Systems, 23(9):14128-14147, 2022.
Jyun-Li Lin, Wei Hung, Shang- Hsuan Yang, Ping- Chun Hsich, and Xi Liu. Escaping from zero gradient: Revisiting action-constrained reinforcement learning via frank-wolfe policy optimization. In Uncerlainly in Artificial Intelligence, pp. 397-407, 2021.
Zhenyang Lin, Yurou Chen, and Zhiyong Liu. Hierarchical human-to-robot imitation learning for long- horizon tasks via cross-domain skill alignment. In International Conference on Robotics and Automation, 2024.
Fangchen Liu, Zhan Ling, Tongzhou Mu, and Hao Su. State alignment-basod imitation learning. International Conference on Learning Representations, 2020.
Jinxin Liu, Li He, Yachen Kang, Zifeng Zhuang, Donglin Wang, and Huazhe Xu. CEIL: Generalized con- textual imitation learning. Advances in Newral Information Processing Systems, 2023a.
Zixuan Liu, Liu Liu, Bingzhe Wu, Lanqing Li, Xueqian Wang, Bo Yuan, and Peilin Zhao. Dynamics adapted imitation learning. Transactions Machine Learning Research, 2023b.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Jiafei Lyu, Chenjia Bai, Jing- Wen Yang, Zongging Lu, and Xiu Li. Cross-domain policy adaptation by capturing representation mismatch. In International Conference on Machine Learning, 2024a.
Jiafei Lyu, Kang Xu, Jiacheng Xu, Jing- Wen Yang. Zongzhang Zhang, Chenjia Bai, Zongqing Lu, Xiu Li, et al. ODRT: A benchmark for off-dynamics reinforcement. learning. Advances in Neural Information Processing Systems, 2024b.
Yecheng Jason Ma, Andrew Shen, Dinesh Jayaraman, and Osbert Bastani. Versatile offline imitation learning via state-occupancy matching, In ICLR 2022 Workshop on Generalizable Policy Learning in Physical World, 2022.
Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martin-Martin What matters in learning from offine human demonstrations for robot manipulation. In Conference on Robot Learning, pp. 1678-1690, 2022.
Liyuan Mao, Haoran Xu, Xianyuan Zhan, Weinan Zhang, and Amy Zhang. Diffusion-ICEE: In-sample diffusion guidance for offline reinforcement learning. Advonces in Neural Information Processing Systems, 2024.
Khanh Nguyen, Debadoepta Dey, Chris Brockett, and Bill Dolan. Vision-basod navigation with language- based assistance via imitation learning with indirect intervention. In IEEE/CVF Conference on Compuler Vision and Pattern Recognation, pp. 12527-12537, 2019.
Tuan Nguyen, Trung Le, Nhan Dam, Quan Hung Tran, Truyen Nguyen, and Dinh Phung. TIDOT: A teacher imitation learning approach for domain adaptation with optimal transport. In International Joint Conference on Artificial Intelligence 2021. Association for the Advancement of Artificial Intelligence, 2021.
Kuan-Chen Pan, MingHong Chen, Xi Liu, and Ping-Chuu Hsieh. Survive on Planet Pandora: Robust cross- domain rl under distinct stale-action representations. In JCML 2024 Workshop: Aligning Reinforcement Learning Experimentalists and Theorists, 2024.
Karl Pertsch, Ruta Desai, Vikash Kumar, Franziska Meier, Joseph J Lim, Dhruv Batra, and Akshara Rai. Cross-domain transfer via semantic skill imitation. In Conference on Robot Learning, 2022.
Dripta S Raychaudhuri, Sujoy Paul, Jeroen Vanbaar, and Amit K Shy-Chowd,.ry. Cross-domain imitation from observations. In International conference machine learning, 2021.
Fumihiro Sasaki and Ryota Yamashina. Behavioral cloning from noisy demonstrations. In International Conference on Learning Representations, 2021.
Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-contrastive networks: Self-supervised learning from video. In International Conference on Robotics and Automation, 2018.
Voot Tangkaratt, Bo Han, Mohammad Emtiyaz Khan, and Masashi Sugiyama. Variational imitation learning with diverse-quality demonstrations. In International Conference on Machine Learning, 2020.
Emanuel Todorov, Tom Erez, and Yuval Tassa, MuJoCo: A physics engine for model- control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012.
Yunke Wang, Chang Xu, Bo Du, and Honglak Lee. Learning to weight imperfect demonstrations. In International Conference on Machine Learning, 2021.
Hayato Watahiki, Ryo Iwase, Ryosuke Unno, and Yoshimasa Tsuruoka. Cross-domain policy transfer by representation alignment via multi-domain behavioral cloning. In Conference on Lifelong Learning Agents, 2024.
Xiaoyu Wen, Chenjia Bai, Kang Xu, Xudong Yu, Yang Zhang, Xuelong Li, and Zhen Wang. Contrastive representation for data filtering in cross-domain offline reinforcement learning. In International Conference on Machine Learning, 2024.
15

---

Published in Transactions on Machine Learning Research (02/2026)
Yueh-Hua Wu, Nontawat Charoenphakdee, Han Bao, Voot Tangkaratt, and Masashi Sugiyama. Imitation learning from imperfect demonstration. In International Conference on Machine Learning, 2019.
Haoran Xu, Xianyuan Zhan, Honglei Yin, and Huiling Qin. Discriminator-weigghhe weighted offiine imitation learning from suboptimal demonstrations. In International Conference Machine Learning, 2022.
Kang Xu, Chenjia Bai, Xiaoteng Ma, Dong Wang, Bin Zhao, Zhen Wang, Xuelong Li, and Wei Li. Cross- domain policy adaptation via value-guided data filtering. Advances Neural Information Processing Systems, 2023.
Zhao-Heng Yin, Lingfeng Sun, Hengbo Ma, Masayoshi Tomizuka, and Wu-Jun Li. Cross domain robot imitation with invariant representation. In International Conference on Robolics and Aulomalion, 2022.
Lantao Yu, Tianhe Yu, Jiaming Song, Willie Neiswanger, and Stefano Ermon. Offine imitation learning with suboptimal demonstrations via relaxed distribution matching. In Associalion for the Advancement of Artificial Intelligence, 2023.
Qiang Zhang, Tete Xiao, Alexei A Efros, Lerrel Pinto, and Xiaolong Wang. Learning cross-domain corre- spondence for control with dynamics cyclo-consistency. In International Conference Learning Repre- sentations, 2021.
Ruiqi Zhu, Tianhong Dai, and Oya Celiktutan. Cross domain policy transfer with effect cycle-consistency. In IEEE International Conference on Robotics and Automation, 2024.
Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martin, Abhishek Joshi, Soroush Nasiriany, and Yifeng Zhu, robosuite: A modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009. 2000:2009.1293,, 2020.
Konrad Zolna, Scott Reed, Alexander Novikov, Sergio Gomez Colmenarejo, David Budden, Serkan Cabi, Misha Denil, Nando De Freitas, and Ziyu Wang. Task-relevant adversarial imitation learning. In Confer- ence on Robot Learning, 2021.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Appendices
A DemoDICE Algorithm
17
B Supporting Lemmas
18
C Convergence of AdaptDICE with Weighting Factor B())
22
D Detailed Experimental Sctup
24
D.1 Settings of Evaluation Environments
24
D.2 Dataset Configuration of Each Experiment
26
D.3 Training Curves
................................
27
D.4 Effect of Dataset Configurations Baseline Methods
29
D.5 Computational Resources and Hyperparameters..
................................
31
D.6 Implementation of Baseline Methods
31 ................................
D.7 Additional Experimental Results
32
D.8 Normalizing Flows for the Mapping Functions
36
A DemoDICE Algorithm
This section details the pseudo code of DemoDICE (Algorithm 2), an offline imitation learning method that trains policy using expert (DE) and imperfect (D') datasets, as formalized in Section 2.2. The algorithm
proceeds in three stages: (1) pretraining discriminator derive pseudo-reward r(s, a); (2) updating
pscudo value network (t) via gradient descent; and (3) optimizing policy network m()) through weighted
behavior cloning. Below, we present the algorithm and explain each stage.
Algorithm DemoDICE
Require:
Expert dataset DE, imperfect dataset DJ, union dataset DU = DE U DL 1: Initialize discriminator network c: S x A + [0,1]
2: Update + arg min
(E(s,)) [log c(s, (s,a)]= (logg) c(s,
Compute r(s, a) = - log(1
(o((1/c(ss /c(s, a) 1) for (s. DU
4: Initialize critic network (0), policy network x(0)
5: for iteration t == 1,...,,77 do Sample batch from D Update (1() (lt-1) v/V, (.vlt(((
Update (!) arg ming LBC (T; (())
return Trained policy x(T)
Pretraining the Discriminator for Reward
The discriminator network c: S x - [0, 1] initialized and trained distinguish expert state-action pairs from those in the union dataset D D D by minimizing the binary cross-entropy loss: Je(DE, DU) :=
- F DD (log c(s,a) a)] D [log(1 a)s, The resulting pseudo-reward is defined r(s,a) a)
17

---

Published in Transactions on Machine Learning Research (02/2026)
(s.a) - 1). For the optimal discriminator c²(x,a)= = d"(s,a)ta we have
(1/c(s, -log(1/c(3)) - log
1 (s, - -1) (c((s00-1)
d (")ssa)
- log
log
(23)
(s.
which approximates the log-density ratio between expert and union distributions, providing reward signal that emphasizes expert-like behaviors.
Training the Critic Network
As described in Section 2.2, DemoDICE optimizes argmax, DKL d=)) =dd d yielding the dual problem with objective [(v;r). The critic w(9) updated for 1..... T by sampling a batch B c DU and performing gradient descent:
ym vlt-1) v/ V ,r,
(24)
where the empirical pseudo-value loss is defined as:
Ea Ba (x)) + (1 + a)) log ( exp (Au-n
[(v(t-1); = (1 7)
(25)
1 1
with A n (s,a) := r(,,)) + pl (utt (uit- (8') (l)))((s).
Optimizing the Policy Network
The policy (()) is updated by minimizing:
LBC(T; (t) (E (s)))(ss a) log (als))
(26)
where (s, == exp(A (s, (1 + a)). This weighted behavior cloning loss, derived from the density ratio in Equation (3), aligus (0) with the expert distribution over T iterations, yielding T(T)
B Supporting Lemmas
In this section, we present the useful properties of the DemoDICE algorithm for the convergence analysis in the sequel. Recall that v + R denotes the stato-dependent value function, r(s, a) = log pscudo reward, the u-induced advantage function A,(s, a) := r(s, + + (()))( [u(s)] - v(s), (*.0)) is the initial state distribution, d A(S x A) is the underlying occupancy measure of the union dataset, a 2 0 is the KT-regularization coefficient, and x 2 0 is the kzrreularrzation parameter. As described in Section 2.2, the training loss of DemoDICE is
(x.a))) is the
A(S)
L(ver) = - Esmpll)) + (1 + a)) log E E(sna)au exp ( 1 (A.6
(27)
To simplify the notation, we use L(v) as the shorthand L(v; r) in the sequel when the context is clear. For case of exposition, we assume that both the state and action spaces are finite, and the results can be extended to continuous spaces with appropriate where y² is minimizer of L and 1s is the all-ones vector over S (Kim et al., 2022, Lemma 2). Denote the orthogonal complement of the span of S* in |R as ( and 1Is- (v) == arg min ||| ylla, we assume that L satisfies the quadratic growth on {v Vv & 5*.. i.e., For all v & S* there exist a constant such that L(v) L 2 p (.)
reasure-theoreti adjustments. Let S* {v" C. 1s C E R},
Lemma 1 Properties of the DemoDICE Loss). L satisfies the following properties:
(a) I is conver with respect to
18

---

Published in Transactions on Machine Learning Research (02/2026)
(b) is sm-shhhh where L, [1+>]
I+a
(c) VL(v) 1s = for all v E Ills
Proof. To establish (a), note that the convexity of L with respect to v follows directly from Proposition 2 in (Kim et al., 2022).
To establish (b), we start by decomposing the loss function L(v) Li(() + L2(e), where
[v( ----7 -))
(28)
Ax(8, a) 1 + a
Iz(v) = (1 + a)) log (a exp
(29)
We can check the Hessian LVV... Note that (2 = 0. As for the Hessian of L2(v), we first rewrite [2(1) = (1 + ar) (v)) where
g(v) (((o))a exp (Al(saa))
(A((s,a)
1ta
Σ
d (s) exp
(30)
1 a
(x,a)
and define h(v) = log (ogg(v). Then, we have (22 = (1 a1) (2)(v). To simplify notation, let b r and rewrite a, = r(s,a) + (,,,,, where ((s,a) (P( e, RI and e, is the standard basis vector with at state Moreover, define waa = (s, a) and Us.a b(((s, a) laav). Then, we have
g(v)= =
Σ
Wao (xp()
(31)
(s.a)ESAA
For each s, a, we define Ps.a Ws.a cxp(xx.a\/(()), and {Po.a} forms probability distribution over the state-action pairs. The Hessian of h(v) can be derived
Σ Papalosa Σ Persllasa 7
V²h(y) = 62 Σ Pr
(32)
To bound Q, we can compute its trace, i.C.,
traco(Q) Eorallaaal3
Σ Pr.ats,a 8.a
Σ
Ps.a Ps 11.
max 11.. I12 3,a
(33)
5.a
5,a
where the last incquality follows from that E. =
By Cauchy-Schwarz inequality, we have Q 2 0, and its largest eigenvalue satisfies Amax S trace(Q) S max,,a ||,,a 12. This implies that
Q < max | I.
(34)
Morcover, we have
|| 11 = ||/P({|s,a) e. = 72 Σ P(s'|s,))2 -2)p({2sssss) +1.
(35)
Since P(5'(8,a) 2 and Σ. P(s'ls, = we have En a() < and |1..0133 = (1++)) (1+7)2
19

---

Published in Transactions on Machine Learning Research (02/2026)
This also implies that
L2 (1+?)²
(36)
1+a
(1+2)² Ita
Therefore, is Ly-smooth with T
To establish (c), show that the gradient of L is orthogonal to the all-ones vector 1s. Recall the decom- position L(v)= = (v)) + L2(v) from the proof of (b):
Li(P) (1- 191-11 7)p= v,
(37)
L2(v) (1 a.) log g(v),
(38)
where g(v) = ((.a)ssxa "a,a exp(xs.a), with Isaa b(r(s, + t,,a)), and b = 1 1-a = -P((ss,,) Cs.
Then we have the gradient VL(v) (L) VL2(V).
For (L)(v), we have
VLI(v) 1s (1 -1-71) -7); 1s 7
(39)
Since j is a initial probability distribution.
For L2)),, we have (z2(v) = (1 + a)
Eglu) g(r) and inner sum is
Vg(v) 1s
VL2(1) 1s= a) g(v)
(40)
Then we differentiating g(v) gives
0g(r) Ou()')
E(maa) Wapa exp(xx,0) (s)) and
Σ 0g(v)
Vglv) 1s
= Σ
Ws,a exp(xs,0) (la(6)) (s,a)
(41)
Ov((")
s'ES
s'ES
Since the inner sum Eves (s') in Equation (41)
El,a('') = Σ
P()|s,,)) - -1=9-1
(42)
S'ES
s'ES
where Eves a() = 1.Therefore,
Vglv) 1s bb6 1 1) Σ Wa,a exp(xs.a) = b(7 1)g(v).
(43)
Substituting back to Equation (40),
= (1) b(7 - 1)g(v)
=7 1.
(44)
g(v)
Finally, we have
1s= + VL2(v) = (1 (1-7) (7 - 1)
(45)
The result holds for all v e RIS),
Lemma 2 (Convergence of DemoDICE). Let vit) and with) be the sequences generated under DemoDICE, with [S] < oo, and learning rale 1/Lg. Then:
(s, a) (s, S Cn exp Cw (C - v² v*1l2) Ve
1 (1:00) - v² v²112,
(46)
where v² = IIs- (())), and C.. := 2(147)/1
voll+a)
20

---

Published in Transactions on Machine Learning Research (02/2026)
Proof. The proof begins by showing the convergence of the visitation distribution plt) to the optimum v² == IIs. (1)),, followed by the convergence of the density ratio (s,a) to w (s) Recall that S* := f C' RJ denotes the optimal level set of L. The projection Hls. (-) is taken with respect to the æy-norm onto this affine subspace. Without loss of generality, we assume y(0) satisfies vl0) & S We start by establishing the convergence bound for [| - v²1
Since we assume L is quadratic growth on {u lv $ S*} in the beginning of this section, and gradient descent with step size 1j = L/L By the result of Lemma 1(c), have
Iss- (u())) = IIs- (v) =²; vk 2 0.
(47)
Therefore, there exist a constant >> such that for all t2 0
L(v(1)- L(v") 7 $11(0 II.- (vl - C [|ol00 101
(48)
Morcover, applying the result of (Bubeck et al., 2015, Theorem 3.3)
-(v(0)- - L(v*) 2 (T v²13
(49)
1
we have
2 [| -2 ((v0))-L(0) - S Ahg ||(00] v*
(50)
which proves the first part.
Next, we establish the convergence of the density ratio (s)) a) to w (s, a). Their difference can be written
as
(Arv(ssa))=
(An((3,0) a
(s, as (s) exp
w0
eex
(51)
l+a
Applying the Mean Value Theorem to the function F(x)= == exp
[Ita] for each (s, a) there exists (s,a) =
g(+) A,co a) (1 o(t)) (s, a) for some g(t) [0, 1] such that
(Aro((s,a)
(An-(s,a) = exp ( (s,
Ast a) A.- (s.
exp
I+a
exp
1+a
1+a
1+a
(52)
Using w- as, = exp (Av. (s, (),aa)((++))))))))) aa), we have
Ap ( an) - Au-
(s, a) wn (s, = w w²(ssa) exp
exp
(53)
+
1
To bound the difference (A, Aur a)) in Equation (53):
(s, (s, = ? Σ T(s'ss ( -² (s) (0))) s (7+ + 1) max (s) v²((l
(54)
With the finite state space |S| xo and the established convergence of || 111, we have
max (s)- v*()) s (| v²112 s V (1111110) v² 2/L1s |1000) v²l12
AALs
(55)
c.
Vc
Vt
8
Therefore,
2(1 + 7), Ly [|(00] - n" 112
(A - Av-
(56)
Vc
V1
21

---

Published in Transactions on Machine Learning Research (02/2026)
Next, we bound the exponential term exp (s in Equation (53). Since (s, 1+a
convex
combination of Auto (s, a) and Au- (8, we have
2(1 + +1
12 v l2
(s.a) (s. s as, Av-
(57)
vc
vt
where the last inequality follows from Equation (56). Consequently,
(00) (s, Au- (8,
2(1 7), Tup (|
VTg
v²12
cxp
S cxp
(58)
1 +
vd(1 + +a)
Vt
Substituting these bounds into Equation (53), we have
(0, a) - w" (s, a)) I 2w (s, (1+ exp vo(1 +a)
2(1+7)V |1 v² 1112
[
v0112
(59)
va(+a)
Vt
v8
= Cw (s,
( Vt
Cw
[|||00 (111011-10 -v'lla)
- -8 [ (110)~'' v²" v21
1
(60)
where C == 211 v((+ta)
C Convergence of AdaptDICE with Weighting Factor 3())
In this section, we analyze the convergence of the AdaptDICE. algorithm. Recall the definitions in Section 3.2, the cross-domain density ratio is given by weross a) ((t))arr(((i))) (s, a)) + (1 - (())) (s, a), which combines a pre-trained source-domain density ratio and a target-domain density ratio through the weighting factor ((t), where 3(t) : N + [0, 1] denotes a weighting factor that balances the contributions from the source and target domains. Where the mapping functions G, H are optimized minimize the cross-domain loss LMAP Equation (4).
We further define the source and target domain error terms follows:
A (w src (s, a) = |Waro (G (1), (s, a)) - (s, ( (s, == [0 (s, a) - (s,
(61)
and
(t)
sEc = Drrr Ware (G H()) (s, a)) - - (8, Aw = Ber.a) lw (8, - wiat (8, a)l
(62)
To analyze convergence, we adopt the notations and assumptions from Section B, and define the optimal solution set yaar as Star = (Mfrr C. 1 C R. and orthogonal projection in target domain Il s- arg minyes;,, || y/12-
Theorem 1. [Upper Bound of Cross- Mooss-Domain Density Ratio Error Under AdaptDICE) Under AdoptDICE with learning nate 1/L,, for each (s, a), the cross-domain density ratio error is bounded as follows:
(s, - (s, a)) (2)) + (1-3(1)) Cn (u. exp (Ca ||:0000 viarll2
||veor viarlle
Vt
(16)
where Is is the smoothness constant of LDICE, Viar II (v(0)), and C., is constant. Moreover, by selecting 3(t) as
0,
if A S A!
B(1) { otherwise
22

---

Published in Transactions on Machine Learning Research (02/2026)
the expected cross-domain density ratio error satisfies
(s,a)) (0) crass (s, a) - (t a)l S min( (Aw arer (arr).
(17)
Proof. The cross-domain density ratio error is bounded using the triangle inequality:
[s Wear (s, a)l - ((t) (G (H) Winr (s. (1 B(t)) ((t))lu/((sss whar (8, a)l
(63)
(1 ((t)) (s)
(64)
For the second term in Equation (64), we bound (1) (s, a). It follows from Lemma 2 that:
C | viarllz v
| (s, a) - (s, (s, & Cu W (s, exp
V Ilp (0) - varll2,
(65)
where C 2(1+7)yLf
ve(l+a)
We then combine the bounds using B(t). Substitute Equation (65) into Equation (64):
(n(t)ss (8) .) (s, <B(1) Aw! (1 3(1)) Caa wir ax
CCw 11 vear l1| Vl
||
(0)
V
l1
Vt
(66)
Having established the pointwise convergence bounds for both X 0 and X = 0, next show that the expected error satisfies (e.. coo (8, a) Wear(s, I min(A min(A Am(ll) Taking the expectation over Equation (64), we have
Desr [1o a) wast (s, a)l B(a.a)-D... [((t)Aw + (1 ((t)) (s.
(67)
By linearity of expectation, this becomes:
((t) Aw are (s, a)] + (1- (..)). (s, ((t) are + - -(()))
(68)
Choosing B(t) Amild
A Amaar
obtain:
If Aw!t) 2 ( then B(1) = 0, and the expected error is Aw!!) min( Wares Aw
(t)
(t)
If Am!!
(() tar: then (()) = 1, and the expected error Aw!!) min( ( Aae Aw(())
(t)
Thus, the expected error satisfies D(s. (t) (s, a) (s, s min (Amfe, Aw())
(1)
Furthermore, when (t
(t) < Aw!!) selecting B(t) 1 yields an expected error bound of Awlid, which is
smaller than Aw the error obtained by training solely in the target domain with DemoDICE, This implies
a faster convergence rate for Ada in such cases.
23

---

Published in Transactions on Machine Learning Research (02/2026)
D Detailed Experimental Setup
D.1 Settings of Evaluation Environments
To evaluate our proposed AdaptDICE framework, inspired by (Pan et al., 2024; Chen et al., 2026), we conduct experiments in MuJoCo (Todorov et al., 2012) and Robosuite (Zhu et al., 2020), two physics- simulation frameworks widely used for reinforcement and imitation learning. MuJoCo offers locomotion tasks with rcalistic dynamics, while Robosuite provides robotic manipulation tasks, together serving as complementary benchmarks for lesting offline cross-domain imitation learning (CDIL) algorithms.
To systematically examine AdaptDICE under varied cross-domain discrepancies, design source target environment pairs that differ in morphology, observation space, and control dimensionality, while maintain- ing identical lask objectives. For MuJoCo lasks, the domain gap is introduced by modifying the agent's embodiment (e.g., adding legs or joints), which alters both the dynamics and state-action representations.
Hopper with an extra thigh: We add an extra thigh body attachod the torso. following analogous modifications used in (Xu al., 2023). Detailed modifications of the XML file are:
<body name="thighi" pos="0 1.45">
<joint axis="O -1 0" name="thigh ttageehhgghhittiiiii joointn" pos="O 0 1.45" range="-150 o" type="hinge">>
<geom friction="O. fromto="0 1.45 1.85" waaih_hhihhhgeoe size="O.00 type="capsule"/>
</body>
Ant with an extra leg: We add fifth leg ("middle leg") to the central body, inspired by the cross-domain transfer settings in (Zhu et al., 2024), which similarly modifies agent morphology to induce representation and dynamics gaps. Detailed modifications of the XML file are:
<body name="middle, _e pos="0 0">
<geon fromto="O.0 0.0 0.0 0.0 -0.28 0.0" name="aux_5 Bacuux____eee 5_geom" size="0.08" type="capsule">
<body name="aux_6 pos="O.0 -0.28 0.0">
<joint axis= 0 1" name="hip_5 0os-"0.0 0.0 0.0" range="-10 10" type="hinge">>
<geom fromto="0.0 0.0 0.0 0.0 -8.28 0.0" name="middle degeee"ddddleeeeeeeeeeee geg_ size="0.08"
type="capsule"/>
<body pos="0.0 -8.28 0.0">
<joint axis="1 1 0" name="ankle_e 5" pos="O.0 0.0 0.0" range="30 70" type="hinge"/>
<geom frosto="O.8 0.0 0.0 0.0 -0.56 0.0" name="fifth Baccsaannkle__ee geom" size="0.08"
caps="caps
</body>
</body>
11
</body>
HalfChectah with extra back leg: We follow the morphological modification introduced by (Zhang et al., 2021), adding second back leg set (thigh, shin, foot) to create embodiment mismatches while preserving the original locomotion objective. Detailed modifications of the XML. file are:
<body name="bthigh2" pos="-,5 .06 0"> <joint axis="O 0" damping="6" name= "thighh2" pos="O 0 0" range="-,52 1.05" stiffness="240 type="hinge"/>
<geon axisangle="1 -1 3.8" name-"bthigh2" pos=", .1 0 -.13" size="0.00 145" type="capsule"/>
<body name="bshin2" pos=". .06 -.25">
<joint axis=*0 0" damping="4.55 name="bshin2" pos="O 0" range="- .78 785"
stiffness="180" type="hinge"/> < ggeom axisangle="0
-2.03" name="bshin2" pos="" .14 0 -.07" rgba="0.9 0.6 0.6 1"
size="0. 046 115" type="capsule"/>
<body name="bfoot2" pos=*-.28 -.1
<joint axis="0
damping="3" name="bfoot2" pos="0 0" range="- .785" stiffness="120* type="hinge"/>
<geom axisangle="O -.27" namo="bfoot2" pos=". .03 -.097" rgba="0.9 0.6 0.6
size="0.0df 046 094" capee""apsule"/>
24

---

Published in Transactions on Machine Learning Research (02/2026)
10 </body>
11 </body>
12
</body>
For Robosuite tasks, we evaluate transfer across robot arms (Panda UR5e) performing identical manipula- tion tasks, introducing embodiment and kinematic mismatches while keeping object configurations and goals consistent. This ensures that performance differences arise from cross-domain transfer capability rather than task definition. The corresponding state and action dimensions of cach source-target pair are summarized
in Table
Table 5: State and action dimensions of the source and target domains.
Environment
Source
Target
State/ Action
State/ Action
Hopper
11
3
13 /
4
HalfCheetah
17
6
23 9
Ant
27 / 8
31
/10
BlockLifting
42 / 8
47 / 7
DoorOpening
46 / 8
51 / 7
TableWiping
37 / 7
/
Below, we present the target environment pairs used in our experiments. For each task, the left image shows the source domain, and the right image shows the corresponding target domain.
Hopper and Three-Thigh Hopper:
Ant and Five-Leg Ant:
HalfCheetah and Three-Leg HalfCheetah:
Panda and UR5e BlockLifting:
25

---

Published in Transactions on Machine Learning Research (02/2026)
Panda and UR5e DoorOpening:
Panda and UR5e TableWiping:
D.2 Dataset Configuration of Each Experiment
Table 6 summarizes the larget-domain dalaset configurations used in our experiments. Each dataset. is constructed by mixing small number of expert trajectories with a larger set of sub-optimal trajectories, which consist of expert demonstrations and purcly random rollouts. We categorize the datasets based either the proportion of expert versus sub-optimal trajectories (Default, Expert Rich, Sub- Sub-Optimal Rich) or the overall scale of available data (Data Rich). For all experiments, used the same dataset setting of source domain, as described in Table 7.
Table 6: Configurations (including Data Rich) of the target-domain dataset in various scenarios.
Environment
Dataset Set
Expert Traj. Sub-optimal Traj.
Sub-optimal Composition
Hopper/ TableWiping
60
10 expert + 50 random
Defanlt
Others
101
1 expert 100 random
Expert Rich
60
10 expert + 50 random
Sub-Optimal Rich
300
50 expert + 250 random
Expert Rich
101
expert + 100 random 5 expert 500 random
Others
Sub- Optimal Rich
505
All
Data Rich
10
800
400 expert + 400 random
Others includes Ant, HalfCheetah, BlockLifting, and DoorOpening.
Table 7: Configuration of the source-domain dataset.
Expert Traj.
Sub-optimal Traj. Sub-optimal Composition
400
2000
400 expert + 1600 random
Regarding the acquisition of expert demonstrations, in addition to the official MuJoCo hdf5 datasets provided by D4RL (Fu et al., 2020), the remaining expert datascts were generated by training policies with Soft Actor- Critic (SAC) (Haarnoja al., 2018). Specifically, we train an SAC agent until reaches expert-level performance, and then the trained policy to collect expert trajectories.
26

---

Published in Transactions on Machine Learning Research (02/2026)
D.3 Training Curves
We provide the training curves evaluation, ablation study, and data configuration experiments described in Section 4.
Evaluation Results. The experimental results in Figure 2 demonstrate that AdaptDICE achieves the superior and stable performance across all testing environments. Compared to baseline methods such SMODICE, GWIL, and IGDF+IQ-Lcam, AdaptDICE not only exhibits faster convergence during the carly stages of training but also attains significantly higher final average returns; while other methods largely struggle to learn effective policies in these settings, AdaptDICE maintains robust performance.
Ablation Study, Based on the results Figure 3, our ablation study confirms the criticality of combining both source and target domain weights. While the Target-Only variant suffers from severe performance collapse in environments such HalfChoetah, the Source-Only variant yields low returns due to limited transferability. However, despite the poor performance of the Source-Only baseline, it is essential for the stability of our method. The proposed hybrid density Weross effectively leverages this source information to anchor the learning process, ensuring that AdaptDICE maintains robust performance in the later stages of training without diverging.
Effect of Dataset Configurations. The results Figure 4 indicate that expanding the dataset scale en- hances model performance; both increasing expert data (Expert-Rich) and sub-optimal data (Sub-Optimal Rich) yield improvements over the Default configuration. Notably, utilizing large amount of sub-optimal data (indicated by the blue line) achieves the best performance in most tasks. This demonstrates Adapt- DICE's strong capability to extract useful information from imperfect demonstrations to refine its policy.
2503
1200
502
10
10
20
(a) Hopper
(b) Ant
(c) HalfCheetah
Blodksiting
PeorOpentty
Tackeei cing
375
355
322
300
72
22
-25
20
70
50
Trainin Steps Oclof)
Training Steps (xtp1)
Traiiing Stose (x20")
(d) BlockLifting
(e) Door Openingg
(f) TableWiping
AdaptDICE
SMODICE
GWIL
IGDF+1Q-Learn
Figure 2: Training curves of AdaptDICE and the baseline methods in the Default setting: (a)-(c) MuJoCo locomotion tasks; (d)-(f) Robot arm manipulation tasks in Robosuite.
27

---

Published in Transactions on Machine Learning Research (02/2026)
hosper
3200
4500
2503
2203
3500
g 3000
1590
5003
1202
2002
300
2500
2000
10
30
30
20
30
10
20
30
60
50
Traiiing
Traiiig
Trakiigg Stape
Hopper
(b) Ant
(c) HalfCheetah
DecrOeating
Tackon
375
100
25
-20
Training Staps (ciof)
Trariin Stoss (420")
Trarning Stass (x20")
(d) BlockL.ifting
(e) DoorOpening
(f) TableWiping
Target-Only
Source-Only
AdaptDICE
Figure 3: Ablation study: Training curves of the full AdaptDICE and its two ablation variants (í.e., using only Ware or Wrar).
3203
Ast
HalCheraa
12000
2302
5000
10000
2303
6000
8000
1502
5000
0503
4000
2000
Tranning Stass (x207)
Trairing Stoze (x20f)
Training Stazs (x101)
(a) Hopper
(b) Ant
(c) HalfCheetah
120
Bacidittig
DecrOparing
200
1.50
375
50
20
5c
-20
20
52
10
23
10
70
50
Trainin Stopa (11311
Truining Stoss 3420%)
Training Stase (420")
(d) BlockLifting
(e) DoorOpening
(f) TableWiping
Default
Expert-Rich
Sub-Opt Bub-Optimal Rich
Figure 4: Effeec of dataset configurations under AdaptDICE: We compare performance across the Expert Rich and Sub-Optimal Rich regimes, showing that both types of data expansion lead to improvements over the Default dataset setting.
28

---

Published in Transactions on Machine Learning Research (02/2026)
D.4 Effect of Dataset Configurations Baseline Methods
In the main text, we presented the analysis of AdaptDICE under varying dataset configurations, highlighting that both Expert Rich and Sub-Optimal Rich regimes improve performance compared to the Default. To assess whether other baseline methods (SMODICE, GWIL, and IGDF+1QLear)) also benefit from increased data quantities, we conduct similar experiments for these algorithms, using the Data Rich configurations outlined in Table 6.
The training curves in Figures 5 to 7, as well as those final performance values in Table and Table 9, illustrate the impact of incrcased dataset sizes on the performance of SMODICE, GWIL, and IGDF+IQ- Learn across the evaluated environments. Compared to the Default settings, SMODICE and IGDF+IQ- Learn exhibit consistent performance improvements under the Data Rich settings. In contrast, GWIL shows limited improvement under Data Rich settings, with consistently poor performance across all environments. This can be attributed to GWIL's reliance on online learning and its unsupervised Cross- Domaan Imitation Learning (CDIL. assumption of isomorphic state-action spaces. The isomorphic assumption often fails in diverse environments with varying dynamics, leading to poor generalization. Moreover, GWIL's online learning nature limits its ability to effectively utilize larger offline datasets, as struggles to adapt to increased data variety without explicit supervision or alignment with target domain dynamics.
These results highlight that methods like SMODICE and IGDF+IQ-Leane which are designed to handle offline data efficiently, benefit significantly from increased dataset sizes, while GWIL's performance is con- strained by its methodological assumptions and online learning requirements.
Hosser
Hollhhothh
2502
2200
1303
500
Tranning Stoes (4.07))
Triring Stess (X101)
Trining Stopp Ixto")
(a) Hopper
(b) Ant
(c) HalfCheetah
Bocktiling
Dasrdparing
Taskon clta
120
200
1.00
80
150
z0.
40-
50
20-
13
20
40
30
23
43
30
30
40
50
Trainig Steps 019"1
Tranning Staps 3610")
Training Stess (410")
(d) BlockLiffing
(c) Door Opening
TableWiping
Default
Data-Rich
Figure 5: Effect of dataset configurations under SMODICE.
29

---

Published in Transactions on Machine Learning Research (02/2026)
Nos2s!
HellShatteh
1.0
-200
-2000
-400
60
900
20
-3000
-1000
20
30
20
30
30
20
35
30
Trainig Stope
Traning (410))
Trining ExEof)
(a) Hopper
(b) Ant
(c) HalfCheetah
BlaceLiting
PoorGpen
Tackowi
The Aurmin
42
-20
-20
-20
-30
Tranning Staps (K20T)
Trainin Staps (xto1)
Trariing Stats (x207)
(d) BlockL.ifting
(e) Door DooOOngging
TableWiping
Default
Data-Rich
Figure 6: Effect of dataset configurations under GWIL.
Biozper
Art
HalCherrat
2009
14000
1753
4000
5 1590
3200
0030
6020
4030
-1000
-2000
10
20
Training Stope (x10%)
Trahing Steps C4t0f)
Training Stape (x10f)
(a) Hopper
(b) Ant
(c) HalfCheetah
200
30
125
60
300
75
50
-40
20
20
(d) BlockLifting
(e) DoorOpening
TableWiping
Default
Data-Rich
Figure 7: Effeec of dataset configurations under IGDF+1Q-Leand Learn.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Table 8: Experimental results other baseline methods in Data Rich setting.
Environment.
SMODICE
GWTL
IGDF+IQTearm
Hopper
1996.0 219.1
6.4 6.1
466.3 + 153.3
Ant
5467.9 2673.1
-3177.1 + 1559.0
-551.6 = 582.7
HalfCheetah
2718.7 5468.2
-713.8 307.8
8184.5 + 2083.3
BlockLifting
47.3 32.2
0.5 - 0.1
71.5 = 15.3
DoorOpening
184.0 16.6
0.3
162.8 33.6
TableWiping
24.6 + 9.5
7.6 + 3.8
5.66 + 22.1
Table 9: Experimental results of other baseline methods in Default setting,
Environment
SMODICE
GWIL
IGDF
Hopper
2046.6 + 53.3
9.8 0.03
286.6 + 78.6
Ant
86.3 30.6
-8991.6 17.4
-358.3 1106.6
HalfCheetah
-15.1 + 42.5
-800.7 228.8
2179.5 2422.1
BlockLifting
0.8
0.001
7.4
DoorOpening
9.6 + 14.6
34.5+46.0
7 52.8
TableWiping
9.8 + 7.6
0.6
+15.5
D.5 Computational Resources and Hyperparameters
All experiments were conducted on a high-performance computing cluster. The training jobs were executed on a server equipped with an Intel Xeon Gold 6154 CPU (3.0 GHz, 36 cores) and eight NVIDIA Tesla V100 GPUs, cach with 32 GB of memory. The software environment compatibility with the GPU architecture. This computational setup provided sufficient resources to support large- batch training and efficient parallelization across multiple GPUs, which was essential for handling the complexity of our experiments.
based on CUDA version 12.9 to ensure
Table 10: Hyperparameters used in experiments.
Component
Setting
7 (discount factor)
0.99
a (regularization coefficient)
0.05
Policy hidden dim
(256, 256)
Critic hidden dim
(256, 256)
Cost hidden dim
(256, 256)
Batch size
512
Optimizer
Adam
Learning rate
10 4, 1 x 10 4]
Gradient penalty coeffs
(0.1, 1x10 4)
Actor T.2 penalty
1x10
Mapping networks (decoder / action decoder)
(256, 256), Adam, 3x
D.6 Implementation of Baseline Methods
In this subsection, we provide the detailed implementation of the baseline methods considered in our exper- iments as follows:
DemoDICE. DemoDICE (Kim et al., 2022) is an offline imitation learning algorithm that directly opti- mizes a density ratio estimator to recover the pscudo reward function. It has been shown be effective in settings with limited demonstrations by exploiting unlabeled trajectories. We used the official PyTorch implementation from https:/ / gig......... http:///ggibbb...ogge/aaeeeeeeee hyeonn/hyyn///// htwpeen/giiiiaattnnniiiii.
31

---

Published in Transactions on Machine Learning Research (02/2026)
SMODICE. SMODICE (Ma al., 2022) is offiine imitation learning algorithm that performs state- occupancy matching via distribution correction estimation, suitable for cross-domain transfers. We used the official PyTorch implementation from nttps://githe httpa///gituub..o////ooattiii for our CDIL experiments, we trained the discriminator using expert observations from source domain and oflline dataset from target domain to estimate state occupancy distributtioss. The policy was trained using target-domain imperfect demonstrations, as these provide diverse behavior for robust learning. Contrary the original paper's assumption of identical state-action spaces, we applied zero-padding truncation to align state-action vectors and ensure dimensional consistency.
//MDDIC.. To adapt SMODICE
GWIL. GWIL (Fickinger et al., 2022) is a cross-domain imitation learning method that employs Gromov- Wasserstein optimal transport to align behavioral distributions between source (expert) and target (Jearner) domains, enabling transfer without requiring paired data dimension matching. We used the official Py- Torch implementation from https://githu:
com/facebokkoseearch hegee /ggl.. Originally designed for online settings, we adapted GWTI. to offline CDIL by replacing the online replay buffer with an offline dataset of imperfect demonstrations. Consistent with the original paper, which uses only source demonstration for training, we also employed 1 source domain expert trajectory our experiments
IGDF +IQ-Len-a IGDF (Wen et al., 2024) a cross-domain offline reinforcement learning method that uses cont rastive learning learn invariant state-action representations for data filtering across domains. We adapted it offline imitation learning (IL) by integrating IQ-Learn (Garg et al., 2021), which optimizes poli- cics directly from demonstrations without adversarial training. We used the official Py PyTorch implementations (IGDF) and (IQ-Learn). In the encoder phase, we trained contrastive encoder using imperfect demonstrations from both source and target domains. In the IL phase, we filtered high-quality source domain expert trajectories (5 - (5=0.5 0.5, selecting the top-50% based on representation similarity) and used target domain expert demon- strations to train the policy via IQ-Learn. Despite the paper's assumption of identical state-action spaces, we applied zero-paddin pro-ppaddng and truncation align dimensions.
from https: //gt thub. http///attleeee/eeeee
com/Div- Inf
D.7 Additional Experimental Results
Cross-domain transfer under domain shifts in transition dynamics. To evaluate domain shifts with mismatches in transition dynamics, we conducted additional experiments using the Off-Dynamic Reinforce- ment Learning (ODRL) benchmark proposed by (Lyu et al., 2024b). This benchmark is explicitly designed to assess robustness under dynamics mismatches. In our experiments, we evaluated AdaptDICE in target domains with modified transition dynamics, including Hopper with gravity scaled to twice the original value and Ant with ground friction reduced to 0.1x the original. The results (Figure 8) show that AdaptDICE consistently outperforms baseline methods under these dynamics shifts.
beceogge Nenum
2920
2000
15:0
0-
1020.
-1800
-2000
10
10
20
30
50
Thiniing Skape oit0f)
Trahing DdL0f)
Hopper
(b) Ant
AdaptDICE
SMODICE
GWIL
IGDF+10-Learn
Figure 8: Experiments Hopper and Ant in the off-dynamics settings.
32

---

Published in Transactions on Machine Learning Research (02/2026)
Cross-domain transfer from only one source-domain expert trajectory. In Figure evaluated Adap using only single expert trajectory from the source domain. Notably, even in this low-data regime, AdaptDICE achieves competitive performance relative to GWIL, highlighting that its gains are not solcly attributable to access to larger source datasets.
Iogoer
HapCCeeeth
1200
4003
6030
2003
6030
2010
400
2003
10
20
20
20
Trasiin Stage (5201)
Traning Slaps (62e
Thennig
Hopper
(b) Aut
HalfCheetah
Enorboaring
Thekeff ping
70-
80.
Ausraggs Resumn
200
Tra King Stege (c22))
Training Stece idaa
Traking Seep (oCo))
(d) BlackT.ffiig
(e) DoorOpening
(f)
TableWiping
AdaptDICE
GWIL
Figure 9: Performance comparison of AdaptDICE and GWIL with single source-domain expert trajectory.
2003
2000
1300
3000
4030
1000
2000
2010
000
1000
10
20
10
30
12
30
52
20
30
50
Towinng Staps Ool0fs
Strpp
(a) Hopper
(b) Ant
(c) HalfCheetah
Techot
Auange Raturs
100
75
50
25
-00
20
22
Wine Stepe (410%)
Trshi ng Step-
Trahing Stype (020F)
(d) BlockLifting
(e) Ooeninggin
(f) TableWiping
source expert
100 source expert
400 source expert
Figure 10: AdaptDICE's performance with pre-training with different amounts of source data (1, 100, 400 expert trajectories).
33

---

Published in Transactions on Machine Learning Research (02/2026)
Table 11: The final performance pre-trained models using different source-domain dataset configurations.
Environment
source expert trajectory
100 source expert trajectories
400 source expert trajectories
Hopper
2384.1
3024.9
3568.6 6
Ant HalfCheetah
1786.2
3262.2
5023.6 6
-5.7
8233.0
11158.5
BlockLiftinn
28.4
165.7
193.5
DoorOpening
13.9
164.5
203.0
TableWiping
80.8
85.4
Scaling effect of source-domain data. In Figure 10, we conducted an empirical study source data scaling across multiple environments, evaluating AdaptDICE with varying numbers of expert trajectories (1, 100, and 400). The results indicate that, in most environments, performance is relatively insensitive to the amount of source data. This behavior is consistent with the adaptive weighting mechanism in AdaptDICE, which down-weights unroliable source information and mitigates negative transfer.
Adaptive versus fixed B. We also compared the performance of AdaptDICE with our adaptive weight- ing scheme against that with fixed / values (0.3, 0.5, and 0.7) in the Hopper and Ant environments. The results in Figure 11 show that smaller fixed B values generally perform better under large domain dis- crepancies, they bias learning toward the target-domain density ratio whar However, the adaptive B(t) consistently matches or outperforms the best fixed setting by dynamically adjusting throughout training. We also visualize the evolution of 3(t), confirming its adaptive behavior.
Sensitivity to th We further evaluated the sensitivity to this parameter by testing a E {0.5, 0.9, 0.99} on Hopper and Ant (w = 0.9 the default value used throughout the experiments). The results in Figure indicate minimal performance variation across these values, supporting our claim that does not require task-specific tuning and does not function critical hyperparameter in practice.
Hopzer
Honner
15:0
Thet Aver
2.0
1020
0.3
Trkning Stups (4101)
Training Stens (x20P)
(a) Hopper
(b) ((t)
0.5
6030
0.7
3030
2000
0.4
2020
0.3
1500
10
Thining Stape ccteft
Traising Steps c410")
(c) Aut
(d) 3(1)
B 0.3
0.5
B= 0.7
Adaptive 8
Figure 11: (a), (c): Comparison of AdaptDICE with adaptive 3 and various fixed / values (0.3, 0.5, and 0.7) on Hopper and Ant; (b), (d): Evolution of (tt) under the adaptive weighting scheme on Hopper and Ant.
34

---

Published in Transactions on Machine Learning Research (02/2026)
2020
10:00
2210
2020
4000
33.00
30
30
13
23
30
50
Toknnng Staps
Traiiin Ixtef)
(a) Hopper
(b) Ant
0.5.5
0.9
4= 0.99
Figure 12: Evaluation of AdatDICE with different &b values on Hopper and Ant.
Loss curves of LMAP. In Figure 13, we report the training curves of the mapping quality measured by the proposed mapping loss IMAP.
In most environments, we observe a clear decrease in LMPP over training, indicating progressively im- proved alignment between the source and target domains. These cases also coincide with the strong final performance of AdaptDICE, suggesting a positive correlation between improved mapping quality and successful transfer, which is consistent with the intuition behind Theorem 1.
On the other hand, in Ant and HalfCheetah, the LMAP values remain relatively flat and do not exhibit a clear downward trend. This behavior is closely aligned with the ablation study results reported in Appendix D.3 (Figure 3). In these environments, the source-domain component Wsrc fails to enable effective transfer and does not achieve meaningful performance when used alone (cf. the "Source-Only" curves in Figure 3). As result, the source domain provides limited useful information to the target domain, which inherently limits both the improvement of the learned mappings and the potential benefits of cross-domain transfer.
wwww
3.2
3.0
2.8
5.4
ic3o")
Hopper
(b) Aut
(c) HalfCheetah
Raritittng
DeneChenirg
4.0
3.5
3.0
1.5
0.
1.0
0.0
Trahn ng Steps 032"1
Traiinn Stope Icto":
Tra ring Stepe ic1o"
(d) BlockLifting
(c) DoorOpening
TableWiping
Figure 13: Loss curves of LMAP.
35

---

Published in Transactions on Machine Learning Research (02/2026)
hosper
HallCteral
1010
12020
3000
10030
2303
0030
1503
6000
5000
1202
0030
300
2000
2000
10
20
30
30
20
30
30
20
30
50
Traiiing (4207)
Trairing Steps (4309)
Traning (420f)
(a) Hopper
(b) Ant
(c) HalfCheetah
Expert-Rick Rich
Cross Fitting
Figure 14: Experimental results of AdaptDICE with cross-fitting on B(t) and Wear-
Calculation of (1) through cross-fitting. Recall from Equation (19) and Equation (21) that in practice B(t) needs to be estimated from statc-action pairs and involves another stochastic estimate Wtar To mitigate the potential circular dependencies, enhancement is to decouple the selection of B(t) from the estimation of wis through cross-fitting, Figure 14, we conducted an additional experiment using cross-fitting: We split the target-domain dataset Diar into two disjoint halves (D, and D2). First, we estimated Wsre and Waar on D1, while computing the proxy errors Aware and Awili (for B(t)) on the held-out Dz. For the policy update via Wtross (Fquation (6)); we combined ((t) from DD with Ware, Whar from D, Similarly, we also performed the reverse: estimating Wsre and wtar on D2 and ((t) on D1. Then, we combine both losses to compute the final LBc (Equation (9)). This ensures B()) validated on data that is not used in learning the density ratios.
We evaluate this variant of AdaptDICE Hopper, Ant, and HalfCheetah under the Expert-Rich setting (which has more target-domain trajectories and hence is more appropriate for this data splitting than the Default setting). We observe that there appears to be improvement in the early training steps in some of the tasks (e.g., Hopper) and rather mild differences in the final performance between those with and without cross-fitting.
D.8 Normalizing Flows for the Mapping Functions
Following (Brahmanage et al., 2023) the action-constrainne RL literature (Lin et al., 2021; Hung et al., 2025), employ normalizing flows to ensure that the outputs of G and H lie within the source-domain feasible region, without relying on projection that involves an online optimization procedure (e.g., quadratic program). Specifically:
Neural network architecture of G and H: In constructing the mapping functions G and H, we use several fully-connected layers to first map the target-domain states/actions to some latent dummy region, followed by the normalizing flow model that transforms these latent vectors into the source-domain feasible region.
Latent dummy region: We define a simple latent dummy region as a N-dimensional hypercube (c.g., 1)*), where N denotes the dimensionality. This latent region serves as the domain of the input distribution of the normalizing flow.
Flow model pre-training and usage: We employ a conditional RealNVP flow consisting of six affine coupling layers with 256-unit MLPs. The flow is pre-trained completely offline by maximizing the log- likelihood of feasible samples, which are collected using Hamiltonian Monte Carlo (HMC). Pre-training is performed using Adam optimizer for 5,000-20,000 epochs, depending
36

---

Published in Transactions on Machine Learning Research (02/2026)
on the environment. During target-domain learning, the parameters of the flow model are kept fixed, and only the weights of the prepended fully-connected layers are updated.
37


Reviews:
[{"review_id": "Kbabr17Lk1", "reviewer": "Reviewer_MikH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a new algorithm named AdaptDICE (Adaptive Cross-Domain Imitation Learning with Distribution Correction Estimation) to address the current challenges of Cross-Domain Imitation Learning (CDIL), specifically on \"Semi-Supervised\" setting (SS-CDIL) where the target domain has very limited labeled expert demonstrations but access to unlabeled, imperfect trajectories.\n\nThe proposed AdaptDICE method extends the offline imitation learning method DemoDICE to the cross-domain setting using three main components:\n- Cross-Domain Mapping Loss ($L_{MAP}$): A loss function that learns state ($G$) and action ($H$) mappings by minimizing the Bellman error in the mapped source-domain space, utilizing a pre-trained source critic ($Q_{src}$).\n- Hybrid Density Ratio: The extraction of the target policy utilizes a hybrid density ratio ($w_{cross}$) that combines the pre-trained source density ratio and the learned target density ratio.\n- Adaptive Weighting ($\\beta(t)$): An adaptive mechanism that dynamically balances the influence of the source and target domains based on their relative estimation errors, removing the need for manual tuning.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper presents sufficient evidence through theoretical derivations and empirical evaluation to support its claims:\n- The central claim that AdaptDICE works with minimal supervision is strongly supported by Table 2, where the method significantly outperforms baselines (SMODICE, GWIL) in various environments using the \"Default\" dataset setting (limited target expert data).\n- The ablation study in Section 4.3 and Table 3 clearly isolates the contribution of the hybrid approach. The results show that using only the source density ratio ($w_{src}$) or only the target density ratio ($w_{tar}$) leads to performance collapse or suboptimal returns in complex tasks, validating Eq. (6)\n- The claim that the adaptive weighting factor $\\beta(t)$ stabilizes training and outperforms fixed weights is supported by Figure 11, which compares the adaptive method against fixed values, showing that the adaptive mechanism converges to the optimal weighting strategy dynamically.", "audience_yn": "Yes", "audience_explanation": "The paper is relevant to researchers working on reinforcement learning, imitation learning, robotics and/or transfer learning.", "requested_changes": "- In Section 3.5, the authors mention using \"normalizing flows\" to ensure mapping functions $G$ and $H$ map to a feasible region15. This is a critical implementation detail that is currently under-explained. Please expand on how the dummy region is defined and how the flow is pre-trained or updated during the main loop.\n- The method relies on a pre-trained source-domain pseudo Q-function $Q_{src}$16. While the paper assumes this is \"optimal,\" in practice, source policies may be suboptimal. A brief discussion or a small experiment on how AdaptDICE behaves if $Q_{src}$ is noisy would add significant value.", "broader_impact_concerns": "None - the paper does not leverage any sensitive data or explicitly enable harmful dual-use applications beyond general control capabilities.", "additional_comments": "Overall, the paper is well-structured and the complex methodological contributions are easy to follow.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CSVHsfhvXS", "reviewer": "Reviewer_QVu5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "I think the main contribution of this work is a new machine learning setting, i.e., SS-CDIL, that formalizes semi-supervised cross-domain imitation learning where you have a small set of target expert demos plus many unlabeled/imperfect target trajectories, and optionally source-domain offline data. \n\nOther contributions include:\n- Algorithm (AdaptDICE): proposes an offline CDIL method that extends DICE-style distribution correction to cross-domain transfer with state/action/dynamics mismatch using (i) a cross-domain Bellman-consistency mapping loss + a hybrid density ratio for policy extraction + adaptive weighting.\n- Practical recipe: implements mappings via normalizing flows to hit \"feasible regions\" and uses a moving-average estimator for the adaptive weighting in practice.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The main claims are mostly backed by the evidence provided. 2 minor points need scoping:\n- \"without requiring...assumptions on isomorphism\": while you don’t assume explicit isomorphism, learning G and H implicitly assumes there exists some mapping that makes Bellman consistency meaningful and that your function classes/optimization can find it. That’s still an assumption, just weaker/implicit.\n- \"hyperparameter-free adaptation\" is misleading: there are still many knobs (network architectures, optimizers, learning rates, flow training, smoothing, DICE temperature/regularization, etc)", "audience_yn": "Yes", "audience_explanation": "This work will be of interest to Imitation Learning and Robot Learning researchers.", "requested_changes": "- Replace \"hyperparameter-free\" with \"no manual tuning of $\\beta$\" (or self-tuned mixing?) and explicitly list remaining hyperparameters.\n- Decouple $\\beta(t)$ selection from $w_{\\mathrm{tar}}^{(t)}$ by computing the proxy error on a held-out target split (or via cross-fitting/ensemble disagreement), so $\\beta(t)$ isn’t “validated” by the same estimate it helps produce.  \n- State explicitly that transfer is via querying source signals on mapped tuples $(G(s_{\\mathrm{tar}}), H(s_{\\mathrm{tar}},a_{\\mathrm{tar}}))$ (i.e., using $w_{\\mathrm{src}}$/$Q_{\\mathrm{src}}$ as priors) while $w_{\\mathrm{tar}}$ and $(G,H)$ still must be learned from target data to avoid degenerate mappings.", "broader_impact_concerns": "No pressing concerns on ethics and societal impact come to my mind, but I recommend that the authors include a broader impact statement section.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bVD8dvwSY3", "reviewer": "Reviewer_89eE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents AdaptDICE, an algorithm for policy learning in the specific setting of semi-supervised cross-domain imitation learning (SS-CDIL), as introduced in the paper. The authors emphasize why CDIL is important, the two domains often available: source domain and target domain, the main approaches often taken: CDIL with proxy tasks(a supervised setting) and unsupervised CDIL,  and why SS-CDIL is an acceptable approach that gets the best of both worlds. In SS-CDIL, AdaptDICE has three main parts: a mapping loss, density ratios from the source and task domains, combined into a hybrid density ratio, and a weighting factor $\\Beta$(t) that allows adaptive blending of the two domains. \n\nThe paper presents a theoretical convergence guarantee analysis for the density ratio in AdaptDICE, practical implementation guides for AdaptDICE, and then experiments on MuJoCo and Robosuite simulators benchmarking AdaptDICE against GWIL, SMODICE, and IGDF+IQ-Learn. Each of these three algorithm benchmarks is adjusted to work in the SS-CDIL setting before being compared with AdaptDICE. AdaptDICE is shown to overall outperform the three benchmarks across six tasks: Hopper, HalfCheetah, Ant, BlockLifting, DoorOpening, and TableWiping. \n\nKey strengths of the paper include the theoretical proofs and derivations (including a convergence guarantee) in both the main paper and the appendix, and the breadth of experimental design and ablations. A primary weakness of the paper is the omission of certain tasks in some experiments, without clarification of why those choices were made.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper robustly and clearly makes its case for why SS-CDIL is needed, supported by motivation, theoretical proofs, evaluations, and a literature review, and systematically builds the various components of AdaptDICE.", "audience_yn": "Yes", "audience_explanation": "The findings of this paper would be of interest to researchers and audience members in the fields of Imitation Learning (IL), Reinforcement Learning (RL), and Robot Learning.", "requested_changes": "1. In the ablations of AdaptDICE (Section 4.3) and the effect of dataset configurations (Section 4.4), why are the Hopper and TableWiping tasks omitted? The authors should either include them or provide sufficient justification for why they are omitted, given that the prior evaluations included them. \n\n2. Similarly, in Appendix D4, while considering the effect of Dataset Configurations for the benchmarks, Ant and Table Wiping are omitted. First, the resultant task set is not the same as in Section 4.4. Secondly, authors should either include the missing tasks or provide sufficient justification for omitting them. \n\n3. The authors should say more about how they made the design choices for the changes in the target domains (versus the source domain MuJoco. Zhang et al. (2021) only include the HalfCheetah task. How did the authors determine what to change across domains for the other two tasks, and do we know if these choices affect the algorithm performance?", "broader_impact_concerns": "The paper has no Broader Impact statement section.\n\n\nBeyond the known concerns about training costs for machine learning projects, no broader, unique impact concern comes to mind.", "additional_comments": "On a minor note, the paper contains a few grammatical or formatting errors that need correction and would benefit from general proofreading.", "extra_scores": {}, "extra_text": {}}, {"review_id": "l5wkaSYd38", "reviewer": "Reviewer_PiJH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Semi-Supervised Cross-Domain Imitation Learning (SS-CDIL) to enable cross-domain policy transfer by combining small amounts of target-domain expert demonstrations with unlabeled “imperfect” trajectories. The authors propose AdaptDICE, an algorithmic framework for SS-CDIL, which learns cross-domain state-action mappings and adaptively combines source and target domain knowledge. The method is evaluated on MuJoCo locomotion and Robosuite manipulation tasks, demonstrating consistent improvements over baselines. While I have a few questions about the baseline comparisons and some parameter choices, the paper is clearly written, novel, and well-motivated.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The main claims about AdaptDICE's performance are generally well-supported by performance on a wide range of tasks. The ablation studies effectively demonstrate that the approach outperforms using either source or target knowledge alone. However, some claims could be made stronger with additional support:\n\n- The experimental comparison doesn’t fully disentangle algorithmic advantages from data availability advantages. AdaptDICE uses substantially more source data (400 expert trajectories, Table 7) than baselines (especially for GWIL, which uses 1 trajectory).\n\n- The authors claim \"hyperparameter-free adaptation\", but then there is a moving average parameter psi=0.9, which looks like a hyperparameter. The paper also lacks ablation against intermediate fixed β values to demonstrate that adaptivity is necessary.\n\n- The theoretical bound depends on the quality of the learned cross-domain mappings, but the paper does not empirically characterize what counts as low error, or how much performance degrades as error increases.", "audience_yn": "Yes", "audience_explanation": "This paper is motivated by a very practical/prevalent constraint, which is that in many domains, expert demonstrations in the target environment are expensive to collect, while related unlabeled data is much easier to obtain. An approach that can make use of both will be appealing to a wide range of researchers across the TMLR audience, especially given the demonstrated improvements on a wide range of tasks. Overall, the combination of a new problem setup and a concrete, theoretically supported algorithm fits well within TMLR’s scope.", "requested_changes": "Critical for acceptance: None. The paper meets TMLR's acceptance criteria as-is.\n\nThere are a few additions that I think would strengthen the work: \n\n1. The experimental setup raises fairness concerns. AdaptDICE uses 400 expert + 2000 sub-optimal source trajectories for pre-training, while GWIL uses only 1 source expert trajectory (a 400× difference), with other baselines falling in between. Several baselines are adapted from their original contexts, and some of the poor baseline performance may stem from these mismatches rather than fundamental algorithmic differences. The substantial source-data advantage should be more clearly acknowledged, and the authors could potentially add a source data scaling analysis. E.g., provide experiments showing AdaptDICE’s performance when pre-trained with different amounts of source data (e.g., 1, 10, 50, 100, 400 expert trajectories) to clarify whether the gains stem from algorithmic design or simply from access to more source data.\n\n2. While Table 3 ablates β = 0 and β = 1, several key design choices are not tested. The method is described as using “hyperparameter-free adaptation,” but β(t) depends on a moving-average parameter ψ = 0.9, which effectively acts as a hyperparameter. The paper does not compare against intermediate fixed β values, and it does not explore alternatives to the mapping loss or analyze the learned mappings G and H. It would be helpful to add experiments comparing adaptive β(t) to fixed values such as 0.3, 0.5, and 0.7, and show the trajectory of beta(t) during training. For ψ, it could be helpful to evaluate a few different values of ψ  or clarify why ψ  = 0.9 is not considered a hyperparameter.\n\n3. Theorem 1 provides a convergence guarantee, but its practical implications are unclear. The bound depends on the quality of the learned cross-domain mappings, yet the paper does not evaluate when these mappings are accurate or how performance changes as the mapping quality worsens. Without empirical characterization, it is hard for practitioners to assess when AdaptDICE is likely to help or when domain mismatch will limit its utility. It would be helpful if the authors could report the LMAP values achieved in each environment and examine whether they correlate with final performance.", "broader_impact_concerns": "None. The work does not raise ethical concerns requiring a Broader Impact Statement.", "additional_comments": "I enjoyed reading the paper, it would be great to have access to the code to facilitate reproducibility and out of sample testing on data from different domains.", "extra_scores": {}, "extra_text": {}}, {"review_id": "qJlVfL28cD", "reviewer": "Reviewer_KGE7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. The paper is generally well-written, with clear motivation and positioning, though the methodology section could benefit from improved exposition.\n\n2. The experimental evaluation is extensive and thoughtfully designed, covering diverse domains and providing strong empirical support for the proposed method.\n\n3. The proposed semi-supervised setting is both practical and impactful, addressing a key limitation in current cross-domain imitation learning approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s claims are substantiated through rigorous theoretical analysis and comprehensive empirical validation.", "audience_yn": "Yes", "audience_explanation": "The semi-supervised Cross-Domain Imitation Learning (CDIL) setting addresses a practical and compelling problem with strong real-world relevance. It is likely to attract interest from the research community working on data-efficient imitation learning and transfer across domains.", "requested_changes": "1. It is recommended to use vector graphics (e.g., PDF or SVG) for figures such as Figure 1 to ensure clarity and scalability, especially for publication-quality rendering.\n\n2. The authors state that the goal is to “learn a policy $\\pi$ that can effectively mimic the expert policy based on data from both domains.” However, if the two domains differ significantly in their transition dynamics, the optimal policies in each domain may diverge substantially. How does the method reconcile these differences to produce a single effective policy?\n\n3. In Algorithm 1, the definition of $\\omega_{\\text{src}}$ is missing. Additionally, the authors should reference the corresponding equations when introducing loss functions, for example, Line 2 should point to Equation (5), and Line 7 should clarify how $\\beta(t)$ is computed. These adjustments would significantly enhance readability.\n\n4. In Section 3.2, $H$ is defined as a mapping from $S_{\\text{tar}} \\times A_{\\text{tar}}$ to $A_{\\text{src}}$. The motivation and interpretation of this mapping are unclear. Why is it necessary to map target state-action pairs to source actions, rather than simply mapping $S_{\\text{tar}}$ to $S_{\\text{src}}$? A more intuitive explanation would be helpful.\n\n5. The methodology section, particularly Section 3.2, is difficult to follow. The algorithm's components are introduced somewhat independently and abruptly. A clearer writing strategy that introduces each component sequentially, alongside their motivations and interdependencies, then an overall algorithm, would greatly improve comprehension.\n\n6. In Section 4.1, the paper should explicitly describe the corresponding target-domain configurations (e.g., modified parameters or morphologies), either in the main text or in the appendix. This transparency is important for reproducibility and clarity.\n\n7. The current experiments primarily evaluate domain shifts caused by changes in embodiment. It would strengthen the paper to also consider other types of domain shifts, such as changes to the transition function (e.g., added noise or even dynamics inversion), to test the generality of the proposed approach.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)