EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data
Authors: Ryan Punamiya
Dhruv Patel
Patcharapong Aphiwetsa
Pranav Kuppili
Lawrence Y. Zhu
Simar Kareer
Judy Hoffman
Danfei Xu
Venue: NeurIPS 2025 poster ()
Keywords: Robot Learning
Imitation Learning
Cross-Embodiment Learning
Learning from Human Data
Domain Adaptation
Primary area: reinforcement_learning
TL;DR: We present EgoBridge, a unified imitation learning framework that uses optimal transport domain adaptation to enable knowledge transfer between human and robot, resulting in generalization to new objects, scenes and tasks seen only in human data.

Abstract: Egocentric human experience data presents a vast resource for scaling up end-to-end imitation learning for robotic manipulation. However, significant domain gaps in visual appearance, sensor modalities, and kinematics between human and robot impede knowledge transfer. This paper presents EgoBridge, a unified co-training framework that explicitly aligns the policy latent spaces between human and robot data using domain adaptation. Through a measure of discrepancy on the joint policy latent features and actions based on Optimal Transport (OT), we learn observation representations that not only align between the human and robot domain but also preserve the action-relevant information critical for policy learning. EgoBridge achieves a significant absolute policy success rate improvement by 44% over human-augmented cross-embodiment baselines in three real-world single-arm and bimanual manipulation tasks. EgoBridge also generalizes to new objects, scenes, and tasks seen only in human data, where baselines fail entirely. Videos and additional information can be found at https://ego-bridge.github.io/

EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data
Ryan Punamiya' Lawrence Y. Zhu!
Dhruv Patel
Patcharapong Aphiwetsa' Judy Hoffman"
Pranav Kuppili' Danfei Xu
Simar Kareer 1*
'Georgia Institute of Technology 'Equal advising
rpunamiya6@gatech.omm
Abstract
Egocentric human experience data presents a vast resource for scaling up end- to-end imitation learning for robotic manipulation. However, significant domain gaps in visual appearance, sensor modalities, and kinematics between human and robot impede knowledge transfer. This paper presents EgoBridge, unified co- training framework that explicitly aligns the policy latent spaces between human and robot data using domain adaptation. Through measure of discrepancy on the joint policy latent features and actions based on Optimal Transport (OT), we learn observation representations that not only align between the human and robot domain but also preserve the action-relevant information critical for policy learning. EgoBridge achieves significant absolute policy success rate improvement by 44% over human-aupe augmented cross- cross-embodiment baselines in three real-world single-arm and bimanual manipulation tasks. EgoBridge also generalizes to new objects, scenes, and tasks seen only in human data, where baselines fail entirely, Videos and additional information can be found at httpp:://eo-bbrdgggpp g
Finite Robot Teleop Data
Scalable Human Experience Data
Scanes Behaviors Seon Only Human Data
EgoBridge
Lumann
Dett
Optinal Tranaport aigeeentt
Haseas Rakkot
Figure 1: EgoBridge enables rich knowledge transfer from human robot, based key hypothesis: aligned latent representations yield stronger transfer. Our algorithm, which adapts optimal transport, aligns behaviors which are similar across embodiments. This enables EgoBridge to generalize to objects, scenes and even motions demonstrated only in human data.
Introduction
Supervised imitation learning methods such as behavior cloning have emerged promising path to scaling robot performance across diverse objects, tasks, and environments. However, while large-scale models in vision and language have achieved remarkable generalization through Internet-soureet data, replicating this success in robotics remains challenging due to the labor-intensive nature of collecting teleoperated demonstrations. Deploying physical robots many new environments to collect data with enough coverage and diversity is economically and practically intractable.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this work, we aim to enable robots to learn from egocentric recordings of natural human behavior, collected by increasingly ubiquitous wearable devices (e.g.. XR devices and smart glasses). Without a robot in the loop, such data is cheap and scalable to collect and captures natural human interactions with the world. More importantly, it reflects the embodied human experience, as it contains both observations (e.g., egocentric RGB images) and actions (e.g., hand motions). Unlike unstructured data sources such as Internet videos, the rich embodied information allows us to treat human data and robot data equal parts in continuous spectrum of demonstration data and potentially learn from both with a unified learning framework.
However, the multitudes of domain gaps between human and robot pose significant challenges in designing such a framework. Human bodies and robots have different visual uppearances. Even within a shared action space, kinematic differences can lead to behavior distribution shifts. Robots also have additional sensing modalities such as wrist cameras that are often missing from embodied human data. While recent works such as EgoMimic [ have attempted to bridge the embodiment gaps with techniques such as visual masking, data normalization, and motion retargeting, such domain gaps still largely remain. More broadly, simply co-training from cross-domain data does not automatically yield effective knowledge transfer, as suggested by recent studies (2). Such challenges prevent policies from scaling their performance primarily with human data.
We formalize the human-rokh roooot pross-embodiment learning problem as a domain adaptation problem, where human and robot data represent two labeled distributions with significant covariate shifts in observations due to embodiment gaps. Standard domain adaptation approaches often rely on global distribution alignment techniques such as adversarial training [3 and maximum mean discrepancy minimization [4. However, they primarily address high-level tasks such as image classification and fail to preserve detailed action-relevant information a critical requirement for robot learning where actions and observations are temporally correlated under compounding covariate shift.
To address these challenges, we propose EgoBridge, a novel domain adaptation approach that uses Optimal Transport (OT) to align latent representations from human and robot domains as part of the policy co- co-training objective. Unlike conventional domain alignment methods, our OT formulation explicitly exploits the inherent relationship between motion similarities in human and robot domains to form pseudo-pair as supervision for the adaptation process. Concretely, we use the dynamic time warping (DTW) distance among human and robot motion trajectories to shape the OT ground cost. This encourages the transport map to find minimal-cost coupling between human and robot data exhibiting similar behaviors. As such, EgoBridge aligns policy representations across domains via a differentiable OT loss (Sinkhorn distance), while preserving action-relevant information for policy learning. Importantly, we show that EgoBridge learns a shared latent representation that generalizes beyond the paired data. This enables the policy to learn behaviors observed only within the human datuset, effectively enabling the policy to scale primarily with human data.
We evaluate EgoBridge on both reproducible simulation benchmark task and three challenging real-woris manipulation tasks. Our results show that EgoBridge consistently improves policy success rates compared to human-augmented cross-embodimg rate improvement, and effectively transfers behaviors from diverse human demonstrations to robotic execution in tasks requiring spatial, visual, and task generalization.
baselines, for up to 44% absolute success
Related Work
Supervised Imitation Learning. Supervised Imitation Learning (SIL), notably Behavior Cloning, leverages expert demonstrations for policy learning and has achieved significant success in robotics, particularly with large-scale datasets [5
13 14. State-of-the-art Vision- II integrate Vision- Lision-Language Models (VLMs) with action decoders, enhancing generalization by incorporating semantic understanding from internet- scale data. Despite these advances, even top-performing models like Pi 0.5 [I]] require extensive labeled robot manipulation data for robust physical world interaction and broad capabilities. Given the intractability of scaling robot teleoperation data, alternative sources such as more accessible human data are being explored. Consequently, our work leverages scalable human demonstrations alongside in-domain robot data to improve learning outcomes.
6
7
g
10 II
12
Language-Action (VLA) models 100 8 12
Learning from Human Data. Human data presents two main opportunities for robot learning: abundant unlabeled online videos and curated, labeled demonstrations [15 16 I] Unlabeled web

---

videos, though plentiful, require pseudo-labeling of actions via inverse dynamics models [7 or point tracking [18] 19 20 for policy training, forming a basis for some foundation models [2]] yet often still necessitating in-domain robot data. Alternatively, labeled human demonstrations can be co-trained with robot data as distinct embodiments 21 22], enhancing robustness and scene understanding. However, generalizing to novel behaviors observed only in human data remains challenging. To address this limitation, we propose a novel learning framework for jointly aligning observation-action spaces across human and robot embodiments to improve generalization.
Domain Adaptation and Optimal Transport. Domain Adaptation (DA) aims to reduce reliance on target-specific data by leveraging labeled source domain data to bridge distribution gaps and improve performance on unlabeled target domains. In coosseembod learning. DA has been applied for shared dynamics modeling [23), unsupervised reward modeling (24]. and high-level planning [5]. However, many DA methods primarily focus on global distribution alignment, which can neglect fine-grained action information crucial for transfer across robot embodiments. To address this, computer vision research introduced Optimal Transport (OT) as a loss function for DA to align both local and global distributions [26 27 28 Building these insights, we propose an action-aware DA approach using OT to learn shared representations across embodiments, thereby improving observation and behavior generalization.
3 Preliminaries and Problem Statement
3.1 Optimal Transport
Optimal Transport for Domain Adaptation. Optimal Transport (OT) offers principled framework for comparing probability distributions by considering the geometry of their sample spaces. Given two distributions, As (source) and pyr (target), over common metric space X, and a cost function C(x$, , measuring the effort to move mass from x E X' to x² E X, OT finds a probabilistic coupling 7 P(X' X') that minimizes the expected transport cost:
arg
min (E)(
IC ..=")"""))) ),
where 1[ pT) is the set of all joint distributions whose marginals are μs and μr. For discrete empirical distributions from Ns source samples {x;}} and Nr target samples {x]] J. the cost matrix
C2, C(x$. )T)), and the total cost is (7, C) Eig Eig Cij.
Differentiable Optimal Transport as Loss Function. When used as cost function align
representations, the standard OT problem is often regularized. The Sinkhorn algorithm (29) introduces an entropic regularization term the OT objective, yielding differentiable approximation Tet to the optimal transport plan:
T arg
min TEl(
Bia5 . (TT -
where > is the regularization strength and H is the entropy of the coupling. This regularization makes the problem strictly convex and efficiently solvable. The resulting regularized optimal transport cost, E,, (T²) "1T"))n Cij. is differentiable with respect to the cost matrix C. This allows OT to serve as loss function within deep learning frameworks, enabling the learning of feature encoders that map inputs space X' where their distributions are aligned by minimizing this transport cost.
3.2 Human and Robot Data Sources
We consider egocentric human data (Dx) and teleoperated robot data (DR). == ((o a H consists of Nur egocentric human demonstrations, where off OH are observations from wearable sensors (c.g., head-maor mounted cameras) and af E A are human actions in common action space (c.g., robot end-effector and human hand poses). This data abundant and captures natural, diverse behaviors. Conversely, DR = {fof" ( ) comprises NR robot experiences, typically from teleoperation, with of E R being robot sensor observations (e.g., ego-centric/wrist cameras, joint states) and aft E AA the robot actions. This data is often scarce. We describe how each data source is captured and processed in more detail Sec. We assume actions are in trajectory chunks, which is shown to improve prediction temporal consistency of the trained policies [5] 6).
3

---

3.3 Cross-Emoodiment Imitation Learning: Challenges and Objectives
Our primary goal is to effectively leam from both limited robot demonstrations (Dr) and more abundant, diverse egocentric human demonstrations (Dr). We train feature encoder So O# U
OR + 2 to project observations from both human (O#) and robot (OR) into the shared latent space 2. We jointly train a policy xy that maps these learned latent representations & E 2 to actions a E A.
Co-Training. A popular approach I 21] involves training the policy end-to-end using a standard Behavior Cloning (BC) loss on the aggregated dataset:
LBC (0, 0) DDD (Cnc(CC((ool))))amm DaaaaooDD
To effectively learn from both data sources, critical assumption is that shared latent space Z would naturally emerge where the mapping from latent states to actions is
resulting in Pr((alfo(oR) (ir(a[[s(on)) for observations on and on from aligned underlying states.
Challenge: Observation Covariate Shift. However, we argue that, without explicit mitigation, the induced marginal distributions over these latents, μи (Of)( and μæ P(f ).")). will exhibit a significant covariate shift (μæ * (g).. This shift arises from inherent domain gaps in observations (e.g., differing visual appearances, viewpoints, sensor modalities like robot wrist cameras absent in human setups) and embodiment kinematics. We also empirically show that such co-trained representations often form disjoint latent clusters (Sec. (..3.. This covariate shift in the marginal latent distributions undermines the foundational assumption of consistent conditional action distributions across domains, thereby limiting effective knowledge transfer from human robot.
This motivates our method that aims at joint domain adaptation (Sec.4.) 4.1). where we explicitly seek to align the latent representations from human and robot data while preserving action-relevant information. Successfully addressing this latent misalignment should enable two crucial levels of generalization: First, for tasks present in both DH and DR. the system must achieve observation generalization. This involves effectively bridging visual and sensor gaps, which include appearance changes that do not affect behaviour. Second, and more ambitiously, the system should enable behavior generalization (Beh. Gen.) allowing the robot to perform tasks or handle novel situations (e.g. task variations) observed only in DH- This requires the learned encoder Sa to generalize beyond scenarios with paired human and robot data which require motion information to be transferred, such spatial variations in goal pose.
Goal: Generalizable Cross-Embodiment Transfer.
EgoBridge
EgoBridge is co-training framework designed effectively imitate embodied human demonstra- tions and robot demonstrations. It explicitly addresses the domain gap between human and robot experiences through Optimal Transport (OT)-based domain adaptation mechanism integrated into the policy learning process. The core of EgoBridge lies in aligning the joint distributions of latent policy features and corresponding actions across the human and robot domains. The following sections detail this joint domain adaptation formulation (Sec. 4.1. the design of its OT cost function (Sec. and the overall training process and system details (Sec. 4.3
4.1 Joint Domain Adaptation via Optimal Transport
To address the latent covariate shift (Sec.
3.3) 3 and generalizable cross-embodiment transfer, EgoBridge builds on Optimal Transport (OT, Sec.
3.1 to directly shape the shared feature encoder fo. Unlike standard domain adaptation techniques 30 that often aligns only the marginals (O))(O)), which can discard action-relevant information, EgoBridge optimizes fa to align the joint distributions of its output latent features and their corresponding actions, i.c., ((f,
Given mini-batches of human data {(of aaf ) and robot data {[oft a "" where a represents a temporally-extendee action trajectory, we define an OT-based loss to guide the learning of Sa- The differentiable Sinkhorn OT formulation (29) allows us to compute loss based on the alignment of
the empirical distributions of (fo(o"), ) ") and (Jo (o(oR),aF))
COT-joiot (0) Σ (T)) C ((folo")) ). (.))))

---

Here, (T) is the optimal transport plan coupling the i-th human and robot (latent, action) pairs. The cost function -(:,)) measures the dissimilarity between these joint entities. Its design crucial for capturing meaningful behavioral similarities across domains, which we detail in Sec. 4.2
Minimizing LoT juin directly influences the parameters & of the encoder fe- The gradients from this loss encourage Jo produce latent features fo(of" ) and fo(oft that minimize the transport cost required to align them, especially when their associated actions a and ag are behaviorally similar (as determined by C). At each step transport plan is computed which influences the feature encoder to couple the action pairs This iterative process shapes the latent space to be Nomain-invariane with respect the joint observation-action manifold.
4.2 Designing OT Cost Function for Action-Aware Joint Adaptation
(= (=R, a))) measure Our joint OT formulation (Section 4.1 relies cost function C((=H ([(ss,aa),;
the dissimilarity between joint human and robot latent feature-action pairs. A critical challenge is designing this cost to be robust to inherent domain differences. Specifically, we aim account for temporal misaligruments, where human and robot often execute the same task at of different speed, e.g., humans might be 2-3 times faster than teleoperated robots, and kinematic variations, where even within a shared SE(3) end-effector action space and hand-eye alignment through an egocentric coordinate frame (Sec.
minor kinematic differences exist.
Dynamic Time Warping. To identify behaviorally similar action sequences while accounting for these differences, we propose to leverage Dynamic Time Warping (DTW) to guide the OT alignment. DTW [31 has been effective in prior work to compare time series data and trajectories. Formally, given two action sequences a (af :...., aff ) from human data and a (af. ..... aB) from
robot data of identical length T', DTW finds an alignment path C (l,.... 7} x (1, T that
minimizes the cumulative distance:
DTW(a" aaa
min xEA(())
Σ ((.))
la." - 1
where A(T) the set of admissible monotonic alignments constrained to start at (1, 1) and end at (T, T), while allowing small local shifts to account for temporal variations.
Soft Supervision. With DTW, we identify highly correlated samples from both domains. However, directly utilizing the DTW cost is noisy and instead is much stronger measure of relative pairing between human and robot samples. As such the DTW cost can be used to pseudo-pair DH and DR. On a mini-batch of size B of sampled ground-truth state-action pairs from Ds and Dr, we form DTW cost matrix A E R Here, A., DTW(a (a a²).. The row-wise minimum cost gives us the most behaviorally similar human pseudo-pate for each robot sample i (G) arg min, Auj.
Given the standard OT Euclidean distance cost Dij ) fo(of)) )l² we define the joint cost C((folof) ), (folof) (f for Lor as:
- A ifi=
otherwise
where 0 < X << is small scalar. This cost function strongly incentivizes OT to match robot samples with their behaviorally closest human pseudo-pair (identified by DTW) by significantly reducing the cost for these pairs. For all other pairs, the cost is simply the distance in the latent feature space. This soft supervision from DTW guides the latent space alignment towards behaviorally relevant correspondences across embodiments.
4.3 Putting it all together: EgoBridge
With all the ingredients for joint distribution adaptation using OT with joint policy co-training, we present EgoBridge as a unified cross-embodimen imitation learning algorithm and describe its corresponding robot learning system and policy architecture.
Policy Co- Co-Training with Joint Adaptation. EgoBridge jointly optimizes the feature encoder fo and тæ, with the joint OT loss applied on the feature encoder and the BC co-training loss applied end-to-end through both components: Laata = LBC-comann (ő, 0) CoTijooint with tunable weight a. Detail of the algorithm and hyperparameter choices are described in Appendix.
5

---

Egocentric Human Data. We largely follow EgoMimic [T] and leverage wearable smart glass platform Meta Project Aria 12 as our main data collection platform. The platform allows us to collect exteroceptive, egocentric first person POV RGB images (eeo)), and pro- prioceptive data (q ), cartesian pose for both arms (E) (3). We take inspiration from EgoMimic to construct stable reference frames form action sequences of cartesian pose (a in the egocentric camera frame.
Lor-joott
Decoder Head
tokkess
Encodirr
f6
cornect tokens
Stem
Storm
Stem
Human Robot
Teleoperated Robot Data. We base robot platform on the open-source Eve robot [. In partieular, we leverage Aria glasses as the main egocentric perception sensor for the robot and mount it in way that emulates the hand-eye configuration of a human adult (22..). This effectively mitigates the human- robot camera device gap, allowing us to specif- ically study the appearance, kinematic and behaviour gaps. The robot additionally provides RGB streams from its two RealSense D405 wrist cameras, I The actions consist of a sequence of corresponding future end-effector poses, a² E (3)3) x SE(3).
Figure 2: EgoBridge policy co-training with joint adaptation. The encoder fo consists of modality- specific input stems and the encoder trunk, while the policy Ty consists of a shared multi-block trans- former decoder. Lor-joint optimizes the encoder while LBC-cotrain optimizes the entire network.
Shared Policy Architecture. Inspired by recent cross-embodiment policy learning 33 and DETR- style architectures [34), our policy employs a shared transformer encoder "trunk" (fo) and shared transformer policy decoder "head" (T²) (Fig. 2p. We perform embodiment-speciff gaussian normal- ization to the proprioception and actions. The encoder fg begins with stems stems-shalle shallow networks that tokenize observations; notably, shared vision stem processes main egocentric RGB images (Tego) from both human and robot to enforce visual aligmment, while separate stems handle robot wrist camera inputs (Twrist). The subsequent multi-layer encoder trunk processes these concatenated tokens, along with M prepended learnable context tokens upon which the OT loss is applied. The multi-block decoder head then generates actions by attending this encoded context, utilizing T learnable action tokens and injecting context through alternating self and cross-attention blocks.
5 Experiments
In this section, aim to validate three core hypotheses. HI: EgoBridge improves co-training perfor- mance for scenarios present in both human and robot data. H2: EgoBridge enables generalization to scenarios only seen in human data. H3: EgoBridge learns shared latent space where human and robot data are aligned in task-relevant manners. We validate the hypotheses through a standard simulation benchmark task (Sec. 5.1) and three complex real-world manipulation tasks (Sec. 5.2).
5.1 Simulation Evaluation
To facilitate reproducible study and eliminate the confounding factors in real robot systems, we study a well-explored planar pushing task [6). where the goal is to push a T-shaped object to desired goal location. We emulate "human" (source) data through a blue circle pusher and "robot" (target) data through a salmon triangle pusher (Fig. 3 with lower floor friction. The differences aim to analogize the appearance and agent-envirenmenn dynamics gaps between human and robot data.
Source and Target Domain Data. our "robot" target domain, we collect demos in the standard push-T setting, but in our "human" source domain, alter the background color to purple and change the T configuration to be mirrored, requiring a new motion to slot into place 3 The change in background color is analogous to the human data containing new visual scenery. and the change in starting configuration analogous to the human demonstrating new motions in their demonstrations.
Training and Evaluation. To eliminate factor from model design (Sec. 4.3 we choose standard ResNet-UNet Diffusion Policy 6 and apply the OT-joint loss on the feature outputs of the ResNet encoder. We perform standard action normalization and co-train the policy both the triangle and

---

Training Distribution: Source ("Human") Target Robot")
PushT Evaluation Results
-141
Col Mirres
-17%
39%
Success Rate
-39%
--50%
57%
Evaluation Cases
149%
Target-BCC
MMD
Standard OT
Co-train
Egoltridge
Method
Figure 3: In the simulated Push-T experiments, probe toy version of visual and motion level generalization from human to robot. We have narrow target "robot" data represented by the triangle pusher on a white background, and diverse source "human" data represented by the circle pusher with changes in background color and configuration. We test our "robot" on the diverse human scenarios, and find that EgoBridge outperforms traditional Domain Adaptation baselines.
Table 1: Real World Evaluation Results: In-Distribution and Generalization
Method
Scoop Coffee (SR)
Drawer (SR)
Laundry (Pts SR)
In-Dist.
Obj. Gen. Scene+Ohj Gen.
Total (Pts SR)
Place Toy Beh. Gen,
Robot-only BC
33%
40%
7%
38 19%
28%
0%
38 | 28%
Co-train
53%
46%
0%
55 22%
42%
0%
41 I 33%
EgoMimic |
60%
53%
0%
49 14%
39%
0%
38 I 33%
MimicPlay
25
33%
27%
0%
33 14%
22%
0%
188
ATM [9]]
47%
33%
0%
6%
17%
28%
EgoBridge 67% %
60%
27 %
77 447
72%
33%
48 I 17%
circle pusher data. We evaluate the policy 3 cases: Triangle in the standard setting, Triangle with purple background, Triangle with purple background and Triangle with purple background and flipped T. We evaluate total of 100 fixed seeds across all the models and report the mean reward (max IoU with goal) and the success rate (reward 2 0.9).
Baselines. In the more controlled simulation settings, we choose compare EgoBridge against conventional domain adaptation baselines. We choose Maximum Mean Discrepancy (MMD) (30) alternative domain adaptation loss to Jo on the feature encoder. We also test Standard OT, which performs marginal alignment instead of joint aligmment. The Co-train baseline trains on data from both domains without alignment loss. Finally, Target-only is control study which trains the policy only on the target (triangle) data.
5.2 Real World Evaluation
We evaluate EgoBridge on three challenging real-world manipulation tasks, as illustrated in Fig.
Drawer: The robot interacts with a 6x4 drawer array, tasked to pick a toy, place it into pre-opened drawer, and close it. Robot data (144 demonstrations) covers three of the four array quadrants, each quadrant being a 3x2 arrangement. Human data (1 hour) covers all four quadrants, providing demonstrations for motions into the fourth, robot-unseen quadrant. This setup specifically tests behavior generalization to drawer locations only seen in human data. Points (Pts) are awarded for successful completion of each stage and trial is considered success only if the robot completes all actions. Evaluation uses 48 trials (2 rollouts for each of the 24 drawers).
Scoop Coffee: The robot uses its left arm to scoop coffee beans with spoon and empty them into a target. Robot data (50 demonstrations) involves specific target (can) in one scene. Human data (2 hours) includes demonstrations with both the can and a new target (grinder), across two distinct scenes, one which is novel to the robot. Target object positions are randomized (30x23 em area). We evaluated observation generalization for: (1) the
grinder target, and (2) the new scene with the new target, that is, scooping to the grinder in the new scene, seen in human data only. Performance measured by success rate 15 rollouts across distinct target locations.
Laundry: This is bimanual task where the robot needs fold the shirt in 50 x 22 cm range with rotation range + 30 degrees. The robot uses both arms to fold the right sleeve, the left sleeve, and
7

---

Training Distributions: Human Robot vs. Human Only
Training
(8) Drawer
(b) Scoop Coffee
(c] Laundry
Rollout
Legend:
(Humen
In-distribution
OOD
In-distribution
OOD
In-distribution
Figure 4: Training Data and Evaluation Settings. We show the distribution of human and robot training data (top) and evaluation setting, where in-distribution scenarios are in both human and robot data, while mistr--distriuution (OOD) scenarios seen only in human data.
then the final stage to fold the shirt in half. We award Pts for each successful stage and consider it success if all the individual stages are successful. We collect hours robot data which include 300 demonstrations across 3 shirts, and 2 hours of human data comprising approximately 700 demonstrations. We conduct 18 evaluations with diverse shirt initial placement and colors.
Baselines. We adopt the following baselines for real-world tasks. Co-train: Direct co-training of the robot and human data using BC loss, without any latent alignment. EgoMimic: [ Co-training with explicit vision and action-space alignment using masking, shared end-effector pose head (human and robot) and a separate joint-space head for robot. Mimicplay: [25] A hierarchial policy with a latent high-level planner co-trained on human and robot data, and action decoder fine-tuned on robot data. Any-Point Trajectory Modeling (ATM): [19] A hierarchical policy where the high-level planner is initially co-trained 2D point tracks derived from both robot and human video data. These point tracksare obtained via Co-tracker [35). Following this, high-level planner is frozen and an action decoder fine-tuned specifically on robot data. Target-only BC: Trained only on robot data.
5.3 Results
EgoBridge improves in-domain task performance (H1). Over our set of in-domain tasks, we observed an improvement of 7-44% absolute task success rate over both human-augmented imitation learning baselines and robot-only behaviour cloning policies. While methods like naive co-training, EgoMimic and ATM also improve in-domain performance across all tasks, EgoBridge consistently outperforms them. We hypothesize that the strongly aligned latent representations facilitate better cross-embont cross-embodimenl transfer.
EgoBridge enables generalization to objects and scenes only seen in human data (H2). While it difficult to collect robot data across diverse scenes and objects, is trivial to do for human data, so it's critical that we can transfer this knowledge from human to robot, inspiring our experimental setup. Specifically, in the Scoop Coffee task, our human data introduces new coffee grinder, table, lighting and height variations, completely unseen to the robot. We find EgoBridge outperforms all baselines when tested on the new coffee grinder (7-33%). Further, most methods fail entirely when tested on the new grinder scene, but EgoBridge retains performance of 27%. We observe similar robustness trends in our simulated benchmark Fig. 3 where EgoBridge enables generalization in the push-T task to new background and starting configuration, outperforming all basclines.
EgoBridge enables generalization new behaviors only seen in human data (H2). In our most challenging setting, we seek to show that we can learn entirely new motions from human data alone. In the drawer task, the robot data covers 3/4 drawer quadrants, whereas the human data covers all quadrants. We evaluate our policy's 's performance able to generalize to these locations with success rate of 33%, whereas most methods fail entirely (Tab. []. While all the methods were exposed to the same human data, only EgoBridge was able to effectively transfer the human motion to a novel robot action. We attribute this success to the well aligned latent representations, which enables human to robot knowledge interpolation. We also
these new drawers, and find that EgoBridge is

---

Dietribution
Generalization
Rebot
Egolridgge
Egodanige
Haman
Humman
Mimic
Himiclay
Mivis
MiskiPaay
Figure 5: Visualization of TSNE plots encoded features for EgoBridge and baselines, with the mean Wasserstein-2 distance and KNN pairs of aligned human-robot data visualized.
observed similar trend in the simulated benchmark where EgoBridge faces the lowest performance drop of 14% compared to all baselines in the mirrored-''s + background colour reflected in Fig. 3
EgoBridge learns shared latent space that aligns human and robot data task relevant manner (H3). We hypothesize that an ideal latent space for transfer would jointly embed human and robot data into a space with high overlap and semantic interoperability. To probe this, we create TSNE visualization of the action tokens from transformer backbone. EgoBridge not only exhibits the highest latent overlap between human and robot as measured by Wasserstein distance as seen in Fig. [S] but also upon inspecting K-nearest neighbor pairs in latent space. exhibits the most semantically similar neighbors. For instance, we see the human and rohot performing the same phase of given task, whereas in baselines like MimicPlay that aligns marginals with KI--div, the semantic similarity is lacking. This result is highly correlated with the task success rates for in-distribution and generalization where baselines with poor alignment perform lower consistently across all evaluations.
Ablation. We ablate three key components of our method 1) replacing our DTW-based pairing metric to instead use simple MSE, 2) replacing the joint OT ob- jective Lor-joint with standard marginal alignment, and 3) removing any auxiliary alignment objectives (direct co-training). We find that replacing the cost function with MSE leads to the largest performance drop for in- distribution policy success rate from 47% to 17%, seen in Tab. [ which emphasizes the importance of creating semantically similar pseudo-pairs. Ablating Joint-OT also shows a large performance drop in both in-distribution and generalization cases which empha- sizes how naive marginal alignment cannot transfer knowledge from human data effectively. Ablating an auxiliary alignment loss also shows a significant performance drop for in-distribution success rate and leads failure in generalization cases, emphasizing the need for joint distribution alignment.
Table 2: Ablation Results (Drawer)
Method
Drawer (SR) Beh. Gen. (SR)
EgoBridge MSE Standard-OT
%
33%
14%
17%
33%
17%
Co-train
22%
0%
Conclusion and Limitations
We presented EgoBridge, a novel co-training framework designed to enable robots to learn effectively from egocentric human data by explicitly addressing domain gaps. By leveraging Optimal Transport on joint policy latent feature-action distributions, guided by Dynamic Time Warping cost on action trajectories, EgoBridge successfully aligns human and robot representations while preserving critical action-relevant information. Our experiments demonstrated significant improvements in real-world task success rates (up to 44% absolute gain) and, importantly, showed robust generalization to novel objects, scenes, and even tasks observed only in human demonstrations, where baselines often failed.
While EgoBridge demonstrates promising single-task transfer, the Dawebassed action alignment cost may not be as informative in multi-task joint domain adaptation settings. In future works, we seek to adopt more generalizable alignment cost such as natural language embedding distances (36] obtained from Vision-Language models and visual features extracted from foundation models. Other avenues for future work include extending the joint distribution adaptation to multiple embodiments and leveraging Internet-soureee human data without action labels.

---

7 Acknowledgment
This work was supported in part by research gift from Meta Platforms, Inc, the Defense Advanced Research Projects Agency (DARPA) under the TIAMAT program and by the National Science Foundation under Grant No. 2144194, Any opinions, findings, and conclusions recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the supporting organizations.
References
[1] Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video, 2024.
[2] Adam Wei, Abhinav Agarwal, Boyuan Chen, Rohan Bosworth, Nicholas Pfaff, and Russ Tedrake. Empirical analysis sim-and-real cotraining of diffusion policies for planar pushing from pixels. arXiv preprint arXiv:2503. 22634,,,,,, 2025.
[3] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation, 2017.
[4] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. Deep transfer learning with joint adaptation networks, 2017.
[5] Tony Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine- ane-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304. 13705, 2023.
[6] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024.
[7] Authony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv: 12.11 2022.
[8] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alex Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang- Wei Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao. Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In arXiv preprint MxXiv:2307.15816 2023.
[9] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-ctiion model. arXiv preprint arXiv:2406.09246. 2024.
[10] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. T0: A vision- vision-language- language- flow model for general robot control, 2024. URL https://arxiv. org/abs/2410.2411 24164444, 2024.
[11] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Devin LeBlane, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair. Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. no.5: vision- laiion-anguage-accte model with open-world generalization, 2025.
10

---

[12] NVIDIA, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang. Dieter Fox, Fengyuan Hu. Spencer Huang, Joel Jang, Zhenyu Jiang. Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang. Jianman Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang. Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. GrOOt nl: An open foundation model for generalist humanoid robots, 2025.
[13] Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee. Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration O. In 2024 IEEE International Conference on Roboties and Automation (ICRA), pages 6892-6903. IEEE, 2024.
[14] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudcep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.1m 12945, 2024.
[15] Shikhar Bahl, Abhinav Gupta, and Deepak Pathak. Human-to-robot imitation in the wild, 2022.
[16] Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Hu- manoid shadowing and imitation from humans, 2024.
[17] Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, and Minjoon Seo. Latent action pretraining from videos, 2024.
[18) Homanga Bharadhwaj. Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. In European Conference on Computer Vision (ECCV), 2024.
[19] Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. 2023.
[20] Juntao Ren, Priya Sundaresan, Dorsa Sadigh, Sanjiban Choudhury, and Jeannette Bohg. Motion tracks: A unified representation for human-robot transfer in few-shot imitation learning, 2025.
[21] Ri-Zhao Qiu, Shiqi Yang, Xuxin Cheng, Chaitanya Chawla, Jialong Li, Tairan He, Ge Yan, David J. Yoon, Ryan Hoque, Lars Paulsen, Ge Yang, Jian Zhang, Sha Yi, Guanya Shi, and Xiaolong Wang. Humanoid policy human policy. arXiv preprint 2rXiv:2503.1344: 2025.
[22] Marion Lepert, Jiaying Fang. and Jeannette Bohg. Phantom: Training robots without robots using only human vidcos, 2025.
[23] Sean J. Wang and Aaron M. Johnson. Domain adaptation using system invariant dynamics models. In Ali Jadbabaie, John Lygeros, George J. Pappas, Pablo A. Parrilo, Benjamin Recht, Claire Tomlin, and Melanie N. Zeilinger, editors, Proceedings of the 3rd Conference on Learning for Dynamics and Control, volume 144 of Proceedings of Machine Learning Research, pages 1130- 1141. PMLR, 07 08 June 2021.
[24] Laura Smith, Nikita Dhawan, Marvin Zhang, Pieter Abbeel, and Sergey Levine. Avid: Learning multi-stage tasks via pixel-level translation of human videos, 2020.
[25] Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anandkumar. Mimicplay: Long-horizon imitation learning by watching human play. arXiv preprinl 2222 2023.
[26] Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation, 2017
[27] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation, 2016.

---

[28] Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. 2018.
[29] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transportation distances, 2013.
[30] Arthur Gretton, Karsten Borgwardt, Malte J. Rasch, Bernhard Scholkopf, and Alexander J. Smola. A kernel method for the two-sample problem, 2008
[31] Hiroaki Sakoe and Seibi Chiba. Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26(1): 43-49, 1978.
[32] Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talatoff, Arnie Yuan, Bilal Souti, Brighid Meredith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel Denone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Duncan Frost, Edward Miller, Elias Mueggler, Evgeniy Oleinik, Fan Zhang, Guruprasad Soma- sundaram, Gustavo Solaira, Harry Lanaras, Henry Howard-Jenkins, Huixuan Tang. Hyo Jin Kim, Jaime Rivera, Ji Luo, Jing Dong, Julian Straub, Kevin Bailey, Kevin Eckenhoff, Lingni Ma, Luis Pesqueira, Mark Schwesinger, Maurizio Monge, Nan Yang, Nick Charron, Nikhil Raina, Omkar Parkhi, Peter Borschowa, Pierre Moulon, Prince Gupta, Raul Mur-Artal, Robbie Pennington, Sachin Kulkarni, Sagar Miglani, Santosh Gondi, Saransh Solanki, Sean Diener, Shangyi Cheng, Simon Green, Steve Saarinen, Suvam Patra, Tassos Mourikis, Thomas Whelan, Tripti Singh, Vasileios Baintas, Vijay Baiyya, Wilson Dreewes, Xiaqing Pan, Yang Lou, Yipu Zhao, Yusuf Mansour, Yuyang Zou, Zhaoyang Lv, Zijian Wang. Mingfei Yan, Carl Ren, Renzo De Nardi, and Richard Newcombe. Project aria: A new tool for egocentric multi-modal ai research, 2023.
[33] Lirui Wang, Xinlei Chen, Jialiang Zhao, and Kaiming He. Scaling proprioceptive--isual learning with heterogeneous pre-trained transformers, 2024.
[34] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, 2020.
[35] Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together, 2024.
[36] Albert Yu, Adeline Foote, Raymond Mooney, and Roberto Martin-Mant Natural language can help bridge the sim2real gap. arXiv preprint arXiv:2405. /0020, 2024.
[37] Jean Feydy, Thibault Séjourné, François- Caancois-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2681-2690, 2019.
[38] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yun- liang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An
generalist robot policy, 2024.
A Algorithm Pseudocode
Overview. Algorithm describes the joint policy co-training of the human and robot data with the joint OT loss.
Mini-batch sampling. At each iteration we draw two size B mini-batches, one from the human dataset D and one from the robot dataset DR (Lines 3-4).
Shared encoder. Both batches are fed through a shured encoder So that maps raw observations o to latent embedding e d (Line 6).
12

---

Algorithm EgoBridge Co-Training for Human ++ Robot Imitation
Require: Human demos Dr. robot demos DR. OT loss weight a, DTW discount X, Sinkhorn regularization E, LR schedule
Ensure: Encoder f and policy decoder T
Initialize network parameters Ø,
2: while training not converged do
3: Bи =(o (6) 1 B
ai 1i=0 SAMPLEBATCH(Dn,))
sample human data
4: Bи = =oo ag 1 t SAMPLEBATCH(DI, .B)
B
> sample robot data
5:
encode raw observation batch into shared latent space for all E (0, .......,BBB |Bu|-1], {o..... 1.n..,1nn111 do
zH t fo(0H). 2R + fo((o))
6:
7:
Aij < (a aq
(i)
DTW distance between actions
8:
end for
9:
(]) + arg min; A,j Vj E {O,..., |Ba| >> pick the most similar human traj. for each
robot traj.
for all
{0, ..,,.. BBr
1}, j. |Bn |Ba|-13 do
11:
Dij - |= R 11
(1)
(5)
D observation feature cost
12:
if = (1) then
13:
Oig >Di
discount cost if DTW says the pair matches
14:
else
15:
Oig t Dis
16:
end if
17:
end for
18:
JH
[Bl UR Ba
D uniform probability mass the minibatch supports compute differentiable OT loss with entropic reg.
T + SINKIIORN(IHI Mn,, Č,
CoT-joint + Eis Tij Oij
āæ + To(ZH), āR To(2n)
> predict actions for both domains > behaviour-cloning loss across domains
22:
LRC-comain &- LRC(âH, au) + LBC (ăR,
23:
L+ Lac.cotminn + a LoT-jpint
24:
(6.0) OPTIMIZERSTEP (o, 0, Vo.eC,1)
25:
end while
26:
return fōr Rg
Behavioural pairing via DTW. The Dynamic Time Warping (DTW) cost is applied on the action (i)
sequences dH and ag from both domains. The assignment i (j) (Line 11) identifies single pseudo-pair in DH for each robot trajectory j which minimizes the DTW cost.
(
Sinkhorn. The Sinkhorn- -aoppo algorithm is used to compute entropy-regularized Optimal Transport plan from the uniform probability mass on the minibatch supports.
Joint-OT, Latent distances Dij = || 2(4) 1|3 are discounted by À < 1 when (i,j) is behaviourally matched (Lines 12-17), producing shaped cost Ō that biases the OT loss computed from the Sinkhorn OT transport plan.
Behaviour cloning and objective. The decoder ne produces domain-specific actions and trained with supervised loss LBC-comain (Lines 22-24). The overall objective (Line 26)
C LBc-cerain Lor-joint
balances imitation fidelity with latent-space alignment, with an balancing factor between the two losses.
B Network Architecture
The following section describes how fs and my are parameterized in the simulation and real-world experiments. All hyperparameters are summarized in Table Table for real-world and in Table for simulation.
13

---

B.1
Real World Experiments
Table 3: Hyperparameters for Real-World Experiments
Symbol
Value
H W x C
480
640
256
Dstem
8
datm
64
L
16
dg (Robot)
7 or 14 (joint positions)
dg (Human)
6 12 (end effector pose as xyz euler)
D uunk
8
N Nunk
16
d
256
M
8
D Dread
8
Nhead
dhead
64
100
da
7 or 14 (xyz euler angles gripper position)
LBC
(Robot)
SmoothL1(xyz) + +SmoothL((rripeer)) + 0.5 ((Eul)))))
Lsc
(Human)
SmoothLl(xyz)
C Lec-cotrain + a Lor-joint
a 0.7
B.1.1
Observation Encoder: fo
fø consists of components, "stems" for each observation modality and "trunk". The stems are shallow networks that encode heterogenous input spaces into fixed representation to allow joint learning between human and robot observations.
Vision Stem. Given an RGB frame IE #x we normalize using ImageNet normalization and then pass through ResNet-18 encoder truncated before the global-pool layer obtain 7 x 512 feature map. The 49 spatial patches are flattened and projected with single-layer MLP of hidden dimension dproj. producing 49 tokens of size dproj per image. A single multi-head cross-attention block (Dstem heads, per-head width daum) employs L learnable query tokens of dimension d that attend these 49 patch tokens; the resulting L query outputs constitute the vision tokens passed to the shared trunk.
Proprioceptive Stem. The proprioceptive observation vector q E TR. (joint angles, end effector pose etc.) is z-score normalized and passed through single-layer MLP of hidden dimension dpwoj- producing one token kpraprio E Ruru. A single multi-head tross-attention block (Daten heads, per- head width dam)) uses the learnable query tokens as the vision branch to attend to this key/value token; the resulting L query outputs constitute the proprioceptive tokens that are forwarded to the shared trunk.
Trunk. The shared "trunk" is standard multi-block transformer encoder, which consists of Dirunk heads, Ntrwnk blocks, and an embedding dimension of d. Each head has an attention dimension of d/ Dirunk- A token sequence of length L by concatenating the token outputs of m observation encoding stems. set of M learnable context tokens are prepended to the the token sequence. The new sequence of M + L are input into the trunk. The first M extracted from the output sequence and represent the feature output of fo, and consequently where Lor-joint is applied.
B.1.2 Policy:
Ty conditioned the output of fo and is parameterized by a DETR- multi-blos transformer decoder "head" [4].
Head. The head consists of Niedd blocks and Dread heads, a cross-attention dimension of
dhoad Dhend and self-attention dimension of dhead/ Dhead- This hybrid attention designed to promote increased conditioning from the latent k learnable tokens of dhead input into the
14

---

model, where K corresponds to the action chunk length k. Each block in the decoder consists of alternating self-attention on the input tokens, and cross-attention with the context M tokens (z) from the trunk. After the final block, a linear layer projects the output to da corresponding to the cartesian action dimension, which is described precisely in Section [D] Since the human data only comprises of 3D position, the loss is masked loss which is only calculated on the values in the prediction corresponding to the 3D position for human data. The loss Lsc on human data is Smooth LI loss the 3D position, while the loss LBc on robot data is Smooth L1 loss the 3D position, gripper value and MSE loss on the orientation.
Complete Architecture. The network consists of a shared vision stem processes main egocentric RGB images (Zego E "R x3) from both human and robot and produces L tokens. Followed by this are individual embodinent--peciffc proprioceptive stems that map g" and q" proprioceptive features to an additional L tokens. To the vision and proprioceptive tokens, left and right wrist RGB images (Turist #x x3) are mapped to additional tokens by one additional wrist vision stem for single-arn robot data and two additional wrist vision stems for bimanual robot data. M learnable tokens are prepended to the token sequence and passed to the trunk. The OT-joint loss is applied on the first M tokens of the output. The same tokens are decoded by the head into aus++ E SE(3) + gripper, where an action for single arm E ReeT with the orientation expressed as euler angles in the yaw-pitch-roll convention. For bimanual tasks, the action comprises of two such action trajectories concatenated resulting in an output E Rtx11
Training Details. We train the real world EgoBridge model on a single L40s gpu for 100000 iterations on the Drawer task, 110000 iterations on the Laundry task, and 120000 iterations on the Scoop Coffee task, which takes about 24 hours. More details are in Table
Table 4: Training Details for Real World Experiments
Parameter
Value AdamW
Optimizer
Learning Rate
5 x 10-5
Weight Decay
0.0001
Scheduler
Linear
Batch Size
32
Data Augmentations
ColorJitter + ImageNett Normalization (ResNet)
OT Loss
GeomLoss (37
Blur
0.05
Distance
Sinkhorn
B.2 Sim PushT
Data Processing. We follow the PushT environment setup introduced in the Diffusson Policy benchmark suite (6). Each demonstration comprises sequence of RGB observations and associated low-dimensional proprioception, along with corresponding action labels. The RGB observations I, E 9 are normalized using ImageNet statistics. The proprioception consists of 2D end- effector positions (x. y), which are min-max normalized per-dimension using statistics computed over the entire dataset. The action at each timestep is defined as the target 2D position of the end-effector, also normalized with per-dimension min-max normalization. Action chunks of length & = 16 are extracted from each demonstration trajectory. No observation history is used in the input, i.e., the observation context window is 1.
Encoder (fz). We employ the same vision encoder architecture in the original Diffusion Policy paper [6] which uses a truncated ResNet-18 to extract spatial feature maps from 96 x 96 x 3 RGB images. The proprioceptive input (normalized x, y position) is concatenated channel-wise to the image feature maps before being passed as the global conditioning to the UNet-based diffusion decoder. This architecture has been shown effectively fuse visual and proprioceptive features for policy learning in wwwdiiensioonl manipulation tasks like PushT. The Joint-OT loss is applied on the image-proprio concatenated feature output.
Policy (TO). The policy architecture mirrors the conditional diffusion policy design (6. The output
is denoising network trained to generate trajectory segment of length k 16 for 2D end-
15

---

effector coordinates in normalized space. During training, the policy receives noisy version of the ground-trute -tunh action chunk, and learns iteratively denoise the sample using the concatenated latent observations. The diffusion loss is the weighted MSE over all denoising steps in the sampling process. During inference, the policy performs iterative denoising starting from Gaussian noise to generate 21) trajectory samples conditioned on the current observation.
Training Details. We train the simulation EgoBridge model a single A40 GPU for 130000 iterations, which corresponds to around 2 hours of training time. The hyperparameters and training details are summarized in Table
Table 5: Training Details for Simulation Experiments (PushT)
Parameter
Value
Optimizer
AdamW
Learning Rate
x 10-4
Weight Decay
1 10-6
Scheduler
Cosine
Warmup Steps
500
Iterations
130,000
Batch Size
32
Exponential Moving Average (EMA)
Power = 0.75
Data Augmentations
ImageNet Normalization (ResNet)
Le (Triangle Circle)
MSE(xy)
C == Lac-cotrain + a Lor-joint
0.2
OT Loss
GcomLoss 37
Blur
0.01
Distance
Sinkhorn
Baselines
C.1 Real World Experiments
All baselines instantiate the encoder fo and decoder my architecture described in Section and with modifications to input and output spaces.
Robot-only Behavioral Cloning (Robot-only BC). This baseline is trained only on robot observa- tions. The encoder includes vision stems for egocentric and wrist camera inputs and proprioceptive stem for joint-space inputs. The resulting tokens and M context tokens passed to the transformer trunk. The decoder head uses K learnable tokens to produce future actions, where each action is 7-dimensional Cartesian command: 3D position, 3D orientation, and scalar gripper. The loss is computed using Smooth LI for position and gripper, and MSE for orientation.
Co-train. Co-train uses both human and robot demonstrations during training and represents ablated version of the EgoBridge model, where the joint-OT loss (Cor-joim) is not applied.
EgoMimic [1. EgoMimic follows the Co-train architecture but includes two modifications: (1) The egocentric RGB input is masked with an overlay and red line augmentation, following (0 before input into the encoder, and (2) The decoder uses two heads: one for robot joint-space commands E Rd and another for shared cartesian pose E SE(3) + gripper for human and robot data. The joint-space predictions are supervised using Smooth L1 loss on the entire prediction vector, while cartesian pose predictions are supervised by the loss described in Section B During training, the appropriate head is selected based on the data source, where robot data provides gradient updates through the joint predictions and cartesian predictions, while the human data provides gradient updates for the model through the cartesian predictions. The identical overlay is applied during evaluation.
ATM 19). For fair comparisons, we implement ATM with an identical architecture as our method and extend it with an additional head designed to predict future dense 2D motion trajectories in the image plane, as described in the original ATM paper. The goal is to test whether access to pixel- level dynamics improves downstream behavior cloning performance via improved latent structure.
16

---

Ground-truth 2D pixel tracks are generated using CoTracker 35 and organized into temporal stack of length 10, covering 100 future frames. The action-trac head outputs future 36 keypoints across 6 x 6 grid over the image plane. Training proceeds in two stages: first, the the trunk, stems, and the 2D track head are pre-trained for 500 epochs to jointly predict human and robot 2D) action tracks; second, the trunk and 2D head are frozen, and only the cartesian action head is fine-tuned on robot data. We opt for stacked, non--utooegreesivee predictions of the full temporal keypoint trajectory. diverging from the sequential autoregressive formulation in the original ATM [9], in order to isolate the representational benefits of point-track conditioning on downstream imitation performance.
MimicPlay [25]. MimicPlay is implemented as hierarchical policy which consists of a "high-level" policy which is co-trained on both human and robot data and "low-level" policy which trained only on robot data and is conditioned on the context output of the "high-level" policy.
High-level policy. We parameterize the high level policy of MimicPlay with shared vision stem for Tego for both human and robot and a shallow "trunk" which consists where Ntrank = 2, while the other hyperparameters for the trunk remain the same to match the parameter size of the original high-level in MimicPlay. 2 tokens corresponding to the high level context are prepended to the vision tokens. The first output tokens are the latent context Zhigh that are passed the low-level policy, A Fhigh- A Gaussian Mixture Model decoder with identical parameters the MimicPlay paper, takes znigh and predicts the mixing coefficient, mean, and the variance of GMM. With these predictions, a sequence of 100 3D positions sampled from the GMM for given input observation and supervised with the ground-truth actions for the observation with a negative log likelihood loss. The high-level planner is trained until convergence which around 60000 iterations.
KI--divergence loss is applied between the robot data Fhigh and the human data =H
Low-level policy. The low level policy is parameterized the Robot-only BC model. The high-level policy is frozen and the latent output Zhigh is concatenated to the M tokens produced by fo which are then decoded into cartesian actions supervised by LB on robot data.
C.2 Sim PushT
In the more controlled Sim PushT environment, we benchmark EgoBridge against standard domain adaptation techniques commonly used in feature-level alignment. All baselines instantiate the same encoder la and decoder my described in Section B.2 with differences only in the training objective and data source used.
Maximum Mean Discrepancy (30]. We evaluate domain aligmment using the MMD loss 30 as an
alternative to the Joint OT loss. This baseline uses the same training configuration EgoBridge but replaces the Joint-OT objective with the MMD loss applied on the latent embeddings from the shared encoder. The overall training objective is:
C
LRC-courain + AMMD LMMD: where
AMMD 1.0
The MMD loss is computed as:
LMDD
Σ
k(2H 1
Σ
k(ER
2R
(
2
Σ
k(2
1 22
n
m²
Tum
i.j
1
where k(-,)) is Gaussian RBF kernel:
|| (-
v 20
k(x, y) cxp
and a is the kernel bandwidth hyperparameter.
Standard OT. This baseline implements marginal alignment using unshaped Optimal Transport cost matrix without any Dasdd behavioural pairing. It follows the same formulation as Ego- Bridge (see Algorithm IT but removes the DTW pairing step (Lines 11 11-17), using direct squared lg distance Cij Dij for all (i.j) in the batch. This serves an ablation that tests the benefit of behaviourally guided pairings in EgoBridge.
Co-train. Similar to the real-world setting, the co-train baselines ablates the Joint OT loss from the sim EgoBridge architecture to evaluate the baseline performance of training a shared policy on both embodiments.
17

---

Targef-only BC. This baseline is trained using only robot demonstrations from the triangle pusher in the white background with original configuration (100 trajectories total). It uses the same architecture as EgoBridge but is never exposed to data from the circle pusher (source domain).
D Robot Data
Bimanual Manipulator. To effectively utilize egocentric human data for manipulation, the robot hardware platform must resemble human sizes and kinematic workspaces. Drawing inspiration from the "Eve" robot platform introduced in EgoMimic [T. we develop custom mobile manipulator that comprises of two 6-DoF ViperX 300s mounted in an identical inverted configuration on height-adjustable rig. Similar to Eve, we propose to leverage the Project Aria glasses [32] as the main egocentric perception sensor for the robot and mount it in way that emulates the hand-eye configuration of a human adult. This effectively mitigates the human-robot camera device gap and reduces the manspllata kinematic gap. Each arm is equipped with an Intel Realsense D405 on its wrist to facilitate precise near-range manipulation. The raw RGB data from the aria glasses is undistorted linear camera model, resized and reshaped to 480 x 640 x 3. Fig. 6 shows an annotated picture of the bimanual manipulator. The bimanual manipulator is telcoperated using followrf system similar to ALOHA [5]. where two WidowX 6-DoF arms in inverted configuration as seen in Fig.
Data Processing. Each robot demonstration comprises synchronized streams of joint-space pro- prioception, RGB image observations, and action labels recorded at 50Hz. each timestep t, we collect the current joint configuration q E IRd,, where dy denotes the number of actuated joints for one or two robot arms. RGB images are recorded from two sources: (I) an egocentric camera (I ego, E R 2480661033 mounted using Project Aria glasses in head-like configuration, and (2) wrist-mounted RGB cameras (wristt
IR 430x placed near the end-effectors.
The raw action at each timestep is defined as the commanded joint position a,aw, R represent- ing the control input issued at time I. To obtain cartesian actions used for training, we use analytical forward kinematics to compute the (3) cartesian pose. The cartesian pose is then projected into the image frame via the Aria glasses extrinsics (Thr50) obtained through hand-eye calibration, which represent the transformation between the robot base frame and the Aria glasses camera frame. The ground-truth actions a!" E IRd, are obtained through
nt a
(I
FKK(Fa R
(a
E
Embodied human data
glasses to collect human embodied experience data. The glasses are accompanied by the Aria Machine Perception Services (MPS). The raw data from the Aria glasses consist of an RGB camera stream, SLAAM cameras, IMU, eye cameras. This raw data is uploaded to a cloud service provided by Project Aria, known as the MPS. The service returns device pose estimated using the SLAM camera, hand tracking relative to the device frame, a semi-dense point-cloud of the environment and eye gaze estimation. This processed data is obtained as CSV files with aligned timestamps. The hand tracking is in the form of 3D positions p" E SE(3) x SE(3) for both hands in device frame, while the head pose T E S (3) in world frame. Each hand position p is in device frame Teria for timestep f. The raw RGB data is undistorted to a linear camera model and reshaped to 480 640 x 3. Constructing human actions. One challenge with unifying the reference frames for joint policy learning is that robot actions are typically in a fixed reference frame (usually the base frame), while human hand tracking in the device frame, which moves. Following the idea of predicting action chunks 6 5l, we aim to construct action chunks d Taking the single-arm case without loss of generality, the raw trajectory is SE(3) poses [ytl //4 14411.p422 pf+ ... r where
We use the Project Aria 32
each hand position p. is in device frame Teria We choose to create pseudo-reference frames taking inspiration from [1]. where we construct an action o& by projecting & future hand positions into the device frame at timestep This allows the policy to predict trajectories with respect stable reference at each timestep. As such, the trajectory is constructed by:
H1 a
(I.
Taria 11 Pt+s
1=1
18

---

Aria Glesses
Human Operator
Human Aria Actions
Robet Ara w/ ST Actions
peri Followers
Widowi
Leaders
A/A SLAM
Whit
Camt
Figure 6: We employ the Aria glasses to capture Egocentric RGB images for both human and robot embodiments. The Aria uses its side SLAM cameras to estimate device pose and hand tracking (left). Ground truth (GT) cartesian actions for both embodiments are projected to pixel space using Aria intrinsics and visualized on the images. Our robot system a leader-follower system with two Viper X arms the followers and two Widow X arms as the leaders which teleoperated by the human demonstrator (right).
Data processing and alignment.
38 10 show that individually normalizing proprioception and actions for both embodiments, helps co- co-training. Similarly, we employ z-score normalization by subtracting the per-embodiment dataset mean and dividing by the standard deviation
Recent coossemmbodiment works
[
norm (p) (p (p-Pp)) pp)/op
We normalize actions identically, using per-dimension mean and standard deviation.
For both human and robot, our action chunk size k: = 100. Human data sensor streams (RGB, hand tracking and SLAM) are received 30hz. We construct a human action chunk by taking 10 sample pose values at an interval of frames, and interpolating them to a trajectory of length 100. This corresponds trajectory of length 100 0.9 seconds in real time. We construct a robot action chunk by concatenating the ground-trute actions of 100 successive timesteps which correspond to 2 seconds in real time. The human and robot data sources are visualized in Fig.
G Additional Task Details
G.1 Real World Experiments
Drawer. For this task, we collect one hour of human data and 30 minutes of robot data comprising 144 demonstrations. The human data was equally distributed across all four quadrants, while the robot demonstrations each receiving 48 robot demonstrations (8 demonstrations per drawer). The main failure modes for this task are the inability correctly grasp the toy, taking the toy towards the incorrect drawer and being unable to completely push the drawer shut.
equally divided among three quadrants (excluding the top-right), with
Scoop Coffee. The scoop coffee task emphasizes observation generalization with novel objects and scenes. To address this, we gathered 180 minutes of human data, equally split across three scenarios: the base scene, a new target object (grinder), and new target object (grinder) with new scene. For robot data, we collected 50 demonstrations with the original target object and scene configuration. The two primary failure modes in this task is an inability precisely grasp the scoop and imprecise targeting of the target container.
Laundry. The laundry task demands precise bimanual coordination. We collected similar amount of human data (2 hours) as the scoop task, comprising 700 demonstrations, alongside two hours of robot data (300 demonstrations). Common failure modes involve missing one or both sleeves during any of the three sub-stages, leading to an imprecise or unsuccessful fold. The qualitative task successes are visualized in Fig. and common failure modes for each task are visualized in Fig. while the data mixture is shown Table 6
19

---

a) Drawer
b) Scoop Coffee
LO
c) Laundry
Figure Qualitative success EgoBridge on each our real world tasks.
a) Drawer Failure Modes
Fail to grasp
Incorrect Target
Fail to close drawer
b) Scoop Coffee Failure Modes
c) Laundry Failure Modes
Fail to grasp sconp
Incorrect Target
Incomplete Grasp
Figure 8: Common failure modes for each of the real world evaluation tasks.
20

---

Table 6: Data collection overview for both (H) and Robot(R) data. We report both the number(#) of total task demonstrations and the time(min) took to collect them.
Task
H
H
H #/min
R
R
R #/min
#
min
#
min
Drawer
360
60
6
144
29
Scoop Coffee
720
180
4
50
13.3
3.75
Laundry
700
120
5.8
300
120
2.5
G.2
Sim PushT
We collect a total 350 demonstrations across scenarios using the PyMunk simulator environment published with Diffusion Policy [6]. We collect 100 demonstrations each for the circle pusher and triangle pusher in the original configuration with the white background. In addition, we collect 50 demonstrations each of the circle pusher with original T configuration mirrored T on purple background and mirrored T on white background. We apply a friction coefficient multiplier of 0.7 on the triangle pusher to embody kinematic gap like human and robot.
purple background,
H
Supplementary Simulation Results
We evaluate settings: In distribution, which corresponds to the triangle pusher with the original configuration on a white background; Observation Generalization which corresponds to the previous setting with purple background; Observation and Behaviour Generalization which corresponds to the triangle pusher with mirrored T on purple background. For evaluation, we select a total of 100 seeds between 101 and 9999 using a deterministic random sampling with seed 42. The complete results are summarized in Table The Mean Reward is calculated using the average max Intersection-over-UU Union with the goal across 100 seeds and Success Rate (SR) is computed using number of seeds with reward of over 0.9.
Table 7: Sim PushT results across 3 evaluation settings In-distribution (Mean Reward (R))
Generalization
Method
Purple Bg
Purple Bg + Mirrored
(Mean Reward (R)
(Mean Reward (R)
EgoBridge
0.7605
53.00 %
0.7206
48.00 %
0.6520
39.00 %
Target Only
0.5555
39.00%
0.0904
0.00%
0.0992
0.00%
Cotrain
0.7062
48.00%
0.6899
42.00%
0.6214
31 31.00%
Standard OT
0.7009
38.00%
0.5303
15.00%
0.5109
8.00%
MMD
0.6439
45.00%
0.4867
22.00%
0.5876
14.00%
I Pseudo-pair Visualisation
To show why Dynamic Time Warping (DTW) is ideal cost function to identify behaviourally similar trajectories to align, we visualize randomly sampled mini-batch pairs from the dataset for each of the tasks and gualitatively compare them with the Mean Square Error (MSE) cost function pairs used for the MSE ablation. A pair, as defined carlier, is the row-wise cost function minimum in the cost matrix computed from batch of human and robot data. Qualitatively, DTW is more robust to temporal shifts, viewpoint shifts and is more spatially precise when compared to MSE. Specifically, MSE often pairs the incorrect sub-task tasks like Laundry, and loses precise spatial location of the target container in tasks like Drawer and Scoop Coffee. The few failure modes of DTW are where the trajectory overlaps a few different stages which leads to temporal misalignments and when locations are spatially apart but visually close due to an extreme viewpoint shift. Despite this, DTW enables pairings that are able to capture very fine-gre fine-grained changes trajectory to find the most behaviorally similar between human and robot data. This is visualized in Fig.
21

---

Dynamic Time Warping
Mean Squared Error
Dynamic Time Warping
Mean Squared Error
Dynamic Time Warping
Mean Squared Error
Figure 9: Qualitative visualization of randomly sampled mini-bateh pairing of MSE and DTW.
J Supplementary Real Results
In order to test EgoBridge's ability to generalize to new appearances under more challenging real-world setting, conduct two additional evaluation experiments the Laundry task. The first, Observation Generalization, introduces a new white table surface unseen in robot data, along with 30 minutes of human data recorded this new setup. The second, Combinatorial Observation Generalization, evaluates a harder scenario where the robot must handle a new shirt color on the white surface, with that shirt only in the human data and excluded entirely from robot demonstrations. Both evaluations follow the same protocol as the main experiments, performing 20 rollouts across randomly sampled workspace positions. As summarized Table g EgoBridge achieves substantially higher success rates and sub-task points than the Co-train baseline in both settings, demonstrat- ing stronger robustness visual appearance shifts and compositional variations in object-scene configurations.
Table Generalization experiments on Laundry. Evaluation Co-train and EgoBridge under two generalization settings. We report success rate (SR) and sub-task points.
Obs. Generalization
Comb. Obs. Generalization
Model
SR (%)
Points
SR (%)
Points
Co-train
25
25
7
EgoBridge
50
43
30
35
22

---

EgoBridge
Query Images
Nearest Neighbours
W2 Dist: 0.00
Co-train
Query Imnages
Nearest Neighbours
W2 Dist: 7.642
EgoBridge
Query Images
Nearest Neighbours
Co-train
Query Images
Neighbours
w2 Dist: 10:55
Figure 10: Qualitative visualization of the learned latent space using T-SNE and K-Nearest Neighbors.
K Latent Space Visualisation
We visualize the learned latent feature outputs of fo for the EgoBridge and Co-train models using T-SNE for the Drawer and Scoop Coffee tasks. evaluate EgoBridge's ability to achieve joint distribution alignment, representations to a give robot feature representation. We plot the ground truth actions associated with those observations onto the input images and visualize the nearest neighbors. In addition to an overall lower Wasserstein-2 distance, which indicates global alignment, EgoBridge also aligns observations that correspond similar behaviors in the latent space. This is visualized in Fig 10
use K-Nearest Neighbors to identify the closest learned human feature
23

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The experiment section show real world and simulation evaluation settings that support the claims made within the abstract and the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
24

---

Justification:
We have described our limitations alongside the conclusion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The purposed method and corresponding optimal transport theory contains detailed proof and is included in the supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The EgoBridge framework details and hardware used in experiments are discussed in the methods and experiments section.
Guidelines:
25

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: No. But plan to release after acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
mublic//guides//odeSuuubi wtt.ssubiiiiioooooooooooooooe for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https CodeeSumiiii wwwiiissssossiiiiio
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
26

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Detailed information regarding data collection, splits, hyperparameter tuning and training details of EgoBridge is mentioned in our supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: In real-world robot learning experiments 25 19], conducting the numerous
repetitions on physical robots required for robust error bar computation is prohibitively expensive and time-consuming. incurring significant hardware and human supervision costs. Moreover, repeated trials can induce hardware wear, introducing uncontrolled variability that hinders achieving consistent experimental setups. Consequently, reporting standard error bars is often impractical. Our evaluation prioritizes full task and sub-task success and qualitative results over statistical variance, aligning with standard evaluation methods in this domain.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
27

---

Answer: [Yes]
Justification: Detailed instructions of compute resources to reproduce our work are provided in the supplementary material
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:////aaaaresccccpppp/cchcaaa
Answer: [Yes]
Justification: We do not conduct formal human study. Demonstration data (human and robot) is collected by authors with explicit consent. We have reviewed the NeurIPS Code of Ethics and ensured full compliance with its guidelines for this research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work advances co-training methodologies for cross-embodied human and robot datasets, representing small increment foundational robot learning research. As such, it does not present direct societal impacts at this stage.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
28

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks. mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: We have not released our data models and we believe potential future release of this paper does not pose any such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly cited all existing code package and datasets and mentioned their licenses in the supplementary material.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwppsaiithhhcooooooo thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This work does not release assets.
Guidelines:
29

---

The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human partici- pants. All human demonstration data collected by authors with explicit consent.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human partici- pants. All human demonstration data is collected by authors with explicit consent.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development this research does not involve L.MMs as any important, original, or non-standard components.
30

---

Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy http:///eeeriisscccooo/eeeeeeers for what should or should not be described.
31


Reviews:
[{"review_id": "V38nA0XPNn", "reviewer": "Reviewer_nDAS", "summary": "The paper proposes EgoBridge, an approach to improving co-training on teleop data and human data via domain adaptation. Specifically, the authors identify that the current naïve robot-human data co-training has the problem of misaligned latent representations of two types of data, and the authors propose EgoBridge that explicitly uses Optimal Transport to align the latent representation of two kinds of data. The authors show that EgoBridge can help improve the performance of co-training and enable the policy to complete tasks that have been seen in robot data and only seen in human data.", "questions": "I have listed my main concerns in Weakness section. Apart from that, I also have an additional question on using action similarity as soft supervision. What if we directly use it as a hard supervision instead of soft supervision? For example, for each human demo, we find the robot demo that has the highest similary to it, and then directly minimizes the distance between their latent representation. This is equivalent to directly setting T* to be 1 for the most similar pair and 0 other wise. In this way, we don’t need to update T* during training and it might make training more stable. I wonder how would the performance be if we do this?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\nThe proposed approach is well motivated, theoretically solid, and shows great performance. The idea of using action similarity as soft supervision to construct the cost function in OT is very clever. The experiments are structured, well thought, and shows promising results.\n\nWeakness:\n\n1.\tOne main concern I have about the paper is using action similarity as soft supervision in the cost function of OT. The intuition here is when two trajectories have similar actions, they are more likely to be the same task and in the same environment setting, and thus their latent representation of the observations should also be similar. However, this assumption can be broken sometimes. For example, consider two tasks of picking a cup and opening a drawer. If the cup and the drawer are at similar position, then most part of the trajectories is very similar (the part of reaching to the object) and maybe only the last part of the trajectories (how to manipulate the object) is different. In this case, the similarity between the actions of two trajectories can be high, but we should have similar observation representation of these two trajectories because they are different tasks and have different objects. What if, instead of using action similarity as soft supervision, we use similarity of visual observation (e.g., similarity between their DINOv2 features), or similarity between language instructions, or some kind of combination of these three? Will it make the soft supervision more reliable?\n2.\tIt seems the authors haven’t provided many details about the efficiency of training OT. One main concern is that, since the OT loss is dependent on the optimal transport plan T* and T* is the result of a optimization problem dependent on the distribution of latent representation of human and robot data, then 1) is it costly to solve the optimization problem and find T*? 2) when we are updating the model during training, the distribution of the latent representation is changing and thus the T* might also change. How to we track the changing optimality of T*? Do we continuously update T* during training and how? Is it expensive to continuously update it during training?", "was_revised": true, "final_justification": "The rebuttal from the authors has resolved all my concerns, and I will remain my rating of Accept, considering the paper should have good impact on robot-human data co-training in robotics.", "rebuttal": "We thank the reviewer for their positive evaluation of our proposed method. We are encouraged that they found our method clever and results promising!\n\n***\"For example, consider two tasks of picking a cup and opening a drawer. If the cup and the drawer are at similar position, then most part of the trajectories is very similar (the part of reaching to the object) and maybe only the last part of the trajectories (how to manipulate the object) is different.\"***\n\nWe thank the reviewer for the insightful comment. We acknowledge that this is a valid concern especially in multi-task settings where DTW-based action pairings may lead to incorrect or degenerate pairings, as discussed in L365-368 of the main text. However, the generality of our joint domain adaptation formulation allows for the inclusion of additional heuristics and labels. For instance, language embedding distances (as explored in adjacent works[1]) and dense language annotations of actions are becoming the norm in large scale imitation learning and would allow us to extend EgoBridge to more complex and multi-task settings in future works.\n\n***\"is it costly to solve the optimization problem and find $T^{\\ast}$?***\n\nThanks for the clarification question. We use the geomloss [5] library which is a kernel optimized sinkhorn process that runs in parallel on GPU. The run-time of this algorithm is $O(N^2)$ where N is the batch size. In practice finding $T^{\\ast}$ is a fraction of the time compared to the model forward pass. To empirically validate this claim, we use a single L40s GPU, with batch size 32 and torch.cuda.synchronize() to profile over 10 batches. \n\n*Average forward pass time (human + robot) :* 0.344373s \\\n*Average OT Loss time :* 0.005504s\n\nWhich makes the $T^{\\ast}$ computation ~1.6% of the forward pass time.\n\n***“when we are updating the model during training, the distribution of the latent representation is changing and thus the $T^{\\ast}$ might also change. How to we track the changing optimality of $T^{\\ast}$?”***\n\nWe thank the reviewer for the insightful comment. $T^{\\ast}$ computed on the policy latent is indeed evolving with the training process. $T^{\\ast}$ is computed for each batch at each training step. Since we don’t use $T^{\\ast}$ as a push forward but instead as a differentiable objective guiding the direction of moving probability mass from the two distributions, it acts as the supervision to the encoder to move the latent representations such that it reaches a cost-minimizing $T^{\\ast}$ for all sampled batches across the dataset. \n\nAdjacent to this, the reviewer asks ***“What if we directly use it as a hard supervision instead of soft supervision? For example, for each human demo, we find the robot demo that has the highest similarity to it, and then directly minimizes the distance between their latent representation?”***\n\nThis is precisely an alternate deterministic formulation of Optimal Transport Plan, the Monge map. We choose to use the probabilistic “soft” version of OT (Kantarovich) [27] as in a mini-batch setting, it is robust to pairing / label noise, and works better with behavior cloning since it retains the marginal distributions of human and robot data distributions. This design decision has also been inherited from related work in joint domain adaptation that leverage OT [2, 3, 4].\n\n**References**\n\n    [1] Yu, A., Foote, A., Mooney, R., & Martín-Martín, R. (2024). Natural language can help bridge the sim2real gap. arXiv preprint arXiv:2405.10020.\n\n    [2] Courty, N., Flamary, R., Habrard, A., & Rakotomamonjy, A. (2017). Joint distribution optimal transportation for domain adaptation. Advances in neural information processing systems, 30.\n\n    [3] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation, 2016\n\n    [4] Damodaran, B. B., Kellenberger, B., Flamary, R., Tuia, D., & Courty, N. (2018). Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European conference on computer vision (ECCV) (pp. 447-463).\n\n    [5] Feydy, J., Séjourné, T., Vialard, F. X., Amari, S. I., Trouvé, A., & Peyré, G. (2019, April). Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics (pp. 2681-2690). PMLR."}, {"review_id": "8V82T4kduv", "reviewer": "Reviewer_kgpd", "summary": "The paper presents EgoBridge, a framework for co-training robot manipulation policies using both egocentric human data and robot teleoperation data. The key innovation lies in using Optimal Transport-informed cost function to explicitly align joint latent representations (observation embeddings and actions) across human and robot domains. This alignment facilitates effective cross-embodiment policy learning and generalization. Extensive evaluations in simulation and three real-world tasks demonstrate the co-training approach significantly improves policy success rate, particularly in settings where objects, scenes, or behaviors appear only in human data.", "questions": "See the strengths and weaknesses section.\nAdditional questions:\n- Currently the human data is not very diverse. If human data collection can be scaled up and includes more diverse scenarios, would the policy be able to generalize to scenarios neither seen in human or robot data?\n- Have you compared to contrastive or VLM-based alignment methods as an alternative to OT?\n- How expensive is the OT computation per training step? Does it impact training scalability in larger datasets?\n- How critical are the wrist camera inputs in robot data compared to the egocentric view? Would EgoBridge still be effective without them?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The approach is sound and well-motivated. By aligning joint (latent policy feature, action) distributions using differentiable OT, the approach preserves action-relevant information—addressing limitations of prior domain adaptation strategies that align marginal distributions only. Experiments are thorough and ablations are well-executed.\n- EgoBridge shows impressive results, it can improve in-domain task performance, generalize to objects and scenes only seen in human data, and motions only seen in human data.\n- The paper is well-written and the structure is clear. The method is described in detail, including the formulation of the OT cost and its integration into the training objective. Figures (e.g., t-SNE plots) are informative and intuitive to understand.\n\nWeaknesses\n- The reliance and generalizability on DTW for action similarity may become impractical in multi-task or more diverse data regimes, as acknowledged in the paper.\n- While EgoBridge outperforms baselines and ablations, the overall task success rates are still pretty low. What is preventing the tasks to get 80-90% success rates? More failure case analysis would be helpful.\n- Although EgoBridge enables some extent of generalization to new scenarios only seen in human data (e.g., unseen objects, scenes, motions), the “new” scenarios are not very different from robot training distribution. Can the policy generalize to objects of different shapes and sizes (not only appearance), that require different contact modes? Can the policy generalize to new strategies or modes only seen in human data?\n\nMinor paper writing issues\n- In Fig 2, the arrows are not aligned.\n- In Fig 4, the legends are confusing. The blue \"robot\" legend does not show up anywhere in the images. In (a), there are two shades of purple outlines, it is unclear what is the difference between them.", "was_revised": true, "final_justification": "The authors clarified the computation cost and limitations of the approach during rebuttal. All my concerns are addressed and I lean towards accept.", "rebuttal": "We thank the reviewer for their positive review. We are encouraged that they find our results impressive and find the paper to be well written.\n\n***“The reliance and generalizability on DTW for action similarity may become impractical in multi-task or more diverse data regimes”***\n\nWe thank the reviewer for the insightful comment. We acknowledge that this is a valid concern especially in multi-task settings where DTW-based action pairings may lead to incorrect or degenerate pairings, as discussed in L365-368 of the main text. However, the generality of our joint domain adaptation formulation allows for the inclusion of additional heuristics and labels. For instance, language embedding distances (as explored in adjacent works[1]) and dense language annotations of actions are becoming the norm in large scale imitation learning and would allow us to extend EgoBridge to more complex and multi-task settings in future works.\n\n***“What is preventing the tasks to get 80-90% success rates?”*** \n\nWe deliberately chose challenging long-horizon tasks with considerable reset variation as these are challenging settings that require a large amount of teleoperated demonstrations and hence can benefit the most from human data. The laundry task involved manipulation of a deformable object, the coffee task involves fine-grained grasping of a scoop with considerable reset variation and the drawer task requires very precise placement of the object. We could further improve the performance with additional robot data, but herein lies the unique benefit of EgoBridge of leveraging human data in resource-constrained settings, where we can get 70+% success in challenging tasks such as Laundry with only 300 demonstrations across a large workspace.\n\n***“Currently the human data is not very diverse. If human data collection can be scaled up and includes more diverse scenarios, would the policy be able to generalize to scenarios neither seen in human or robot data?”***\n\nOur work focuses in establishing joint domain adaptation as a method for imitation learning to align data from robot and human domains to enable transfer of knowledge from the source (human) to target (robot) domain. As such, we chose to focus our experiments on validating and studying the transfer of knowledge and whether the policy can generalize to scenarios only shown in human data. While achieving zero-shot generalization to new scenarios is an important problem for the field, we consider it to be an orthogonal direction that would require a dedicated, full-fledged research study.\n\n***“Can the policy generalize to objects of different shapes and sizes (not only appearance), that require different contact modes?”***\n\nGrasp primitives are not captured in the action space (cartesian). Hence given the  large embodiment gap (parallel jaw v/s dextrous) in grasp, we hypothesize that grasping new object shapes will be challenging.\n\n***Can the policy generalize to new strategies or modes only seen in human data?”*** \n\nWe thank the reviewer for this question. We acknowledge that generalization to new modes is a challenging task. We show this in EgoBridge, through the drawer task where the robot needs to place the object in target drawers where there is no spatial robot data, hence achieving motion generalization through human data. Future work will study other modes and behaviours we can transfer from human data.\n\n***“How expensive is the OT computation per training step? Does it impact training scalability in larger datasets?”***\n\nThanks for the clarification question. We use the geomloss [2] library which is a kernel optimized sinkhorn process that runs in parallel on GPU. The run-time of this algorithm is $O(N^2)$ where N is the batch size. In practice OT computation is a fraction of the time compared to the model forward pass. To empirically validate this claim, we use a single L40s GPU, with batch size 32 and torch.cuda.synchronize() to profile over 10 batches. \n\n*Average forward pass time (human + robot) :* 0.344373s \\\n*Average OT Loss time :* 0.005504s\n\nWhich makes the $T^{\\ast}$ computation roughly 1.6% of the forward pass time and hence it doesn’t impact training scalability with larger batch sizes and datasets.\n\n***“Have you compared to contrastive or VLM-based alignment methods as an alternative to OT?”***\n\nYes, we have experimented with contrastive-based alignment methods as an alternative. We empirically observed that when naively applied, contrastive learning results in unstable training. Specifically, we utilized the same DTW action supervision to form positive pairs and used the InfoNCE [3] loss on policy latents. We hypothesize that this instability is a result of contrastive learning not retaining the marginal distributions of the human and robot, which is crucial for behavior cloning. Additional research is required to correctly adapt contrastive learning for joint domain adaptation. \n\n***“How critical are the wrist camera inputs in robot data compared to the egocentric view? Would EgoBridge still be effective without them?”***\n\nWe thank the reviewer for the insightful comments. We had a similar hypothesis that removing wrist camera may improve alignment, but we empirically found that the wrist camera is critical to manipulation steps that require precision, such as grasping and alignment, as echoed by many other research [4, 5] . Hence we designed the architecture to handle heterogeneous sensor modalities from the human and robot data.\n\nWe thank the reviewer for identifying presentation issues, which we will address in the next revision of the paper. Specifically, we will align the arrows in Figure 2, replace the purple legend with Human + Robot, remove the robot legend and make the colors consistent in Figure 4.\n\n**References**\n\n    [1] Yu, A., Foote, A., Mooney, R., & Martín-Martín, R. (2024). Natural language can help bridge the sim2real gap. arXiv preprint arXiv:2405.10020.\n\n    [2] Feydy, J., Séjourné, T., Vialard, F. X., Amari, S. I., Trouvé, A., & Peyré, G. (2019, April). Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics (pp. 2681-2690). PMLR.\n\n    [3] Oord, A. V. D., Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.\n\n    [4] Liu, Y., Romeres, D., Jha, D. K., & Nikovski, D. (2020). Understanding multi-modal perception using behavioral cloning for peg-in-a-hole insertion tasks. arXiv preprint arXiv:2007.11646.\n\n    [5] Mandlekar, A., Xu, D., Wong, J., Nasiriany, S., Wang, C., Kulkarni, R., ... & Martín-Martín, R. (2021). What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108.03298."}, {"review_id": "WNxEjfF0lU", "reviewer": "Reviewer_3a7U", "summary": "This paper presents EgoBridge, a domain adaptation framework for enabling robots to learn manipulation tasks from egocentric human and robot demonstration data together. The key innovation lies in using Optimal Transport (OT) to align the joint distributions of latent policy features and actions between human and robot domains, rather than just aligning marginal distributions as in conventional domain adaptation. The method employs Dynamic Time Warping (DTW) to identify behaviorally similar human-robot action pairs, which guides the OT cost function to create meaningful correspondences across embodiments. EgoBridge addresses significant domain gaps including visual appearance differences, sensor modality mismatches, and kinematic variations between humans and robots. The framework is evaluated on both simulation and real-world manipulation tasks, demonstrating substantial improvements in success rates and crucially enabling generalization to novel objects, scenes, and behaviors that are only observed in human demonstrations.", "questions": "- Why does the OT cost $C_{ij}$ only apply scalar reduction to the minimum DTW pairs? Have you explored alternative strategies like graduated scaling based on $A_{ij}$ ranking order?\n- For the Drawer experiments, how do the following factors affect performance: (1) varying the proportion of OOD drawers beyond 1/4, (2) different spatial arrangements of OOD drawers (random vs. clustered), and (3) different overlap ratios between human and robot training data?\n- What is the Laundry task performance under out-of-distribution scenarios such as different tables, unseen shirt types, or other environmental changes?\n- Given the focus on scalable human data, what is the relationship between human data proportion and robot policy performance? Also please compare with other methods across varying human-robot data ratios?", "limitations": "# Limitations\nThe authors acknowledge limitations in their paper; I provide additional concerns in the **Strengths And Weaknesses** and **Questions**.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\n- **Strong empirical results**: Achieves significant performance improvements over established baselines.\n- **Generalization capabilities**: Demonstrates the ability to transfer to scenarios only seen in human data, including new objects, scenes, and behavioral patterns.\n- **Well-motivated technical design**: The use of DTW for identifying behaviorally similar pairs provides a robust way to handle temporal and kinematic differences between human and robot execution.\n\n# Weaknesses\n\n- **Limited Embodiment Generalization and Scalability Concerns**: The approach necessitates collecting human demonstrations using specialized equipment with careful calibration to ensure action space alignment, semantically similar task execution, and comparable environmental conditions between human and robot domains. This dependency means that **when switching to a different robot embodiment**, a completely new human dataset may be needed with embodiment-specific calibration and task adaptation, which significantly reduces the claimed scalability advantages over direct robot teleoperation. The paper would benefit from evaluation across different robot embodiments (e.g., different humanoid platforms or gripper configurations) using the same human dataset to demonstrate true cross-embodiment generalization.\n\n- **Presentation Issues**: Figure 3 contains apparent errors in the evaluation case labels. The middle image in the \"Evaluation Cases\" section appears to be mislabeled - it should likely read \"BG Color + Mirrored T\" if following the authors' described experimental setup.", "was_revised": true, "final_justification": "After reviewing the authors' rebuttal, I raise my score to **Borderline accept** for the following reasons:\n\n> The authors have significantly clarified their proposed methodology, and the additional experiments they provided strengthen the evidence for their method's effectiveness. The rebuttal successfully addressed several of my initial concerns and enhanced my understanding of the approach.\n\n> However, I maintain some reservations that prevent me from giving a firm **Accept**. The scaling experiments comparing human data and robot data remain incomplete, which the authors acknowledged is due to computational equipment malfunctions that prevented training new models during the rebuttal period. While I understand these technical constraints, the absence of these critical experiments limits my confidence in the work. This evaluation is particularly important because, as the authors themselves state, the core hypothesis underlying this research direction is that human data scalability surpasses robot data scalability. Without comprehensive scaling experiments, this fundamental claim remains inadequately validated.\n>\n> If the paper is accepted, I strongly encourage the authors to supplement their work with the missing scaling experiments, as these would provide crucial evidence for their central thesis.\n\nI appreciate the authors' diligent efforts in addressing the reviewers' questions despite the technical challenges they faced.", "rebuttal": "We thank the reviewer for their review. We are glad that they acknowledge our strong empirical results and think that our technical design is well motivated. \n\n***“[EgoBridge] necessitates collecting human demonstrations using specialized equipment with careful calibration to ensure action space alignment, semantically similar task execution, and comparable environmental conditions between human and robot domains.”***\n\nIn regards to “using specialized equipment with careful calibration”, XR / smart wearables are growing ubiquitous. Our work simply requires some form of relative device localization at minimum. Many research devices such as Project Aria, and products such as Quest, Vision Pro with this capability are readily available. There are also many open source algorithms such as MANO and HAMER that provide 3D hand pose estimation. Large scale datasets with similar devices such as Nymeria, HOT3D, EgoExo4D, HOI4D, EgoDex etc. also leverage similar devices to collect human experience data. \n\nUsing camera frame to align action spaces is a very common design choice made in prior work [1, 4, 5]. In addition hand-eye calibration to obtain the transformation between base and camera frame is a very mature and common optimization based computer vision technique which can be applied trivially on any robot embodiment and dataset. \n\n***“when switching to a different robot embodiment, a completely new human dataset may be needed with embodiment-specific calibration and task adaptation.”***\n\nWe respectfully disagree with the comment. To clarify, the human data we collect is general and **does not require any robot-specific setup or calibration.** As stated in L214-224, the data consists of first-person RGB images and proprioceptive hand tracking poses obtained directly from the Project Aria glasses. None of these assume a specific robot embodiment for datasets construction in the co-training setting. Task adaptation is also not required since we do not constrain how the human data is collected. Hence, our human data can be directly applied to co-train with data from any robot embodiment. \n \n***“comparable environmental conditions between human and robot domains”***\n\nAs stated in L289-291 and shown in Figure 4 of the paper, we collect human data on an entirely different table, lighting conditions and background for the observation generalization result where EgoBridge generalizes to a completely new environment seen only in human data.\n\nWhile we do agree the paper would benefit from “evaluation across different robot embodiments”, we do not have access to collect data and perform evaluation on another robot within the time constraints of the rebuttal.\n\nWe thank the reviewer for pointing out presentation issues, which we will address in the next revision of the paper. Specifically we will replace the bottom right image of Figure 3 with the bottom center image. We will also replace the mislabelled bottom center case with purple background without a mirrored T to match the described evaluation cases.\n\n***“Why does the OT cost only apply scalar reduction to the minimum DTW pairs?”***\n\nWe thank the reviewer for their insightful question. We tried graduated scaling based on $A_{ij}$ by directly adding the $A_{ij}$ distance to the OT cost following prior work [6]. However, we found that this was unstable in training. We hypothesize that this is because of the large label shift between human and robot domains which makes the ordering of $A_{ij}$ noisy. Therefore we proposed a conceptually simple formulation that we also found to be stable during training empirically.\n\n***“For the Drawer experiments, how do the following factors affect performance: (1) varying the proportion of OOD drawers beyond 1/4, (2) different spatial arrangements of OOD drawers (random vs. clustered), and (3) different overlap ratios between human and robot training data?”***\n\nFrom empirical observation and prior work that studies spatial generalization [2, 3], random drawer arrangements would allow the policies to interpolate over the trajectories, especially when using a variational model like Flow matching. Hence we deliberately designed a challenging setting where the test region is held out in robot data to show a clear impact of human data. As shown in both DemoGen [2], this setting is strictly harder than a random configuration. Due to time constraints of the rebuttal, we are not able to collect additional data for the specific requirement and perform training and evaluation.\n\n***“What is the Laundry task performance under out-of-distribution scenarios such as different tables, unseen shirt types, or other environmental changes?”***\n\nTo address this question, we provide an additional observation generalization experiment for the Laundry task. Here, in addition to the robot demonstrations, we add 30 minutes of human data on a **new table with white surface** unseen in robot data (we used a wooden table with brown color in robot data). We evaluate the co-trained baseline and EgoBridge on the **new table**. We perform 20 rollouts across 5 randomly sampled positions on the workspace. We calculate success rate and sub-task points as described in L292-297 of the paper. The results (success rate and points) are summarized in the table below.\n\n| Model     | Success Rate (%) | Points |\n|-----------|------------------|--------|\n| Co-train  | 25               | 25     |\n| **EgoBridge** | **50**               | **43**     |\n\nIn addition, we perform an additional experiment, where we test a more challenging case of combinatorially generalizing to a **new shirt color** on the **white surface** where the new shirt is seen on the original table in the human data and excluded from robot data entirely. We use an identical evaluation protocol all the previous experiment and report the results in the table below.\n\n| Model     | Success Rate (%) | Points |\n|-----------|------------------|--------|\n| Co-train  | 5               | 7     |\n| **EgoBridge** | **30**               | **35**     |\n\n***“Given the focus on scalable human data, what is the relationship between human data proportion and robot policy performance? Also please compare with other methods across varying human-robot data ratios?”***\n\nPrior work (EgoMimic [1]) with a similar co-training setup show a favorable scaling law for human data as compared to teleoperated robot data. We anticipate our method to show a similar or superior scaling law as we build on top of EgoMimic and show favorable performance across all evaluated tasks compared to it. We would like to support this with empirical results, however we are unable to perform this evaluation within the time constraints of the rebuttal.\n\n**References**\n\n    [1] Kareer, S., Patel, D., Punamiya, R., Mathur, P., Cheng, S., Wang, C., ... & Xu, D. (2024). Egomimic: Scaling imitation learning via egocentric video. arXiv preprint arXiv:2410.24221.\n\n    [2] Xue, Z., Deng, S., Chen, Z., Wang, Y., Yuan, Z., & Xu, H. (2025). Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning. arXiv preprint arXiv:2502.16932.\n\n    [3] Saxena, V., Bronars, M., Arachchige, N. R., Wang, K., Shin, W. C., Nasiriany, S., ... & Xu, D. (2025). What matters in learning from large-scale datasets for robot manipulation. arXiv preprint arXiv:2506.13536.\n\n    [4] Bahl, S., Mendonca, R., Chen, L., Jain, U., & Pathak, D. (2023). Affordances from human videos as a versatile representation for robotics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13778-13790).\n\n    [5] Wen, C., Lin, X., So, J., Chen, K., Dou, Q., Gao, Y., & Abbeel, P. (2023). Any-point trajectory modeling for policy learning. arXiv preprint arXiv:2401.00025.\n\n    [6] Damodaran, B. B., Kellenberger, B., Flamary, R., Tuia, D., & Courty, N. (2018). Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European conference on computer vision (ECCV) (pp. 447-463)."}, {"review_id": "cFyngwCfOZ", "reviewer": "Reviewer_Eu5H", "summary": "This paper explores methods to bridge the distribution gap between teleoperated robot data and human demonstration data, which is crucial due to differences in visual appearance, sensor modalities, and kinematics which impede effective knowledge transfer. By addressing these challenges, the study demonstrates how human demonstration data can be effectively utilized to enhance robotic learning.", "questions": "1. If I understand correctly, the authors use DTW to search for the most similar trajectory between the human and robot trajectories in Section 4.2. In soft supervision, the most similar trajectory should have a low transport cost. However, each trajectory consists of many steps. How are the steps aligned between the trajectories? Do the trajectories have the same length? Then, we directly align step 1 in the human trajectory to step 1 in the robot trajectory?\n2. How are the trajectories divided into different steps? Human experience data might be continuous, while teleoperated data and human experience data may have different step structures to complete a task. How are these differences handled?\n3. If OT is considered a type of trajectory selection method, then we align the representations between the selected teleoperated and human data. Can we directly align trajectory i with i*(j)?\n4. Teleoperated data is certainly more challenging to collect. In the experiments, is the teleoperated data smaller than the human data?", "limitations": "See Question", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper is well-written and well-organized. The performance demonstrated is impressive, and the use of OT appears justified based on the experimental results presented. However, I think in this method, the human data and robot data should be collected from the same task. Although the experiments show the method's applicability to unseen tasks, these tasks are notably similar to those used during training. I am concerned about whether this method would perform effectively if a large amount of human data and only a minimal amount of robot data were collected on a completely new task.", "was_revised": true, "final_justification": "After reading the rebuttal, my concerns have been addressed.", "rebuttal": "We find it encouraging that the reviewer finds that our paper is well-written and well-organized and that our experimental results are impressive. \n\n***“[Would] this method would perform effectively if a large amount of human data and only a minimal amount of robot data were collected on a completely new task.”***\n\nWe assume that sufficient human and robot data are required for domain alignment. The scope of our work to show that in single task settings we can resolve the domain gap between human and robot for\nSuperior transfer for in-domain tasks over other human data augmented baselines\nGeneralization to new appearances (objects and scenes)\nGeneralization to new behaviours and motions that are only seen in human data\nWe do agree that multi-task learning may allow one to collect less robot data on a completely new task, however the study of this is beyond the scope of this work. Our vision is that existing efforts to collect broad and diverse robot data can be augmented by an even larger distribution of human data of similar tasks across different scenes since human data with wearable devices can be collected with ease using EgoBridge.\n\n***“How are the trajectories divided into different steps? Human experience data might be continuous, while teleoperated data and human experience data may have different step structures to complete a task. How are these differences handled?”***\n\nWe thank the reviewer for this question. To clarify, we do not perform alignment at an episodic level. Instead, as described in L164-167 we align individual samples of (observation, action chunk) pairs from the human and robot dataset. An action chunk is a finite sequence of atomic actions. In our case, as mentioned in L234-235 in the paper, it is a sequence of future cartesian end-effector poses. Therefore, given a sampled mini-batch of human and robot (observation, action chunk) pairs, we identify the most similar samples with DTW and use that as soft supervision for OT to align latent embeddings computed from the samples in a differentiable manner. \n\nThe reviewer asks ***“In soft supervision, the most similar trajectory should have a low transport cost. However, each trajectory consists of many steps. How are the steps aligned between the trajectories? Do the trajectories have the same length? Then, we directly align step 1 in the human trajectory to step 1 in the robot trajectory?”*** \n\nThe human and robot trajectories have the same length of 100. DTW captures spatial similarity between two trajectories while allowing for differences in execution speed. Each timestamp in the first trajectory is mapped to a corresponding (but potentially different) timestamp in the second and the distance is aggregated across timestamps to produce a cumulative loss. This is described in L185-192 of the paper. Qualitatively, this heuristic cumulative loss works well in finding similar trajectories as seen in the Appendix Section I.\n\n***“If OT is considered a type of trajectory selection method, then we align the representations between the selected teleoperated and human data. Can we directly align trajectory $i$ with $i^{\\ast}(j)$?”***\n\nOT is not a trajectory selection method in our method. Instead, it is a differentiable training objective which encourages the feature encoder to align the latent space of human and robot data (L171-176). This results in a coupling feature distribution which permits transfer of knowledge. The reviewer astutely observes that it may be possible to directly align the positive pairs through deterministic mapping. This is precisely an alternate deterministic formulation of Optimal Transport Plan, the Monge map. We choose to use the probabilistic “soft” version of OT (Kantarovich) [27] as in a mini-batch setting, it is robust to pairing / label noise, and works better with behavior cloning since it retains the marginal distributions of human and robot data distributions. This design decision has also been inherited from related work in joint domain adaptation that leverage OT [1, 2, 3].\n\n***“Teleoperated data is certainly more challenging to collect. In the experiments, is the teleoperated data smaller than the human data?”***\n\nYes, collecting teleoperated data is much more challenging to collect and in the experiments the human data is much larger than the teleoperated data. This is summarized in the Appendix Table 4, where, for the Drawer task, human data is 2.5x the amount of robot data (360 demo v/s 144 demo), the Scoop Coffee task, human data is 14x the amount of robot data (720 demo v/s 50 demo) and Laundry task, human data is 2.33x the amount of robot data (700 demo v/s 300 demo).\n\n**References**\n\n    [1] Courty, N., Flamary, R., Habrard, A., & Rakotomamonjy, A. (2017). Joint distribution optimal transportation for domain adaptation. Advances in neural information processing systems, 30.\n\n    [2] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation, 2016\n\n    [3] Damodaran, B. B., Kellenberger, B., Flamary, R., Tuia, D., & Courty, N. (2018). Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European conference on computer vision (ECCV) (pp. 447-463)."}]

Author rebuttal:

Meta-review: The paper proposes EgoBridge to improve co-training on human and robot data via domain adaptation. It explicitly uses Optimal Transport to align the latent representations of the two types of data. 

The initial major concerns from the reviewers include:
- Using action similarity only as soft supervision in the cost function of OT (nDAS)
- Lack of details on the efficiency of training OT (nDAS,kgpd )
- Generalization and scalability (kgpd, 3a7U)

The rebuttal adequately addressed these concerns, and the final ratings of the paper are 2 Accepts and 2 Borderline Accepts. The AC agrees with the reviewers' assessment and recommends acceptance. The authors should revise the paper accordingly in the final version.
Decision: Accept (poster)