MonoLift: Learning 3D Manipulation Policies from Monocular RGB via Distillation
Authors: Ziru Wang
Mengmeng Wang
Guang Dai
Yongliu Long
Jingdong Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Robotic Manipulation
Cross-Modal Knowledge Distillation
Policy Learning
Primary area: applications

Abstract: Although learning 3D manipulation policies from monocular RGB images is lightweight and deployment-friendly, the lack of structural information often leads to inaccurate action estimation. While explicit 3D inputs can mitigate this issue, they typically require additional sensors and introduce data acquisition overhead. An intuitive alternative is to incorporate a pre-trained depth estimator; however, this often incurs substantial inference-time cost. To address this, we propose MonoLift, a tri-level knowledge distillation framework that transfers spatial, temporal, and action-level knowledge from a depth-guided teacher to a monocular RGB student. By jointly distilling geometry-aware features, temporal dynamics, and policy behaviors during training, MonoLift enables the student model to perform 3D-aware reasoning and precise control at deployment using only monocular RGB input.
Extensive experiments on both simulated and real-world manipulation tasks show that MonoLift not only outperforms existing monocular approaches but even surpasses several methods that rely on explicit 3D input, offering a resource-efficient and effective solution for vision-based robotic control. The video demonstration is available on our project page: https://robotasy.github.io/MonoLift/.

MonoLift: Learning 3D Manipulation Policies from Monocular RGB via Distillation
Ziru Wang', Mengmeng Wang 2.1, Guang Dai', Yongliu Long", Jingdong Wang"
SGIT Al Lab, State Grid Corporation of China ZZhejiang University of Technology $7hejiang University "Baidu
Abstract
Although learning 3D manipulation policies from monocular RGB images is lightweight and deppoymment-rriendly. the lack of structural information often leads to inaccurate action estimation. While explicit 3D inputs can mitigate this issue, they typically require additional sensors and introduce data acquisition overhead. An intuitive alternative is incorporate pre-trained depth estimator; however, this often incurs substantial inference-time cost. To address this, we propose MonoLift, tri-level knowledge distillation framework that transfers spatial, temporal, and action-level knowledge from depth- -uiiddgudddd teacher to a monocular RGB student. By jointly distilling geometry-awarr features, temporal dynamies, and policy behaviors during training, Monol.ift enables the student model to perform 3D- aware reasoning and precise control at deployment using only monocular RGB input. Extensive experiments on both simulated and real-world manipulation tasks show that MonoLift not only outperforms existing monoeular approaches but even surpasses several methods that rely on explicit 3D input, offering a resource-efficient and effective solution for vision-based robotic control. The video demonstration is available on our project page: ftops://robotaaa httpps///robttasyiiiiubbb.b giitgub.tiiii io/ MonoLift/
1 Introduction
Robotic manipulation demands precise spatial understanding to support reliable 3D actions generation. While existing methods leverage multi-view images (I 2]. point clouds [ D. RGB-D (5) sensors to encode 3D structure, they typically require specialized hardware and complex preprocessing steps, including calibration, alignment, and filtering. These constraints limit scalability and hinder deployment in real-world, enoource-constriing settings (see Figurd a)).
As practical and lightweight alternative, recent efforts have increasingly focused learning 3D) manipulation policies directly from monocular RGB images 7 8 9 TO However, fundamental gap remains between 2D visual perception and 3I) action reasoning: visual similarity in 2D does not imply consistency in 3D actions. In manipulation tasks, visually similar observations may correspond states that demand distinct actions, which often stem from subtle positional or structural variations. Without access to such structural cues, imitation learning models frequently fail to disambiguate these cases, leading ambiguous suboptimal actions and ultimately hindering the learning of precise sate-action mappings [TT 12 13) Recent methods attempt to address this issue by enriching monocular representations with implicit structural cues (see Figure These efforts span range of strategies, including modeling temporal dynamics through video or image prediction 77 9 and synthesizing novel views from a single frame to inject spatial priors [TO]. While these methods have shown promise, they often adopt multi-stage pipelines, where structural reasoning is performed as an
Cooresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

intermediate step separate from policy learning. Such indirect designs often introduce cumulative errors and weaken the coupling between perception and decision
decision-making making.
A straightforward solution is to integrate depth estimators into the policy architecture, using their outputs as geometry-awar inputs. With recent advances in monocular depth estimation
Point Clouds
Multi-Viww RGB
Single-View RGB
Point Cloud Prepronssing
Projection and Fasion
Image Generater
Policy
15 16 17]. it is now feasible to obtain pseudo-depte maps from RGB images. How- ever, incorporating such estimators into the in- ference pipeline significantly increases computa- tional cost and latency (see Figure cc)), limiting their practicality in real-world deployment. This raises an important question: Can we retain the benefits of depth-guided 3D reasoning without incurring inference-timet cost?
14
Policy
Policy
Trames Impliis
Mand ancess
tratonn dote
Higher rist commllatie
(4)
(h)
Single-View RGB
Vingle-Vizw RGB
Pretiiian
Policy
Depih Exteenerr
Policy
Depei Map Explisii
Knowlolge
Teacher
Coity to dayloy
We answer this with MonoLift, which lifts monocular RGB inputs into 3D-aware percep- tion and control by distilling knowledge from a depth- -uddedd teacher (see digurd[d)). To enable effective distillation, we identify three fundamental challenges in policy learning from RGB-only inputs: (i) difficulty in spatial dis- ambiguation, (ii) limited temporal cues, and (iii) misguided actions due to absent 3D priors.
(6)
(d)
Figure I: Comparison of frameworks for 3D policy learning. Our method (d) leverages 3D cues estimated by off off-thheshelf depth estimators during training via dis- tillation, requiring no external 3D data (as in and no inference-time modules (as in and c). Red indicates additional inputs or modules beyond single-view RGB.
These observations motivate our tri-level knowledge distillation framework, which targets each limi- tations through three complementary components: (i) Spatial Representation Distillation: Transfers fused RGB-depth features from the teacher to help the student disambiguate visually similar yet structurally different observations. (ii) Temporal Dynamics Distillation: Aligns temporal feature trajectories to enable the student to capture motion patterns that reflect underlying 3D structural changes. (iii) Action Distribution Distillation: Transfers action distributions shaped by the teacher's 3D) understanding, guiding the student generate geometry-awen behaviors. Our ablation study in Figure Di a) confirms each distillation level provides distinct gains.
In summary, the contributions of this work are threefold: (i) We propose MonoLift, a resource- efficient policy learning framework that learns from 3D-aware teacher built on a pre-trained depth estimator, enabling monocular RGB agents to perform structured perception and control without extra 3D data or inference overhead. (ii) We design a tri-level knowledge distillation strategy that conveys spatial, temporal, and behavioral cues to improve the student's contextual understanding and rocision-making under limited-moon modalltt constraints. (iii) We validate MonoLift across a wide range of simulated and real-world robotic manipulation tasks, demonstrating its ability to effectively learn 3D-aware policies while maintaining deployment efficiency.
2 Related Work
Leveraging Explicit 3D Inputs for Policy Learning. To improve spatial reasoning and control precision in complex environments, many works incorporate explicit 3D inputs such RGB-D images, multi-view observations, and point clouds. Some methods focus on constructing coherent 3D representations to enhance spatial understanding [18
for example, MV-MWM [T] and SPA 2 utilize multi-view masked autoencoders to learn geometry-aware embeddings. In addition, several methods integrate 3D perception directly into policy generation [22
20
23
24
25
26 For
instance, 3D Diffusion Actor (27) formulates trajectory generation as a diffusion process in 3D space,
while 3D-VLA (88 unifies vision, language, and action modalities support end-to-end planning.
Despite their strong performance. these methods typically require high-cost sensors and complex processes, which limit their scalability. In contrast, our work targets a monocular RGB-only setting, aiming to learn 3D manipulation policies directly from single-view RGB images.
Inferring Implicit 3D Cues for Policy Learning. Policy learning for 3D manipulation under monocular RGB input has received increasing attention. Many methods directly map RGB inputs

---

to actions [TT 12 13 While efficient, they struggle to disambiguate visually similar states with different 3D structures, limiting control performance. To mitigate this, recent work has focused on inferring implicit 3D cues from monocular inputs. One common approach leverages temporal modeling by generating video sequences or intermediate frames to capture inter-frame disparities (29 8 [,0. For example, AVDC 7 extracts actions from generated videos using rigid transformations computed from optical flow, while GROUND [ aligns video models to continuous action spaces via goal-conditioned exploration. Other works synthesize pscudo-novel views from single RGB images using pre-trained diffusion models to inject explicit spatial priors [TO]. Additionally, vision foundation models have been used to extract semantic and structural representations from static images 31
32 33 as demonstrated in MT-R3M [34], which combines R3M embeddings with Transformer- based policy head. Although these methods show promise, they typically rely multi-stage pipelines that introduce error accumulation and weak perception-action coupling. In contrast, our method leverages depth estimators provide explicit geometric priors and adopts multi-level distillation for efficient policy learning.
Problem Formulation
We aim to learn a policy from monocular RGB images, without relying on 3D sensors. Each training demonstration consists of a language instruction and expert trajectory: T, (g', [.]](((, (=)))= where o denotes the language command associated with trajectory is the RGB observation time step i, and aj is the corresponding expert action. The full dataset is denoted as D = {Ti) The conditional policy R0(a, Ot, 9) is modeled distribution over actions given the current observation and instruction enabling uncertainty-aware imitation maximum likelihood estimation over expert demonstrations:
12
13
It is optimized via
Lacter - -B 9>>D [log (a(a (r,g)],
(1)
where 0 denotes the learnable parameters and at is the ground-truth expert action. To enhance 3D awareness, we incorporate a teacher-student distillation framework as an additional training objective. Complementing the actor loss Lactor, which captures expert behaviors, the proposed distillation losses ($ (42) convey spatial, temporal, and action-leve cues from depth-guidet teacher.
Method
The overall architecture and data processing pipeline of MonoLift are presented in Section 4.1
while
the details of the tri-level knowledge distillation mechanism elaborated in Section 4.2
4.1 Data Flow and Model Architecture
Monol ifft consists of two main components: a student model for deployment and teacher model for enhanced policy training (see Fig.
Student Model. The student model processes a sequence of RGB frames (representing the visual history) and language instruction through pipeline composed of the following components:
Spatial Encoding: Encode the RGB history using ResNet-18 encoder to obtain 1S Encode the instruction into a language token using a 6-layer version of MiniLM provided in [35].
Temporal Modeling: Concatenate visual features with the language token and append learnable action tokens form the input sequence. This sequence is fed into causal Transformer decoder, producing temporally-awee
features FiS from observation tokens.
Action Prediction: Use the Transformer outputs corresponding to action tokens as input MLP policy head, which generates the student's action distributions AS
Teacher Model. Used only during training, the teacher enhances visual representations by generating pseudo-depth signals. Its processing consists of:
Spatial Encoding: For each RGB, generate pseudo-depth maps using pretrained depth estimator (Depth Anything V2 [14]). Fuse the RGB-depth pairs using unified encoder with dual-ppth cross- modal fusion module (.4. l to produce features spa for spatial representation distillation.
3

---

Student model (Sec. 4.13
Image
Policy Head
Enc.
Transformer
Fs
(Sec.42.1)
FR
4.2.2)
From
(Sec. 4.2.3) Action
48
Temporal Distillation
Represestation Distillation
Lyp
Distribution
LAD
Flum
Distilationn
AT
Teacher model (Sec.
Depth
Imaage Enc.
Dual-Path Cross- Modal
Fige
F
Policy
Transformer
Head
Fusion
Depth
training and testing
Fiast
training only
Figure 2: Overview of the proposed MonoLift framework. The student uses single-view RGB input, while the teacher incorporates estimated depth to guide spatial, temporal, and action learning. Both models are trained end-to-end with shared encoder, Transformer, and policy head for consistent knowledge transfer. To avoid redundancy, action and language tokens, shared between teacher and student, depicted only in the teacher.
Concatenate each fused visual feature with the language token, and append learnable action tokens to form the input sequence. Pass this sequence through the Transformer decoder to model temporal evolution, yielding fumm for guiding temporal dynamics distillation.
Temporal Modeling:
Action Prediction: Pass the action token outputs through the policy head to produce the teacher action AT which serve behavioral guidance for the student.
4.2 Tri-level Knowledge Distillation Mechanism
Models based on monocular RGB inputs often lack structural awareness, limiting their effectiveness in manipulation tasks. First, without explicit geometric cues, the model struggles infer spatial layouts and object relationships, impairing scene understanding. Second, since state transitions are often accompanied by spatial changes, the lack of structural cues hinders accurate temporal modeling and transition recognition. Third, lacking spatial grounding, the model tends to produce conservative suboptimal actions, particularly in cluttered visually ambiguous environments.
To tackle these limitations, we introduce a tri-level knowledge distillation strategy that transfers spatial, temporal, and action-level knowledge, enabling the student to acquire the teacher's 's 3D-aware capabilities: (i) Spatial Representation Distillation ($4.2.1) minimizes the discrepancy between the student's s spatial features FS and the teacher's depth-enhanced representations FT (ii) Temporal Dynamics Distillation ($4.2.2) aligns the temporal gradients between teacher and student features (V, FV Fiem V. F,FS to enhance transition-aware representation learning; (ûii Action Distribution Distillation (42..33 minimizes the KL divergence between the student's policy act and the teacher's output AT transferring action distributions encourage geometry-awar behaviors.
4.2.1 Spatial Representation Distillation
Monocular RGB lacks key geometric cues, such as depth boundaries and spatial layouts, making it challenging to infer object positions and occlusions. To address this, we adopt spatial representation distillation, where the teacher encodes both RGB and pseudo-depth using Unified RGB-Depe Encoder with Dual- -Path Cross-Modes Fusion module. This process produces fused features FT which help the student learn geometry-awan representations from RGB alone.
Unified RGB-Depth Encoder. To reduce low-level discrepancies between RGB images and pseudo- depth maps, we first render each depth map d, into a pseudo-colored colored heatmap to visually resemble its corresponding RGB frame o,. This preprocessing step enables appearance-leved alignment across modalities. Both RGB and pseudo-depis inputs are then passed through the shared image encoder E(.) with identical weights. Despite parameter sharing, the encoder preserves modality-specifie cues through input-driven activation differences, yielding spatial features ET and FT Fep.t

---

Dual-Path Cross-Modal Fusion. Using the features F and F dep.e. we employ a dual-path fusion strategy that combines attention-based and pointwise interactions to balance global integration and local structural consistency. Specifically,, we concatenate F and F dep., and feed them into a multi-head self-attention module to produce F Fulm.-- We then split the attention output into two modality-specific halves and compute their per-token average to obtain F muttnmmannt' encouraging balanced contributions from both modalities. Simultaneously, we compute the token-wise average of RGB and depth features to obtain Pa "avg.tt which maintains pixel-level alignment. The final fused representation is computed as: F spa.t alignment while enhancing seometry-sensinee
(F attnnmmannt + Fave This fusion scheme retains modality
features, resulting in enhanced spatial representations.
Distillation Objective. The student receives only RGB input and extracts spatial features f's using 8()). To transfer the spatial knowledge encoded in the teacher's fused features F ts define distillation objective that minimizes the discrepancy between the student and the teacher's output:
LSD
1 H
Σ
|| spa,
FTT spa, el|2
(= H+1
where H denotes the length of the historical window, and t' represents the current prediction step. The loss is computed by averaging the feature alignment errors over the H time steps.
4.2.2 Temporal Dynamics Distillation
Manipulation tasks often appear visually continuous, but temporal variations in RGB are insufficient to capture distinct state transitions. To address this, we introduce a temporal dynamics distillation mechanism that uses a temporal modeling module to capture state evolution over time and aligns temporal gradients between the teacher and student. This alignment enables the student to better capture structural changes and transition dynamics.
Temporal Modeling Module. We use a Transformer decoder the temporal backbone for both teacher and student models. It consists of L stacked blocks with masked self-attention, cross-attention, and feedforward layers, following the standard decoder design (3 37, 38) Sharing the decoder ensures consistent temporal encoding and efficient feature alignment without extra parameters.
At each decision step t, the agent processes a history of length H. For each timestep in this history. the decoder input comprises three types of tokens concatenated sequentially: (i) Spatial tokens (F(t) -p spa, t). extracted by fusing RGB and depth inputs (teacher) or from RGB-only inputs (student); (ii) Action tokens [Faction, t- ---........ Faction, t). which are learnable embeddings t initialized randomly and trained jointly to capture autoregressive action dependencies; (iii) Language token Fiext derived from the task instruction.
Fr(+)
The decoder first applies masked self-attention to model temporal dependencies within this to- ken sequence, followed by cross-attention to integrate contextualized observations from encoded perceptual features. The decoder then outputs sequence of contextualized temporal features F() [F()) 1emm, -- tem, to capture the dynamic evolution of visual observations, which serve as the primary representations for temporal dynamic distillation. Additionally, the decoder outputs contextualized action features acc, ta which are passed to the policy head for action prediction.
pe(c)
Distillation Objective. Inspired by temporal gradient modeling in video understanding (39 we treat the variation of observation features over time as a key indicator of dynamic transitions in robotic manipulation. To transfer this capability, we align the temporal gradients of contextualized features output by the Transformer decoder, enabling the student to model the temporal evolution patterns encoded in the teacher's 3D-aware representations. The temporal gradients of the contextualized features from the teacher and student models at timestep t are defined GT = V. G$ V Since the feature sequence is defined over discrete timesteps, the gradients are approximated using central differences. The temporal distillation loss is computed over the central H 2 steps of the temporal window ending at the step t':
LID
1
Σ
| - G
(3)
H-2
t=
H+2
5

---

Table 1: Quantitative results on 8 tasks from two Libero scenes. Input types: S-RGB Single-view RGB,
M-RGB Multi-view RGB, RGB-D RGB+Depth. Task names are ahbreviated: R/W Red/White mug, Y/W
Yellow/White mug, C= Chocolate, Left, Right, P Plate.
Method
Input Type
R-L
R-R
W-L
Y/W-R
C-L
C-R
R-P
W-P
Avg. S.R. (%)
MT-ACT
S-RGB
43.3
20.0
70.0
36.7
20.0
20.0
33.3
40.0
35.4++2.6
RT-1
S-RGB
33.3
36.7
73.3
70.0
30.0
40.0
46.7
50.0
47.5 1227
MT-R3M
S-RGB
36.7
46.7
73.3
60.0
50.0
50.0
36.7
40.0
49.2 11.6
GROUND
S-RGB
38.4
40.8
51.2
38.4
70.4
79.2
72.8
25.6
52.5+8.8
3D-VLA
RGB-D
56.7
46.7
73.3
66.7
86.7
83.3
60.0
76.7
68.71110
SPA
M-RGB
40.0
36.7
70.0
76.7
70.0
76.7
60.0
60.0
61.2.12.7
MonoLift
S-RGB
63.3
83.3
83.3
73.3
80.0
100.0
83.3
80.0
80.8++3.3
where the boundary steps are excluded avoid the influence edge effects, ensuring stable gradient computations over the temporal window.
4.2.3 Action Distribution Distillation
In standard imitation learning, expert labels provide deterministic action targets but fail capture potential uncertainties and correlations among different actions. In contrast, a teacher equipped with RGB-D inputs can generate more informative action distribution that reflects a deeper spatial under- standing and richer 3D structural information, producing more stable and consistent action sequences. Therefore, introduce action distribution distillation, which minimizes the Kullback-Leibler (KL) divergence between the teacher and student action distributions, enabling the student model to learn 3D-aware maaiiinnnmmaiigggg even when only RGB inputs are available.
Policy Head Module. Following temporal modeling, we employ a multi-layer perceptron (MLP) with M linear layers the policy head shared by both teacher and student models. This module takes temporally contextualized action features F=(-) as input and outputs parameters defining a Gaussian action distribution A represented by its mean and standard deviation.
Distillation Objective. To align action distributions, we minimize the KL divergence at each step 4 conditioned on window of H past observations:
CAD = KL (A) A.),
(4)
where A7 and A are the action distributions produced by the teacher and student, respectively.
MonoLift is trained end-to-end using a total loss function that combines the three distillation objec- tives, along with the actor loss defined in Equation
5 Experiments
Environments. We evaluate our method on several simulated benchmarks and real-world robotic manipulation tasks, covering diverse range of challenges: (i) LIBERO-90 (40] for visually ambigu- ous tasks involving subtle structural differences; (ii) Meta-World 44] for fine-grained manipulation; and (iii) LIBERO-LONG 40 for long-horizon tasks. See supplementary materials for details.
Baselines.
We compare MonoLift against multiple baselines, detailed in supplementary materials.
Single-view RGB methods (direct mapping): RT-1 II and MT-ACT [2] adopt direct imitation
learning from single-view RGB observations, without incorporating any form of 3D information. They represent lightweight baselines suited for real-world deployment.
Single-view RGB methods (learned 3D cues): GROUND
(
encourages implicit 3D understand-
ing predicting future visual observations. MT-R3M 32]
incorporates spatial and structural
information extracted by a pretrained visual encoder, R3M [31.
utilize explicit 3D inputs, such as depth maps or multi-view RGB images, to provide geometric priors for policy learning.
Explicit 3D input methods: 3D-VLA (28 and SPA 2

---

Input Type
Avg. (%))
ME-ATT ROACA
KEG SPA
MT-ACT
11.3
RT-1
S-RGB
67.6#1.6
MT-R3M
S-RGB
72.242.7
GROUND
S-RGB
74.8#1.6
3D-VLA
RGB-D
83,2+22.4
SPA
M.RGR
72.842.1
Monotifi (onrs)
MonoLift
S-RGB
87.8
Dey C7)
Madiumm (5)
(3
Ang (15)
Figure 3: Quantitative results and qualitative analysis on Meta-World Success rates including (a) overall average and individual results for easy, medium, and hard tasks. (e) Visualized failure cases of the RT-1, and corresponding successful exccutions
by Monol iift, both under monocular RGB input.
5.1 Performance Comparision
Evaluation on LIBERO-90. We evaluate our method on the LIBERO-90 benchmark (40), where
challenges arise for monocular RGB-based methods due to the difficulty in distinguishing visually similar objects in the absence of 3D information. For a fair comparison, we adopt the same representative tasks from two diverse scenes in LIBERO-90, each paired with 20 expert demonstration trajectories, as outlined in [ We compare our method to three categories of baseline methods, introduced in the previous section under unified experimental setup. As shown in Table [ our approach demonstrates significant performance improvements across most tasks, achieving the highest average success rate among all evaluated methods. Compared to approaches that implicitly infer 3D cues from visual sequences, such as MT-R3M and GROUND, our method avoids multi-stage modeling and the cumulative errors associated with it by directly distilling from depth-guided teacher. Moreover, it outperforms methods relying on ground-trutt 3D inputs, such as 3D-VLA and SPA, which often lack explicit guidance the spatial, temporal, and behavioral dimensions. Our joint distillation strategy combines these aspects into a tightly coupled perception-control framework that enhances both task understanding and execution in complex environments.
Evaluation on Meta-World. To assess the effectiveness of our method on tasks requiring fine- grained control, we evaluate on manipulation tasks from the Meta- World benchmark (4I). covering diverse object interaction and tool-use scenarios. Based on control difficulty and geometric reasoning demands, we categorize the tasks into easy. medium, and hard categories, with each task comprising 20 expert demonstrations (details provided in the supplementary materials). These tasks require complex 3D reasoning, which particularly challenging under monocular RGB input, especially in our multi-task setup. As shown in Figure (a-b), our method achieves consistently strong performance, with especially large gains hard tasks, underscoring its robustness and precision in complex settings. Figure (c) illus- trates typical failure cases of RT-1 in the bin bin_ _iicking. Due to the lack of 3D informa- tion, the model often struggles to localize objects accurately in scenarios with vary- ing target positions, leading to misaligned grasping attempts. In contrast, our method successfully picks the target object under the same input conditions, demonstrating stronger spatial reasoning and robustness target variations. Notably, this achieved without any depth input during inference.
Meta-World
Method
Avg. S.R. (%)
MI-ACT
42.0#:.2
RT-1
33.7+:3.0
w/o TrKD MonoLin
46.52:13
71.7+1.1
Saxco Rute
Mosstih wis TwRD
[c]
Maxutaff (var))
Figure 4: Quantitative and qualitative results on the LIBERO-Long benchmark. (a) Average success rates across all LIBERO- tasks under single-view RGB input. (b) Scene-wise comparison three environments: Kitchen, Liv- ing Room, and Study. (c) Qualitative comparison typical failures from the w/o TriKD variant, with successful comple- tions by MonoLift under the same input.
eval- uate the effectiveness of our proposed MonoLift in long-horizon, multi-stage ma- nipulation tasks, conduct experiments the LIBERO-Long benchmark 40
Evaluation on LIBERO-Long,
Each task in
7

---

(a Modium
Ang (i Sumclh Sene24)) And Miraiiibvani
Tempeni
B Nisochric E-RO B Moacic
Depibeit
Destwii
B Depbwaal
Bea Digea
(4)
(b)
Figure 5: (a) Ablation Study on TriKD. (h) Effectiveness and Transferability TriKD, (c) Impact of Depth Estimation Quality on Distillation Performance.
LIBERO-Lomg comprises a sequence of interdependent subgoals, posing significant challenges for spatial understanding and temporal stability. These experiments focus on evaluating policy performance under single-view RGB input. As shown in Figure A (a-b), our method consistently outperforms existing single-view RGB methods (MT-ACT, RT-
ablated variant without distillation (w/o TriKD) across multiple scenarios. In such complex tasks, baseline methods often struggle correctly transition between subtasks, resulting in unstable execution. Figure (c) shows typical failure cases. Without 3D-aware distillation, the model fails to accurately localize the handle of the moka pot, resulting in the pot being knocked over. Additionally, it also fails to perceive the spatial structure of the microwave, causing the mug to be only partially placed and eventually dropped. contrast, MonoLift demonstrates significantly better spatial understanding and policy stability under the same monocular RGB input.
well
5.2 Ablation Studies and Analysis
Ablation Study on TriKD. We ablate the tri-level knowledge distillation (TriKD) strategy in MonoLift by selectively disabling each guidance branch: spatial, temporal, and action-level. All variants share the same student architecture and training settings to ensure fair comparison. Specif- ically, we consider four model variants: the full TriKD model (ours) and three ablated versions without spatial (w/o Spatial), temporal (w/o Temporal), action-level (w/o Action) distillation. As shown in Figure a{),, the complete TriKD consistently achieves the best performance across all tasks, confirming the complementary nature of multi-level supervision. Removing spatial or temporal distillation leads to notable performance drops, particularly in complex scenarios. Spatial distillation guides the student in distinguishing geometrically distinct states; without it, the model often confuses similar-looking inputs and selects incorrect actions. Temporal distillation ensures consistent state transitions across time steps, while its removal results in unstable and incoherent behavior. Excluding action distribution distillation primarily affects the student's ability to learn the 3D-aware teacher's policy, leading difficulties in selecting the correct actions.
Effectiveness and Transferability of TriKD. To assess the transferability of TriKD beyond MonoLift, we apply it to RT-1, which formulates policy learning as a multi-class classification task by discretizing continuous actions. Since RT-1 lacks continuous action distributions, we remove the action distribution distillation branch and retain only the spatial and temporal objectives. As shown Table b), applying only the spatial and temporal components of TriKD RT-1 still yields performance gains, demonstrating the transfcrability of these objectives. However, the improvement smaller than that observed in MonoLift, which benefits from the full combination of all three objectives: spatial, temporal, and action, that work together to enhance policy learning. In addition to validating the applicability of TriKD, these results also highlight the individual effectiveness of each distillation branch, as well as the complementary advantage of their integration.
Impact of Depth Quality on Distillation Performance. We investigate how the quality of estimated depth affects the effectiveness of our method. Specifically, we employ three variants from the Depth Anything V2 series, vits, vitb, and vitl, which offer progressively better depth estimation [14]. Additionally, we include real depth maps rendered by the simulator for comparison. Unless otherwise specified, vitl is used as the default estimator in all experiments. Student performance improves with teacher depth quality (Figure ())) highlighting the role of accurate geometric cues. Notably, students trained with vitl-based supervision nearly match the performance of those trained with

---

real depth. We attribute the result to two main factors. First, vitl offers high-quality structural cues with clear boundaries and spatial continuity, enabling reliable guidance. Second, by distilling feature representations and action distributions rather than raw pixels, Monol lift is less affected by local errors and can leverage imperfect but structurally meaningful depth estimates.
Deployment Efficiency Evaluation. We evaluate deploy- ment efficiency in terms of inference time and model size. As shown in Table 2 MonoLift incurs lower computational over- head than the RGB+Depth baseline. The latter adopts the same network architecture as MonoLift but takes both RGB images and estimated depth maps (from Depth Anything V2) as inputs, using the same fusion module as the teacher model to combine the two modalities. Unlike Monolift, the RGB+h baseline requires depth inputs during both training and inference. This demonstrates the advantage of conducting depth-guided distillation without introducing additional runtime complexity. Compared to the RGB-only baseline, which is trained without depth or distilla- tion, MonoLift delivers improved task performance without increasing deployment cost. Moreover, as reported in Table in in the appendix, our MonoLift consistently achieves higher success rates than the RGB+ + Depth baseline. These results confirm the efficiency and scalability of our method in resource-constrained environments.
Table 2: Deployment efficiency.
Model
Time (ms) Params (M)
RGB-only RGB+Depth
18.1
8.5
442.8
344.8
Monoliff
18.1
8.5
5.3 World Experiment
We conduct real-world experiments using Franka Research 3 (FR3) arm, with data captured from a fixed frontal viewpoint using an Orbbec camera. The dataset covers six manipulation tasks; (I) press button, (2) push box into a goal, (3) pull out tissue, (4) pick up grapes and place them in fruit plate, (5) lift cup and pour water, and (6) fold a towel. These tasks involve diverse interactive objects and manipulation types. For each task, we collect 10 demonstration trajectories covering varied spatial configurations, recorded at a frame rate of 20 FPS. To evaluate the model's robustness to natural variations and suboptimal executions, we intentionally do not clean the expert demonstrations, which include perception jitter and imperfect actions. The implementation details are consistent with our simulation experiments. During real- wolld evaluation, task success is determined through manual inspection, with each task evaluated over 10 trials.
100
w/o TrKD MinoLift lours)
Press batton
cup pour
25%
00
Monol.ft wio TrikD
Jime
Pack iore dee goad
pell os stante
up ln place
i8t ani
Avomose
Monolift (ours)
Figure 6: (a) Quantitative comparison of real-world task success rates. Monol jift achieves higher average performance than its ablated counterpart (w/o TriKD) across real-world tasks. (b) Qualitative visualization of real-world execution examples comparing MonoLift and w/o TriKD on representative tasks. (left) press a button and (right) lift cup and pour water.
Quantitative Results. As shown in Figure 6 (a), we compare MonoLift with its ablated variant, MonoLift w/o TriKD, across six real-world manipulation tasks. Overall, MonoLift consistently achieves strong and reliable performance. For example, in the lift a cup and pour water task, which requires grasping narrow handle and performing smooth pouring motion, MonoLift demonstrates a clear performance advantage over its ablated variant. The press a button task involves pressing a small black area where even minor pose errors can lead to failure. In this high-precision task, MonoLift attains a perfect 100% success rate, while the baseline achieves only 20%. These improvements mainly result from our proposed TriKD framework, which transfers spatial, temporal, and levion- supervision from depth-guuded teacher. This design enables the student model to learn richer 3D

---

Time Stop
Press button
pick place a plate
Push box into poal
cup pour
Mill
Poll out a tissue
Fold towel
Figure 7: Qualitative visualization of real-world task executions. Trajectories from MonoLift visualized for tasks. Due space constraints, only selected keyframes shown rather than the full sequences.
representations and maintain stable control policies even under monocular RGB input. In contrast, the performance gap is smaller in the fold a towel task, where both methods achieve 20% success rate. This is mainly because deformable object manipulation involves unstable geometry and weak depth cues, while the towel's uniform texture further weakens the guidance from the depth-guided teacher.
Qualitative Results. Figure b()) visualizes representative real-world executions. Consistent with the quantitative trends, MonoLift demonstrates stable and precise control behaviors across different tasks. In the press a button task, MonoLift precisely localizes the button and executes smooth end-effector motions, while w/o TriKD frequently misses or presses inaccurately. In the lift cup and pour water task, MonoLift accurately aligns with the handle, performs a secure grasp, and smoothly modulates gripper rotation achieve controlled pour, showing fine-grained control over spatial alignment and motion timing. In contrast, w/o TriKD often misaligns with the handle due to its weaker 3D reasoning from monocular inputs. Figure Z all six real-world tasks, illustrating the effectiveness and robustness of Monolift in handling diverse manipulation tasks.
further presents successful executions across
Conclusion
In this work, we propose MonoLift, efficient framework for policy learning that bridges the gap between 2D perception and 3D action reasoning. Unlike prior approaches that rely on explicit 3D inputs multi-stage pipelines to extract 3D cues, MonoLift adopts training-only distillation strategy that leverages pseudo-depee depth maps generated by a pretrained depth estimator. This enables structured and efficient decision decision-making using only RGB inputs, without incurring any inference-timg overhead. To facilitate 3D-aware policy learning, we adopt a tri-level knowledge distillation strategy that transfers spatial representations, temporal dynamics, and action distributions from depth- teacher to a lightweight RGB-only student. Experimental results confirm that each distillation level contributes complementary benefits, leading consistent improvements in overall performance. MonoLift provides a scalable and deployable solution for real-world robotic manipulation tasks, especially in scenarios where 3D sensors or heavy computational resources are unavailable.
Limitations and Future Work. Although Monol.ff ifft achieves strong performance across multiple tasks through its tri-level distillation strategy, it still exhibits a notable limitation. While the training phase incorporates a depth-guided teacher model, the framework does not explicitly model the uncertainty or potential errors in the estimated depth. In real-world settings, depth estimators could produce inaccurate or unstable outputs due to factors such as occlusion, specular reflection, transparent materials, or sensor interference. While MonoLift directly leverages estimated depth signals to guide the student's 's spatial and temporal representation learning, these signals may introduce slight noise or uncertainty, potentially posing challenges for geometric understanding under visually complex or ambiguous conditions. Future work could explore an uncertainty-awan distillation mechanism that adaptively adjusts the depth guidance based on its estimated reliability, enhancing the robustness in such challenging scenarios.
10

---

Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant 6o.62203429) and the Natural Science Foundation of Zhejiang Province (Grant No.LON25F030009),
References
[1] Younggyo Seo al. "Multi-view masked world models for visual robotic manipulation". In: International Conference on Machine Learning- PMLR. 2023, pp. 30613-30632.
[2] Haoyi Zhu et "SPA: 3D Spatial-Awarenes Enables Effective Embodied Representation" In: The Thirteenth International Conference on Learning Representations. 2025. URT.: https
/ /pppnreeiee
[3] Shizhe Chen al. "PolarNet: 3D Point Clouds for Landuage-Guided Robotic Manipulation". In: Conference on Robot Learning. PMLR. 2023, pp. 1761-1781
[4] Minghua Liu al. "Frame Mining: Free Lunch for Learning Robotic Manipulation from
Point Clouds". In: Conference on Robot Learning. PMLR. 2023, pp. 527-538.
[5] Wanying Wang et al. "Visual robotic manipulation with depth-aware pretraining". In: arXiv
preprint ww..iv:240..00000 000938 (2024).
[6] Qingwen Bu et al. "Closed-Looop Visuomotor Control with Generative Expectation for Robotic Manipulation". In: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024. URT.: https: //ppr//v///eeeeeeeeeeeeeee net/ pett/forrmiidd1ptdkk//e/
[7] Po-Chen Ko et al. "Leearning to Act from Actionless Videos through Dense Correspondences" In: The Twelfth International Conference Learning Representations. 2024. URL: https
//openreview.c Sopperrevee..ne//Eorunnno htt..attee..ooounddddaaafft.....
[8] Achint Soni et al. "Videoagent: Self-improving video generation". In: arXiv preprint arXiv:2410. 10076 (2024).
[9] Yunhao Luo and Yilun Du. "Grounding Video Models to Actions through Goal Conditioned Exploration". In: The Thirteenth International Conference Learning Representations, 2025.
URL.: https: // /atp//veeeeeeeeee
[10] Stephen Tian et al. "View-Invariant Policy Learning via Zero-Shot Novel View Synthesis", In: 8th Annual Conference on Robot Learning. 2024. URT.: https:///ppenrevier httpeeewwwwwwwwweeeeeeee net/forum?
id=tqsQGrmVEa
[11] A. Brohan al. "RT-1: Robotics Transformer for Real-World Control at Scale", In: Proceed- ings of Robotics: Science and Systems (RSS). 2023.
[12] Homanga Bharadhwaj et al. "Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking". In: 2024 IEFE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 4788-4795
[13] Siddhant Haldar, Zhuoran Peng, and Lerrel Pinto. "BAKU: An Efficient Transformer for Multi-Task Policy Learning". In: The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024. URT.: https::///// ptttps://openreviee
[14] Lihe Yang et al. "Depth anything v2". In: Advances in Neural Information Processing Systems (2024), pp. 21875-21911.
[15] Lihe Yang al. "Depth anything: Unleashing the power of large-scale unlabeled data". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 10371-10381.
[16] Luigi Piccinelli et al. "UniDepth: Universal monocular metric depth estimation". In: Pro- ceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition. 2024,
pp. 10106-10116.
[17] Yiquan Duan, Xianda Guo, and Zheng Zhu. "Diffusiondepth: Diffusion denoising approach for monocular depth estimation". In: European Conference on Computer Vision. Springer. 2024,
pp. 432-449.
[18] Shengyi Qian et al. "3d-mvp: 3d multiview pretraining for robotic manipulation". In: arXiv preprint arXiv:2406 18158 (2024).

---

[19] Shizhe Chen et al. "Sugar: Pre-training 3d visual representations for robotics". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 18049- 18060.
[20] Haoyi Zhu et al. "Point cloud matters: Rethinking the impact of different observation spaces on robot learning". In: Advances in Neural Information Processing Systems 37 (2024), pp. 77799- 77830.
[21] Xi Zhu et al. "Nemo: Neural map growing system for spatiotemporal fusion in bird Sard's-eye-view and bdd-map benchmark". In: arXiv preprint arXiv:2306. 04540 (2023).
[22] Hao-Shu Fang et al. "Anygrasp: Robust and efficient grasp perception in spatial and temporal domains". In: IEEE Transactions on Robotics 39.5 (2023), pp. 3929-3945.
[23] Ankit Goyal al. "Rvt: Robotic view transformer for 3d object manipulation". In: Conference
on Robot Learning. PMLR. 2023, pp. 694-710.
[24] Ankit Goyal al. "RVT-2: Learning Precise Manipulation from Few Demonstrations". In: RSS 2024 Workshop: Data Generation for Robotics.
[25] Zhou Xian and Nikolaos Gkanatsios. "Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation" In: Conference on Robot Learning/roceeddnng
of Machine Learning Research. Proceedings of Machine Learning Research. 2023.
[26] Cheng Chi et al. "Diffusion policy: Visuomotor policy learning via action diffusion". In: The International Journal of Robotics Research (2023), p. 02783649241273666.
[27] Tsung- Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. "3D Diffuser Actor: Policy Diffusion with 3D Scene Representations". In: 8th Annual Conference on Robot Learning. 2024.
[28] Haoyu Zhen et al. "3D-VLA: A 3D Vision-Language-Actiion Generative World Model"". In: International Conference on Machine Learning. PMLR. 2024, pp. 61229-61245
[29] Chuan Wen et al. "Any-point trajectory modeling for policy learning". In: arXiv preprint vrXiv:2401.0002. (2023).
[30] Chrisantus Eze and Christopher Crick. "Learning by watching: review of video-based learning approaches for robot manipulation". In: arXiv preprint arXiv:2402.07127 (2024).
[31] Suraj Nair et al. "R3M: Universal Visual Representation for Robot Manipulation". In: Conference on Robot Learning. PMLR. 2023, pp. 892-909.
[32] Yecheng Jason Ma et al. "VIP: Towards Universal Visual Reward and Representation via IaluecImplicit nre-Training". In: The Eleventh International Conference on Learning Repre- sentations.
[33] Arjun Majumdar et al. "Where are we in the search for an artificial visual cortex for embodied intelligence?" In: Advances in Neural Information Processing Systems 36 (2023), pp. 655-677 677.
[34] Hongtao Wu et al. "Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation" " In: The Twelfth International Conference on Learning Representations. 2024. URL: https: http:///pppeveeeeeee
[35] Nils Reimers and Iryna Gurevych. "Sentence-bert: Sentence embeddings using siamese bert- networks". In: arXiv preprint arXiv: /908. 10084 (2019).
[36] Ashish Vaswani al. "Attention is all you need". In: Advances in neural information process-
ing systems 30 (2017).
[37] Nur Muhammad Mahi Shafiullah et al. "Behavior Transformers: Cloning $k$ modes with stone". In: Advances in Neural Information Processing Systems. Ed. by Alice H. Oh et al. 2022. URL: https:///// ///pp/reee//eriieee wwtp:/eeeeeeoeioeeeeeeeeeaa www.foorrmniiigrrrreorrrra vRQsa
[38] Zhiwei Dong al. "Bezierformer: A unified architecture for 2d and 3d lane detection". In: 2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE. 2024, pp. 1-6.
[39] Junfei Xiao et al. "Learning from temporal gradient for semi-supervised action recognition". In: Proceedings of the IEFF/CVF conference on computer vision and pattern recognition. 2022, pp. 3252-3262.
[40] Bo Liu et al. "Libero: Benchmarking knowledge transfer for lifelong robot learning". In: Advances in Neural Information Processing Systems 36 (2023), pp. 44776-44791.
12

---

[41] Tianhe Yu et al. "Meta-worke A benchmark and evaluation for multi-task and meta reinforce- ment learning" In: Conference on robot learning. PMLR. 2020, pp. 1094-1100.
[42] Tongzhou Wang and Phillip Isola. "Understanding contrastive representation learning through alignment and uniformity on the hypersphere". In: International conference on machine learning. PMLR. 2020, pp. 9929-9939.
[43] Xinyin Ma. Gongfan Fang. and Xinchao Wang. "Deepeache: Accelerating diffusion models for free". In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024, pp. 15762-15772.
[44] Xumpeng Huang et al. "Reverse transition kernel: A flexible framework to accelerate diffusion inference". In: Advances in Neural Information Processing Systems 37 (2024), pp. 95515- 95578.
[45] Yuanhao Zhai et al. "Motion consistency model: Accelerating video diffusiin with disentangled motion-uppearance distillation". In: Advances in Neural Information Processing Systems 37 (2024), pp. 111000-111021 1021.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] is generally preferable to "[No]" it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims regarding the proposed Monol.nf .ift framework and the tri-level knowledge distillation strategy are clearly stated in both the abstract and introduction. These claims are consistently supported by the experimental results and ablation studies presented in Section 5. The paper does not overstate the contributions and properly distinguishes between achieved results and future goals.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
14

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Due to space limitations, the main paper does not include an explicit "Limita-
tions" section. However, we discuss key limitations in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not contain any formal theoretical results or proofs. It fo-
cuses on algorithmic design and empirical evaluation of distillation framework for policy learning.
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
15

---

Justification: The paper provides detailed descriptions of the model architecture, datasets, and evaluation protocols in Section 4 and 5. Additional implementation details, hyperparam- eters, and environment configurations are included in the supplementary material to ensure full reproducibility of the main results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We plan to release them after proper organization and preparation.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https) for more details.
/nips.cc.. wttp://pcpppppccceeeeeeocceeeeee /tt.:u/deeeeeeeeeuuuuaiiiiiiaaoa
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
16

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper specifies key experimental settings such as model architecture, training objectives, and evaluation protocols Section 4. Additional details including optimizer configurations, and hyperparameters are provided in the supplementary material to ensure completeness.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports standard deviation across multiple random seeds for key experimental results to reflect variability due to training stochasticity.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
17

---

Answer: [Yes]
Justification:
The paper specifies the GPU type used for all experiments (Section 5).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
The research adheres to the NeurIPS Code of Ethics. It does not involve human subjects, sensitive data, or any application with foresecable ethical concerns.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper includes brief discussion of societal impact. The proposed method could potentially improve the efficiency and robustness of policy learning from limited sensing modalities, which may benefit real-world robotics applications. We do not foresee significant negative societal impacts, as the work does not involve human subjects, privacy-sensitive data, or deployment-critical systems.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
18

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks. mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve any models datasets that pose high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All third-party assets used in the paper, including datasets and pretrained models are properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwpprswiithhccooommoo thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not introduce any datasets, models, benchmarks.
Guidelines:
19

---

The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve any crowdsourcing research with human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The research does not involve any human subjects and therefore does not require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core methodology of this research does not involve the use of large language models (LLMs).
20

---

Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy bttesss//eerriissscc///oferreeee for what should or should not be described.
21


Reviews:
[{"review_id": "b6O3nx426i", "reviewer": "Reviewer_zdQB", "summary": "This paper proposes MonLift, which learns a single-view manipulation policy with 3D awareness by distilling information from a teacher model during training. The proposed method uses three kinds of knowledge distillation:  spatial representation distillation by aligning feature to a teacher model that encodes both RGB and estimated depth; temporal dynamics distillation by aligning the temporal student feature towards teachers by matching the temporal gradients; the  action distribution distillation that matches the student's action distribution to the teacher's. During deployment, only the student policy with single-view image observation is needed. \n\nExperiments are conducted on Libero and Metaworld benchmarks. The proposed method is shown to outperform several prior methods that use either explicit 3d information at test time or first reconstruct 3d information at test time instead of distilling the 3d knowledge at training time.", "questions": "- In figure 1 of the paper it is mentioned that  Multi-View RGB requires preprocessing and alignment. What preprocessing and alignment are needed? In my understanding, most modern multi-view RGM imitation learning algorithms do not need any specialized ways of fusing the multiview RGB images -- simply concatenating them channel wise or concatenating the feature vector is enough, e.g., as done in Diffusion Policy, OpenVLA, Pi-0, etc.\n- Regarding Temporal dynamics distillation – an ablation on using l2 loss between teacher and student features instead of gradient matching may help validate this choice.\n- Can the authors add a discussion on why is Action Distribution Distillation is needed if we already have expert action labels and the standard imitation learning loss using the expert action labels? Why additionally imitate the teacher actions would help?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n- This paper is well-motivated. 3D information is essential for manipulation and how to infer that purely from single view RGB image is important.\n- The paper is generally well-written and easy to follow.\n- The experiments are relatively comprehensive and show good results of the proposed method.\n\nWeakness:\n- I think the paper is missing important comparison to some well-established multi-view RGB imitation learning baselines, such as Diffusion Policy, OpenVLA, and Pi-0. These methods use multi-view RGB images as input and has demonstrated strong performance on many imitation learning benchmarks. E.g. Table 12 in OpenVLA shows that OpenVLA and Diffusion Policy can achieve strong results on the Libero Benchmarkings. It is unclear how the proposed method compares to these stronger baselines, which makes it hard to access the true merits of the proposed method.  \n- Some details and ablations are missing from the paper. See the questions section below.", "was_revised": true, "final_justification": "The authors have cleared my concerns during the rebuttal and thus I have raised my score from borderline reject to borderline accept.", "rebuttal": "Thank you for your feedback and recognition of our motivation, clarity, and experimental contributions. We have carefully responded to each point raised and provided a detailed reply to every comment.\n>W1: I think the paper is missing important comparison to some well-established multi-view RGB imitation learning baselines, such as Diffusion Policy, OpenVLA, and Pi-0. These methods use multi-view RGB images as input and has demonstrated strong performance on many imitation learning benchmarks. E.g. Table 12 in OpenVLA shows that OpenVLA and Diffusion Policy can achieve strong results on the Libero Benchmarkings. It is unclear how the proposed method compares to these stronger baselines, which makes it hard to access the true merits of the proposed method.\n\nThank you for pointing this out. To address the concern, we added comparisons to Diffusion Policy and OpenVLA on the four benchmarks from Table 12 of the OpenVLA paper: Libero-goal, spatial, object, and long. Libero-long was already part of our original evaluation; the other three were newly added for completeness. For consistency, we report Diffusion Policy and OpenVLA results directly from Table 12 of the OpenVLA paper. **MonoLift achieves the highest average performance across these benchmarks.** Public evaluation results of Pi-0 on the Libero suite are currently unavailable, and due to the limited rebuttal timeline, we were unable to reproduce them reliably for a comparison.\n\nMethod|Libero-Goal|Libero-Spatial|Libero-Object|Libero-Long|Average\n-|-|-|-|-|-\nDiffusion Policy from scratch|68.3±1.2%|78.3±1.1%|92.5±0.7%|50.5±1.3%|72.4%\nOpenVLA fine-tuned|79.2±1.0%|84.7±0.9%|88.4±0.8%|53.7±1.3%|76.5%\nMonoLift from scratch (ours)|85.3±1.2%|78.6±0.9%|96.3±1.2%|71.7±1.1%|**83.0%**\n\nWe will incorporate the full comparison and analysis into the final version of the paper. \n\n>W2: Some details and ablations are missing from the paper. See the questions section below.\n\nThank you for the comment. We address all questions below and will include the details and ablations in the final paper and appendix.\n\n>Q1: In figure 1 of the paper it is mentioned that Multi-View RGB requires preprocessing and alignment. What preprocessing and alignment are needed? In my understanding, most modern multi-view RGM imitation learning algorithms do not need any specialized ways of fusing the multiview RGB images -- simply concatenating them channel wise or concatenating the feature vector is enough, e.g., as done in Diffusion Policy, OpenVLA, Pi-0, etc.\n\nThank you for the feedback. Our original intention was to refer to techniques for projecting and fusing multi-view inputs, including spatial projection, cross-view attention, and self-supervised feature fusion. We recognize that the original wording may have been ambiguous and will revise it to *“projection and fusion”* in the final version.\n\nIt's worth noting that concatenation is a simple form of fusion. In Figure 1, our intent was to emphasize approaches that explicitly model inter-view relationships.\n\nFor example, in embodied intelligence:\n- RVT [1] and HiveFormer [2] apply cross-view attention to enhance contextual modeling; SPA [3] aggregates multi-view features into unified 3D feature volumes; MV-MWM [4] uses self-supervised view masking and video reconstruction for improved 3D world modeling. \n\nSimilar trends appear in other domains: \n- 3DMV-VQA [5] builds voxel-grid feature volumes from multi-view inputs; UniFusion [6] jointly models spatial and temporal information to improve cross-view understanding. \n\n*These approaches suggest that projection and fusion could lead to richer representations and improved performance, especially in tasks requiring complex spatial reasoning.*\n\n\n\n[1] Goyal A et al. RVT: Robotic view transformer for 3D object manipulation. CoRL, 2023: 694–710.  \n[2] Guhur P L et al. Instruction-driven history-aware policies for robotic manipulations. CoRL, 2023: 175–187.  \n[3] Zhu H et al. SPA: 3D spatial-awareness enables effective embodied representation. ICLR, 2025.  \n[4] Seo Y et al. Multi-view masked world models for visual robotic manipulation. ICML, 2023: 30613–30632.  \n[5] Hong Y et al. 3D concept learning and reasoning from multi-view images. CVPR, 2023: 9202–9212.  \n[6] Qin Z et al. UniFusion: Unified multi-view fusion transformer for spatial-temporal representation in bird’s-eye-view. ICCV, 2023: 8690–8699.\n\n\n>Q2: Regarding Temporal dynamics distillation – an ablation on using l2 loss between teacher and student features instead of gradient matching may help validate this choice.\n\nThank you for the suggestion. We performed this ablation study during the model design phase and are glad to present the results here. \n\nSpecifically, we replaced the temporal gradient matching term in MonoLift with a standard L2 loss and evaluated the model on Libero-Long and Libero-Object. As shown below, **temporal gradient loss consistently outperforms L2 loss across both benchmarks**.\n\n-|Libero-Long|Libero-Object\n-|-|-\nL2 Loss|62.3±1.2%|93.7±0.9%\nTemporal Gradient Loss (ours)|**71.7±1.1%**|**96.3±1.2%**\n\nThese results highlight a key insight: **static feature alignment (e.g., via L2 loss) is insufficient for capturing temporal dynamics critical to manipulation tasks**. With RGB-only input, adjacent frames often appear visually similar, making it difficult to detect meaningful state changes—this can lead to unreliable actions. In contrast, our temporal gradient loss proves more effective, enabling the student model to track how features evolve over time, thereby improving performance.\n\nWe will include these findings in the final version  to clarify the benefit of our design.\n\n>Q3:Can the authors add a discussion on why is Action Distribution Distillation is needed if we already have expert action labels and the standard imitation learning loss using the expert action labels? Why additionally imitate the teacher actions would help?\n\nWhile standard imitation learning uses expert action labels as direct supervision, we introduce Action Distribution Distillation (ADD) to enhance policy learning in several ways:\n\n**Learning from a More Informed Teacher:** While expert labels offer deterministic targets, they convey limited information regarding uncertainty and the interdependencies among possible actions. In contrast, a teacher model equipped with RGB-D input yields a more informative action distribution, shaped by a deeper spatial understanding. \nDistilling this distribution helps the student acquire both action preferences and uncertainty estimates, similar to how soft labels have been proven effective in enhancing student models in previous studies [1,2].\n\n**Enhancing Policy Stability:**\nSingle-view RGB policies are often sensitive to irrelevant appearance changes due to the lack of geometric cues, resulting in unstable actions. Our RGB+Depth teacher incorporates richer 3D structure to produce more robust and consistent actions, guiding the student via ADD as a form of regularization.\nTo support this, we measured action variance under fixed inputs: the teacher exhibited a lower standard deviation (0.17) compared to the student (0.21), indicating smoother responses. This effect is supported by prior findings that soft labels in distillation reduce output variance [3].\n\n**Empirical Support:** \nAblation results in Figure 5(a) in the main text confirm the effectiveness of ADD—removing it leads to performance drops, highlighting its additional benefit beyond standard imitation learning loss.\n\n[1] Phuong et al. Towards Understanding Knowledge Distillation. ICML, 2019.\n\n[2] Liang et al. Knowledge Consistency Between Neural Networks and Beyond. ICLR, 2019.\n\n[3] Zhou et al. Rethinking Soft Labels for Knowledge Distillation: A Bias–Variance Tradeoff Perspective. ICLR, 2021."}, {"review_id": "ckAVi6tF6X", "reviewer": "Reviewer_8Sa5", "summary": "In this paper, the authors propose a distillation framework for manipulation policy learning, where a teacher model that incorporates estimated depth, temporal dynamics is used; and a student model which only uses RGB image is used to distill the policy. The authors conduct extensive experiments on LIBERO-90, Meta-World and LIBERO-LONE. The proposed method outperforms baseline methods on these benchmarkes. Ablation studies are conducted to validate the model design choice.", "questions": "1. What are the weights of the three losses? How are they determined?How sensitive is the method to the weights?\n\n2. More results of the teach models can be added, including performances, learning curves.\n\n3. For the real-world experiments, the baseline performances are unexpected low. More explaination can be added.", "limitations": "The limitations are discussed in the supp.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Stength:\n\nThe proposed method is simple yet effective, which can be promising and valuable for the community. The authors conduct extensive experiments and ablation studies to validate the effectiveness of the method. \n\nThe paper is well written and structurd. Important details are provided for reproducibility.\n\nWeakness:\nThe effectiveness of the method on challenging tasks, where the depth estimation can be wrong, such as transparent and specular objects, is not shown.", "was_revised": true, "final_justification": "My question have all been well addressed in the rebuttal. I'll keep my score of accept.", "rebuttal": "Thank you for your kind and positive feedback—we truly appreciate your recognition of our method’s effectiveness, clarity, and thorough experimentation.\nBelow, we address each of your questions one by one, and we will carefully revise both the main text and the appendix according to your valuable suggestions.\n> Weakness: The effectiveness of the method on challenging tasks, where the depth estimation can be wrong, such as transparent and specular objects, is not shown.\n\nThank you for this point. Transparent and specular surfaces are indeed challenging for robotic manipulation. While our method does not explicitly target such cases, its design helps reduce the impact of imperfect depth in three ways:\n\n* **Robustness via Depth-Free Inference**:\n  Our method uses a teacher with access to predicted depth during training, but the student relies only on RGB at test time. This avoids direct dependence on depth quality and improves robustness when depth predictions are unreliable.\n\n* **Imperfect Depth Still Provides Structure**:\n  Even when depth is noisy, it often preserves useful cues like occlusion boundaries or coarse geometry. The teacher can still extract spatial structure from this signal, allowing the student to benefit from it during distillation.\n\n* **Expert Supervision Remains a Core Signal**:\n  In addition to distillation, the student is trained with expert action labels via behavior cloning. This provides a strong and reliable learning signal, ensuring stable policy optimization.\n\n**Empirical results:** To evaluate robustness to varying depth quality, we used three pretrained depth estimators from Depth Anything v2 [1], corresponding to low-, medium-, and high-quality outputs. Even with the weakest model (S), the distilled RGB-only student clearly outperforms the RGB-only baseline. Performance further improves as depth quality increases, showing that our method is tolerant to lower-quality depth.\n\n| Depth Estimator      | NYU-δ1 ↑ | NYU-AbsRel ↓ |Success Rate|\n|- | - | - | - |\n| No depth (RGB-only baseline) | –        | –            | 53.7                     |\n| S (low quality)          | 0.961    | 0.073        | 67.5 (**+13.8%**)         |\n| B (medium)               | 0.977    | 0.063        | 70.0 (**+16.3%**)         |\n| L (high quality)         | 0.984    | 0.056        | 80.8 (**+27.1%**)         |\n\n*Note: δ1 and AbsRel are reported on NYU-D dataset following [1].*\n\nWhile our current work does not target transparent or specular objects, we agree this is an important challenge and plan to address it in future work.\n\n[1] Yang L et al. Depth Anything v2. NeurIPS, 2024, 37: 21875–21911.\n\n>Q1:What are the weights of the three losses? How are they determined?How sensitive is the method to the weights?\n\nThank you for the question. We set the loss weights as follows: Spatial Distillation (1.0), Temporal Distillation (1.0), and Action Distribution Distillation (0.1). These weights were chosen based on empirical observations from preliminary experiments: we explored several reasonable configurations and found that this setting performs consistently well across benchmarks. We use a lower weight for action distribution distillation as it directly affects the final policy output and is more sensitive to the modality gap between RGB and RGBD inputs. \n\nTo assess robustness to weight choices, we conducted ablation experiments on the Meta-World benchmark by varying each loss weight in {0.1, 1.0, 2.0}, keeping the other two fixed. The ablation study shows that **performance stays consistently above the non-distilled baseline (75.6%)** and is **stable across a range of weight values**, indicating robustness.\n\nWeight Setting|0.1|1.0|2.0|\n-|-|-|-|\nSpatial Representation Distillation|+8.0%|+12.2%|+11.7%\nTemporal Dynamics Distillation|+8.7%|+12.2%|+11.4%\nAction Distribution Distillation|+12.2%|+10.1%|+9.0%\n\nWe will include these settings and results in the final version of the paper.\n\n>Q2:More results of the teach models can be added, including performances, learning curves.\n\nThank you for the suggestion. In response, we provide results on the teacher models, focusing on two aspects.\n\nWe first present results showing how different teacher models—equipped with pretrained depth estimators of varying quality from Depth Anything v2 [1]—affect student performance. As teacher quality improves, the distilled RGB-only student also achieves higher success rates.\n\n| Teacher Configuration  | NYU-δ1 ↑ | NYU-AbsRel ↓ |Success Rate|\n|- | - | - | - |\n| No depth (RGB-only baseline) | –        | –            | 53.7                     |\n| With Depth Estimator (S)  | 0.961    | 0.073        | 67.5 (**+13.8%**)         |\n|With Depth Estimator (B)| 0.977    | 0.063        | 70.0 (**+16.3%**)|\n|With Depth Estimator (L) | 0.984    | 0.056        | 80.8 (**+27.1%**)|\n\nIn addition, we report the results of teacher model trained and tested directly with RGB and depth as input, without distillation. Interestingly, our MonoLift consistently outperforms the teacher across all benchmarks. This is mainly because: 1) Our tri-level distillation effectively transfers structured spatial, temporal, and action-level knowledge from the depth-guided teacher. 2) The student relies only on RGB at inference, avoiding direct dependence on noisy depth and improving robustness.\n\nMethod|Libero-90 (%)|Libero-Long (%)| Meta-World (%)\n-|-|-|-\nTeacher model|70.3±3.7%|53.0±2.4%|85.4±2.6% \n**MonoLift (ours)**|**80.8±3.3%**|**71.7±1.1%**|**87.8±2.3%** \n\nWe'll include the teacher training curves in the final appendix to show convergence and stability. Due to rebuttal constraints, we can't add figures here. Thank you again for the helpful suggestion—it helped us improve the completeness of the paper.\n\n>Q3:For the real-world experiments, the baseline performances are unexpected low. More explaination can be added.\n\nThank you for the helpful observation. The relatively low success rates of the baseline model in our real-world experiments are largely attributable to two main factors:\n\n1. **Real-World Tasks Demand High-Precision Localization:**  \nReal-world tasks require accurate precise spatial reasoning and fine-grained temporal control, which are difficult for RGB-only models. The baseline struggle with weak 3D understanding, while our method improves robustness by distilling from a depth-informed teacher into an RGB-only student.\nThis difference is clear in tasks like Press a Button and Lift a Cup and Pour Water. The first task involves pressing a small black area, where minor pose errors can cause failure. The second requires grasping a narrow handle and pouring smoothly.\n\n1. **Imperfect Demonstrations Expose Weaknesses in the Baseline:**  \nWe didn’t clean the expert demonstrations to better reflect real-world conditions, including perception jitter and imperfect actions. This contrasts with some prior work that extracts only key frames for training [1]. RGB-only methods are more vulnerable to such noise, whereas our method learns more stable behavior through multi-level guidance during training.\n\n[1] Jia et al, Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation. CVPR, 2025.\n\n**We will include this analysis and discussion in the final version to better contextualize the real-world performance gap and highlight the benefits of our design.**\n\n>Limitations: The limitations are discussed in the supp.\n\nThank you for the suggestion. We will move the main content of the limitations section into the main text to improve clarity and transparency."}, {"review_id": "fwcqoY62aD", "reviewer": "Reviewer_TXrx", "summary": "This paper proposes MonoLift, a policy-distillation framework that applies tri-level knowledge distillation: (1) Spatial Distillation (matching the student’s RGB-only features to the teacher’s depth-enhanced features), (2) Dynamic Distillation (aligning temporal feature gradients), and (3) Action Distillation (minimising the KL divergence between student and teacher action distributions). The authors present an extensive experimental evaluation in both simulation and the real world, showing that MonoLift consistently outperforms a broad set of baselines.", "questions": "1. As mentioned in Weakness, what precisely is the “RGB + Depth” baseline in Table 2? Does it use the same architecture as MonoLift but with RGB + estimated depth inputs?\n2. Using a small MLP as the policy head feels limiting, especially given the recent advances with ACT and Diffusion Policy heads. Implementing MonoLift with one of these stronger heads might further raise performance.\n3. In the ablation study, did the authors remove the corresponding distillation loss in the Student for each of the ablated variations? \n4. There is a typo `$M$ linear layers` in line 218.\n5. A video showcasing the real-world experiments would help readers understand qualitative performance.\n6. Classical distillation schemes first train (and freeze) the teacher before optimizing the student. Did the authors try this sequential alternative? What advantages led you to prefer the end-to-end joint approach reported in the paper?", "limitations": "Although the authors discuss limitations, the section is relegated to the appendix. Moving it into the main text in the final version would improve transparency and balance.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The paper is well-motivated and timely: RGB-only perception is increasingly popular, and the proposed approach retains depth-related spatial cues during training while keeping test-time inference lightweight.\n2. The experimental design is thorough, spanning several environments and a wide array of baselines. The empirical results are convincing.\n\nWeakness\n\nMy main concern is the absence (or, at least, the lack of clarity) of a baseline that passes estimated depth to the policy at inference time instead of relying on distillation. The introduction claims that attaching a depth estimator `significantly increases computational cost and latency`, which is the core motivation for spatial distillation. Yet it is unclear whether such an RGB + Depth baseline is actually evaluated. Table 2 seems intended to quantify this cost, but it is not clear from lines 358–368 what this baseline contains. Moreover, including a clearly defined RGB + Depth baseline in the main quantitative results (Table 1 and Fig. 3a) would also let readers weigh accuracy against compute.", "was_revised": true, "final_justification": "The authors addressed most of my concerns in the rebuttal. I believe the paper is solid and has the potential to have a significant impact on the robot learning community, particularly in the field of policy learning using RGB-only inputs.", "rebuttal": "Thank you for your thoughtful feedback—we appreciate your recognition of the motivation and the strength of our experimental design and results. We address each of your comments in detail below and will incorporate your suggestions into the final version of the paper.\n> Weakness: My main concern is the absence (or, at least, the lack of clarity) of a baseline that passes estimated depth to the policy at inference time instead of relying on distillation. The introduction claims that attaching a depth estimator significantly increases computational cost and latency, which is the core motivation for spatial distillation. Yet it is unclear whether such an RGB + Depth baseline is actually evaluated. Table 2 seems intended to quantify this cost, but it is not clear from lines 358–368 what this baseline contains. Moreover, including a clearly defined RGB + Depth baseline in the main quantitative results (Table 1 and Fig. 3a) would also let readers weigh accuracy against compute.\n\nThis is a good point. We address this in two parts below.\n\n**Clarifying the RGB + Depth Baseline in Table 2:**\nThe “RGB + Depth” baseline refers to a model that takes as input both RGB images and monocularly predicted depth maps (from Depth Anything v2) during both training and inference. It employs the same architecture as the teacher model used in MonoLift.\nTable 2 in the main paper reports its compute cost, including the inference-time depth prediction. Compared to MonoLift (8.5 M / 18.1 ms), the RGB+Depth baseline is much heavier (344.8 M / 442.8 ms), highlighting the substantial deployment overhead our method avoids.\n\n**Updated Results in the Main Table:**\nWe originally reported this baseline in Table 2 to highlight compute cost. As suggested, we now include its performance in the main results table for a clearer comparison.\nWe report the average success rates across benchmarks here:\nMethod|Libero-90|Libero-Long| Meta-World\n-|-|-|-\nRGB + Depth|70.3±3.7%|53.0±2.4%|85.4±2.6% \n**MonoLift (ours)**|**80.8±3.3%**|**71.7±1.1%**|**87.8±2.3%**\n\nAlthough the RGB+Depth baseline benefits from richer inputs at inference time, it still underperforms MonoLift—especially on the most challenging Libero-Long benchmark. We attribute this to our multi-level distillation design, which makes more effective use of depth signals by transferring structured spatial, temporal, and behavioral knowledge into the RGB-only student.\n\nWe will incorporate these comparisons and clarifications into the final version of the paper. Thank you again for this valuable suggestion—it significantly improves the completeness and clarity of our evaluation.\n\n>Q1:As mentioned in Weakness, what precisely is the “RGB + Depth” baseline in Table 2? Does it use the same architecture as MonoLift but with RGB + estimated depth inputs?\n\nYes. The “RGB + Depth” baseline in Table 2 uses the **same architecture** as MonoLift, with both RGB and estimated depth maps (from Depth Anything V2) as inputs, along with the same simple fusion module used in our teacher model to combine the two modalities. Unlike MonoLift, this baseline requires depth inputs at both training and inference time.\n\nWe will clarify this in the revised text to avoid confusion.\n\n>Q2:Using a small MLP as the policy head feels limiting, especially given the recent advances with ACT and Diffusion Policy heads. Implementing MonoLift with one of these stronger heads might further raise performance.\n\nThank you for the insightful suggestion. Stronger policy heads such as those in ACT and Diffusion Policy are indeed effective. In our case, we adopted a lightweight MLP primarily due to efficiency and deployment constraints.\n\n| Policy Head      | Libero-Goal| Libero-Spatial| Libero-Object\n| - | -| -- | -\n| MonoLift (MLP)| 85.3±1.2%              | 78.6±0.9%     | 96.3±1.2%            \n| MonoLift (Diffusion Policy)| 80.0±1.1% | 82.5±0.8%| 96.2±1.4%   \n\nTo test a more expressive head, we integrated a diffusion-style policy head into MonoLift. While not all tasks showed consistent improvement, **Libero-Spatial exhibited clear performance gains**, confirming that **MonoLift is compatible with more expressive policy heads and can benefit from them**.\n\nWhile the diffusion-style head improves task success on certain benchmarks, it also incurs noticeably higher inference latency—from **18.1ms** (MLP) to **221.6ms** (diffusion). That said, we recognize the rapid progress in accelerating diffusion inference [1,2,3], and see these advances as complementary to MonoLift. Exploring their integration is an exciting direction for future work.\n\n[1] Ma X et al. Deepcache: Accelerating diffusion models for free. CVPR, 2024: 15762–15772.  \n[2] Huang X et al. Reverse transition kernel: A flexible framework to accelerate diffusion inference. NeurIPS, 2024, 37: 95515–95578.  \n[3] Zhai Y et al. Motion consistency model: Accelerating video diffusion with disentangled motion-appearance distillation. NeurIPS, 2024, 37: 111000–111021.\n\n>Q3:In the ablation study, did the authors remove the corresponding distillation loss in the Student for each of the ablated variations?\n\nYes, we removed the corresponding distillation loss for each ablated variant, confirming that each component individually contributes to performance.\n\n>Q4:There is a typo \\$M$ linear layers in line 218.\n\nThank you for your careful review and for pointing this out. We have corrected the formatting error in the revised version.\n\n>Q5:A video showcasing the real-world experiments would help readers understand qualitative performance.\n\nThank you for the helpful suggestion. We will create a project homepage with videos of our real-world experiments and include the link in the final version of the paper to facilitate qualitative assessment.\nAs external links are not allowed during the rebuttal phase, we will ensure the final version clearly references the video materials.\n\n>Q6:Classical distillation schemes first train (and freeze) the teacher before optimizing the student. Did the authors try this sequential alternative? What advantages led you to prefer the end-to-end joint approach reported in the paper?\n\nThank you for the thoughtful question.\n\nWe tested the standard sequential distillation setup: train and freeze the teacher, then train the student under fixed supervision. Despite using the same architectures, **joint distillation consistently outperforms sequential setup**. Additionally, sequential training significantly increases overall training time due to its two-phase structure, while our joint approach is simpler and more efficient to train.\n\nSetup|Libero-Long|Libero-Object\n-|-|-\nSequential Distillation|57.3±3.6%|90.7±1.2%\nJoint Distillation|71.7±1.1%|96.3±1.2%\n\nBeyond empirical gains, joint training offers several benefits in our cross-modal distillation setting (RGB+Depth → RGB):\n1. **Bridging the Modality Gap via Feature Alignment:**\n   In cross-modal distillation, the teacher and student operate on different input modalities, often leading to mismatches in feature distributions. Joint training helps bridge this modality gap by allowing both models to co-adapt their representations during optimization, improving alignment and transferability \\[1,2].\n2. **Adaptive, Student-Aware Supervision:**\n   Fixed supervision from a pretrained teacher fails to align with the student’s evolving capacity. Joint training enables the teacher to adapt its guidance based on the student’s learning progress, leading to more effective knowledge transfer—an effect also supported by prior studies [3].\n\n3. **Improved Efficiency and Stability:**\n   Sequential distillation fixes the teacher, meaning it cannot adjust to the student’s training dynamics.  In contrast, the teacher and student are trained jointly, resulting in a more stable and efficient distillation process—particularly beneficial in noisy or long-horizon tasks [4].\n\n[1] Park D Y et al. Learning student-friendly teacher networks for knowledge distillation. NeurIPS, 2021, 34: 13292–13303.  \n[2] Huo F et al. C2KD: Bridging the modality gap for cross-modal knowledge distillation. CVPR, 2024: 16006–16015.  \n[3] Sengupta A et al. A good learner can teach better: Teacher-student collaborative knowledge distillation. ICLR, 2023: 25–29.  \n[4] Ypsilantis N A et al. UDON: Universal dynamic online distillation for generic image representations. NeurIPS, 2024, 37: 86836–86859.\n\n> Limitations: Although the authors discuss limitations, the section is relegated to the appendix. Moving it into the main text in the final version would improve transparency and balance.\n\nThank you for the suggestion. We agree and will move the limitations section from the appendix into the main text in the final version to improve transparency and completeness."}, {"review_id": "45FmlXptJA", "reviewer": "Reviewer_fcUe", "summary": "This paper presents MonoLift, a tri-level knowledge distillation framework that enables learning 3D manipulation policies from monocular RGB images by transferring spatial, temporal, and action-level knowledge from a depth-guided teacher to an RGB-only student model.", "questions": "Please refer to my comments above.", "limitations": "Please refer to my comments above.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "1.\tWhile the paper claims to eliminate inference-time overhead, it still requires access to high-quality depth estimation during training. This dependency on pre-trained depth estimators may limit the method's applicability in scenarios where obtaining reliable depth estimates during training is challenging or expensive, somewhat undermining the core motivation of making 3D manipulation learning more accessible.\n2.\tThe paper lacks theoretical analysis of what specific 3D knowledge is being transferred through the tri-level distillation and why this particular combination of spatial, temporal, and action distillation is optimal. While the empirical results are promising, the absence of theoretical justification makes it difficult to understand the fundamental principles and predict when the approach might fail.\n3.\tAlthough the results are consistent across tasks, the absolute performance gains are often modest. For instance, on Meta-World, MonoLift achieves 87.8% vs 83.2% for the best 3D baseline - a meaningful but not transformative improvement. The gains over RGB-only methods are more substantial but vary significantly across tasks, suggesting limited robustness.\n4.\tThe demonstrated tasks, while spanning multiple benchmarks, are primarily focused on relatively standard manipulation scenarios. The evaluation lacks more complex manipulation tasks that would truly validate the claimed advantages of 3D-aware reasoning, and the real-world experiments are limited to basic manipulation primitives.\n5.\tWhile the tri-level distillation framework is well-designed, the individual components largely build upon existing techniques (standard depth estimation, transformer architectures, knowledge distillation). The primary contribution is the integration and application to manipulation learning rather than fundamental algorithmic innovations.", "was_revised": true, "final_justification": "I appreciate the authors' diligent efforts in the rebuttal to address several concerns. However, key issues regarding the “module integration weakens innovation” remain inadequately resolved. Consequently, my rating stays at borderline accept.", "rebuttal": "Thank you for your detailed and constructive feedback. We appreciate your time and effort. Below, we respond to each of your comments and will incorporate the relevant revisions into the final paper.\n>While the paper claims to eliminate inference-time overhead, it still requires access to high-quality depth estimation during training. This dependency on pre-trained depth estimators may limit the method's applicability in scenarios where obtaining reliable depth estimates during training is challenging or expensive.\n\nThanks for your thoughtful question. We clarify that our training setup is low-cost and robust to depth quality. We support this from the following perspectives:\n\n**Depth comes from reliable and off-the-shelf estimators.**\nOur method does not require depth ground truth during training. Instead, we obtain predicted depth from a publicly available monocular depth estimator trained on large-scale data, specifically Depth Anything v2 [1]. This large-scale trained estimator provides predicted depth without requiring specialized hardware, **generalizes robustly across datasets, and serves as a practical, inexpensive alternative**—as supported by recent work [2,3,4].\n\n**Our method remains effective with low-quality depth.**\nBy distilling high-level spatiotemporal and behavioral patterns rather than pixel-level details, the student avoids overfitting to noisy regions—allowing imperfect depth to offer useful guidance. We confirmed this by testing with smaller, less accurate models; even with degraded depth quality, our method outperformed the RGB-only baseline, demonstrating robustness for real-world deployment.\nDepth Estimator|NYU-δ1↑| NYU-AbsRel↓|Success Rate\n-|-|-|-\nNo depth|–|–|53.7 \nS (low quality)|0.961|0.073|67.5(**+13.8%**)\nB (medium)|0.977|0.063|70.0(**+16.3%**)  \nL (high quality)|0.984|0.056|80.8(**+27.1%**) \n\n*Note: δ1 and AbsRel are reported on NYU-D dataset following [1].*\n\n**Once-and-Done Depth Estimation.**\nIn practice, we estimate depth once before training to prepare the teacher’s inputs. The depth estimator is not used afterward, keeping the training and inference pipeline lightweight.\n\nWe will incorporate these clarifications into the revised version. Thank you again for pointing out this concern.\n\n[1] Yang L et al. Depth Anything v2. NeurIPS, 2024\n\n[2] Ren X et al. Gen3C. CVPR, 2025\n\n[3] Wang H et al. Levitor: 3D trajectory oriented image-to-video synthesis. CVPR, 2025 \n\n[4] Izquierdo S et al. MVSAnywhere. CVPR, 2025\n>The paper lacks theoretical analysis of what 3D knowledge is transferred and why the chosen spatial, temporal, and action distillation combination is optimal. This limits understanding of the method’s principles and failure cases.\n\nThank you for the thoughtful question. While a full theoretical analysis is challenging, we provide conceptual motivation and empirical evidence to support the rationale behind the chosen combination and help address your concerns.\n\n**Design Rationale and Supporting Evidence:**\n- Unlike static perception tasks, robotic manipulation involves dynamic decision-making shaped by spatial structure, temporal dynamics, and policy behavior—requiring accurate perception, temporal reasoning, and stable action generation. This motivates our tri-level distillation design across spatial, temporal, and action levels.\n- This formulation is also supported by prior work. For example, [1,2,3] shows that jointly modeling spatial and temporal features benefits dynamic tasks, and [4,5] demonstrate that multi-level distillation can significantly improve student model stability and generalization.\n\n**Empirical Validation:**\n- We validated each component through ablations (Fig. 5(a) in our paper), which show that removing any single objective leads to a drop in performance.\n- To further illustrate the role of each component, we added additional analytical experiments. While figures cannot be included in this rebuttal, we report three key metrics aligned with our distillation objectives: (1) Correlation between RGB features and depth (↑ 257%) — indicates how well the student captures 3D spatial structure., (2) Temporal difference (↑ 57.6%) — reflects the ability to capture dynamic information over time, and (3) Action std (↓ 19.1%) — measures policy stability. \n\nWe believe that theoretical support also boosts the explanation, which is an interesting future direction but is not currently our focus.\n\n[1] Lin J et al. TSM. ICCV, 2019\n\n[2] Pan J et al. ST-Adapter. NeurIPS, 2022\n\n[3] Wang M et al. A multimodal, multi-task adapting framework for video action recognition. AAAI, 2024\n\n[4] Jin Y et al. Multi-level logit distillation. CVPR, 2023\n\n[5] Li M et al. Correlation-decoupled knowledge distillation for multimodal sentiment analysis with incomplete modalities. CVPR, 2024\n\n>Although the results are consistent across tasks, the absolute performance gains are often modest. For instance, on Meta-World, MonoLift achieves 87.8% vs 83.2% for the best 3D baseline-a meaningful but not transformative improvement. The gains over RGB-only methods are more substantial but vary significantly across tasks, suggesting limited robustness.\n\nWe thank the reviewer for raising this point. **Performance gains vary depending on task difficulty, with greater improvements observed on harder tasks.**\n\nUsing Meta-World as an example, we grouped tasks into three difficulty levels and observed the following patterns:\n* **Easy tasks** generally involve only coarse spatial reasoning and already perform well with RGB input, leaving little room for improvement.\n* **Medium tasks** show consistent and stable improvements with MonoLift.\n* **Hard tasks** benefit the most: they demand precise control and stronger spatial understanding. MonoLift outperforms the best 3D baseline by up to **+16.1%** in this category.\n  \n*Notably, the strongest 3D baseline uses depth at inference, while MonoLift relies only on RGB—yet still outperforms it, especially on harder tasks, demonstrating both efficiency and effectiveness.*\n\nIn addition, we evaluated the robustness of our method across different tasks by comparing the task-wise standard deviation of MonoLift with that of the best-performing 3D baseline:\n* **Meta-World**: MonoLift: **14.4%** vs. 3D Baseline: **25.7%**\n* **Libero-90**: MonoLift: **9.7%** vs. 3D Baseline: **12.9%**\n\nIn both benchmarks, MonoLift **exhibits lower standard deviations across tasks**, demonstrating greater robustness.\n*We appreciate the opportunity to clarify this point and will incorporate this explanation in the final version of the paper.*\n>The tasks focus on standard manipulation scenarios, with limited evaluation on more complex ones that would better validate the benefits of 3D-aware reasoning. Real-world experiments also remain basic.\n\nWe appreciate your comment and take this opportunity to clarify our benchmark selection and how task complexity is reflected in our evaluation.\n\nWe chose Meta-World and Libero as benchmarks due to their **popularity and widespread use** in evaluating manipulation performance [1,2]. To introduce greater complexity, **we added Libero-Long, which requires long-horizon decision-making and fine-grained spatial reasoning**.\nMonoLift’s strong performance demonstrates its ability to handle complex tasks.\n\nOur real-world tasks **demand high spatiotemporal reasoning**, and we intentionally avoided cleaning or manual selection of expert demonstrations, **preserving noise** to better reflect real-world conditions. These challenges are reflected in the poor performance of the baseline, as shown in Appendix Figure 8. *Press a Button* is depth-sensitive—slight over-pressing can cause collisions. *Lift a Cup and Pour Water* demands accurate grasping and smooth pouring.\n\nThis work focuses on developing the core method and validating it on widely adopted benchmarks. Informed by your feedback, we plan to extend it to more complex, unstructured settings in future work.\n\n[1] Yunhao Luo and Yilun Du. Grounding Video Models to Actions through Goal Conditioned Exploration. ICLR. 2025\n\n[2] Jia et al, Lift3D Foundation Policy. CVPR, 2025.\n>The tri-level distillation is well-designed but mainly integrates existing techniques, with the core contribution lying in application rather than novel algorithms.\n\nThank you for the comment. We would like to clarify two core aspects of our method:\n1. Our method is not a simple combination of existing components—it **extends prior work to address key challenges in dynamic manipulation**: weak spatial perception, limited temporal modeling, and unstable policies, which are respectively targeted by our tri-level distillation over spatial, temporal, and action dimensions.\n\n    Empirically, we found that **existing distillation strategies fail to address the core challenges**. We compared: (1) feature distillation, (2) action distillation, and (3) pretrain-and-freeze.\n    All variants underperformed due to key limitations: feature distillation weakens the perception–action link; action distillation misses intermediate features and cross-modal alignment; and pretrain+freeze prevents co-adaptation, limiting transfer.\n    -|No Distillation|Feature Distillation|Action Distillation| Pretrain+Freeze |MonoLift (Ours)\n    -|-|-|-|- |-\n    Libero-Long|46.5±1.3|61.0±1.6| 57.7±2.1|57.3±3.6|71.7±1.1\n2. While monocular depth estimator can produce depth maps, **their direct use in manipulation policies is often inefficient and suboptimal**. We introduced an RGB+Depth baseline that uses both RGB images and depth maps (from Depth Anything v2) during both training and inference. Compared to MonoLift (8.5M / 18.1 ms), this baseline is significantly heavier (344.8M / 442.8 ms) and still underperforms—despite having richer inputs at inference time.\n\n    Method|Libero-90|Libero-Long|Meta-World\n    -|-|-|-\n    RGB + Depth|70.3±3.7%|53.0±2.4%|85.4±2.6% \n    MonoLift (ours)|80.8±3.3%|71.7±1.1%|87.8±2.3%"}]

Author rebuttal:

Meta-review: This paper proposes MonoLift, a tri-level distillation framework that transfers spatial, temporal, and action-level knowledge from a depth-guided teacher to a monocular RGB student, enabling efficient 3D-aware manipulation without explicit depth at deployment. The method is well motivated and clearly written, and the experiments across LIBERO-90, LIBERO-LONE, Meta-World, and real-world tasks are comprehensive. Results show MonoLift consistently outperforms monocular baselines and even some methods using explicit 3D input.

**Strengths**:
- Clear writing and strong motivation (Reviewers zdQB, TXrx).
- Comprehensive experiments, with ablations validating design choices (Reviewers zdQB, 8Sa5, TXrx).

**Weaknesses**:
Real-world results are less convincing compared to simulation ones, with no videos provided.

**Rebuttal**:
The authors provided a thorough response with additional experiments, including:
- Comparisons with multi-view baselines (Diffusion Policy, OpenVLA).
- Ablations on temporal dynamics distillation, teacher depth quality, distillation strategies, and head architectures.
- An RGB+depth baseline.

These additions are insightful and address reviewer concerns, though real-world validation could still be strengthened.

**Suggestions for Improvement**:
- Include videos showcasing real-world experiments (Reviewer TXrx).
- Incorporate additional experiments into the main paper or appendix for clarity and completeness.
Decision: Accept (spotlight)