Title: Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations
Venue: NeurIPS 2025
Accepted: yes
Keywords: Imitation learning from videos

Online policy learning

Behavior cloning

Unsuerpvised learning

Visual reinforcement learning
Abstract: Humans can efficiently extract knowledge and learn skills from the videos within only a few trials and errors. However, it poses a big challenge to replicate this learning process for autonomous agents, due to the complexity of visual input, the absence of action or reward signals, and the limitations of interaction steps. In this paper, we propose a novel, unsupervised, and sample-efficient framework to achieve imitation learning from videos (ILV), named Behavior Cloning from Videos via Latent Representations (BCV-LR). BCV-LR extracts action-related latent features from high-dimensional video inputs through self-supervised tasks, and then leverages a dynamics-based unsupervised objective to predict latent actions between consecutive frames. The pre-trained latent actions are fine-tuned and efficiently aligned to the real action space online (with collected interactions) for policy behavior cloning. The cloned policy in turn enriches the agent experience for further latent action finetuning, resulting in an iterative policy improvement that is highly sample-efficient.
  We conduct extensive experiments on a set of challenging visual tasks, including both discrete control and continuous control. BCV-LR enables effective (even expert-level on some tasks) policy performance with only a few interactions, surpassing state-of-the-art ILV baselines and reinforcement learning methods (provided with environmental rewards) in terms of sample efficiency across 24/28 tasks.  To the best of our knowledge, this work for the first time demonstrates that videos can support extremely sample-efficient visual policy learning, without the need to access any other expert supervision.


Peer reviews:
comments: SUMMARY:
This paper introduces a method for sample-efficient imitation learning from videos. This works by first training an encoder on videos by using a contrastive loss between the frame and a shifted version of the frame, as well as a temporal loss for the next frame feature similarity. This paper also uses a world model to learn a latent action representation, which is then mapped to a real world action using a few expert trajectories. The paper evaluates on video-game tasks and compares to many prior works.

STRENGTHS AND WEAKNESSES:
The paper is well written, easy to understand, and presents an approach to a important problem. The results are good, showing more sample efficient learning than prior works. The idea is interesting, and learning from videos without requiring action annotations is a hard problem.


There are a few weaknesses, mostly in that the training of the encoder seems limiting, and I have doubts it will generalize to more complex and realistic environments. Same for the mapping from the latent actions to real actions. The main novelty of the approach is in taking these parts and making them work for the task, ie. using the contrastive pre-training + world model + latent action + action mapping. So if these methods don't generalize to real world tasks, it will greatly limit the impact of the paper.

QUESTIONS:
How general is the learned encoder? Is the shift image + contrastive loss learn useful features for non-video games, say real world robotics tasks? Or will this only work for relatively simplistic videos/environments?

How well does the mapping from latent action to action work in more complex environments, e.g., robot controls rather than just video game controls?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-24

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method for Imitation Learning from Videos, that is, the problem of learning a policy from videos of expert demonstrations with no action information. To do this, the authors propose \"Behavior Cloning from Videos via Latent Representations\" (BCV-LR).\n\nThe method: 1. learns latent features from video through self-supervision over video frames; 2. then learns a separate world model on the latent features, together with a latent action model; 3. online finetunes the latent action model and a learns a latent action decoder; 4. simultaneously to (3.), trains a latent policy that uses the latent action decoder to act.\n\nThe authors conduct experiments across discrete control and continuous control benchmarks, comparing their method to other ILV and RL based baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* The paper focuses on an important and timely problem, as imitation learning becomes widespread state of the art for many real world robotics applications in manipulation and beyond. Many are attempting to learn from large scale video, as it promises to be an extremely cheap and scalable data source of scarce robotics data.\n* The paper operates with state of the art methods in representation learning (self-supervision, latent actions, etc) and provides additional evidence towards a \"representation-learning first\" approach to solving traditional RL tasks and benchmarks. Specifically, the method's performance rivals or beats that of pure RL methods that have access to environmental rewards.\n* The comparisons with state of the art methods are extensive (within the chosen eval suite) and prove that the method beats alternatives.\n\nWeaknesses:\n* The overall method is complex, consisting of many stages that all have to work for the method to achieve its performance. The ablation study does however demonstrate that each component is required.\n* Stage 3. (as per the summary above), in which the latent action model is finetuned and the action decoder is trained, requires real actions to be observed via environment interaction. It is unclear what is the impact of the schedule of environment interactions, how exactly the phases are optimally alternated so as to sample iteratively from the policy learned online via the concurrent step 4. Does this matter?\n* The method as requested does require a source of ground truth actions from the videos, it is thus not strictly suitable for solving pressing hard problems for real world robotics, e.g. learning from real human videos and then transferring the policies to robots. This is also related to the benchmarks being limited to classic \"toy RL\" simulation environments.\n* How does the method compare to performing the steps in a fashion akin to LAPA? (No online learning, only imitation learning alignment on ground truth expert action?\n\nQUESTIONS:\n* As above: how exactly are phases 3 and 4 optimally alternated? Does this matter?\n* How does the method compare to performing the steps in a fashion akin to LAPA? (No online learning, only imitation learning alignment on ground truth expert action?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, it presents a new framework, Behavior Cloning from Videos via Latent Representations (**BCV-LR**), that performs imitation learning directly from expert videos without access to rewards or action labels. The method works in two stages: (1) an offline pretraining stage that learns a latent representation of video frames via first self-supervised representation learning and then extracts latent actions through a dynamics-based objective, and (2) an online finetuning phase where latent actions are aligned with real actions using minimal interactions in a reward-free environment. Experimental results demonstrate that BCV-LR achieves superior sample efficiency and outperforms state-of-the-art imitation learning and reinforcement learning baselines across 24 visual control tasks, including both discrete (Procgen) and continuous (DMControl) settings.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The paper presentation is good and easy to follow\n2. Good motivation to learn from action-free videos\n3. Good empirical performance with great sample efficiency: the proposed BCV-LR framework demonstrates impressive sample efficiency in both discrete and continuous control tasks. It performs competitively with or surpasses methods that have access to ground-truth rewards or expert actions.\n\n**Weaknesses**:\nThe scope of the experiments are relatively limited. All experiments are conducted in simulation/\"toy\" environments (Procgen and DMControl). While these are standard benchmarks, it is unclear how well BCV-LR generalizes to internet-scale human videos and real-world robotics applications.\n\nQUESTIONS:\n1. For continuous control tasks, they are relatively easy with low-dimensional action spaces. I'm curious how would the proposed methods work for environments with high-dimensional action spaces. (For example, take humanoid & dog from Deepmind Control Suite)\n2. The author mention that during self-supervised representation learning phase, it also learns some temporal dynamics by aligning representation of $o_t$ with $o_{t+1}$, but this seems not appearing in Equation 1. So what's the overall objective for this stage. Is it Equation 1 (contrastive loss with image reconstruction ) plus temporal loss?\n3. The reconstruction loss of feature learning seems suboptimal especially for environments with distracting backgrounds. The authors mention that they get better performance with this loss added. But is it because the visual observations of the environments it tests on are too simple?\n4. A very relevant work [1] is missing.\n\n[1] Ye et al. Become a Proficient Player with Limited Data through Watching Pure Videos, ICLR 2023\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes BCV-LR to conduct behavior cloning with expert videos and several online interactions. BCV-LR learns the dynamics model and inverse dynamics model on latent actions during the offline phase. In the online phase, it achieves behavior cloning of expert videos by aligning latent actions to true actions through interaction. Without action labels, experiments conducted on Procgen and DMControl demonstrate BCV-LR  with good sample efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe presentation of the paper is quite clear. The experimental performance of BCV-LR is excellent, especially considering that only a small number of online steps are used.\n\nWeaknesses:\nI am not sure whether the offline stage is individually conducted on each task or jointly conducted on a multi-task dataset. The practical significance of video behavior cloning in a single-task setting is limited. The LAPO in the baseline is designed for the pre-training on multi-task data, which may affect comparisons. Additionally, incorporating extra ablation studies and baselines could further solidify this work. Please refer to the Questions part.\n\nQUESTIONS:\n1.  Does the phrase \"without the need to access any other supervision\" include the supervision of actions during the online phase? The supervision on the actions directly injects the environment's inverse dynamics knowledge and potentially adjusts the distribution of latent actions through alignment with ground-truth actions. If the phrase refers to merely using the latent reward information implicit in expert videos without online expert rewards for video behavior cloning, JEPT[1] may be an uncited but related work. Similarly, without using an explicit reward signal, JEPT can achieve generalization of one-shot visual imitation in some tasks with a mixture dataset.\n2.  The design of BCV-LR is very similar to FICC[2] (mainly in the offline part), while FICC is not properly referenced. FICC should be a comparable baseline in the discrete setting.\n3.  I am curious whether the effectiveness of direct behavior cloning in the online phase is related to the single-task setting or the quality of expert data. The offline phase of BCV-LR is quite similar to LAPO and FICC, while both LAPO and FICC avoid direct behavior cloning in the online phase and choose to use online rewards for policy adjustment in the multitask setting. It would be better if a direct online BC variant of LAPO were involved in the comparison, which means the online reward will also be excluded in this setting, and may further illustrate this aspect.\n4.  The paper lacks experiments in a multi-task setting or a discussion on the relationship between the expert video scale and the performance in the single-task setting. Can BCV-LR generalize to new tasks through multi-task data? Or, for new tasks lacking sufficient data, how much offline data is required for BCV-LR to fulfill behavior cloning? Otherwise, the prerequisite of obtaining sufficient data often implies having a well-performing in-domain policy on the task, reducing the significance of behavior cloning from expert videos.\n5.  Adding some ablation experiments would be more helpful in explaining what factors make BCV-LR more effective than previous works. For example, a) the impact on performance of whether the representation learning part is directly frozen after pre-training or updated with $L_{la}$ or $L_{ft}$; b) the impact on performance of not updating the latent action predictor in the online phase.\n\n[1] Learning Video-Conditioned Policy on Unlabelled Data with Joint Embedding Predictive Transformer. Hao Luo, Zongqing Lu. ICLR 2025.\n\n[2] Become a proficient player with limited data through watching pure videos. Weirui Ye, Yunsheng Zhang, Pieter Abbeel, and Yang Gao. ICLR 2023.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}