Title: Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution
Venue: NeurIPS 2025
Accepted: yes
Keywords: Imitation Learning

Out-of-Distribution Generalization

Robotic Manipulation
Abstract: Visuomotor policies trained via behavior cloning are vulnerable to covariate shift, where small deviations from expert trajectories can compound into failure. Common strategies to mitigate this issue involve expanding the training distribution through human-in-the-loop corrections or synthetic data augmentation. However, these approaches are often labor-intensive, rely on strong task assumptions, or compromise the quality of imitation. We introduce Latent Policy Barrier, a framework for robust visuomotor policy learning. Inspired by Control Barrier Functions, LPB treats the latent embeddings of expert demonstrations as an implicit barrier separating safe, in-distribution states from unsafe, out-of-distribution (OOD) ones. Our approach decouples the role of precise expert imitation and OOD recovery into two separate modules: a base diffusion policy solely on expert data, and a dynamics model trained on both expert and suboptimal policy rollout data.  At inference time, the dynamics model predicts future latent states and optimizes them to stay within the expert distribution. Both simulated and real-world experiments show that LPB improves both policy robustness and data efficiency, enabling reliable manipulation from limited expert data and without additional human correction or annotation. More details are on our anonymous project website https://latentpolicybarrier.github.io.

Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution

Abstract

Visuomotor policies trained via behavior cloning are vulnerable to covariate shift, where small deviations from expert trajectories can compound into failure. Common strategies to mitigate this issue involve expanding the training distribution through human-in-the-loop corrections or synthetic data augmentation. However, these approaches are often labor-intensive, rely on strong task assumptions, or compromise the quality of imitation. We introduce Latent Policy Barrier, a framework for robust visuomotor policy learning. Inspired by Control Barrier Functions, LPB treats the latent embeddings of expert demonstrations as an implicit barrier separating safe, in-distribution states from unsafe, out-of-distribution (OOD) ones. Our approach decouples the role of precise expert imitation and OOD recovery into two separate modules: a base diffusion policy solely on expert data, and a dynamics model trained on both expert and suboptimal policy rollout data. At inference time, the dynamics model predicts future latent states and optimizes them to stay within the expert distribution. Both simulated and real-world experiments show that LPB improves both policy robustness and data efficiency, enabling reliable manipulation from limited expert data and without additional human correction or annotation.

1 Introduction

In control theory, Control Barrier Functions (CBFs) offer a principled mechanism to enforce safety in autonomous systems by explicitly defining a safety set and ensuring states remain within it during execution [2]. Motivated by the conceptual clarity of CBFs, we seek a similar mechanism for visuomotor policies learned purely from data, where explicit analytical definitions of safety constraints and system dynamics are typically unavailable. In learning-based visuomotor control, the analogous challenge arises from covariate shift [55]: minor deviations from demonstrated expert behaviors can quickly compound, pushing agents into out-of-distribution (OOD) states and causing task failures. Traditional behavior cloning (BC) methods are particularly vulnerable to covariate shift, severely limiting their effectiveness and reliability in real-world applications.

Failure

Suboptimal

(Prior works)

Dynamics-guided Correction

(Ours)

Rollout distribution

Policy rollout

Expert demo distribution

Dynamics steering

Current approaches to address covariate shift primarily involve expanding the training distribution . One common approach is human-in-the-loop interventions, such as DAgger [11, 55, 36, 29, 40, 39], which repeatedly collect corrective demonstrations whenever agents deviate from expert behaviors 39th Conference on Neural Information Processing Systems (NeurIPS 2025).

Caption: similar size to the legend

(Left)

Prior methods using interactive expert corrections (e.g., DAgger). The expert shaded region) is expanded through iterative human-in-the-loop corrections (pink tra

trained on this expanded distribution (pink region) may lead to suboptimal trajectorie covariate shift (gray trajectories).

(Right)

Our approach, Latent Policy Barrier, decou

Ex

Ex di

(Figure 1 left). Another strategy involves synthetic data augmentation methods that leverage taskspecific invariances [17, 42]. However, these approaches are either labor-intensive or rely heavily on strong prior assumptions about the environment. Moreover, both approaches risk introducing inconsistent or suboptimal demonstrations into the training dataset, potentially degrading overall policy performance [41]. This limitation exposes a fundamental trade-off in robust visuomotor policy learning: precise imitation benefits from consistent, high-quality expert datasets, whereas robustness inherently demands exposure to diverse and often suboptimal data. Constructing a single dataset that balances these competing needs is inherently challenging.

To resolve this tension, our key insight is to explicitly decouple these conflicting objectives (Figure 1 right). Inspired by CBFs, we propose Latent Policy Barrier (LPB) , which implicitly treats the latent expert demonstration distribution itself as a barrier that separates safe, in-distribution states from unsafe, out-of-distribution regions [7]. Unlike analytically defined CBFs, LPB does not require explicit safety sets or system dynamics. Instead, it maps high-dimensional expert states into a learned latent space and uses their embeddings to define a barrier for detecting and correcting deviations.

To decouple precise expert imitation and OOD recovery, LPB leverages two complementary components: (a) a base diffusion policy trained exclusively on consistent, high-quality expert demonstrations, ensuring precise imitation; and (b) an action-conditioned visual latent dynamics model trained on a broader, mixed-quality dataset combining expert demonstrations and automatically generated rollout data [4, 45, 9, 62]. We collect the rollout data by executing intermediate checkpoints saved during base policy training. Importantly, the rollout data naturally covers diverse deviations around the policy's own distribution without requiring explicit success labels, task rewards, or additional human teleoperation. At inference time, LPB ensures that the agent stays within the expert distribution by performing policy steering in the latent space. LPB uses the dynamics model to predict future latent states conditioned on candidate actions output from the base policy. Then LPB minimizes the distance between the predicted future latent states and their nearest neighbors from the expert demonstrations in the same latent space. This latent-space steering approach simultaneously achieves high task performance and robustness, resolving deviations without compromising imitation precision.

In summary, we introduce Latent Policy Barrier in the context of behavior cloning. LPB offers the following advantages: 1) improves sample efficiency by decoupling expert imitation from out-ofdistribution correction - enabling the policy to focus on learning from a small amount of high-quality human demonstrations; 2) enhances robustness through the use of a dynamics model trained on inexpensive, lower-quality policy rollout data; and 3) plug-and-play compatibility with off-the-shelf pre-trained policies, improving their robustness without requiring policy retraining or fine-tuning.

Experimental results across simulated and real-world manipulation tasks demonstrate that LPB is able to enhances both the robustness and sample efficiency of visuomotor policy learning. Code and data for reproducing the result will be made publicly available.

2 Related Work

Mitigating Covariate Shift in Imitation Learning : Behavior Cloning (BC), despite its simplicity, remains a strong baseline for working solely with expert demonstrations [51]. An extensive body of works mitigates covariate shift of BC by expanding the expert distribution. Prior works use interactive expert interventions [55, 36, 29, 40, 14, 39, 25] that are iteratively integrated into the policy learning loop to improve performance. Alternative ways to expand the training distribution include synthetic data generation with task invariances [17, 42, 65], noise injection [36, 27], or dynamics model-guided state-action pair generation [49, 28]. Instead of focusing on the data, Inverse Reinforcement Learning combats covariate shift by alternately collecting on-policy rollouts and updating reward function that penalizes deviations from expert trajectories, pulling the learner's state distribution toward the expert's [68, 24, 18]. Other lines of work include training a recovery policy that automatically pushes the agent back to the in-distribution region [53] and incorporating training objectives that penalize distribution divergence [43]. Our method differs from these strategies by performing inference-time steering in latent space, bypassing the need for additional expert queries or heavy data augmentation.

Policy Learning from Suboptimal Data : Offline RL algorithms prevents the learned policy from drifting into OOD states by imposing an explicit regularization. This can take the form of policy regularization that uses a penalty to keep the learned policy close to the behavior policy or a BC prior [60, 34, 20, 19, 50], conservative value estimation that pessimistically down-weights outof-distribution actions [35, 33], or model-based uncertainty quantification that learns a dynamics

Base Policy Training (on expert data)

Expert

Data

Base

Policy

Dynamics Model Training

(on both expert and rollout data)

Dynamics

Model

Training Phase

Base

Policy

Dynamics

Model

Expert

Data

Back to In-Distribution

Steering

Base policy denoising

future latent state

Nearest

neighbor

Inference Phase

End of

training

Gradient

Minimize

distance

Rollout

Data

t

0

ckpt

t

0

+Δ

t

ckpt

t

final

ckpt

……

……

Repeat

K

times

if

Rollout

Data

Rollout

Data

入

A8(2t+h)

[Eo(Ag)

A

Zt+h

model and uses an uncertainty penalty to discourage trajectories from unfamiliar regions [64, 30, 8]. These strategies can exploit large, mixed-quality datasets, but they assume an external reward. Methods that learn from suboptimal demonstrations fit a reward function via enforcing trajectory rankings or regressing returns against noise-perturbed rollouts [5, 10]. Recent work shows that an agent can exploit its own rollouts - using them to forecast and refine future latent trajectories [62], to autonomously accumulate new task executions [4], or to label and filter heterogeneous demonstrations [9] - to supplement the limited expert data. In contrast, we repurpose policy rollout data to learn a dynamics model that, at inference time, nudges the agent back toward the expert distribution.

Inference-Time Policy Steering: A growing line of work improves a pre-trained policy at inference time by steering, without requiring additional data or fine-tuning. Guided denoising biases diffusionbased policies toward specified goals or reward signals by injecting gradient guidance into the denoising process [26, 1, 54, 46]. Value-based filtering samples actions from a generalist policy and executes the one ranked highest by a value function [47]. Human-in-the-loop steering treats user provided sub-goals, corrections, or preferences as constraints injected into the policy's sampler [58, 61]. Model-predictive refinement combines a base policy with a dynamics model to improve task performance or respect safety constraints [67, 56, 52, 48]. Similarly, LPB also uses a latent visual dynamics model [22, 21, 23, 59, 66] to steer the policy. Unlike prior works, LPB performs gradientbased action corrections using gradients from the latent dynamics model. The corrections pull predicted future states toward the expert manifold, unifying classifier guidance with model-predictive foresight while needing no explicit goals, rewards, or human input.

3 Method: Latent Policy Barrier

We consider the problem of visuomotor policy learning with behavior cloning. Given a demonstration dataset of observation-action pairs D expert = { ( o t , a t ) } t , the goal of the base policy is to learn a π : O → A that maps observations to actions by imitating expert behavior. Following the notation introduced by [12], at each time step t , the policy π θ ( A t | O t ) receives the most recent T o observations O t = { o t -T o +1 , . . . , o t } and predicts the next T p actions A t = { a t , . . . , a t + T p -1 } . Among these predicted actions, the first T a steps, { a t , . . . , a t + T a -1 } , are executed without replanning. The process then repeats after receiving new observations.

The goal of Latent Policy Barrier is to learn such a robust policy π θ from a limited set of high-quality expert demonstrations, capable of consistently performing long-horizon robot manipulation tasks with minimal compounding errors and drifting into out-of-distribution states (Figure 2). In the following subsections, we detail our approach: base policy and dynamics model training §3.1, and test-time optimization to avoid deviations §3.2.

3.1 Policy and Dynamics Model Training

We train the policy π θ ( A t | O t ) using a diffusion model [12] on a limited set of high-quality expert demonstrations. Given the high-dimensional visual observations, the policy uses a visual encoder

h θ ( O t ) that maps raw image observations into latent representations. The visual encoder and the noise-prediction network are trained end-to-end with behavior cloning loss.

To enhance dynamics model's generalization beyond the distribution of expert demonstrations, we collect an additional dataset of exploratory rollouts. These trajectories are automatically generated by rolling out intermediate checkpoints of the diffusion policy saved during training. Concretely, after an initial warm-up phase of t 0 training epochs, during which the base policy is still highly random, we save policy checkpoints at fixed intervals. Every ∆ t epochs we save a checkpoint ckpt t 0 + n ∆ t and roll it out for N full episodes in the evaluation environment, recording all transitions regardless of task success or failure. This schedule continues until the final training epoch t final. The resulting rollout dataset covers both early exploratory and near-convereged behavior, yielding a much wider state-action distribution than the original expert demonstrations. The diverse transitions in the rollout dataset enables the dynamics model to generalize more effectively, especially to novel states encountered at test time. Importantly, the rollout data is collected without human correction or reward annotations. Since it is generated as a natural byproduct of base policy training, no additional teleoperation or manual labeling is required. This makes it a inexpensive source of training data for learning a generalizable dynamics model that can steer the policy back toward expert-demonstrated states. We ablation the choice of data source for dynamics model training in Appendix A.

The visual latent dynamics model, denoted as d ϕ , is trained to predict future latent observations given current observations O t and a sequence of future candidate actions A t . It consists of two components: a frozen visual encoder h θ , shared with the base policy, and a learnable dynamics predictor f ϕ . The full model is thus written as:

We reuse the visual encoder h θ from the behavior cloning policy, which is trained end-to-end to optimize for task execution. This design choice is motivated by the following consideration: during test-time optimization, the dynamics model is used to predict future latent states in the same embedding space that the policy relies on to make action decisions. Freezing the encoder ensures consistency between the latent representations used by the policy and those optimized through the dynamics model (3.2). It also stabilizes training by preventing collapse of the representation space. The dynamics predictor f ϕ is implemented as a decoder-only transformer ([57, 44, 6, 66]. Given encoded latent representations of current observations and a future action sequence, the dynamics model predicts the encoded future latent observation at a specified prediction horizon.

The model is trained using a latent-space mean squared error (MSE) loss between the predicted future latent and the ground-truth latent of the future observation:

3.2 Test-Time Optimization

At inference time, our goal is to mitigate compounding errors by steering the policy back toward in-distribution expert states whenever deviations from those states are about to occur. We detect OOD states using the latent OOD score, which measures how far the current observation lies from the expert data distribution in the latent space. Specifically, given an observation o , we first encode it into a latent representation z = h θ ( o ) using the frozen visual encoder h θ from the base policy. To quantify distributional shift, we identify the nearest neighbor of z in the expert latent space, denoted as z NN ∈ D expert. The latent OOD score δ ( z ) is defined as:

A higher δ value indicates that z is farther from the expert data manifold and is thus more likely to be out-of-distribution for the base policy. At inference timestep t , we employ the learned dynamics model d ϕ to refine the stochastic denoising process (Algorithm 1). At timestep t , the agent observes O t , which is input to the base policy's denoising process. We first compute the latent OOD score for the current observation. If it's lower than a pre-defined threshold τ , then the action generated by the base policy is executed. Otherwise, we refine the denoising process using gradient guidance. Taking inspirations from classifier guidance in diffusion models ([15]) and its applicatons in robotics-related domains ([1, 26, 63]), we take the gradient of latent OOD score of predicted future latent state through the dynamics model and use this gradient to refine the action denoising process. Denote the noisy action sample at timestep t and at denoising iteration k as A k t , we extend classifier guidance to refine

A k t by minimizing the latent OOD score of predicted future state z t + h = d ϕ ( h θ ( O t ) , A k t ) . We define the modified noise prediction as

where η is the guidance scale. The classic classifier guidance approach requires training a classifier on noisy data samples, which is less practical in robotics settings where executing random actions can be unsafe or infeasible. Our rollout data offers a practical alternative: actions in policy rollouts naturally exhibit greater variability than expert demonstrations. As a result, the dynamics model trained on this data is well-equipped to predict the outcomes of noisy action samples. To avoid unreliable guidance from highly stochastic early denoising steps - where actions diverge significantly from the expert manifold - we restrict gradient-based guidance to only the final K guide denoising steps, where samples are more structured. For instance, Diffusion Policy often uses up to 100 denoising steps with DDPM, we only choose to refine only a subset of the denoising steps, e.g., the last 10 steps. By reweighting the action sampling distribution in favor of actions that lower the latent OOD score of predicted future states, the gradient guidance effectively steers the agent toward expert-like states. To better understand the effectiveness of the gradient guidance strategy, we also compare against alternative optimization methods, including direct gradient descent on the action sequence and model predictive control (MPC). Ablation results are provided in Appendix A.

Algorithm 1 Latent Policy Barrier (Inference time)

Require: Base policy π θ , dynamics model d ϕ , visual encoder h θ , expert dataset D expert, latent OOD

4 Experiments

Our evaluation focuses on two questions: 1) Does LPB improve sample efficiency and robustness of visuomotor policy learning compared to baseline methods? 2) When the agent drifts, can LPB detect the deviation at the right moment and steer the agent back to expert-like states? We benchmark across a suite of challenging robotic manipulation tasks in both simulated environments and on a real robot (Figure 3). In simulation, we benchmark on three suites: Push-T [12, 16], Robomimic [41], and the multi-task, language-conditioned Libero10 [38]. For Robomimic, we select its three most challenging tasks, Square , Tool Hang , and Transport . For the real robot experiment, we test on the Cup Arrangement task from [13] and the Belt Assembly task from the NIST board assembly challenge [32].

4.1 Simulation Benchmarks

To evaluate the sample efficiency of our method, we focus on a limited demonstration regime, where suboptimal rollout data can play a significant role in improving performance. For each Robomimic task ( Square , Tool-Hang , Transport ) and for Push-T , we keep 20% of the original expert demonstrations. For each task, a base diffusion policy is trained on these demonstrations. For Libero10 , we use

Push-T

Square

Tool Hang

Transport

Cup Arrangement

Libero-10

Belt Assembly

all 50 provided demonstrations for each of the ten tasks to train a language-conditioned, multi-task base diffusion policy. During policy training, we save intermediate checkpoints at fixed intervals and use them to collect additional rollouts. See Appendix B for further implementation details.

We compare LPB against the following baselines:

Expert BC [12]: Diffusion policy trained on only expert data.

Mixed BC : Diffusion policy trained on both rollout data and expert data.

Filtered BC [9, 45, 4]: We augment the expert data with successful rollout trajectories and re-train the diffusion policy on the aggregated dataset. We use the success criteria defined by the original benchmarks.

CQL [35]: Conservative Q-Learning (CQL) is an offline RL algorithm that learns a value function that explicitly penalizes overestimation of unseen actions. CQL requires task rewards. For Push-T , we use the dense reward from [12]; for Robomimic and Libero10 tasks, we use the sparse reward, which gives 1 on success and 0 otherwise.

CCIL [28]: CCIL enhances the robustness of behavior cloning by generating corrective data to augment the original expert data. Since the original method was proposed for tasks with low dimensional state space, we adapt it to tasks with image observations by encoding images with the frozen visual encoder in the base policy trained with behavior cloning loss.

As shown in Table 1, under the limited-demonstration setting, LPB matches or exceeds every baseline on all simulated tasks, showing strong sample efficiency. The largest gains appear on the longhorizon, precision-sensitive tasks, Tool-Hang and Transport , highlighting LPB's ability to correct minor action deviations that otherwise compound over time. On the most challenging Tool-Hang task, Filtered BC offers only marginal improvement, indicating that even "successful" rollouts still contain suboptimal or inconsistent actions that hurt BC. CQL obtains zero reward on Robomimic and Libero10 tasks, possibly due to the absence of a dense reward function, consistent with results reported in the Robomimic paper [41].

Inference-time Robustness. To investigate how well LPB mitigates covariate shift, we inject noise perturbations to the output action at inference time. At every time step, with probability p ∈ { 0 . 0 , 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 } , a Gaussian noise is added to the output action. We evaluate LPB alongside Filtered BC , Expert BC , and CCIL on the Transport task. Figure 4a shows that LPB maintains the highest success rate across all noise levels. While the baselines exhibit worsening performance as p increases, LPB degrades the least, indicating robustness to inference-time perturbations.

Sample Efficiency on Expert Demonstrations. We next vary the fraction of expert demonstrations used to train the base policy on Tool-Hang , from 20% up to the full dataset, to see whether LPB continues to add value as more expert data becomes available. Figure 4b shows that LPB's advantage is largest in the low-data regime (up to 60% demos), the margin narrows as the dataset approaches 100% , reflecting performance saturation.

















































































✎

✏

✑

✒

✓

✑

✓

✦

★

✦

★

✩✪✫ ✬✛✭✤✢✮✢✯ ✰✱ ✩✰✪✣✢✭✛✲✢✫

✩✳✫ ✴✵✰ ✩✶✷✮✣✫

✦

★

✦

★

✔

✕

✆

✄

☎

✆

Impact of Rollout Data Quantity. To evaluate how the performance of LPB depends on the amount of rollout data used for dynamics model training, we conduct experiments on the Tool Hang task (Figure 4c). We vary the number of rollout trajectories used to train the dynamics model: 0 , 50 , 150 , 300 , and 500 . When using 0 rollouts, the dynamics model is trained solely on expert demonstrations. We compare LPB against Filtered BC , which filters successful trajectories from the same rollout data and combines them with demonstrations to retrain a BC policy. Due to inconsistencies and suboptimal behaviors present even in the successful rollouts, Filtered BC yields little improvement upon the base policy. In contrast, LPB consistently improves as more rollout data is added, suggesting that the dynamics model benefits from the broader state-action coverage despite the suboptimality. Performance gains of LPB saturate around 300 rollouts.

Latent OOD Score Visualization and Analysis. To validate our design choice of the latent space and the latent OOD score formulation, we plot the latent OOD score over time for LPB and the Filtered BC baseline on a representative Transport rollout (Figure 5). As defined in 3.2, the latent OOD score is the ℓ 2 distance between the encoded observation and its nearest expert neighbor, where the encoder comes from the base policy. In the rollout generated from the Filtered BC baseline, the two arms miss the hammer hand-over, leading to unrecoverable failure (frames with red bounding box). At these failure timesteps, the latent OOD score spikes above the threshold τ , indicating the

/a0

✁

✂

✂

✄

☎

✆

✝

✞

✠

✠

✡

☛

☞

✌

✍

✝

✝

✖

✗

✘

✙

✂

✄

Success Rate

In-distribution

Initial Poses

Out-of-distribution

Initial Poses

LPB (Ours)

Base Policy

In-distribution Initial Poses

Out-of-distribution Initial Poses

1.0

0.8

0.80

0.85

0.85

0.30

0.2

0.0

Success Rate

LPB (Ours)

Base Policy

(1) Init

(2) Hook on 1st Pulley

(3) Hook on 2nd Pulley

(4) Release & Finish

1.0

0.75

0.55

0.2

0.0

state has drifted far from the expert manifold. In the rollout generated by LPB the latent OOD score rises as the hand-over phase begins but stays near τ ; gradients from the dynamics model guides the policy back toward expert-like states (frames with green bounding box). Throughout the episode the score remains mostly below τ , showing that LPB keeps the agent inside the demonstrated distribution. These observations support that (1) the visual encoder trained end-to-end with behavior cloning loss induces a latent space in which the nearest-expert distance reflects how far the current state has drifted from demonstrated behavior, and (2) this ℓ 2 distance serves as a reliable signal for OOD detection and correction, enabling effective latent-space steering by LPB. See Appendix A for results on alternative latent space choices.

4.2 Real-World Evaluation

4.2.1 Cup Arrangement with an Off-the-Shelf Pretrained Policy

We first assess LPB's ability to improve the robustness of an off-the-shelf pretrained policy on a real robot. As the availability of such pretrained policies continues to grow [31, 3, 37], the plug-and-play nature of LPB and its ability to enhance their robustness makes it especially valuable.

Task Setup. Specifically, we use the pre-trained diffusion policy for the Cup Arrangement task as our base policy [13]. In this task, a robot arm with a wrist-mounted RGB camera and a compliant finray gripper needs to first rotate the cup so its handle faces right, then pick up the cup, and finally place the cup upright on the saucer.

Base Policy and Dynamics Model. The base diffusion policy checkpoint, trained on in-the-wild expert trajectories, serves as the base policy for LPB. To collect training data for the dynamics model, we roll out the pre-trained policy from deliberately out-of-distribution initial poses, gathering 80 trajectories. Because intermediate policy checkpoints are unavailable, we augment the dataset with additional 40 human play trajectories recorded via the handheld UMI device [13].

Results. We perform two groups of experiments: in-distribution initial poses , where the wrist camera initially observes both the cup and the saucer (Figure 6 left), and out-of-distribution initial poses , where the camera initially sees neither object (Figure 6 middle). As shown in Figure 6 right, LPB matches the base policy on in-distribution initial poses and substantially outperforms it on OOD initial poses, confirming that latent-space steering is able recover from distribution shifts without compromising nominal performance.

To illustrate how the dynamics model corrects distribution shift, Figure 8 plots four frames sampled from rollouts under the base policy (top row) and LPB (middle row), with LPB paired with its nearest expert demonstration image (bottom row) and the corresponding latent OOD scores. Both rollouts

/a0

✁

✂

✄

☎

✆

✝

✞

✠

✡

☎

/a0

☛

☞

✌

✍

✂

✎

✏

✏

✞

✑

✘

✙

✛

✙

✘

✘

✙

✛

✘

✙

✛

✛

start from the same wrist-camera pose showing neither cup nor saucer. Guided by the dynamics model, LPB tilts the camera downward to reveal the objects, driving its OOD score below the threshold and successfully completing the task. By contrast, the base policy continues moving in the wrong direction with its observations remain OOD throughout. It fails to locate the objects and terminates at a joint-limit failure. Although the demonstration data contains only in-the-wild images and the latent space is noisier than simulated tasks, the latent-space nearest neighbor selected by LPB still redirects the robot to in-distribution views, showing that LPB is robust to visual clutter and attends to task-relevant features.

4.2.2 Belt Assembly

We further evaluate on the Belt Assembly task from the NIST task board.

Task Setup. As illustrated in Figure 7, the task begins with the belt already grasped by the robot's gripper. First, the robot positions the belt to hook it over the small pulley. Next, it moves downward while stretching the belt to loop its opposite side around the large pulley. The robot then performs a 180 ◦ rotation around the large pulley to thread the belt. Finally, it lifts the gripper to release the belt.

Base Policy and Dynamics Model. We collect 200 expert demonstrations, each performed with slight variations in the board position while keeping the initial robot pose fixed. To train the dynamics model, we additionally collect 400 rollout trajectories executed under randomly initialized board positions and initial robot poses.

Results. This task poses a significant challenge for naive behavior cloning due to its contact-rich nature and the high precision required for successful execution. At test time, we perform 40 rollouts for both the base policy and LPB, varying the initial robot pose and board position. All evaluations use the same set of test cases and initial robot configurations for fair comparison. The base BC policy achieves a success rate of 0 . 55 . As shown in Figure 7 (right), LPB improves this performance to 0 . 75 . Figure 9 shows representative rollout frames for both the base policy and LPB, along with their corresponding latent OOD score plots. At the start of the rollout, the observation lies out-ofdistribution, reflected by a high latent OOD score. LPB first recovers the robot to an in-distribution pose, enabling the base policy to subsequently complete the task. In contrast, the base policy alone fails to recover, resulting in task failure.

✒

✓

✔

✕

✖

✕

✄

✍

✁

✁

✕

✑

✕

✖

✡

✁

✄

☞

☞

✗

✆

✍

✄

✕

✑

✕

✖

✡

✁

✄

☞

☞

✗

✆

✍

✄

Base Policy

LPB (Ours)

NN in Expert Data

Latent OOD Score

Latent OOD Score

1

2

3

4

2

3

4

1

1

2

3

4

1

2

3

4

5 Conclusion and Discussion

In this paper, we propose a novel method, Latent Policy Barrier, that transforms behavior cloning from a passive imitator into an active, self-correcting controller with a dynamics model. By treating the expert manifold as a latent barrier and steering a base policy with a learned dynamics model, LPB achieves reliable imitation and eliminates costly human corrections. Extensive simulated and real-robot experiments show that LPB surpasses or matches baselines when trained on just a small slice of the demonstration set, while maintaining robust performance under severe perturbations, highlighting a practical path toward reliable visuomotor learning.

Limitations and Future Work. Currently, LPB only corrects short-term, local deviations around the expert demonstration. Future work can expand the dynamics model training distribution and employ long-horizon reasoning to recover from more aggressive deviations. In addition, the visual latent dynamics model in LPB is trained per task, limiting reuse and requiring new data for each domain. Future work can explore training multitask dynamics models that capture shared dynamics structure, offering zero-shot generalization and amortized training cost across diverse robotics domains. Moreover, the latent OOD score in LPB assumes access to expert training data to define the in-distribution manifold; in scenarios where only a pretrained base policy is available without paired training data, this metric cannot be directly computed. Future work could investigate distribution-free uncertainty quantification to enable robust policy learning in the absence of expert training data.

References

Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657 , 2022.

Aaron D Ames, Samuel Coogan, Magnus Egerstedt, Gennaro Notomista, Koushil Sreenath, and Paulo Tabuada. Control barrier functions: Theory and applications. In 2019 18th European control conference (ECC) , pages 3420-3431. Ieee, 2019.

Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π 0 : A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint arXiv:2410.24164 , 2024.

Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X Lee, Maria Bauzá, Todor Davchev, Yuxiang Zhou, Agrim Gupta, Akhil Raju, et al. Robocat: A selfimproving generalist agent for robotic manipulation. arXiv preprint arXiv:2306.11706 , 2023.

Daniel Brown, Wonjoon Goo, Prabhat Nagarajan, and Scott Niekum. Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations. In International conference on machine learning , pages 783-792. PMLR, 2019.

Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning , 2024.

Fernando Castaneda, Haruki Nishimura, Rowan Thomas McAllister, Koushil Sreenath, and Adrien Gaidon. In-distribution barrier functions: Self-supervised policy filters that avoid out-ofdistribution states. In Learning for Dynamics and Control Conference , pages 286-299. PMLR, 2023.

Jonathan Chang, Masatoshi Uehara, Dhruv Sreenivas, Rahul Kidambi, and Wen Sun. Mitigating covariate shift in imitation learning via offline data with partial coverage. Advances in Neural Information Processing Systems , 34:965-979, 2021.

Annie S Chen, Alec M Lessing, Yuejiang Liu, and Chelsea Finn. Curating demonstrations using online experience. arXiv preprint arXiv:2503.03707 , 2025.

Letian Chen, Rohan Paleja, and Matthew Gombolay. Learning from suboptimal demonstration via self-supervised reward regression. In Conference on robot learning , pages 1262-1277. PMLR, 2021.

Sonia Chernova and Manuela Veloso. Interactive policy learning through confidence-based autonomy. Journal of Artificial Intelligence Research , 34:1-25, 2009.

Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research , page 02783649241273668, 2023.

Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. arXiv preprint arXiv:2402.10329 , 2024.

Shivin Dass, Karl Pertsch, Hejia Zhang, Youngwoon Lee, Joseph J Lim, and Stefanos Nikolaidis. Pato: Policy assisted teleoperation for scalable robot data collection. arXiv preprint arXiv:2212.04708 , 2022.

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems , 34:8780-8794, 2021.

Pete Florence, Corey Lynch, Andy Zeng, Oscar A Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning. In Conference on robot learning , pages 158-168. PMLR, 2022.

Peter Florence, Lucas Manuelli, and Russ Tedrake. Self-supervised correspondence in visuomotor policy learning. IEEE Robotics and Automation Letters , 5(2):492-499, 2019.

Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adversarial inverse reinforcement learning. arXiv preprint arXiv:1710.11248 , 2017.

Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learning. Advances in neural information processing systems , 34:20132-20145, 2021.

Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International conference on machine learning , pages 2052-2062. PMLR, 2019.

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603 , 2019.

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning , pages 2555-2565. PMLR, 2019.

Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955 , 2022.

Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems , 29, 2016.

Nils Ingelhag, Jesper Munkeby, Michael C Welle, Marco Moletta, and Danica Kragic. Real-time operator takeover for visuomotor diffusion policy training. arXiv preprint arXiv:2502.02308 , 2025.

Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991 , 2022.

Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattacharjee, Byron Boots, and Siddhartha Srinivasa. Grasping with chopsticks: Combating covariate shift in model-free imitation learning for fine manipulation. In 2021 IEEE international conference on robotics and automation (ICRA) , pages 6185-6191. IEEE, 2021.

Liyiming Ke, Yunchu Zhang, Abhay Deshpande, Siddhartha Srinivasa, and Abhishek Gupta. Ccil: Continuity-based data augmentation for corrective imitation learning. arXiv preprint arXiv:2310.12972 , 2023.

Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, and Mykel J Kochenderfer. Hgdagger: Interactive imitation learning with human experts. In 2019 International Conference on Robotics and Automation (ICRA) , pages 8077-8083. IEEE, 2019.

Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, and Thorsten Joachims. Morel: Model-based offline reinforcement learning. Advances in neural information processing systems , 33:21810-21823, 2020.

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246 , 2024.

Kenneth Kimble, Karl Van Wyk, Joe Falco, Elena Messina, Yu Sun, Mizuho Shibata, Wataru Uemura, and Yasuyoshi Yokokohji. Benchmarking protocols for evaluating small parts robotic assembly systems. IEEE robotics and automation letters , 5(2):883-889, 2020.

Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169 , 2021.

Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing offpolicy q-learning via bootstrapping error reduction. Advances in neural information processing systems , 32, 2019.

Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems , 33: 1179-1191, 2020.

Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, and Ken Goldberg. Dart: Noise injection for robust imitation learning. In Conference on robot learning , pages 143-156. PMLR, 2017.

Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. arXiv preprint arXiv:2410.18647 , 2024.

Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems , 36:44776-44791, 2023.

Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot learning on the job: Human-in-the-loop autonomy and learning during deployment. The International Journal of Robotics Research , page 02783649241273901, 2022.

Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, and Silvio Savarese. Human-in-the-loop imitation learning using remote teleoperation. arXiv preprint arXiv:2012.06733 , 2020.

Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108.03298 , 2021.

Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17596 , 2023.

Shaunak A Mehta, Yusuf Umut Ciftci, Balamurugan Ramachandran, Somil Bansal, and Dylan P Losey. Stable-bc: Controlling covariate shift with stable behavior cloning. IEEE Robotics and Automation Letters , 2025.

Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world models. arXiv preprint arXiv:2209.00588 , 2022.

Suvir Mirchandani, Suneel Belkhale, Joey Hejna, Evelyn Choi, Md Sazzad Islam, and Dorsa Sadigh. So you think you can scale up autonomous robot data collection? arXiv preprint arXiv:2411.01813 , 2024.

Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In Conference on Robot Learning , pages 2905-2925. PMLR, 2023.

Mitsuhiko Nakamoto, Oier Mees, Aviral Kumar, and Sergey Levine. Steering your generalists: Improving robotic foundation models via value guidance. arXiv preprint arXiv:2410.13816 , 2024.

Kensuke Nakamura, Lasse Peters, and Andrea Bajcsy. Generalizing safety beyond collisionavoidance via latent-space reachability analysis. arXiv preprint arXiv:2502.00935 , 2025.

Jung Yeon Park and Lawson Wong. Robust imitation of a few demonstrations with a backwards model. Advances in Neural Information Processing Systems , 35:19759-19772, 2022.

Seohong Park, Qiyang Li, and Sergey Levine. Flow q-learning. arXiv preprint arXiv:2502.02538 , 2025.

Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems , 1, 1988.

Han Qi, Haocheng Yin, Yilun Du, and Heng Yang. Strengthening generative robot policies through predictive world modeling. arXiv preprint arXiv:2502.00622 , 2025.

Alfredo Reichlin, Giovanni Luca Marchetti, Hang Yin, Ali Ghadirzadeh, and Danica Kragic. Back to the manifold: Recovering from out-of-distribution states. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 8660-8666. IEEE, 2022.

Moritz Reuss, Maximilian Li, Xiaogang Jia, and Rudolf Lioutikov. Goal-conditioned imitation learning using score-based diffusion policies. arXiv preprint arXiv:2304.02532 , 2023.

Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages 627-635. JMLR Workshop and Conference Proceedings, 2011.

Zhanyi Sun, Yufei Wang, David Held, and Zackory Erickson. Force-constrained visual policy: Safe robot-assisted dressing via multi-modal sensing. IEEE Robotics and Automation Letters , 2024.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Yanwei Wang, Lirui Wang, Yilun Du, Balakumar Sundaralingam, Xuning Yang, Yu-Wei Chao, Claudia Perez-D'Arpino, Dieter Fox, and Julie Shah. Inference-time policy steering through human interactions. arXiv preprint arXiv:2411.16627 , 2024.

Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Daydreamer: World models for physical robot learning. In Conference on robot learning , pages 2226-2240. PMLR, 2023.

Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361 , 2019.

Yilin Wu, Ran Tian, Gokul Swamy, and Andrea Bajcsy. From foresight to forethought: Vlm-inthe-loop policy steering via latent alignment. arXiv preprint arXiv:2502.01828 , 2025.

Amber Xie, Oleh Rybkin, Dorsa Sadigh, and Chelsea Finn. Latent diffusion planning for imitation learning. arXiv preprint arXiv:2504.16925 , 2025.

Xiaomeng Xu, Huy Ha, and Shuran Song. Dynamics-guided diffusion model for robot manipulator design. arXiv preprint arXiv:2402.15038 , 2024.

Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, and Tengyu Ma. Mopo: Model-based offline policy optimization. Advances in Neural Information Processing Systems , 33:14129-14142, 2020.

Allan Zhou, Moo Jin Kim, Lirui Wang, Pete Florence, and Chelsea Finn. Nerf in the palm of your hand: Corrective augmentation for robotics via novel-view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17907-17917, 2023.

Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. Dino-wm: World models on pre-trained visual features enable zero-shot planning. arXiv preprint arXiv:2411.04983 , 2024.

Guangyao Zhou, Sivaramakrishnan Swaminathan, Rajkumar Vasudeva Raju, J Swaroop Guntupalli, Wolfgang Lehrach, Joseph Ortiz, Antoine Dedieu, Miguel Lázaro-Gredilla, and Kevin Murphy. Diffusion model predictive control. arXiv preprint arXiv:2410.05364 , 2024.

Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. Maximum entropy inverse reinforcement learning. In Aaai , volume 8, pages 1433-1438. Chicago, IL, USA, 2008.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The evaluation section 4 on both simulated environments and on a real robot demonstrates that our proposed method achieves higher samples efficiency and robustness.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations are explicitly discussed in the conclusion section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This paper does not contribute theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Experiment setups are described in the experiment section 4 in the main paper and in full detail in Appendix B. Code is provided in the supplemental and a cleaned version will be released in the near future.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: For simulation experiments, the demonstration data is taken from public datasets ([41, 12]). The rollout data can be generated by executing checkpoints saved during expert policy training. For real robot experiment, the demonstration data is taken from [13]. The additional rollout data and play data will be released in the near future. A working copy of the code is provided in the supplemental material, and a cleaned version will be released in the near future.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experiment settings are discussed in the experiments section 4 in the main paper and full details are in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For simulation experiments, we report success rate with std. deviation over 3 policy checkpoints. For real robot experiments, we do not have intervals due to the intensive resource requirements. We note that it is common for experiments with real robots.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The full details of compute requirement is provided in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We confirm that the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the broader impacts of the paper in Appendix C.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper uses public datasets and models. The learned policies and dynamics models do not have risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cited all original owners of assets mentioned in the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: A pre-release anonymous version of the code with documentation is provided in the supplemental material.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The paper doesn't not use LLMs apart from assisting with visualization code and grammar checks.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Additional Experiments and Results

A.1 Ablation: Gradient Guidance Hyperparameter Choice

To understand how the gradient guidance hyperparameters in LPB affect the final task success rate, we run a set of experiments on the Transport task. We focus on the two hyperparameters: the guidance scale η and the number of denoising steps that receive gradient guidance, K guide. As shown in Figure 10 left, when η is very small, the improvement over the base policy is minimal because the guidance signal is too weak. As η increases, the improvement becomes more pronounced; however, once η reaches 0.3, performance drops slightly, indicating an overshooting effect caused by overly strong guidance. Figure 10 right shows that LPB is generally robust to the choice of K guide: even when gradients are applied only during the last 35 denoising steps, LPB still outperforms the base policy, demonstrating that the dynamics model can provide meaningful guidance even when the noisy action samples are far from valid expert actions. Across all tasks, we find that injecting gradient guidance during the final 10 denoising steps is sufficient for a substantial performance boost.

















































A.2 Ablation: OOD Threshold

We analyze the effect of varying the OOD threshold for the Transport and Tool-Hang tasks. From Figure 11, we observe that setting too high results in fewer corrections being triggered, leading to performance drops. Conversely, setting too low can cause the dynamics model to get stuck in local optima and stall task progress. This effect is particularly pronounced in the Tool-Hang task, where the robot may get stuck at inserting the hook into the base.







































A.3 Ablation: Data Source for Dynamics Model Training

We evaluate different data sources for training the dynamics model on the Tool-Hang and Transport task. LPB leverages policy rollout data as an inexpensive and scalable way to improve the dynamics

model's generalization, but alternative sources are also available. We compare against two such baselines: (1) Expert Demonstration with Noise Injection. Gaussian noise is injected into expert demonstrations with a probability of 0.3 to generate diverse, perturbed trajectories. (2) EpsilonGreedy Exploration. A base policy is first trained on limited expert data, then rolled out from randomized initial configurations with Gaussian noise added to the output action at a probability of 0.3 for exploration. As shown in Table 2, LPB, which relies on policy rollouts, achieves the highest success rate. We also note that rollout collection does not require manual tuning of noise magnitudes or injection probabilities, which simplifies the data collection procedure in practice. Moreover, it naturally captures a broad spectrum of data, spanning early exploratory trajectories to those generated by a converged policy.

A.4 Ablation: Action Optimization

We ablate the action optimization strategies used in LPB on the Tool-Hang and Transport tasks. LPB adopts a classifier guidance-style approach for test-time action refinement. We compare this with two alternatives: (1) LPB-MPC , which uses Model Predictive Control with stochastic action sampling, and (2) LPB-GD , which directly optimizes actions via gradient descent. As shown in Table 3, LPB achieves the highest success rate. LPB-MPC also improves over the base policy, but not as much as LPB - likely because it relies on sampling, which can fail to capture rare but optimal actions if they are underrepresented in the action distribution. By contrast, LPB nudges each sampled action toward expert-like directions using a differentiable guidance signal, enabling exploration that remains on-manifold while still biasing toward high-value corrections. LPB-GD performs the worst, as its unconstrained optimization can produce actions that are out-of-distribution for both the base policy and the dynamics model.

A.5 Ablation: Choice of Latent Representation

To study how the choice of latent space affects deviation detection and recovery toward expert states, we evaluate different latent representations on the Square and Transport tasks. We compare LPB, which uses the base policy's encoder, against two alternatives: (1) DINOv2 Encoder and (2) Encoder Trained with Reconstruction . For (1), we use a pretrained DINOv2 model as the observation encoder for the dynamics model; it outputs 256 patch embeddings of dimension 384. This representation preserves fine-grained visual details, but may include task-irrelevant information, as it is not directly trained with task supervision. For (2), we pretrain a ResNet-18 as an autoencoder using a reconstruction loss and then freeze it to serve as the encoder for dynamics model training. As shown in Table 4, both the base policy's encoder and DINOv2 improve over the base policy alone, with the base policy's encoder outperforming DINOv2. This may be because DINOv2 encoder, trained in a task-agnostic manner, may focus on irrelevant features, while the base policy's encoder, trained via behavior cloning loss, is more aligned with task-relevant information. The reconstruction-based encoder performs the worst, suggesting that the reconstruction objective alone does not produce a latent space suitable for effective action optimization.

B Implementation Details

B.1 Implementation Details of Latent Policy Barrier

Base Policy. We adopt Diffusion Policy as the base policy in LPB due to its strong capability in modeling complex robot action distributions. It uses a ResNet-18 as the image encoder and a U-Net as the noise prediction network, with FiLM layers to condition the denoising process on both observation features and the current diffusion timestep. For all tasks, the base policy is trained using 20% of the available expert demonstrations. Task-specific and shared hyperparameters are provided in Table 5 and Table 6, respectively.

Dynamics Model. Once base-policy training passes an initial warm-up of t 0 epochs, during which the policy remains highly exploratory, we begin saving checkpoints at fixed intervals. Concretely, every ∆ t epochs we store ckpt t 0 + n ∆ t and roll it out for N complete episodes from randomly initialized configurations. All transitions, whether successful or not, are retained to train the dynamics model. This procedure continues until the final epoch t final. The values of t 0 , ∆ t , t final , N , and the resulting total number of rollout trajectories for each simulated task are summarized in Table 7.

The dynamics predictor f ϕ is implemented as a Vision Transformer (ViT). It receives the concatenation of the latent observation token, the proprioception state token, and an action token, and predicts the next latent observation and proprioception state tokens. Training hyperparameters for f ϕ are provided in Table 8.

Test-time Optimization. At test time, we denoise actions with a DDPM scheduler. For each timestep t we first compute the latent OOD score; if the score is below the threshold τ , we run standard denoising and execute the resulting action A 0 t . If the score exceeds τ , we apply latent steering during the final 10 denoising steps. Specifically, the current observation and the intermediate noisy action samples at denoising timestep k , A k t , are fed to the dynamics model, which predicts the future latent state z t + h . We then measure the Euclidean distance between z t + h and its nearest expert state in latent space; this distance is back-propagated through the dynamics model, and the resulting gradient with

respect to the noisy action samples provides the guidance signal. The OOD threshold τ is chosen empirically by rolling out the final policy checkpoint, while the guidance scale η is selected via a grid search. Both η and τ for each task are listed in Table 9.

Compute Resources. All simulated experiments are run on a single NVIDIA L40S GPU (46 GB VRAM). Base policies require roughly 24-48 h to converge, while the dynamics models converge within 24 h. For each simulated task, we evaluate the final three checkpoints, each spaced 10 training epochs apart; the results reported in Table 1 are averages over those three checkpoints. In the realrobot setting, the base policy is pretrained and thus incurs no additional training cost. The dynamics model is trained in parallel on six NVIDIA L40S GPUs and converges in approximately 36 h.

B.2 Implementation Details of Baselines

Filtered BC. This baseline uses the same rollout data collected for LPB. We discard trajectories that do not meet the task-specific success criteria and merge the remaining successful rollouts with the original expert demonstrations. The diffusion policy is then retrained from scratch on this augmented dataset.

CQL. We adopt the CQL implementation provided in the Robomimic codebase without modification.

CCIL. CCIL was originally designed for tasks with low-dimensional state inputs. To extend it to high-dimensional image observations, we cache latent representations produced by the base policy's encoder. Specifically, we first train a base diffusion policy π θ on the limited expert demonstrations, identical to the setup used for LPB. We then encode every image observation in the expert dataset with π θ 's encoder and store the resulting latent transition pairs. CCIL's dynamics model is trained on these low-dimensional latents, after which CCIL's corrective-label generation procedure is applied to augment the expert data. Finally, we fine-tune π θ on the augmented latent dataset. We use the authors' original implementation for all CCIL components.

B.3 Real Robot Experiment Setup

B.3.1 Cup Arrangement

Setup. We use a 6-DoF ARX5 robot arm with 3D-printed soft compliant fingers and a wrist-mounted GoPro camera, mirroring the sensor configuration of the original training data. Low-level controller is provided by the Universal Manipulation Interface (UMI) stack ported to the ARX5 platform https://github.com/real-stanford/umi-arx. The cup arrangement task consists of three sequential stages: (i) rotate the cup so its handle points right, (ii) grasp and lift the cup, and (iii) place it upright on a saucer.

Pre-trained Base Policy. We start from the publicly released diffusion policy checkpoint in the Universal Manipulation Interface repository https://github.com/realstanford/universal_manipulation_interface. The base policy was trained on 1447 in-the-wild demonstrations of the cup arrangement task collected in diverse environments. The test environment is not included in the training data. At each timestep, the policy receives a single RGB observation plus the past two proprioceptive states and predicts a 16 -step action sequence; the first 12 actions are executed each control cycle.

Dynamics Model Training. To collect training data for the dynamics model, we roll out the pretrained policy from deliberately out-of-distribution initial poses, gathering 80 trajectories. Because intermediate policy checkpoints are unavailable, we augment the dataset with additional 40 human play trajectories recorded via the handheld UMI device. These play trajectories are intended to broaden state-action coverage rather than solve the task, for example, by sweeping the gripper

through random motions to capture wide-angle views. These 120 unlabeled trajectories, together with the original demonstrations, are used to train the dynamics model.

Evaluation Protocol. We evaluate two initial-pose regimes: (i) in-distribution initial poses and (ii) out-of-distribution initial poses . For each group we conduct 20 trials with both the base policy and LPB, using identical initial robot and object poses for fair comparison. During deployment we denoise actions with a DDIM scheduler (16 diffusion steps) and apply gradient guidance during the final five steps.

B.3.2 Belt Assembly

Setup. We use a 6-DoF UR5 robot arm with 3D-printed soft compliant fingers and a wrist-mounted OAK-D camera. Low-level controller is provided by https://github.com/yifanhou/hardware_interfaces.

Base Policy Training. We collect 200 expert demonstrations, each performed with slight variations in the board position while keeping the initial robot pose fixed. We train the base policy for 800 epochs. At each timestep, the policy receives a single RGB observation plus the past two proprioceptive states and predicts a 32 -step action sequence; the first 16 actions are executed each control cycle.

Dynamics Model Training. To train the dynamics model, we collect 400 rollout trajectories executed under randomly initialized board positions and initial robot poses. We use hyperparameters t 0 = 200 ∆ t = 200 , t final = 800 , and N = 100 . We train the dynamics model with the combined 600 trajectories.

Evaluation Protocol. We perform 40 rollouts for both the base policy and LPB, varying the initial robot pose and board position. All evaluations use the same initial robot and object poses for fair comparison. During deployment we denoise actions with a DDIM scheduler (16 diffusion steps) and apply gradient guidance during the final five steps.

C Broader Impact

Our work contributes to improving the robustness of visuomotor policies in robotic manipulation settings by introducing a test-time optimization framework that leverages learned dynamics and latent-space guidance. This has the potential to reduce failure rates in deployment, especially in out-of-distribution scenarios, which is critical for the safe and reliable operation of robots in humancentered environments. However, as with any data-driven system, care must be taken to ensure that failure modes are thoroughly evaluated before real-world deployment. Future extensions could investigate formal guarantees for test-time optimization mechanisms.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an approach to keep diffusion policies in distribution during execution. The core idea is to learn a local dynamics model by gathering data using policy rollouts from a trained policy (or checkpoints from the policy during training). The trained dynamics model is then used as a form of classifier guidance to keep the observed latents close to a set of latent states gathered from an expert's demonstration. This helps keep the policy close to the support of the expert training data, and improves robustness. Results show this reduces the amount of data required for robust model training. Experiments are conducted in simulation and on real robots.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe idea is simple, and results show that it is robust to noise, and policies require fewer expert demonstrations to be effective.\n\nI think it is a sensible approach, and like that it is modular and can be attached to a pre-trained policy. \n\nThis meets my internal criteria of would I send it to a PhD student.\n\nThe paper is well written, and experiments are nicely executed.\n\nWeaknesses:\n\nThe data collection process for dynamics means that only a small local model can be trained, this means that the approach may struggle in complex environments with greater environmental dynamics and for longer horizon tasks. This is discussed in the limitations.\n\nMany of the demonstration tasks show the approach successfully driving the robot to the expert data distribution in areas like free space motion, but I would have liked to see an example of this happening in a more complex interactive setting like contact, where classical planning techniques couldn't work. \n\nThis is nitpicking, but the distance used to determine whether something is out of distribution is an l2 distance between the nearest expert latent. Given that the policy being corrected is a probabilistic one, it seems that a less heuristic approach would be to directly determine where a latent lies in distribution or not, although this may require solving the flow equation.\n\nQUESTIONS:\nIn Fig 4c. How is the impact of rollout data computed and contribution to success rates? Eg. Where does the 0.4 number come from at 300 trajectories.\n\nI may have missed this, but what sort of coverage do you need for the expert latents? How big is the buffer, and do you do anything special to ensure they adequately cover the space of interest?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address the covariate shift problem that is prevalent in visuomotor policies trained by behavior cloning (BC). The authors propose a new framework called Latent Policy Barrier (LPB). The core idea is to regard the embedding distribution of expert demonstrations in the latent space as an implicit \"safe\" region, and inspired by the control barrier function (CBF), ensure that the agent does not deviate from this region during policy execution. A key design of this framework is to decouple the two core goals in imitation learning - accurate imitation and out-of-distribution (OOD) recovery. It uses a base policy (the diffusion model in this paper) trained only on high-quality expert data to ensure the accuracy of imitation, and uses a dynamics model trained on expert data and suboptimal policy rollout data to learn how to recover from deviations. In the inference stage, the dynamics model \"guides\" the actions of the base policy through online optimization so that its predicted future latent states remain within the manifold of the expert data distribution, thereby improving the robustness of the policy without sacrificing the quality of imitation. Experiments on simulation and real robots demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe core idea of ​​the paper cleverly generalizes and applies the concept of \"control barrier function\" (CBF) in control theory to data-driven imitation learning, and proposes to use \"latent space distribution of expert data\" as an implicit barrier, which is a novel and insightful analogy and framework.\n\nThe paper contains in-depth qualitative and quantitative analysis. For example, the analysis in Figure 5 associates the latent space OOD score with the actual task failure moment, providing strong intuitive evidence for the core mechanism of the LPB method. Its ability to improve the \"out-of-the-box\" pre-trained model is verified on a real robot, which enhances the credibility and practical value of the paper.\n\nWeaknesses:\n\nStrictly speaking, the various components of the algorithm (diffusion strategy, latent space dynamics model, gradient-based guidance) are all existing technologies. The originality of this work is mainly reflected in the combination of these components and the guiding ideas behind them (CBF analogy and decoupling principle).\n\nThe method introduces some hyperparameters that need to be adjusted, such as the threshold $\\tau of the latent space OOD score and the guidance strength \\eta$. The sensitivity analysis of these hyperparameters is not sufficient in the main text of the paper, which may affect its reproducibility in different tasks.\n\nThe guidance mechanism during inference (Formula 4) combines the ideas of diffusion model denoising and gradient optimization. It is conceptually dense and may require readers to have a certain understanding of the relevant background to fully digest it.\n\nQUESTIONS:\nOn the quality of the latent space: The effectiveness of the entire LPB framework relies on a high-quality latent space. Can you discuss under what circumstances the visual encoder learned by the BC strategy may not form a \"good\" latent space (i.e., a space in which distances do not effectively reflect the degree of OOD)? For example, how does the performance of LPB change in the presence of a large amount of visual distractors that are irrelevant to the task?\n\nOn the sensitivity of hyperparameters: The setting of the threshold $\\tau is crucial for the algorithm to trigger. How should this value be set in practice? Is it a sensitive parameter that needs to be fine-tuned for each task, or is it robust over a wide range? Can you provide more sensitivity analysis on \\tau and the guidance strength \\eta$?\n\nComparison with explicit recovery strategies: Your method can be viewed as an online, implicit recovery strategy. Another idea is to explicitly train a separate recovery strategy using the same rollout data. Can you fundamentally analyze the advantages and disadvantages of your proposed \"online guidance\" method compared with the \"offline training recovery strategy\" method and what are the applicable scenarios?\n\nLIMITATIONS:\nYes. The authors clearly discuss the limitations of the current work in the conclusion section, including that the current method mainly deals with local deviations, the dynamics model is task-specific, and the method requires access to expert demonstration data to define the inner manifold of the distribution. The discussion is sufficient and honest.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new approach to improve OOD recovery of the diffusion policy. The authors achieve this by learning a separate dynamics model and using classifier guidance to guide the generation process to in-distribution states to encourage recovery. The result is verified on well-designed experiments both in simulation and in the real world.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Presented a novel approach to recover from OOD situations in diffusion policies.\n2. Well-designed experiments that showcase the effectiveness of their work.\n3. Very effective improvements in the low (expert) data situations compared to previous works.\n4. Overall clearly written and presented paper.\n\nWeaknesses:\n\n1. Classifier-based guidance in diffusion policy has been widely used already.\n2. The proposed method claims to be Plug-And-Play. However, it still requires non-trivial data collection and training for the dynamics model.\n\nQUESTIONS:\n1. How does the method compare to using a two-stage framework? For example, how about using planners such as MPPI with the learned dynamics model to push back into the expert distribution first, then utilize the expert diffusion model to do inference? Is this the MPC baseline in your appendix?\n2. How much overhead (both memory for loading the expert dataset latents and computation for calling the learned dynamics model and calculating the gradient-based guidance) does the method add to the original diffusion policy inference pipeline?\n3. When expert data size is scaled up, it seems both the proposed approach and the base expert policy lead to similar performance. I am curious what is the trend of the correction rate (i.e., percentage of times that the guidance is activated) as one increase the amount of expert data.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Latent Policy Barrier (LPB), a novel framework for robust visuomotor policy learning via behavior cloning. LPB decouples precise expert imitation from out-of-distribution (OOD) recovery by combining: (1) a base diffusion policy trained exclusively on high-quality expert demonstrations to ensure task performance, and (2) a visual latent dynamics model trained on both expert and automatically collected rollout data to detect and correct deviations at inference time. At test time, LPB computes a latent OOD score-the $l_2$ distance between the current latent embedding and its nearest expert neighbor-and, if it exceeds a threshold, performs gradient-based steering in latent space to pull predicted future states back toward the expert manifold. Experiments on four simulated robotic manipulation tasks (Push-T, Square, Tool-Hang, Transport) and a real-world cup-arrangement task demonstrate that LPB achieves up to a 34% success-rate improvement over strong baselines under limited expert data and maintains robustness under action noise and significant distribution shifts.\n\nSTRENGTHS AND WEAKNESSES:\n# Quality\n\n**Strengths**: the empirical evaluation is thorough, covering both simulated and real-robot benchmarks. LPB consistently outperforms baselines-including Behavior Cloning variants, Filtered BC, CCIL, and CQL-especially in low-data regimes (20% demos) and under inference-time perturbations (up to 0.4 action noise).\n\n**Weaknesses**: the paper lacks theoretical guarantees on convergence or safety margins. Evaluation is limited to a narrow set of manipulation tasks; generalization to other domains (e.g., mobile navigation) remains untested.\n\n# Clarity\n\n**Strengths**: the conceptual analogy to Control Barrier Functions and the separation of training vs. inference components are well motivated. Figures (e.g., rollout comparisons in Figure 5) and Algorithm 1 clearly illustrate the LPB pipeline.\n\n**Weaknesses**: Key hyperparameters, such as the OOD threshold $\\tau$, guidance scale $\\eta$, and selection of nearest-neighbor search strategy, are deferred to the appendix, which may hinder immediate reproducibility. The presentation of computational overhead at inference (gradient-guided denoising) could be more detailed.\n\n# Significance\n\n**Strengths**: Covariate shift is a central challenge in imitation-based visuomotor control. LPB offers a plug-and-play approach that enhances robustness without additional human labeling or policy fine-tuning, potentially benefiting a broad class of pretrained vision-based policies.\n\n**Weaknesses**: While impactful within robotic manipulation, the approach's scalability to environments with very high-dimensional observation spaces or multi-agent settings is unclear.\n\n# Originality\n\n**Strengths**: LPB is the first to frame expert demonstrations as an implicit latent barrier and to perform gradient-based inference steering in a learned visual latent space for behavior cloning policies. This unifies ideas from diffusion-policy classifier guidance and model-predictive refinement in a novel way.\n\n**Weaknesses**: Related works on guided denoising and model-based policy refinement exist; the incremental novelty lies primarily in applying these to latent-space neighbor distances rather than explicit reward signals.\n\nQUESTIONS:\n1. How sensitive is LPB's performance to the choice of latent-OOD threshold $\\tau$? Can you provide guidance or automate its selection?\n\n2. What is the real-time computational cost of LPB's gradient-based steering during inference, and how might this scale with higher action dimensions or longer denoising chains?\n\n3. Storing all expert latent embeddings for nearest-neighbor search can be memory-intensive. Have you considered approximate methods (e.g., k-d trees or learned quantization) and their impact on robustness?\n\n4. LPB focuses on short-term, local corrections. How might the dynamics model be extended for long-horizon planning to recover from more severe drifts?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-14"}