Title: Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies
Venue: NeurIPS 2025
Accepted: yes
Keywords: Imitation Learning; Diffusion Policy; Vector–Jacobian Product; Variational Inference; Latent Stochastic Differential Equation; Contrastive Learning
Abstract: Existing imitation learning methods decouple perception and action, which overlooks the causal reciprocity between sensory representations and action execution that humans naturally leverage for adaptive behaviors. To bridge this gap, we introduce Action-Guided Diffusion Policy (DP-AG), a unified representation learning that explicitly models a dynamic interplay between perception and action through probabilistic latent dynamics. DP-AG encodes latent observations into a Gaussian posterior via variational inference and evolves them using an action-guided SDE, where the Vector–Jacobian Product (VJP) of the diffusion policy's noise predictions serves as a structured stochastic force driving latent updates. To promote bidirectional learning between perception and action, we introduce a cycle-consistent contrastive loss that organizes the gradient flow of the noise predictor into a coherent perception–action loop, enforcing mutually consistent transitions in both latent updates and action refinements. Theoretically, we derive a variational lower bound for the action-guided SDE, and prove that the contrastive objective enhances continuity in both latent and action trajectories. Empirically, DP-AG significantly outperforms state-of-the-art methods across simulation benchmarks and real-world UR5 manipulation tasks. As a result, our DP-AG offers a promising step toward bridging biological adaptability and artificial policy learning. Code is available on our project website: https://jingwang18.github.io/dp-ag.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes perception-action interplay in diffusion policy learning. Instead of conditioning on a static encoding of the observation, the framework will refine the observation in the latent space by using an action-guided SDE. A contrastive learning method is applied to the action denoising module to make sure the latent observation is consistently changing with actions.\n\nEvaluations in both the simulator and the real world show the effectiveness of this method in improving performance and smoothness in manipulation.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n1. The writing is clear and easy to follow.\n2. The theoretical proof and evaluation are thorough.\n\n# Weakness\n1. Some designs of the DP-AG do not seem to align with the motivation. Humans could leverage the observation-action-obsevation loop for adaptive behavior because new information might be discovered through interaction and changing viewing angle. However, this pipeline does not get any additional information in the interplay, instead, it changes the representation of observation through an SDE guided by action denoising. The reason why the representation from ground truth observation should be refined by a prediction needs further justification. Besides, the drift of the latent observation is constrained by contrastive learning. While this prevents the model from collapsing to a trivial solution (e.g., purely imagining the observations), it is unclear to me how to determine the balance of evolution versus keeping similar to the static observation latent (i.e., what is the expected amount and direction of drift that is beneficial for action prediction).\n2. The authors could consider comparing with UWM [1] fine-tuned on the selected tasks, which has a similar motivation of coupling perception and action.\n\n[1] Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets\n\nQUESTIONS:\n1. Could you please explain the motivation for refining the representation of real observations with guidance from predicted actions? \n2. It's unclear to me what drift of observation latent is expected to improve the action prediction. Could the authors consider replacing the variational posterior with a variational autoencoder so we can decode the evolved latent to see whether it has any interpretable semantic meaning?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Action-Guided Diffusion Policy (DP-AG), a novel imitation learning framework that addresses the limitation of static perception in existing methods. The core contribution is a dynamic perception-action loop where the policy's latent observation features evolve during the action generation process, guided by the Vector-Jacobian Product (VJP) of the diffusion model's noise predictions. A cycle-consistent contrastive loss reinforces this interplay, and the method demonstrates significant performance gains over state-of-the-art baselines on a variety of simulation and real-world robotic manipulation tasks\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The core idea of creating a dynamic perception-action loop within a single action generation step is highly novel and significant. It directly addresses the key limitation of static perception in many IL policies.\n2. The empirical validation is extensive, showing strong results on standard benchmarks and introducing a new dynamic task (Dynamic Push-T) that effectively showcases the method's advantages. The model's success in the difficult Peg-in-Hole task (85% vs. 0% for the baseline) provides compelling evidence for the practical benefits of the proposed perception-action interplay.\n3. The method is supported by solid theoretical grounding, including a variational lower bound and a proof connecting the contrastive loss to trajectory continuity, which adds rigor to the paper.\n4. The paper is well-written and clearly presented. The figures effectively illustrate the core concepts.\n\nWeakness\n1. Several simulation benchmarks in Table 2 are near-saturated, with both DP-AG and baselines achieving perfect scores. While this demonstrates non-regression, these tasks don't fully differentiate SOTA methods. The paper's impact is more clearly shown in the more challenging dynamic and real-world settings.\n2. The paper lacks a deep semantic intuition for why the VJP is the ideal guiding force for latent updates. Explaining what this signal represents (e.g., a direction of uncertainty reduction) would improve clarity for readers not versed in stochastic adjoint methods.\n3. The work could be better contextualized against model-based RL methods that also use latent dynamics (e.g., Dreamer). A clearer distinction between DP-AG's intra-policy state refinement and a world model's future state prediction would sharpen the paper's claimed contribution.\n\nQUESTIONS:\n1. Can you provide more semantic intuition for the VJP update term? What does this structured force represent in the context of perception and action, beyond its mathematical formulation?\n2. Regarding the Peg-in-Hole task, what was the primary failure mode for the baseline DP (e.g., incorrect depth estimation, inability for fine-grained correction)?\n3. Why is the relative-similarity-based contrastive loss superior to the absolute-target-based MSE likelihood objective for this task?\n\nLIMITATIONS:\nYes, the authors have adequately addressed the limitations. The Broader Impacts section discusses the reliance on expert demonstrations, which could introduce biases, and correctly points to the need for validating demonstration data. This is a sufficient acknowledgment of the primary limitations of the imitation learning paradigm.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Action-Guided Diffusion Policy (DP-AG), a new imitation learning framework that models a dynamic interplay between perception and action. The encoded observations are evolved with an action-guided SDE, and then construct a cycle-consistent contrastive loss to create a perception-action loop.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n- DP-AG creates a perception-action loop that allows the gradient of the actions to flow back to the encoded observations.\n- DP-AG outperforms baselines in simulated and real-world benchmarks.\n\nCons:\n- The VJP can be very expensive to compute, especially with a high-dimensional action space or encoded observation space.\n- Lack of imitation baselines: DP-AG should include classic Diffusion imitation learning baselines like Diffusion Policy, Diffuser, and Hierarchical Diffuser.\n\n\nVisuomotor Policy Learning via Action Diffusion\n\nPlanning with Diffusion for Flexible Behavior Synthesis\n\nSimple hierarchical planning with diffusion\n\nQUESTIONS:\nI am still confused by the motivation of DP-AG:\n- \"Existing imitation learning methods decouple perception and action\": common IL methods optimize the perception (representation) in an end-to-end manner, where the gradient of the perception comes from the downstream action prediction objective. What do you mean by *decoupled*?\n- Line 130: \"However, it still treats perception as static: once extracted, observation features remain fixed during the corresponding action generation and cannot adapt to refinements\": during the diffusion process $\\epsilon_\\theta\\left(\\hat{a}_t^k, z_t, k\\right)$, why would $z_t$ need to evolve w.r.t. $\\hat{a}_t^k$ as $z_t$ only needs to serve as a compact representation of $o_t$?\n- In Eq. (6), I notice that DP-AG implements the drift coefficient with VJP (or JVP, if more precisely). What is the advantage of such implementation? Also, I notice that DP-AG does not require using the reverse SDE of Eq. (6), then what is the purpose of proposing the forward SDE?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a diffusion policy model where the input conditional observation feature is updated along the action denoising process. Specifically, the method leverages VJP, the Vector Jacobian Product from the action score prediction, to create a dynamic process to update the observation. It was claimed that such an update to the observation feature would result in improved smoothness in action prediction. To ensure the updated feature is not deviating from the original feature, a consistency loss is applied so that the score prediction based on the updated observation and the original observation is consistent. The paper conducts a set of experiments comparing diffusion policy vs action-guided diffusion policy. Experiment results show the benefit of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- The presented experiment results show the benefit of the proposed algorithm over the baseline.\n- The verification is done both using synthetic data and real robots.\n- The proposed approach is clean and can be applied to the existing diffusion policy model.\n\nWeakness\n- It is unclear why an updated observation would lead to better action prediction. Given the observation has been made, the update did not add new information. What has been updated is the observed feature, we are not updating the observation.\n- It is unclear why the proposed algorithm works. I feel the paper would benefit from a clearer explanation of the intuition.\n\nQUESTIONS:\nQ1: Why would the updated observation feature (not updated observation) lead to a better policy model? All the info is already in the observation.\n\nQ2: What is the effect of consistency loss? To me, it seems that the goal is to make Vector Jacobian Product of the action score prediction the \"null space\" of the score estimator. Why?\n\nQ3: Could the achieved stability be achieved with other smoothness techniques? For example, we can simply perturb the input observation and request the score prediction to be consistent.\n\nQ4: The phrase “Act to See, See to Act” seems to be misleading. The environment has been seen. What has been updated is the observed features instead of the observation. There is no new see during the action generation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-05"}

comments: SUMMARY:
DP-AG establishes a dynamic perception-action loop by guiding feature evolution via the VJP of DP’s predicted noise. To reinforce interplay, a cycle-consistent contrastive loss aligns noise predictions from static and evolving features, enabling mutual perception–action influence. By using such method, they successfully transforms diffusion noise gradients into structured perceptual updates.

STRENGTHS AND WEAKNESSES:
Strength: The paper is well-written, and the idea is interesting to me. The wide range of verification scenarios convinces readers of the effectiveness of the proposed method. The experimental results are good with clear and diverse visualization.

Weaknesses: N/A

QUESTIONS:
Overall, I think the writing logic of this article is clear and the theory is reliable. What is particularly commendable is that it includes tests of simulation tasks and real-world robot control tasks. Two curious points are: 

1) Recently, many works related to diffusion model strategies have begun to focus on how to simultaneously accommodate online training. Could the author please elaborate further from a theoretical or experimental perspective on whether your method has the potential in the online mode? 

2) Multimodal decision-making ability - for the same task, can your method fit a variety of effective strategies? This is an important ability in real robot control scenarios and it was discovered that diffusion model strategies have this potential in some recent papers.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05