One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation
Authors: Zhendong Wang
Max Li
Ajay Mandlekar
Zhenjia Xu
Jiaojiao Fan
Yashraj Narang
Linxi Fan
Yuke Zhu
Yogesh Balaji
Mingyuan Zhou
Ming-Yu Liu
Yu Zeng
Venue: ICML 2025 poster (2025)
Keywords: diffusion policies; diffusion distillation; one-step diffusion policy
Primary area: applications->robotics
TL;DR: We introduce One-Step Diffusion Policy (OneDP), a distilled diffusion-based policy that accelerates robotic action generation 40 times..

Abstract: Diffusion models, praised for their success in generative tasks, are increasingly being applied to robotics, demonstrating exceptional performance in behavior cloning. However, their slow generation process stemming from iterative denoising steps poses a challenge for real-time applications in resource-constrained robotics setups and dynamically changing environments.
In this paper, we introduce the One-Step Diffusion Policy (OneDP), a novel approach that distills knowledge from pre-trained diffusion policies into a single-step action generator, significantly accelerating response times for robotic control tasks. We ensure the distilled generator closely aligns with the original policy distribution by minimizing the Kullback-Leibler (KL) divergence along the diffusion chain, requiring only $2\%$-$10\%$ additional pre-training cost for convergence. We evaluated OneDP on 6 challenging simulation tasks as well as 4 self-designed real-world tasks using the Franka robot. The results demonstrate that OneDP not only achieves state-of-the-art success rates but also delivers an order-of-magnitude improvement in inference speed, boosting action prediction frequency from 1.5 Hz to 62 Hz, establishing its potential for dynamic and computationally constrained robotic applications. A video demo is provided at our project page, and the code will be publicly available.

One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation
Zhendong Wang' Zhaoshuo Li Ajay Mandlekar Zhenjia Xu²
Jiaojiao Fan?
Yashraj Narang Yu Zeng?
Linxi Fan"
Yuke Zhu 2 Yogesh Balaji? Mingyuan Zhou?
Ming-Yu Liu
Abstract
text-to-imaaee generation (Saharia et al. 2022 Ramesh
et al.
2022
Rombach et al.
2022). video generation (Ho
Diffusion models, praised for their success in gen- erative tasks, are increasingly being applied robotics, demonstrating exceptional performance in behavior cloning. However, their slow gen- eration process stemming from iterative denois- ing steps poses a challenge for real-time applica- tions in resource-constrainee robotics setups and dynamically changing environments, In this pa- per. we introduce the One-Step Diffusion Policy (OneDP), novel approach that distills knowl- edge from pre-trained diffusion policies into a single-step action generator, significantly accel- erating response times for robotic control tasks. We ensure the distilled generator closely aligns with the original policy distribution by minimiz- ing the Kullback-Leibler (KL) divergence along the diffusion chain, requiring only 2%-10% addi- tional pre-trate pre-training cost for convergence. We eval- uated OneDP on 6 challenging simulation tasks as well as 4 self-designed real-world tasks using the Franka robot. The results demonstrate that OneDP not only achieves state-of-the the-arr- success rates but also delivers an order-of-man ma-giniudd im- provement in inference speed, boosting action pre- diction frequency from 1.5 Hz to 62 Hz, establish- ing its potential for dynamic and computationally constrained robotic applications. A video demo is provided (here, and the code will be publicly available soon.
et al.
2022
OpenAI, 2024). and online/offline reinforce-
ment learning (RI.) (Wang et al. 2022 Chen et al. 2023a
Hansen- Caassen-Estrucheee et al. 2023 Psenka et al. 2023 Chen ct al.
2024). Recently, Chi et al. (0233) ] Team al.
(2024) Reuss
al. 2023); ZZ et al. (2024); Ke al. (2024) ) Prasad et al.
2024 demonstrated impressive results of diffusion models
in imitation learning for robot control. In particular, Chi
ct al. (2023) introduces the diffusion policy and achieves a state-of-the-arr imitation learning performance on a variety of robotics simulation and real-world tasks.
However, because of the necessity of traversing the reverse diffusion chain, the slow generation process of diffusion models presents significant limitations for their application in robotic tasks. This process involves multiple iterations to pass through the same denoising network, potentially thousands of times (Song et al. 2020a Wang et al. (023). Such a long inference time restricts the practicality of using the diffusion policy (Chi et al. 2023), which by default runs at 1.49 Hz, in scenarios where quick response and low com- putational demands are essential. While classical tasks like block stacking or part assembly may accommodate slower inference rates, more dynamic activities involving human interference or changing environments require quicker con- trol responses (Prasad et al. 2024). In this paper, we aim to significantly reduce inference time through diffusion distil- lation and achieve responsive robot control.
Considerable research has focused streamlining the re- verse diffusion process for image generation, aiming to complete the task in fewer steps. A prominent approach in- terprets diffusion models using stochastic differential equa- tions (SDE) or ordinary diffcrential equations (ODE) and employs advanced numerical solvers for SDE/ODE to speed up the process (Song et al. 2020a Liu et al.. 2022 Karras et al.) 2022 Lu al. 2022 Another avenue explores dis- tilling diffusion models into generators that require only one or a few steps through Kullback-Leibbe (KL) optimization or adversarial training (Salimans & Ho 2022 Song et al.)
1. Introduction
Diffusion models (Sohl-Dickstein et al.) 2015 Ho et al.
2020) have emerged as leading approach generative AI, achieving remarkable success in diverse applications such
'The University of Texas at Austin 'NVIDIA. Correspondence to: Zheudong Wang (Work done during an internship NVIDIA) <zhendong, wang @ edus, Yu Zeng <yuzeng noiidi..com >
2023 Luo et al., 2024) Yin et al.] (004)). However, acceler- ating diffusion policies for robotic control has been largely underexplored. Consistency Policy (Prasad al.) 2024 (CP) employs the consistency trajectory model (CTM) (Kim
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

One-Step Diffusion Policy
Iffusion Policy
Perturt
Wait
Siowly move
Hositato
Retry gras
Porturb
Falurel
OneDe
Perturb
Quicky react
Ratry
Pomturt
Quickly react
Retry
Sucoosoo
Policy Frequency
Success Rate (1)
Completion Time
Training Cost (4)
Real Seprimart
Sirulstion Issetmeet
Diper
0.98
40
Seal Expermment
1200
Simulation Exputtment
Desertrent
35-
0.829
Success Rate
25
Epochs
B00
30
600
0.4
15-
10-
0.2
200
100
1.49
Diffusion Policy
OneDP 0.0 Diffusion Palicy
OreDP
Diffusion Policy
One
Diffusion Policy
One
Figure 1: Comparison of Diffusion Policy and One-Step Diffusion Policy (OneDP). We demonstrate the rapid response of OneDP to changes in dynamic environments through real-world experiments. The first row illustrates how Diffusion Policy (Chi et al.) 2023) struggles to adapt to environment changes (here, object perturbation) and fails to complete the task due to its slow inference speed. In contrast, the second row highlights OneDP's quick and effective response. The third row offers a quantitative comparison: in the first panel, OneDP executes action prediction much faster than Diffusion Policy. This enhanced responsiveness results in a higher average success rate across multiple tasks, particularly in wellwwrldd scenarios, as depicted in the second panel. The third panel reveals that OneDP also completes tasks more swiftly. The final panel indicates that distillation of OneDP requires only small fraction of the pre-training cost.
et al.. 2023a) to adapt the pre-trained diffusion policy into a few-step CTM action generator. Despite this, several it- erations for sampling are still required to maintain good empirical performance.
forms CP with a higher success rate across tasks, leverag- ing single-step action generator and achieving x0 faster convergence. A detailed comparison with this approach is provided in Appendix A and Section
In this paper, we introduce the One-Step Diffusion Policy (OneDP), which distills knowledge from pre-trained diffu- sion policies into a one-step diffusion-based action genera- tor, thus maximizing inference efficiency through a single neural network feedforward operation. We demonstrate su- perior results over baselines in Figure [ Inspired by the suc- cess of SDS (Poole et al. 2022) and VSD (Wang et al. 2024) in text-to-3D generation, we propose policy-matching dis- tillation method for robotic control. The training of OneDP consists of three key components: one-step action genera- tor, generator score network, and a pre-trained diffusion- policy score network. To align the generator distribution with the pre-trained policy distribution, we minimize the KL divergence over diffused actions produced by the generator, with the gradient of the KL expressed as a score difference loss. By initializing the action generator and the genera- tor score network with the identical pre-trained model, our method not only preserves or enhances the performance of the original model, but also requires only 2%.10% addi- tional pre- tre-training cost for the distillation to converge. We compare our method with CP and demonstrate that it outper-
We evaluate our method in both simulated and real-world environments. In simulated experiments, test OneDP on the six most challenging tasks of the Robomimic bench- mark (Mandlekar et al.) 202T). For real-world experiments, we design four tasks with increasing difficulty and deploy OneDP on a Franka robot arm. both settings, OneDP demonstrated state-of-the-art success rates with single-step generation, performing 42x faster in inference.
2. One-Step Diffusion Policy
2.1. Preliminaries
Diffussinn models are powerful generative models applied across various domains (Ho et al. 2020 (Soh-DDickstein et al. 2015 Song al., 20206). They function by defining forward diffusion process that gradually corrupts the data distribution into known noise distribution. Given data distribution p(æ), the forward process adds Gaussian noise to samples, x0 ~ p(æ), with each step defined at ax +o where e ~ NN0,, I). The parameters og and
2

---

One-Step Diffusion Policy
o are manually designed and vary according to different noise scheduling strategies.
We define the action distribution generated by Gg as PG. Assuming the existence of pre-trained diffusion policy (A(A defined by Equation (2) and parameterized by Eg, its corresponding action distribution is denoted as Pre Drawing inspiration from the success of SDS (Poole et al.) 2022) and VSD (Wang et al., 2024) in text-to-3D applica- tions, we propose using the following reverse KL divergence to align the distributions PG, and pr
A probabilistic model Po (x=""((=") is trained to reverse this diffusion process, enabling data generation from pure noise. DDPM (Ho et al.
2020) uses discrete-time scheduling with
a noise-prediction model eg to parameterize ps, while EDM 2022) employs continuous-time diffusion with (Karras al.
x" -"-predicioon. We use epsilon prediction Eg in our deriva- tion. The diffusion model is trained using the denoising score matching loss (Ho et al. 2020) Song et al.) 2020b). Once trained, we can estimate the unknown score s (x*)= at a diffused sample x²* as:
D (r..la.llP... = N1o. [og pc. - log Pr, (Ao]O)) Au=Ga(s,0)
It is generally intractable to estimate this loss by directly computing the probability densities, since PG. is an implicit distribution and Pr, involves integrals that are impracti- cal (Equation (2). However, we only need the gradient VoDKL (PG, lIPx.) with respect to to train our generator by gradient descent, with can be computed as follows:
c² (k) ca(a* k) ok
s(x*) =
(1)
ok
where e" e"(x*,, k) is the true noise added at time k and we de- note so (x²h)= = so(a=".A), oN With score estimate, clean data x." can be sampled by reversing the diffusion chain (Song iet al. 20206). This requires multiple iterations through the estimated score network, making it inherently slow.
E s.o.). Ca
[VV (ogPc.(A.)0) VA, log (A)]O))
(4)
Here Spag (Ao) = VA, PG, (Ao () and 8p-g (A.) V A, log Pro OA) the scores of the pG, and Prg re- spectively. Computing this gradient still presents two sig- nificant challenges: First, the scores tend to diverge for samples from Pc, that have a low probability in Pa.. espe- cially when px. may approach zero. Second, the primary tool for estimating these scores, the diffusion models, only provides scores for the diffused distribution.
Wang et al. (022) ; Chi et al. (2023) extend diffusion mod-
els as expressive and powerful policies for offline RL and robotics. In robotics, set of past observation images, O, is used as input to the policy. An action chunk, A, which consists of sequence of consecutive actions, forms the output of the policy. Diffusion policy is represented as a conditional diffusion-based action prediction model,
Inspired by Diffusion-GAN (Wang et al. 2023). which proposed to optimize statistical divergence, such as the Jensen-Shannon divergence (JSD), throughout diffused data samples, we propose to similarly optimize the KI. diver- gence outlined in Equation (4) across diffused action sam- ples using its gradient below,
To AA = / K; 0, IT PoAk O)AA K
A
(2)
The explicit form of To (A) is often impractical due
to the complexity of integrating actions from AK to A!.
However, we can obtain action chunk samples from it by iterative denoising. More details are provided in Appendix B
V VoRaril (D
K... (PG [[Tg
2.2. One-Step Diffusion Policy
E x0
w [kk)(Sppa, (Ab) Spxs (A
Ao=Go(z.0)
Action sampling through the vanilla diffusion policies is notoriously slow due to the need of tens to hundreds of iterative inference steps. The latency issue is critical for computationally sensitive robotic tasks or tasks that require high control frequency. Although employing advanced ODE solvers (Song et al.] 2020a; Karras et al. (022) could help speed up the sampling procedure, empirically at least ten it- erative steps are required to ensure reasonable performance. Here, we introduce a training- cuining-based diffusion policy distil- lation method, which distills the knowledge of a pre-trained diffusion policy into single-step action generator, enabling fast action sampling.
A A As.k)
(5)
where wik) is reweighting function, g is the forward dif- fusion process and sp., (Aa) could be obtained through Equation (1) with cs- In order estimate the score of the generator distribution, Spa," : we introduce auxiliary dif- fusion network (A (O). parameterized by Go- We follow the typical way of training diffusion policies, which opti- mizes vé by treating pG, as the target action distribution Wang et al. 2024).
min V
xe "((a" (oo)).
(A(k) . (1e,(ar k) == (6)
=top Go
We propose one-step implicit action generator Go. from which actions can be easily obtained as follows,
(0,T)
Then we can obtain Spx, (A by applying 6o to Equa- tion
z N( Ag = Go(z. O).
We approximate Spa, (A)) in Equation 5 with
3

---

One-Step Diffusion Policy
Viston
Aj
Ak4 Ag
42
Action Generator Gy
KL Distillation
AB
Forward diflusion
Scores of Pretrained Diftusion Palicy
Scores of OneDP
A
4
Ab
Pre-trained Score Network 7
A
Network.g
Action Prediction
(b) Distillation
Figure 2: Diffusion Distillation Pipeline. a) one-step action generator processes image-based visual observations alongside random noise input to deliver single-step action predictions. b) We implement KL-based distillation across the entire forward diffusion chain. Direct computation of the KL divergence is often impractical; however, we effectively utilize the gradient of the KL, formulated into score-diffen dife-diffeeence loss. The pre-trained score network Ty remains fixed while the action generator Go and the generator score network अ are trained.
Spx, (At). We iteratively update the generator parameters by Equation (5). and the generator score network param- eter by Equation (6. The parameter of the prertrained diffusion policy @ is fixed throughout the training. During inference, we directly perform one-step sampling with Equa- tion 33. We name our algorithm OneDP-S, where S denotes the stochastic policy.
it may exhibit slightly weaker empirical performance. We offer comprehensive comparison between OneDP-S and OneDP-D in Section
Distillation Discussion. We discuss the benefits of optimiz- ing the expectational reverse KL divergence. First, reverse KL divergence typically induces mode-seeking behavior, which has been shown to improve empirical performance in offline RL (Chen et al.) 2023a). Therefore, anticipate that reverse KL-based distillation offers similar advantages for robotic tasks. Second, as demonstrated by Wang et al. (2023), optimizing JSD, a combination of KLs, between dif- fused action samples provides stronger performance when dealing with distributions with misaligned supports. This aligns with our approach performing KL optimization over the diffused distribution.
When we apply deterministic action generator by omitting random noise z, such that A (O). the distribution PG. becomes Dirae delta function centered GO)),, that is. PG, (A)) Consequently, Spa, (A can be explicitly solved as follows:
V A log (A (A
log A Ao)
Tk
where A A. TREP EN (O.
By incorporating Equation 7 into Equation ( we can
Algorithm OneDP Training
have simplified loss function without the need to introduce the generator network:
Inputs: action generator Go, generator score network Tw. pre-trained diffusion policy nT.
VoRRaul [D (PG.. IIPT,,))]
Initializaiton Go TT. म.. Tg.
3: while not converged do
((:))
(E,(AA)))) ex. VoAA
(8)
4: Sample Ag = Go O), ~~ ~
k.0. (z,00(
5: Diffuse A ag Ap + ONER, En ~ (N
A Ag
OneDP-S then
name this deterministic diffusion policy distillation
Update v by Equation (6)
OneDP-D. We illutrate our training pipeline in Figure 2 and summarize our algorithm training in Algorithm
Update 0 by Equation
else if OneDP-D then
10: Update 0 by Equation
Policy Discussion. A stochastic policy, which encompasses deterministic policies, is more versatile and better suited to scenarios requiring exploration, potentially leading to better convergence at a global optimum (Haarnoja et al. 2018). In our case, OneDP-D simplifies the training process, though
11: end if
12: end while

---

One-Step Diffusion Policy
2.3. Implementation Details
4. Experiments
Diffusion Policy. Following Chi et al. (023). 3, we construct diffusion policy using a ID temporal convolutional neural network (CNN) (Janner et al., 2022 based U-Net and a stan- dard 8 (without traininn) (He et al. (016) as the vision encoder. We implement the diffusion policy with two noise scheduling methods: DDPM (Ho et al.; 2020)
We evaluate OneDP on a wide variety of tasks in both sim- ulated and real environments, In the following sections, we first report the evaluation results in simulation across six tasks that include different complexity levels. Then we demonstrate the results in the real environment by deploy- ing OneDP in the real world with a Franka robot arm for object pick-and-place tasks and a coffee-machine manipu- lation task. We compare our method with the pre-trained backbone Diffusion Policy (Chi et al. 2023) (DP) and re- lated distillation baseline Consistency Policy (Prasad et al.) 2024 (CP). We also report the ablation study results in Ap-
and EDM (arrras et al., 2022). We use noise prediction for discrete-time (100 steps) diffusion and x² prediction for continuous-time diffusion, respectively. The EDM schedul- ing essential for Consistency Policy (Prasad et al., 2024) due to the use of CTM (Kim et al.) (023a). For DDPM, we set A(k) = 1 and use the original SDE and DDIM (Song et al. 2020a) sampling. For EDM, we use the default (a),aa²² with dg = 0.5. We use the second-order EDM sampler, which requires two neural network forwards per discretized step in the ODE.
pendix B to present more detailed analyses on our method and discuss the effect of different design choices.
X(k) C
4.1. Simulation Experiments
Datasets. Robomimic. Proposed in (Mandlekar et al. (021). Robomimic is large-scale benchmark for robotic manipu- lation tasks. The original benchmark consists of five tasks: Lift, Can, Square. Transport, and Tool Hang. We find that the the performance of state-of-the-ard methods was already saturated on two easy tasks Lift and Can, and therefore only conduct the evaluation on the harder tasks Square, Transport and Tool Hang. For each of these tasks, the bench- mark provides two variants of human demonstrations: profi- cient human (PH) demonstrations and mixed proficient/non- proficient human (MH) demonstrations. PushT. Adapted from IBC (Florence et al. 2022), Chi et al. (2023) intro- duced the PushT task, which involves pushing a T-shaped block into a fixed target using a circular end-effector. A dataset of 200 expert demonstrations is provided with RGB image observations.
Distillation. We warm-start both the stochastic and de- terministic action generator Gg, and the generator score network, cg. by duplicating the neural-network structure and weights from the pre-trained diffusion policy, aligning with strategies from Luo et al. (2024): Yin et al. (2024); Xuu et al. (2024). The inputs of Gg include pure noise, a fixed time embedding (an initial timestep for DDPM or initial sigma value for EDM), and observations O. The outputs of Gg are formulated as direct action predictions. Following DreamFusion (Poole et al., 2022), we set w(k) = oß. In the discrete-time domain, distillation occurs over [2, 95] diffusion timesteps avoid edge cases. In continuous-time, we employ the same log-normal noise scheduling as EDM (Karras et al. 2022 used during distillation. The generators operate at a learning rate of 1 x 10-6, while the generator score network is accelerated to a learning rate of 2 x 10-5, Vision encoders are also actively trained during the distilla- tion process.
Experiment Setup. We pretrain the DP model for 1000 epochs on each benchmark under both DDPM (Ho et al., 2020) and EDM (Karras et al. 2022) noise scheduling. Note EDM noise scheduling is a requirement for CP (Prasad) al. 2024) to satisfy diffusion boundary conditions. Subse- quently, we train OneDP for 20 epochs and the baseline CP for 450 epochs until convergence. During evaluation, we observe significant variance in evaluating success rates with different environment initializations. We present average success rates across 5 training seeds and 100 different initial conditions (500 in total). We report the peak success rate for each method during training, corresponding to the peak points of the curves in Figure The metric for most tasks is the success rate, except for PushT, which is evaluated using the coverage of the target area.
3. Related Work
Diffusion models have been explored as policy representa- tions for control tasks, demonstrating strong performance in offline RL. and imitation learning (Janner et al. 2022 Wang
ctt al. 2022 Chi et al., [023] Ze et al. 2024). However, their iterative denoising process poses challenges for real-time applications. To address this, diffusion distillation methods
(Salimans & Ho 2022 (Song et al.) 2023 Kim et al. 2023b
Luo et al.] 2024 Yin et al. 2024 Zhou et al. (024) aim
to acceleraaee sampling by training faster student model.
Consistency Policy (Prasad et al. 2024) applies distillation to diffusion policies but suffers from performance degra- dation and slow convergence. We build on these ideas by employing reverse KL optimization distill one-step ac- tion generator, achieving both efficiency and high success rates. See Appendix Afror detailed discussion.
Table [ presents the results of OneDP compared with DP under the default DDPM setting. For DP, we report the aver- age success rate using DDPM sampling with 100 timesteps, as well the accelerated DDIM sampling with 1 and 10 timesteps. Notably, DP fails to generate reasonable actions
5

---

One-Step Diffusion Policy
PushT
Square
ToolHang
Transport
Figure 3: Simulation tasks. We evaluate our method against baselines on the singlc-robot tasks; Push'T, Square, and ToolHang, well dual-robot task Transport. Task difficulty increases from left right.
Table 1: Rohomimic Benchmark Performance (Visual Policy) in DDPM. We compare our proposed OneDP-D and OneDP-S, with DP under the default DDPM scheduling. We report the mean and standard deviation of success rates across different training runs, each evaluated with 100 distinct environment initializations. Details of the evaluation procodure can be found in Section 4.1 Our results demonstrate that One not only matches but can even outperform the pre-trained DP, achieving this with just one-step generation, resulting in an order of magnitude speed-up.
Method
Epochs
NFE
PushT
Square-mh
Square-ph
ToolHang-ph
Transport-mh
Transport-ph
Avg
DP (DDPM)
1000
100
0.863 + 0.040
0.846 0.023
0.926 0.023
0.822 0.016
0.620 + 0.049
0.896 0.032
0.829
1000
10
0.823 0.023
0.850+
0.013
0.918+ 0.009
0.828+ 0.016
0.688 0.020
0.908 0.011
0.836
DP (DDIM)
1000
1
0.0001: 0.000
0.0001:
0.000
0.0001 0.000
0.000: 0.000
0.000 0.000
0.000: 0.000
0,000
OneDP-D
0.802 + 0.057
0.846
0.028
0.926
0.011
0.808 + 0.046
0.676 + 0.029
0.896 0.013
0.826
OneDP-S
20
1
0.816 + 0.058
0.864 0.042
0.926 0.018
0,850 = 0.033
0.690 + 0.024
0,914 0.021
0.843
Table 2: Robomimic Benchmark Performance (Visual Policy) in EDM. We compare our proposed OneDP with CP under the EDM scheduling. EDM scheduling is required in CP to satisty boundary conditions. We follow our evaluation metric and report similar values as [ We also ablate Diffusion Policy with 1. and 18 ODE steps, which utilizes 1, 19 and 35 NFE in EDM sampling.
Method
Epochs
NFE
PushT
Square-mh
Square-ph
ToolHang-ph
Transport-mh
Transport-ph
Avg
1000
35
0.861: 0.030
0.810:: 0.026
0.898:1 0.033
0.828 0.019
0.684 0.019
0.890: 0.012
0.829
DP (EDM)
1000
19
0.851
0.012
0.828 0.015
0.8801: 0.014
0.7944 0.012
0.692 0.009
0.860: 0.013
0.818
1000
0.000 0.000
0.000:: 0.000
0.000 0.000
0.000 0.000
0.000= 0.000
0.000:: 0.000
0,000
CP
20
0.595 0.141
0.120:: 0.165
0.2384 0.219
0,238+
0.163
0,140= 0.148
0.174 0.257
0.251
CP
450
1
0.828 0.055
0.646:: 0.047
0.7764 0.055
0,650:
0.046
0.378= 0.091
0.754: 0.120
0.672
CP
450
3
0.839 0.037
0.710# 0.018
0.874 0.022
0.626
0.041
0.374= 0.051
0.848: 0.028
0.712
OneDP-D
20
0.829 0.052
0.776.
0.023
0,902:: 0.040
0.762+
0.056
0.705 0.038
0.89811 0.019
0.812
OneDP-S
20
0,841 0.042
0.774 0.033
0.910 0.041
0,824 0.039
0.722 0.025
0.910 0.027
0.830
with single-step generation, yielding 0% success rate for all tasks. DP with 10 steps under DDIM slightly outper- forms DP under DDPM. However, OneDP demonstrates the highest average success rate with single-step generation across the six tasks, with the stochastic variant OneDP-S surpassing the deterministic OneDP-D. This superior perfor- mance of OneDP-S aligns with our discussion in Section 2.2 suggesting that stochastic policies generally perform bet- ter in complex environments. Interestingly, OneDP-S even slightly outperforms the pre-trained DP, which is not un- precedented, as shown in cases of image distillation (Zhou et al. 2024) and offline RL (Chen et al. 2023a). We attribute this to the fact that iterative sampling may introduce subtle cumulative errors during the denoising process, whereas single-step sampling avoids this issue by jumping directly from the end to the start of the reverse diffusion chain.
to 1, 19, and 35 number of function evaluations (NFE) in EDM due to second-order ODE sampling. OneDP-S outperforms the baseline CP with single-step and its default best setting of 3-step chain generation. Under EDM, OneDP- S matches the average success rate of the pre-trained DP, while OneDP-D performs slightly worse. We also observe that CP converges much more slowly compared OneDP, as shown in Figure 4 This slower convergence is likely because CP, based on CTM, does not involve the auxiliary discriminator training that is used to enhance distillation performance in CTM.
4.2. Real World Experiments
We design four tasks to evaluate the real-world performance of OneDP, including three common tasks where the robot picks and places objects at designated locations, referred to as pnp, and one challenging task where the robot learns to manipulate a coffee machine, called cof fee. Figure shows the experimental setup, with the first row illustrating
In Table 2 we report a similar comparison under the EDM setting, including CP. We report DP under the same I and 10 DDIM steps, and 100 DDPM steps, which correspond
6

---

One-Step Diffusion Policy
6:75
017
0.25
300
300
223 Epoch
300
306
Epecn
Ex0ch
boiltang-ghh
100
Transport-mh
10
Iransportggh
M1
00
00
04
0.50
425
12
35
20
33
30
30
15
EF
183
253
300
400
325
253
302
400
200
250
366
403
Figure 4: Convergence Comparison. We show our method OneDP converges 20x faster than the baseline method Consistency Policy (CP) under EDM setting,
the pnp tasks and the second row depicting the ca of fee task. We introduce the data collection process and the evalu- ation setup in the following section and provide more details in Appendix C
DDIM (10 steps) to accelerate the real-world experiment.
We compare OneDP against the DP backbone in real- world experiments, focusing on three key aspects: success rate, responsiveness, and time efficiency. Table 3 demon- strates that OneDP consistently outperforms DP across all tasks, with the most significant improvement seen in pnp milk move. This task demands rapid adaptation to dynamic environmental changes, particularly due to sudden human interference. The wall-clock time for action genera- tion is reported in Table The slow action generation of DP hinders its ability to track the moving milk box effectively, often losing control when the box moves out of its visual range, as it is still predicting actions based on outdated in- formation. In contrast, OneDP generates actions quickly, allowing it to instantly follow the box's 's movement, achiev- ing a 100% success rate in this dynamic task. OneDP-S slightly outperforms OneDP-D, aligning with the observa- tions from the simulation experiments.
pnp Tasks. This task requires the robot to pick an ob- ject from the table and put it in box. We design three variants of this task: рпр pnp-milk pnp-au aap-anything and pnp-mi pno-miik-moon move.. In pap-milk, the object is always the same milk box. In pnp-anything
we expand the target to 11 different objects as shown in Figure g For pnp-mi millllk-oeee we involve human interference to cre- a dynamic environment. Whenever the robot gripper attempts grasp the milk box, we move it away, following the trajectory shown in Figure 9 We collect 100 demon- strations each for the pпp-mi k and pmp-anythi ng tasks. Separate models are trained for both tasks, with the pnp-anything model utilizing all 200 demonstrations. The pnp-mi k-move task evaluated using the check- point from the pap-anything model.
Additionally, we measure the task completion time for suc- cessful evaluation rollouts across all algorithms. As shown in Table 4 OneDP completes tasks faster than DP. Both OneDP-S and OneDP-D exhibit similarly-rapid task com- pletion times. The quick action prediction of OneDP re- duces hesitation during robot arm movements, particularly when the arm camera's viewpoint changes abruptly. This leads significant improvements in task completion speed. In Figure 7 we present a heatmap for illustrating the task completion times; lighter colors indicate faster completion times, while dark red demonstrates failure cases. Overall, OneDP completes tasks more efficiently across most loca- tions. Although all three algorithms encounter failures in some corner cases for the coffee task, OneDP-S shows fewer failures.
Coffee Task. This task requires the robot to operate coffee machine. It involves the following steps: (1) picking up the coffee pod, (2) placing the coffee pod in the pod holder on the coffee machine, and (3) closing the lid of the coffee machine. This task is more challenging since it in- volves more steps and requires the robot to insert the pod in the holder accurately. We collect 100 human demonstrations for this task. We train one specific model for this task.
Evaluation. We evaluate the success rate and task com- pletion time from 20 predetermined initial positions for the pnp-mil 1k, pappppppnnn pnp-anythiii ng, and co fee tasks, as well as 10 motion trajectories for the pnp-mill miik-mee --movee task. The left side of Figure [egure/]soww shows the setup of the robot, desti- nation box, and coffee machine, with 20 fixed initialization points. Figure shows the 10 trajectories for evaluating onp-milk move. Details of the evaluation are provided in Appendix For DP, we follow Chi et al. (2023) to use
7

---

One-Step Diffusion Policy
pnp-milk
pmp-anything
pnp-anything
unp-milk-mowr
Pick
Plate
Close
Figure 5: Real-World Experiment Illustration. In the first row, display the setup for the pick-and-pie place experiments, featuring three tasks: pnp-milk, 1k, hap-anythime and рnp-mi. mi -ovve In total, sap-anything handles around 10 random objects as shown Figure X The second row illustrates the procedure the more challenging co fee task, where Franka arm tasked with locating the coffee cup, precisely positioning it in the machine's 's cup holder, inserting it, and finally closing the machine's 's lid.
Table Success Rate of Real-world Experiments. We evaluate the performance our proposed OneDP-D and OneDP-S against the baseline Diffusion Policy in real-world robotic manipulation tasks. The baseline Diffusion Policy was trained for 1000 epochs to ensure convergence, whereas our distilled models were trained for 100 epochs. We do not select checkpoints; only the final checkpoint is used for rate.
evaluation. Performance assessed over 20 predetermined rounds, and report the average
Method
Epochs
NFE
pnp-milk
pup-anything
pnp-milk-move
coffee
Avg
DP(DDMM)
1000
10
1.00
0.95
0.80
0.80
0.83
OneDP-D
1.00
1.00
1.00
0.95
OneDP-S
100
1
1.00
1.00
1.00
0.90
0.98
Table 4: Time Efficiency of Real-world Experiments. We present the completion times for cach algorithm recorded in Table For fair comparison, we report the average completion time (in seconds) for each algorithm across evaluation rounds where algorithms succeeded. Specifically, the tasks pnp-mill 1k, pnp-ann aep--nntthingg pnp-mi mii 1k-move and co fee were averaged over 18, 15, and 13 respective rounds. These times indicate how quickly each algorithm responds and completes tasks in a real-world environment.
Method
Epochs
NFE
pnp-milk
pnp-anything
pnp-milk--ove coffee
Avg
DP(DDIM)
1000
29.74
26.03
34.75
54.92
36.36
OneDP-D
100
23.21
22.93
28.73
33.13
27.00
OneDP-S
100
22.69
22.62
28.15
29.78
25.81
Table 5: Real-world inference speeds. We report the wall clock times for each policy real-world scenarios. The action generation process consists of two parts: observation encoding (OE) and action prediction by each method. All measurements were taken using local NVIDIA V100 GPU, with the same neural network size each method. The policy frequencies, shown in Figure | are based on the values from this table.
We enhanced the slow, iterative action prediction process of Diffusion Policy by reducing it to a single-step process, dra- matically decreasing action inference time and enabling the robot respond quickly to environmental changes. Through extensive simulation and real-world experiments, we demon- strate that OneDP not only achieves slightly higher success rate, but also responds quickly and effectively to environ- mental interference. The rapid action prediction further allows the robot to complete tasks more efficiently.
OE
DDPM (100 steps)
DDIM (10
OneDP
Time (ms)
9
660
66
NTE
100
10
However, this work has some limitations. We did not eval- uate OneDP on long-horizon real-world tasks, and in real- world experiments, constrained the robot's operation frequency to 20 Hz for stability, limiting OneDP 's full
5. Conclusion
In this paper, we introduced the One-Step Diffusion Policy (OneDP) through advanced diffusion distillation techniques.
8

---

One-Step Diffusion Policy
potential. Additionally, the KL-based distillation approach may not be the optimal choice for distribution matching, and incorporating a discriminator term could further enhance distillation performance.
Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310. /95/2, 2023b.
Chen, T., Wang, Z., and Zhou, M. Diffusion policies creat- ing trust region for offline reinforcement learning. arXiv preprint arXiv:2405.
2024.
Acknowledgments
Z. Wang and M. Zhou acknowledge the support of NSF-IIS 2212418 and NIH-R37 CA271186.
Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burch- fiel, B., and Song, S. Diffusion policy; Visuomotor policy learning via action diffusion. arXiv preprint Stti:22303.0413333 2023.
Impact Statement
Florence, P., Lynch, C., Zeng, A., Ramirez, O. A., Wahid, A., Downs, L.,, Wong, A., Lee, J., Mordatch, I,, and Tompson, J. Implicit behavioral cloning. Conference on Robot Learning, pp. 158-168. PMLR, 2022.
This work aims to advance the field of Machine Learning by introducing a more efficient approach for deploying diffu- sion policies in robotics. By accelerating the response time of robotic control tasks through the One-Step Diffusion Pol- icy (OneDP), this research has the potential to enhance the practicality of real-time decision-making making in robotic systems, especially in dynamic and resource-consee constre-conttrained environ- ments.
Haarnoja, T., Zhou, A., Abbeel, P., and Levine, Soft actor-critic: Off-policc maximum entropy deep reinforce- ment learning with a stochastic actor. In International conference on machine learning, pp. 1861-1870. PMLR, 2018.
The broader impact of this work includes improving the effi- ciency and responsiveness of robotic applications in sectors such healthcare, manufacturing, and service industries. However, with any advancement in autonomous systems, there are potential risks, including misuse in surveillance, weaponized robotics, or displacement of human labor. Miti- gating these risks requires responsible deployment, adher- ence to ethical guidelines, and proactive consideration of societal impacts.
Hansen-EEstruch P., Kostrikov, I., Janner, M., Kuba, J. G., and Levine, S. Idql: Implicit q qelernning actor- critic method with diffusion policies. arXiv preprint arXiv:2304.105: 2023.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Overall, this work contributes positively to the field of robotics and Machine Learning, with the potential for sig- nificant benefits in various applications, while mindful of the ethical considerations inherent in deploying autonomous technologies.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in Neural Information Process- ing Systems, 33:6840-688 2020.
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. J. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646. 2022.
References
Ajay, A., Du, Y., Gupta, A., Tenenbaum, J., Jaakkola, T., and Agrawal, P. Is conditional generative model- ing you need for decision-making? arXiv preprint 2215577 2022.
Janner, M., Du Y., Tenenbaum, J. B., and Levine, S. Plan- ning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.0999/ 2022.
Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Zhang, Q.. Kreis, K., Aittala, M., Aila, T., Laine, S., et al. ediff- Text-to-mmage diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:22/1.01322 2022.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidat- ing the design space of diffusion- generative mod- els. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL hhttps::/ppenreeiiees net//
Chen, H., Lu, C., Wang, Z., Su, H., and Zhu, J. Score reg- ularized policy optimization through diffusion behavior. arXiv preprini arXiv:2310.07297 2023a.
forum
Ke, T.-W., Gkanatsios, N., and Fragkiadaki, K. 3d diffuser actor: Policy diffussion with 3d scene representations. arXiv preprint arXiv:2402. 10.4000.0888 10885, 2024.
Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y. Chen, Q., Wang, X., al. Videocrafterl:

---

One-Step Diffusion Policy
Kim, D., Lai, C.-H., Liao, W.-H., Murata, N., Takida, Y., Uesaka, T., He, Y., Mitsufuji, Y., and Ermon, S. Consis- tency trajectory models: Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023a.
OpenAL. Video generation models as world simulators, 2024. URL htthhttss///peeee comm
Poole, B., Jain, A., Barron, J. T., and Mildenhall, B. Dream- fusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. /4988, 2022.
Kim, D., Lai, C.-H., Liao, W.-H., Murata, N., Takida, Y., Ue- saka, T., He, Y., Mitsufuji, Y., and Ermon, S. Consistency trajectory models: Learning probability flow ode trajec- tory of diffusion. In The Twelfth International Conference on Learning Representations, 2023b.
Popov, V., Vovk, I., Gogoryan, V., Sadekova, T., and Kudi- nov, M. Grad-tts: A diffusson probabilistic model for text-to-speech. In International Conference on Machine Learning, pp. 8599-8608. PMLR, 2021.
Prasad, A., Lin, K., Wu, J., Zhou, L., and Bohg. J. Consis- tency policy: Accelerated visuomotor policies via con- sistency distillation. arXiv preprint arXiv:2405.07503, 2024.
Kong, Z., Ping, W., Huang, J., Zhao, K., and Catanzaro, B. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.0976; 2020.
Liu, L., Ren, Y., Lin, Z., and Zhao, Z. Pseudo numer- ical methods for diffusion models on manifolds. In International Conference on Learning Representations, forruumum?
Psenka, M., Escontrela, A., Abbeel, P., and Ma, Y. Learn- ing a diffusion model policy from rewards via q-score matching, arXiv preprint 1rXiv:2312.11753 2023.
2022. URL
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:220..06125, 2022.
Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, DPM-solver: A fast ODE solver for diffusion proba- bilistic model sampling in around 10 steps. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information P Processing Systems, 2022. URL httttps://openreeiiit http:///pppeeeeeeeeeeeeeeeorrro net//orumm 1d= Id=2uAaGw1P V
Reuss, M. Li, M., Jia, X., and Lioutikov, R. Goal- conditioned imitation learning using score-based diffu- sion policies. arXiv preprint arXiv:2334..m 025532, 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Luhman, F. and Luhman, T. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint wxXiv22011.02388 2021.
Luo, W., Hu, T., Zhang, S., Sun, J., Li, Z.. and Zhang, Z. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.1 1487, 2022.
Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M.,
Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., Savarese, S., and Fei-Fei, L. RoboTurr: A Crowdsoure- ing Platform for Robotic Skill Learning through Imitation. In Conference on Robot Learning, 2018.
Salimans, T. and Ho, J. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512 2022.
Diekkteektt,, J., Weiss, E., Maheswaranathan, N., and Ganguli, Deep unsupervised learning using nonequi- librium thermodynamics. In International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
Mandlekar, A., Booher, J., Spero, M., Tung, A., Gupta, A., Zhu, Y., Garg, A., Savarese, S., and Fei-Fei, Scaling robot supervision hundreds of hours with roboturk: Robotic manipulation dataset through human reasoning and dexterity. arXiv preprint arXiv: 1911 1.052,, 2019.
Song. J., Meng, C., and Ermon, S. Denoising diffusion im- plicit models. arXiv preprint arXiv:2010.02502. 2020a.
Mandlekar, A., Xu, D., Wong, J., Nasiriany, S., Wang, C., Kulkarni, R., Fei-Fei, L., Savarese, S., Zhu, Y., and Martín-Martin R. What matters in learning from offline human demonstrations for robot manipulation. arXiv preprint ar arxiv:2200 108 100.0088, 2021.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint M:Xiv:201..3344 2020b.
10

---

One-Step Diffusion Policy
Song, Y., Dhariwal, P., Chen, M., and Sutskever, Consis- tency models, arXiv preprint arXiv:2303.01400 2023. Team, O. M., Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.122:1 2024.
of pretrained diffusion models for one-step generation. In Forty-first International Conference Machine Learn- ing, 2024.
Urain, J.. Funk, N., Peters, J., and Chalvatzaki, G. Se (3) :)-diiffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion. In 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 5923-5930. IEEE, 2023.
Wang, Z., Hunt, J. J., and Zhou, M. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint ar 2i0: 06199,, 2022.
Wang, Z., Zheng, H., He, P., Chen, W., and Zhou, M. Diffusion-gan: Training gans with diffusion. Interna- tional Conference on Learning Representations, 2023.
Wang, Z., Lu, C., Wang, Y., Bao, F., Li, C., Su, H., and Zhu, Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024.
Xiao, Z., Kreis, K., and Vahdat, A. Tackling the generative learning trilemma with denoising diffusion gans, arXiv 2021.
preprint 1r
Xu, Y., Zhao, Y., Xiao, Z., and Hou, T. Ufogen: You forward once large scale text-to-image generation via diffusion gans. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 8196-8206, 2024.
Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W. T., and Park, T. One-step diffusion with distribution matching distillation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6613-6623, 2024.
Ze. Y.. Zhang, G., Zhang, K.. Hu, C., Wang, M., and Xu, H. 3d diffusion policy. arXiv preprint arXiv:2403.03954 2024.
Zhao, T. Z.. Kumar, V., Levine, S., and Finn, C. Learn- ing fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13700 2023.
Zheng, H., Nie, W., Vahdat, A., Arizzadenesheli, K., and Anandkumar, A. Fast sampling of diffusion models via operator learning. In International conference machine learning, pp. 42390-42400 PMLR, 2023.
Zhou, M., Zheng, H., Wang. Z., Yin, M., and Huang, H. Score identity distillation: Exponentially fast distillation

---

One-Step Diffusion Policy
One-Step Diffusion Policy: Appendix
A. Related Work
Diffusion models have emerged as powerful framework for modeling complex data distributions and have achieved groundbreaking performance across various tasks involving generative modeling (Ho et al. 2020 Karras et al. 2022). They operate by transforming data into Gaussian noise through a diffusion process and subsequently learning to reverse this process via iterative denoising. Diffusion models have been successfully applied wide range of domains,
Diffusion Models.
including image, video, and audio generation (Saharia et al.
2022 Ramesh et al. [022 BBalaii al.
2022 Chen et al.
2023b
Ho et al. 2022 Popoy al., 2021 Kong et al. 2020). reinforcement learning (Janner et al.) 2022 Wang et al. 2022
Psenka et al. 2023 and robotics (Ajay al. 2022: Urain et al. 2023 Chi et al. 2023).
Diffusion Policies. Diffusion models have shown promising results as policy representations for control tasks, Janner
2022) introduced a trajectory-Ievel diffusion model that predicts all timesteps of a plan simultaneously by denoising two-dimensional arrays of state and action pairs. [Wang et al. (2022) proposed Diffusion Q-learning, which leverages a conditional diffusion model to represent the policy in offline reinforcement learning. An action-space diffusion model is trained generate actions conditioned on the states. Similarly, Chi al. (2023) used a conditional diffusion model in the robot action space to represent the visuomotor policy and demonstrated a significant performance boost in imitation learning for various robotics tasks. [Ze et al. (2024) further incorporated the power of a compact 3D visual representations to improve diffusion policies in robotics.
et al.
Diffusion Distillations. Although diffusion models are powerful, their iterative denoising process makes them inherently slow in generation, which poses challenges for time-sensitive applications like robotics and real- time control. Motivated by the need accelerate diffusion models, diffusion distillation has become an active research topic in image generation. Diffusion distillation aims to train student model that can generate samples with fewer denoising steps by distilling
knowledge from pre-trained teacher model (Salimans & Ho 2022 Luhman & Luhman 2021 Zheng et al. 2023 Song et al.
2023 Kim et a 2023b). [Salimans& & Ho/ (002)) proposed a method to distill a teacher model into a new model that takes half the number of sampling steps, which can be further reduced by progressively applying this procedure. [Song et al.
(2023)
introduced consistency models that enable fewer step sampling by enforcing self-consistency of the ODE trajectories.
CTM
Kim et al.. 2023b) improved consistency models and provided the flexibility to trade-off quality and speed. (Luo et al.,
2024
Yin et al.| 2024) leverage the success of stochastic distillation sampling (Poole et al. (022) in text-to-3D and proposes
KI.-based score distillation for image generation. Beyond KI., (ioou al. (2024) proposes the SiD distillation technique derived from Fisher Divergence. However, leveraging diffusion distillation to accelerate diffusion policies for robotics remains an underexplored and pressing challenge, particularly for real-time control applications. Consistency Policy (Prasad ct al. 2024) explored applying CTM to reduce the number of denoising steps and accelerate inference of the diffusion policies. It simplifies the original CTM training by ignoring the adversarial auxiliary loss. While this approach achieves a considerable speed-up, it leads to performance degradation compared to pre-trained models, and its complex training process and slow convergence present challenges for roboties applications. In contrast, OneDP employs expectational reverse KL optimization to distill a powerful one-step action generator, achieving comparable or higher success rates than the original diffusion policy, while converging 20x faster.
B. Detailed Preliminaries
Diffusion models are robust generative models utilized across various domains (Ho et al. 2020: Sohl-Dicksee et al., 2015 Song et al.] 2020b). They operate by establishing forward diffusion process that incrementally transforms the data distribution into known noise distribution, such as standard Gaussian noise. A probabilistic model is then trained to methodically reverse this diffusion process, enabling the generation of data samples from pure noise.
Suppose the data distribution is p(z)). The forward diffusion process is conducted by gradually adding Gaussian noise to samples xd ~ p(x) as follows,
o + G N(O,
(<<"|1=)) = NN(a " I)
where dg and og are parameters manually designed to vary according to different noise scheduling strategies. DDPM (Ho
ct al. 2020 is a discrete-timg time diffusion model with E [1..... K}. 3. It can be casily extended to continuous-tim diffusion
from the score-based generative model perspective (Song et al. 2020b Karras et al. 2022) with E [0, 1]. With sufficient
12

---

One-Step Diffusion Policy
amount of noise added, xx (0, I. Ho et al. (2020) propose reverse the diffusion process and iteratively reconstruct
the original sample x by training neural network co(=", k)] to predict the noise E; added at each forward diffusion step (epsilon prediction). With reparameterization
(x
(/ozz)))ox.)) the diffusion model could also be formulated as
x" prrediction process (x*, k) (Karras et al,, 2022, Xiao et al. 2021). We use epsilon prediction in our derivation. The
diffusion model is trained with the denoising score matching loss (Ho et al. 2020),
min Bogtagga -p))) (A(k) ||eo , Ex
where U is uniform distribution over the space, and ((k) is noise-ratio re-weighting function. With a trained diffusion model, we could sample æ² by reversing the diffusion chain, which involves discretizing the ODE (Song et al. 2020b) as follows:
dx = [(k))a
1 ( -
log ((<))))))))))) dk
(9)
where [(k) d'logas and g²(k)) dk
doj. 2 2 oi The unknown score Van log g(x**) could be estimated as follows:
dk
dk
s(x*) Va logg(x²)) c'(x", .k)
cola" (k)
o
where c"(x"", k) is the true noise added at time k, and we let so sxx* = <a(c"* .R)
Wang et al. (2022): Chi et al. 2023) extend diffusion models as expressive and powerful policies for offline RL and robotics. In robotics, set of past observation images O is used input to the policy. An action chunk A, which consists of sequence of consecutive actions, forms the output of the policy. ResNet (He al.) 2016) based vision encoders commonly utilized to encode multiple camera observation images into observation features. Diffusion policy is represented as conditional diffusion- based action prediction model,
k= II Po O,) d.. A=K
To To (A910.)) = J... o.
where O, contains the current and a few previous vision observation features at timestep f, and po could be represented
by Ea as shown in DDPM (Ho et al. 2020). The explicit form of T*(A O,) is often impractical due to the complexity of integrating actions from A, to A.. However, we can obtain an action chunk prediction A by iteratively solving Equation ( from K to O.
C. Real-World Experiment Setup
Robot Setup. The physical robot setup consists of Franka Panda robot arm, front-view Intel RealSense D415 RGB-D camera, and wrist-mounted Intel RealSense D435 RGB-D camera. The RGB image resolution was set to 120x160. The depth image is not used in our experiments.
Teleoperation. Demonstration data for the real robot tasks was collected using phone-based teleoperation system (Man- dlekar al. 2018 2019).
Data Collection. We collect 100 demonstrations for each task separately: pnp-milk 1k, pnp-anything, and co fee. In pnp-mi mik, the target object always the milk box, and the task involves picking up the milk box from various random locations and placing it into a designated target box a fixed location. For pnp- aooonnything we extend the set of target objects 11 different items, as shown in Figure B with the target box location randomized vertically. In the cof fee task, the coffee cup is randomly placed, and the robot is required to pick it up, insert it into the coffee machine, and close the lid,
The area and location for each task are illustrated in the left column of Figure 7 During data collection, target objects are randomly positioned within the blue area; the grid is used for evaluation, as described in the next section. For the pnp tasks, the blue area is a rectangle measuring 23 cm in height and 20 cm in width, while the target box is a square with a side length of 13 em. In the co f fee task, the blue area slightly smaller, measuring 18 cm in height and 20 em in width.
13

---

One-Step Diffusion Policy
Figure 6: Real-world Experiment Setup
Table 6: Real-world experiment demonstrations. In total we collect 300 demonstrations, with 100 demonstrations for each task.
pnp-milk 100
pnp-anything
coffee
Demos
100
100
To ensure a fair comparison between OneDP and all baseline methods, we standardize the evaluation process. For the pnp-m 1k, pnp pnp-anythe anything. and fee tasks, we evaluate each method according to the grid order shown in Figure 7 The target object is placed at the center of the grid to ensure consistent initial conditions across evaluations. For task pnp-anything, the picked object also follows the order shown in Figure g For the dynamic environment task pnp-milk-move we introduce human interference during the evaluation. Whenever the robot gripper attempts to grasp the target milk box, we manually move it away along the trajectory depicted in Figure 9
Evaluation.
Although we aim to maintain consistent conditions during each evaluation, the exact nature of human interference cannot be guaranteed. Some trajectories involve single instance of interference, while others may involve two consecutive human movements.
The original DDPM sampling in Diffusion Policy is too slow for real-world experiments. To speed up the evaluation, follow (Chi et al. 2023) and use DDIM with 10 steps. For OneDP, we use single-step generation. In real-world experiments, we do not select intermediate checkpoints but use the final checkpoint after training for each method.
We record both the success rates and completion times, reporting their mean values. For prp-mi 1 k k-move, evaluations are conducted over 10 trajectories, while for the other tasks, results are obtained from 20 grid points. In Figure we present a heatmap to visualize task completion times, where lighter colors represent faster completions and dark indicates failure cases. Overall, OneDP completes tasks more efficiently across most locations. While all three algorithms experience failures in certain corner cases for the cof fee task, OneDP-S demonstrates fewer failures.
D. Training Details
We follow the CNN-based neural network architecture and observation encoder design from Chi et al. (2023). For simulation experiments, we use a 256-million 336-miilion-paramaee parameter version for DDPM and a 67-million parameter smaller EDM network performs slightly better. In real-world experiments, we also use the 67- paamaetton-parameter version.
- version for EDM, as the
14

---

One-Step Diffusion Policy
DP
OneDP-D
OneDP-5
Robot
41
2-
2-
32
Box
OneDP-D
OneDP-5
Robot
0-
335
2-
2-
Box
3-
1
OneDP-D
OneDP-S
Robot
0-
11
Coffee
2-
3
32
12
Figure Real-World Comparison Illustration. We present the time taken by each algorithm complete tasks from specific starting point in colors. color map on the right side ranges from white red indicating the time in seconds. Dark red signifies that the algorithm failed at that location. The three rows represent tasks pnp-min mi 1k, pap-anythhaaeeeee co fee. Details of the evaluation of pnp-anything titiing can be found in Figure g
Additionally, we adopt the action chunking idea from Chi et al. (2023) and Zhao et al. 2023 using 16 actions per chunk for prediction, and utilize two observations for vision encoding.
We first train DP for 1000 epochs in both simulation and real-world experiments with a default learning rate of le-4 and weight decay of le-6. We then perform distillation using the pre-trained checkpoints, distilling for 20 epochs in simulation and 100 epochs in real-world experiments.
For distillation, we warm-start both the stochastic and deterministic action generators, Gg. and the generator score network, by duplicating the network structure and weights from the pre-trained diffusion-policy checkpoints. Since the generator network is initialized from denoising network, timestep input is required, as this was part of the original input. We fix timestep at 65 for discrete diffusion and choose o 2.5 for continuous EDM diffusion. The generator learning rate to le-6. We find these hyperparameters to be stable without causing significant performance variation. We provide an ablation study that focuses primarily the generator score network's learning rate and optimizer settings in Appendix E We provide the hyperparameter details in Table
E. Ablation Study
As shown in the first panel of Figure 10
we explore a range of learning rates for the generator score network in the grid [1e-6, 1e-5, 2e-5, 3e-5, 4e-5] and find 2e-5 be optimal most cases. higher learning rate for the score network compared
15

---

One-Step Diffusion Policy
Hyperparameters
Values
generator learning rate
Ir-ee-6
generator score network learning rate
1r=2e-5
generator optimizer
Adam([0.0, 0.999])
generator score network optimizer action chunk size
Adam([O.0, 0.999])
n=16
number of observations
n=2
discrete diffusion init timestep
linit=65
discrete diffusion distillation range
[2. 95]
continuous diffusion init sigma
a = 2.5
Table 7: Hyperparameters
the generator ensures that the score network keeps pace with the generator's distribution updates during training. In the second panel, we search for the best optimizer settings, finding that setting 81 to O for both the generator and the generator score network optimizers effective. This approach, commonly used in GANs, allows the two networks to evolve together more quickly.
F. Diseussion
Comparison with VSD. VSD is designed to distill image-level knowledge from powerful 2D priors, specifically pretrained text-to-image diffussin models, to facilitate 3D content generation. Its overarching objective- reverse KL optimization widely applied across multiple domains, including VAEs. In this work, we also apply reverse KL optimization for diffusion policy distillation. However, the implementation and derivation for different domains required major efforts. This extensive process involved adjustments noise scheduling (DDPM and EDM), proper initialization, balancing the convergence of the generator and its score network, tuning parameters, designing experiments in dynamic environments, and conducting both simulated and real-world robotics experiments an undertaking that should not be underestimated. Furthermore, OneDP considered temporal control characteristics by predicting action chunks, each comprising sequence of actions (K=16), This approach addresses the temporal dependencies inherent in many robotics tasks, which are not considered in VSD.
Training Cost Comparison of OneDP-D and OneDP-S. OneDP-S and OneDP-D differ in their computational require- ments. The training cost for OneDP-S is approximately twice that of OneDP-D, due to the inclusion of the generator score network. When accounting for evaluation during training, the total time for OneDP-S about 1.5 times longer than that of OneDP-D. For example, on the small dataset PushT, training and evaluation for OneDP-D take about 30 minutes, while OneDP-S requires approximately 45 minutes. On the larger ToolHang dataset, OneDP-D takes roughly hours, compared to about hours for OneDP-S. These details will be further elaborated in future revisions to provide a comprehensive view of the trade-offs between stochastic and deterministic policies in terms of both performance and computational efficiency.
G. More Dynamic Experiments
We conducted an additional dynamic real-world experiment to evaluate performance under human intervention. During the milk box pick-and- place task, we randomly reset the milk box pose simulate changes in the environment. The process is illustrated in Figure] [I The results indicate that DP achieves a success rate of 28.57% (6/21), while our OneDP significantly outperforms it with a success rate of 76. 19% (16/21), over 21 random initializations. DP fails in most cases due to its slow response to environmental changes, whereas OneDP reacts quickly and achieves a much higher success rate.
16

---

One-Step Diffusion Policy
Figure 8: Evaluation setup for pnp pap-ppnnhiiie anything.
17

---

One-Step Diffusion Policy
Robot
Robot
Robot
Robot
Robot
Box
Box
Box
Box
Robot
Robot
Robot
Robot
Robot
Box
Box
Box
Figure 9: Evaluation trajectories for pnp-mi 1k-move. The box is always the left-hand side of the tested blue area.
Ablation Study on ToolHang-ph
Ablation Study Optimizer on Square-mh
0.5
Success Rate
Success Rate
0.6
0.4
g5.
g5. Ir=2e
g_optim=(0.0 (0.0. 0.999), gs, pp_opcim=(0.0 (0.00, 0.999]
3_0pttm=[0.99 @981_gs_pptim-100 0.999)
0.0
1r-55
0.0
9. 1.00tm-t00.9 0.999).gs_opte 9.9991.5_.pttmm10000 0.999)
0.0
2.5
5.0
7.5
10.0 Epoch
12.5
15.0
17.5
20.0
0.0
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0
Epoch
Figure 10: Ablation studies on the learning rate of the generator score network and optimizer settings.
Figure 11: Dynamic Real- World Experiment: Pose Reset.
18


Reviews:
[{"review_id": "gPbN5F3mR7", "reviewer": "Reviewer_5rTc", "summary": "The paper introduces the One-Step Diffusion Policy, a novel approach that distills a pre-trained multi-step diffusion-based visuomotor policy into a single-step action generator for robot control​. This one-step policy greatly accelerates inference (boosting action output frequency from ~1.5 Hz to 62 Hz) while maintaining state-of-the-art task success rates.\n\n**Strengths**:\n- Order-of-magnitude speed improvement in action inference without performance loss.\n- The distillation process only adds 2%–10% to the additional training cost\n- Efficient Task Completion not only reacts faster but completes tasks in significantly less time.\n\n**Weaknesses**:\n- The study does not assess long-horizon tasks, where cumulative decision-making plays a larger role.\n- Some mode-seeking behaviors inherent in reverse KL minimization could lead to suboptimal exploration in stochastic tasks .\n- No Analysis of Failure Case", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer 5rTc's positive feedback and address your remaining concerns below:\n### **Weaknesses:**  \n1. **Mode-Seeking Behavior in Reverse KL Minimization**  \n   While reverse KL minimization encourages mode-seeking, this does not necessarily lead to mode collapse. In our real-world coffee machine manipulation experiments, we deliberately collected data for closing the lid from two different angles. After distillation, the distilled policy retained this multimodal behavior. Additionally, mode-seeking can be beneficial in imitation learning tasks—once the policy identifies an optimal solution, it helps eliminate noisy low-density actions and enhances stability. In our case, imitation learning data is human-demonstrated and high-quality, ensuring that the pretrained policy converges to optimal solutions. Therefore, mode-seeking is not a critical concern for our approach.  \n2. **Evaluation on Long-Horizon Tasks**  \n   We emphasize that our study explores both simulation and real-world tasks. However, long-horizon tasks require a diffusion policy trained explicitly for long-horizon planning. We acknowledge the importance of extending our approach to such tasks and consider it a promising direction for future work.  \n\n### **Questions:**  \n1. **Comparison with Other Imitation Learning Methods**  \n   Implementing all cited imitation learning baselines within our framework is highly resource-intensive and beyond the scope of this work. Instead, we selected Diffusion Policy and Consistency Policy as our primary baselines, as they are the most representative diffusion-based approaches.  \n2. **Why the Distilled Policy Outperforms the Pretrained Policy**  \n   The distilled policy demonstrates slightly better performance than the original diffusion policy due to two main factors:  \n   - **Error Accumulation in Multi-Step Sampling:** Traditional diffusion policies suffer from accumulated errors over multiple denoising steps, while our one-step approach eliminates this issue.  \n   - **Mode-Seeking Behavior Enhancing Stability:** As previously mentioned, the distilled policy focuses more on successful trajectories, reducing the influence of low-density noisy actions. This results in a more stable and confident policy during evaluation.", "claims_and_evidence": "- The paper presents quantitative results comparing the inference speed of OneDP (62 Hz) vs. Diffusion Policy (1.49 Hz)\n- A wall-clock time comparison (Table 5) shows that OneDP's single-step action generation is much faster than the iterative denoising process in traditional diffusion models.\n---\n- While the improvement is clear, the robot’s control frequency was limited to 20 Hz in real-world settings\n- Success rates are evaluated on a fixed set of tasks. The results may not generalize to more complex or unseen environments.", "methods_and_evaluation_criteria": "**Strengths**:\n- OneDP effectively distills a slow diffusion policy into a single-step generator, achieving ~42× faster inference while maintaining high success rates.\n- Benchmark evaluation on Robomimic and real-world tasks with meaningful metrics (success rate, inference speed, task time).\n- Fair comparison with Diffusion Policy and Consistency Policy, demonstrating superior speed and efficiency.\n\n**Weaknesses**:\n- KL divergence distillation may limit policy diversity; alternative techniques are unexplored.\n- Evaluation is limited to short-horizon tasks; long-term planning and generalization to unseen environments are not tested.", "theoretical_claims": "- The paper’s theoretical foundation is based on KL divergence minimization for distillation.\n- The mathematical derivation follows from established diffusion model literature (Ho et al., 2020; Song et al., 2020), ensuring theoretical validity.\n- No formal proof or bound is given for the error introduced by the single-step approximation compared to multi-step diffusion policies.", "experimental_designs_or_analyses": "- Evaluations conducted on six Robomimic tasks (simulation) and four real-world robotic tasks (Franka arm), ensuring relevance to visuomotor learning.\n- While OneDP theoretically achieves 62 Hz, real-world experiments cap execution at 20 Hz for stability.", "relation_to_broader_scientific_literature": "1. Diffusion Models in Robotics & Policy Learning:\n2. The work is related to Consistency Policy (CP) (Prasad et al., 2024), which also attempts to accelerate Diffusion Policies.\n3. KL-Based Distillation in Generative Models", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "No other strengths and weaknesses.", "other_comments_or_suggestions": "No other comments or suggestions.", "questions_for_authors": "1. Would it be possible to include experiments on other imitation learning methods cited in the paper for comparison?\n2. Is there a more in-depth discussion on the insights behind why the distilled policy can achieve better performance compared to the original diffusion policy?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5cfTtG4enT", "reviewer": "Reviewer_kXET", "summary": "This paper adapts diffusion distillation techniques from text-to-3D generation to achieve one-shot diffusion policies for robotics. The authors compare two types of distillation: 1) distilling to a stochastic one-step diffusion policy, and 2) distilling to a deterministic one-step policy. They show that the stochastic policy has a task performance slightly superior to that of the original DDPM teacher, with a ~40x inference time speedup. The method significantly outperforms a consistency policy baseline, both in terms of final performance and training speed.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer kXET's recognition of our work's value and your insightful suggestion on extending our distillation technique to facilitate online RL fine-tuning. We agree this is a promising future direction. Below, we address your remaining concerns and questions.  \n### **References**  \nWe will include a discussion of *π₀: A Vision-Language-Action Flow Model for General Robot Control* in the related work section.  \n### **Questions & Responses**  \n1. **Deterministic Sampling in OneDP-D (Algorithm 1)**  \n   Thank you for pointing this out. In the OneDP-D case, $z$ is indeed fixed at zero. We will correct the typo in Algorithm 1 accordingly.  \n2. **Distillation Over [2, 95] Diffusion Timesteps**  \n   The reason for avoiding the extreme timesteps (near 0 and 100) is that the distilled policy’s distribution may initially differ significantly from the pretrained diffusion policy.  \n   - If nearly no noise is added, the distilled policy may generate actions that fall in low-density regions of the pretrained policy, leading to unstable distillation loss.  \n   - If excessive noise is added, the score guidance from the pretrained policy becomes nearly uniform across actions, reducing its utility for training.  \n   This aligns with observations from *DreamFusion*, where extreme noise levels were avoided. We quote the original sentence here: “We sample t ∼ U(0.02, 0.98), avoiding very high and low noise levels due to numerical instabilities”. \n3. **Why OneDP-S Outperforms OneDP-D**  \n   This is due to the pretrained diffusion policy having strong performance in learning multi-modal distributions, so the pretrained policy has multi-modality. The stochastic policy benefits from the multimodal nature of the pretrained diffusion policy. As discussed in our response to Reviewer 5rTc, reverse KL divergence encourages mode-seeking behavior, allowing the stochastic policy to converge to multiple optimal action modes.  \n   - In our coffee machine manipulation tasks, we designed scenarios with two distinct successful strategies. The stochastic policy successfully captured both, whereas the deterministic policy followed only one.  \n   - Exhibiting multi-modality enables the robot to solve the request in broader regions with multiple potential solutions so it is more stable than pure deterministic policy that can only follow one specific path. \n\nWe appreciate your thoughtful feedback and will incorporate these clarifications in the final revision.", "claims_and_evidence": "The main claim is that the OneDP dramatically speeds up inference: \"that OneDP not only achieves state-of-the-art success rates but also delivers an order-of-magnitude improvement in inference speed, boosting action prediction frequency from 1.5 Hz to 62 Hz.\" This is well-supported by the results (although see the caveat in \"Experimental Designs Or Analyses\").", "methods_and_evaluation_criteria": "The evaluation settings are thorough, involving both simulated and real-world experiments.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Overall, the experiments look sound. The one comment I'd have is that it seems that the underpowered hardware might accentuate performance differences. A DDPM frequency of 1.5 Hz is much lower than Diffusion Policy (10Hz diffusion), Octo (10Hz diffusion), or $\\pi_0$ (~6Hz flow matching). Nevertheless, I think that ultimately these visuomotor policies will need to be executed onboard with resource-constrained hardware, and this work is correspondingly more valuable.", "relation_to_broader_scientific_literature": "I think that the importance of faster action inference are pretty clear for robotics, especially for systems where compute has to be entirely onboard.\n\nI'd like to comment on another reason I see this paper as being important to the field that I don't see mentioned by the authors. One important challenge with these diffusion policies lies in transitioning from imitation learning pre-training to finetuning with online RL. Obvious online RL candidates like PPO don't immediately apply, as they require the gradient of the action probability w.r.t. the policy parameters; this isn't available in closed-form because the diffusion action probability is a) defined implicitly, and b) involves iterative denoising which would be hard to differentiate through anyways. This paper solves problem (b), and I think opens up a few interesting possibilities for addressing (a), such as having the policy network output a normal distribution centered at the distilled $G_{\\theta}(O)$, or perhaps this idea can be adapted to flow matching where the density at a point can be evaluated. While understandably this extension isn't in scope for the current paper, I think the paper opens up some exciting next steps.", "essential_references_not_discussed": "I'd say that $\\pi_0$ is a fairly important paper that should be discussed: Black, Kevin, et al. \"$\\pi_0 $: A Vision-Language-Action Flow Model for General Robot Control.\" arXiv preprint arXiv:2410.24164 (2024).", "other_strengths_and_weaknesses": "The primary weakness is that the paper is a little lacking in novelty, as it largely adapts ideas from the generative AI literature to robotic policies. But I still think that the paper is a valuable contribution for both its practical impact on the feasibility of diffusion policies and it setting the groundwork for incorporating online RL. The paper is overall well-written and easy to follow.", "other_comments_or_suggestions": "\"Initializaiton\" in Algorithm 1.", "questions_for_authors": "1. In Algorithm 1, should not the sampling of $A_{\\theta}$ be deterministic in the OneDP-D case?\n2. Can the authors elaborate on \"distillation occurs over [2, 95] diffusion timesteps to avoid edge cases\" in line 248? I haven't seen this before.\n3. I'm somewhat suprised that the stochastic policy outperformed the deterministic one. I get that the stochasticity during pre-training is valuable, as human demonstrations aren't deterministic and thus it's difficult to fit a deterministic policy to them. But when we're distilling, it seems that converging to the action mode should be beneficial (as briefly discussed in the \"Distillation Discussion\" section). Could the authors elaborate on why OneDP-S outperforms One-DP-D? The authors write that: \"A stochastic policy, which encompasses deterministic policies, is more versatile and better suited to scenarios requiring exploration, potentially leading to better convergence at a global optimum.\" In RL, I see why exploration is important; but here we're doing imitation learning so I don't see how this applies.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EYcUTKNLzM", "reviewer": "Reviewer_Bfs2", "summary": "The authors introduce One-Step Diffusion Policy (OneDP), which distills pre-trained diffusion policies into single-step generators for robotic control. OneDP achieves 42× faster inference (62.5Hz vs 1.5Hz). Evaluation is performed on six simulation and four real-world tasks.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We find it disheartening that Reviewer Bfs2 rated our paper a 1, accompanied by a very brief review and a complete dismissal of our contributions, despite our focus on a critical problem in learning fast visuomotor policies for robotic control and our efforts to advance the state of the art in this area. \n\nWe respectfully urge Reviewer Bfs2 to reassess our paper more carefully and, if time permits, consider the evaluations provided by the other reviewers. We address your concerns in detail below:\n\n1. Mode-Seeking Behavior in Reverse KL Minimization: While reverse KL minimization encourages mode-seeking, this does not necessarily lead to mode collapse. In our real-world coffee machine manipulation experiments, we deliberately collected data for closing the lid from two different angles. After distillation, the distilled policy retained this multimodal behavior. Additionally, mode-seeking can be beneficial in reinforcement learning tasks—once the policy identifies an optimal solution, it helps eliminate noisy low-density actions and enhances stability. In our case, imitation learning data is human-demonstrated and high-quality, ensuring that the pretrained policy converges to optimal solutions. Therefore, mode-seeking is not a critical concern for our approach.  \n\n2. Our submission primary area is `Applications -> Robotics`. Diffusion distillation has been applied across various domains, including 2D/3D generation, video generation, and diffusion-based language models. However, our contribution focuses on demonstrating its effectiveness in robotics, a field where efficient policy execution is critical. We put significant effort to show the effectiveness of diffusion distillation in robotics and its wild potentials for robotics. We also kindly request Reviewer Bfs2 to review the potentials mentioned by Reviewer kXET.\n\n3. The pretraining + distillation paradigm is widely adopted across AI research, including language models (e.g., GPT-4o with its 4o-mini and o1-mini variants). Distillation plays a crucial role in making inference efficient, reducing computational costs, and enabling deployment on constrained hardware. For robotic control, where policies must run onboard with limited compute, achieving fast inference without sacrificing performance is highly practical and necessary. Given its broad adoption in other domains, distillation should not be viewed as a drawback but rather as a crucial technique for real-world applicability. We will update Fig1 to make it looks better.", "claims_and_evidence": "The results are promising, but I am not entirely convinced. See below.", "methods_and_evaluation_criteria": "Ok", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Ok-ish", "relation_to_broader_scientific_literature": "Ok", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "- My main concern with the method is that it should lose partly/fully the benefit of multimodality that is given by diffusion policy, which is perhaps the biggest strength of diffusion policy.  It would have been interesting to see experiments where multimodality is required, and to see how the distilled one-step policy would deal with them.\n\n- The idea is simple, but not novel (same trick used, e.g., in image generation).", "other_comments_or_suggestions": "- Figure 1, training cost:  I think this is highly misleading.  I understand that the distillation part is much cheaper than the full training of the diffusion policy, showing that oneDP adds little overhead. However, it still requires training the original diffusion policy first", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The method shows a clear 40–42× inference-speed improvement over prior diffusion policies (1.5 Hz → 62 Hz) while matching or slightly surpassing task success rates on six simulated and four real-world Franka tasks. 

Reviewers agree the strong empirical evaluation, modest extra training cost, and potential impact on onboard robotics. 

Reviewer Bfs2 raises two main objections: (i) limited novelty because one-step distillation has appeared in very recent work, and (ii) risk of losing multimodality. The authors convincingly rebut (ii) with evidence that the stochastic variant retains multiple modes in a coffee-machine task and explain why reverse-KL does not force mode collapse. On (i), although one-step distillation itself is not new, applying it rigorously to visuomotor control and demonstrating real-world speedups is novel and practically important for resource-constrained robotics; both other reviewers agree. 

Overall, I recommend acceptance.
Decision: Accept (poster)