CL-DiffPhyCon: Closed-loop Diffusion Control of Complex Physical Systems
Authors: Long Wei
Haodong Feng
Yuchen Yang
Ruiqi Feng
Peiyan Hu
Xiang Zheng
Tao Zhang
Dixia Fan
Tailin Wu
Venue: ICLR 2025 Poster (2025)
Keywords: physical systems control
closed-loop control
PDE
physical simulation
generative models
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a diffusion method with an asynchronous denoising schedule for physical systems control tasks. It achieves closed-loop control with significant speedup of sampling efficiency.

Abstract: The control problems of complex physical systems have broad applications in science and engineering.  Previous studies have shown that generative control methods based on diffusion models offer significant advantages for solving these problems. However, existing generative control approaches face challenges in both performance and efficiency when extended to the closed-loop setting, which is essential for effective control. In this paper, we propose an efficient Closed-Loop Diffusion method for Physical systems Control (CL-DiffPhyCon). By employing an asynchronous denoising framework for different physical time steps, CL-DiffPhyCon generates control signals conditioned on real-time feedback from the system with significantly reduced computational cost during sampling. Additionally, the control process could be further accelerated by incorporating fast sampling techniques, such as DDIM. We evaluate CL-DiffPhyCon on two tasks: 1D Burgers' equation control and 2D incompressible fluid control. The results demonstrate that CL-DiffPhyCon achieves superior control performance with significant improvements in sampling efficiency. The code can be found at https://github.com/AI4Science-WestlakeU/CL_DiffPhyCon.

Published as a conference paper at ICLR 2025
CL- CL-DIFFPHYCON: CLOSED-LOOP DIFFUSION CON- TROL OF COMPLEX PHYSICAL SYSTEMS
Long Wei Haodong Feng Yuchen Yang Ruiqi Feng'
Peiyan Hu 28
Xiang Zheng Tao Zhang' Dixia Fan Tailin Wu
'Department of Artificial Intelligence. Westlake University.
School of Statisties and Data Science, Nankai University,
3 AAcademy of Mathematics and Systems Science, Chinese Academy of Sciences, "Scooo of Future Technology, South China University of Technology (we ong, fenghaodong, wutai in @west ake. edu.
ABSTRACT
The control problems of complex physical systems have broad applications in science and engineering. Previous studies have shown that generative control methods based on diffusion models offer significant advantages for solving these problems. However, existing generative control approaches face challenges in both performance and efficiency when extended to the closed-loop setting, which is essential for effective control. In this paper, we propose efficient Closed- Loop Diffusion method for Physical systems Control (CL-DiffhyConn) By em- ploying asynchronous denoising framework for different physical time steps, CL-DifiPhyCo generates control signals conditioned on real-time feedback from the system with significantly reduced computational cost during sampling. Ad- ditionally, the control process could be further accelerated by incorporating fast sampling techniques, such DDIM. We evaluate CL-DiftPhyCon on two tasks: ID Burgers' equation control and 2D incompressible fluid control. The re- sults demonstrate that CL-DiifPhyCon achieves superior control performance with significant improvements in sampling efficiency. The code can be found at httpp://github. http:thwww...ccnAAciieeeeeeeee.e Westtlakee/CLL DiffPhyyC
INTRODUCTION
The control problem of complex physical systems is critical research area for optimizing quence of control actions to achieve specific objectives. It has wide applications in science and en- gineering fields, including fluid control (Verma al.] (2018). plasma control (Degrave et al.) (022)). and particle dynamics control (Reyes Garza et al.) (023). The challenge in controlling such systems arises from their high-dimensional and highly nonlinear characteristics. Therefore, to achieve good control performance, there is inherent requirement of closed-loop control, which is particularly necessary for control tasks that involve extra challenges, such as stochastic dynamics. Specifically, each control decision should be based on the latest state feedback from the system dynamics, allow- ing for continuous adaptation the control inputs in response to any changes.
Over recent decades, several methods have been developed to address this problem, including clas- sical control methods, recent reinforcement learning approaches, and the latest generative methods. Among them, diffusion models, such as DiftPhyCon (Wei al., 2024), have demonstrated competi- tive performance, often outperforming both classical control and reinforcement learning methods in complex physical systems control. The superiority of diffusion models for general decision-making problems has also been widely demonstrated recently (Ajay et al. 2022 Janner et al. 2022a).
However, these diffusion control approaches encounter significant challenges in handling the closed- loop control problems, due to their reliance on synchronous denoising strategy. The diffusion mod- els start from pure noise to a denoised sample for all physical time steps within the model horizon. Applying a full sampling process at each physical time step can realize the closed-loop control, but
"Equal contribution. Work done an intern Westlake University: 'Corresponding author.

---

Published as a conference paper at ICLR 2025
DiffPhyCon-1
DiffPhyCon-i or Adaptive replan
CL-DiffPhyCon (ours)
TeA-1
System dynamics
System dynamios
System dynamics
Sampling cost
Sampling costo T/h(1 < R)
Sampling cost 7/H
Closed-loop
Closed-loop X
Closed-loop
Sampling
sampling
End sampling
Control signal
System State
Figure 1: Advantages of our CL-DifIPhyCon (right) over previous diffusion control methods (left and middle). The diffusion model horizon is denoted as H and the total number diffusion steps is T. By employing asynchronous denoising framework, loop control and accelerate the sampling process significantly. The notation DiffPhyCon-l means conducting full sampling process including T denoising steps every h physical time steps.
method could achieve closed-
incurs high sampling cost. Moreover, it may disrupt the consistency of the control signals, thereby affecting overall performance (Kaelbling& Lozano-Péree (01I). On the other hand, conducting full sampling process every several physical time steps improves sampling efficiency but no longer conforms to the closed-loop requirement, leading inferior control decisions due to outdated sys- tem states. Although online replanning strategy has been proposed recently to determine when to replan adaptively (Zhou et al., 2024), they do not establish a fully closed-loop framework. In addition, it involves extra computation of likelihood estimation sampling from scratch, with dependence on thresholding hyperparameters. which may vary across different tasks and require experiments or significant amount of tuning to determine.
In this paper, we propose a novel Closed-Loop Diffusion method for Physical systems Control, named as CL-DiffPhyCon. The key idea is decouple the synchronous denoising within the model horizon, allowing different physical time steps to exhibit different noise levels. In this way, closed- loop generation of control sequences is naturally realized: the asynchronous diffusion model out- puts control signals sequentially with increasing levels of noise along physical time steps, which enables utilization of real-time feedback state for control signal sampling each horizon without waiting for all the following control signals in the same horizon to be denoised completely. Then, the feedback serves as the initial condition for sampling subsequent control signals, ensuring they are generated based on this reliable state. Our method can also be seen as seamless replanning approach that leverages fresh observations with minimal sampling costs. Therefore, compared to existing diffusion based control methods (Wei al. 2024 Zhou et al. 2024). approach not only realizes closed-loop control but also achieves significant sampling acceleration. These advantages of our method illustrated in Figure
In summary. we make the following contributions: (1) We propose CL-DiffPhyCon a novel closed- loop diffusion control method for complex physical systems. The core of this method is asyn- chronous diffusion model derived from a theoretical analysis of the target distribution. This model enables parallel denoising across physical time steps, allowing earlier control actions be sam- pled sooner, thereby accelerating the sampling process. Additionally, the control process could be further accelerated by incorporating fast sampling techniques, such as DDIM (Song et al. 2020). (2) We evaluate CI.-DiffPhyCon on the 1D Burgers' equation control and 2D incompressible fluid control tasks. The results demonstrate that CL-DiffPhyCon achieves notable control performance with significant sampling acceleration.
RELATED WORK
Classical control methods like PID Li 2006 and MPC (Schwenzer et al.) 2021) are known for their high efficiency, steady performance, and good interpretabilitty but they face significant
2

---

Published as a conference paper at ICLR 2025
challenges in both performance and efficiency when applied to control high- high-dimensional complex physical systems. Recently. imitation learning and reinforcement learning has shown good perfor-
mance wide range of physical systems (Pomerleau) 1988; Zhuang et al., 2023). including drag
reduction (Rabault et al.
2019 Elhawary 2020] Feng et al. 2023 Wang et al. 2024). heat transfer
(Beintema et al.) 2020
Hachem et al.] and fish swimming Novati et al. 2017 Verma al.,
Feng et al. 2024).. Another category of supervised learning (SL) methods (Holl et al., 2220]]
Hwang et al. 2022) plan control signals by utilizing backpropagation through neural surrogate model. In contrast, our approach does not depend on surrogate models; instead, it simultaneously learns the dynamics of physical systems and the corresponding control sequences. Additionally, physics-informed neural networks (PINNs) (Willard et al., 2020) have recently been utilized for control (Mowlavi & Nabi 2023). but they necessitate explicit PDEs. In contrast, our method is data-driven and has broader applicability.
Diffusion models (Ho et al.) 2020) excel at learning high-dimensional distributions and have achieved significant success in image and video generation (Dhariwal & Nichol 2021 Ho et al.) 2022 weather forecasting (Price et al. 2023), and inverse design (Wu et 2024). to name few.
They have also demonstrated superior capabilities on decision making tasks, such robot control
Janner et al., [202bb] Ajay et al. 2022) and high-dimensional nonlinear systems control (Wei et al.) 2024). compared with widely used reinforcement learning and imitation learning approaches. How- ever, they struggle to balance the conflicting goals of achieving closed-loop control and maintaining efficient sampling for long trajectories. Some previous work has focused on improving the adapt- ability of diffusion generation (Zhou et al. 2024) on decision-making tasks, but it is not closed-loop and needs extra hyperparameters and computations for the decision of replanning. In contrast, our method is a closed-loop approach with an efficient sampling strategy without extra hyperparame- ter. Some recent works also assign varying noise levels different frames within a model horizon (Wu et al. 2023 Ruhe al. 2024). The key differences between our work and these approaches are twofold: (I) Our method focuses on the closed-loop control task of complex physical systems, whereas their methods are geared towards sequential content generation that does not involve inter- action with the external world. (2) The diffusion models we need to learn are derived from the target distribution we aim sample from (see Section 4.1 for details), while theirs are heuristic.
3 BACKGROUND
3.1
PROBLEM SETUP
Given an initial state ug, a system dynamics G, and a specified control objective J, We consider the following complex physical systems control problem:
......, min 11sa(w [J WI, Un .... WN, uy)] s.t. u,+1 G(u,, W.+1:
(1)
Here u, E Rd, and w,, E Rddw are the system state and external control signal at physical time step respectively. The system dynamics G represents the transition of states over time under external control in the system, typically determined by implicit PDEs. G could be stochastic with nonzero random noise & deterministic with & 0. The evolution of states can only be observed through state measurement. The control objective J is defined over a trajectory of length N, representing the performance of the control strategy. For example, can be designed to measure the deviation from a target state u", subject to cost constraints: J = ||ux - u u²1 + Σ w| In this paper, we focus on closed-loop control, which means that the control signal 11+1 in each time step is sampled from a distribution conditioned on the current state u, Unlike open-loop control, which determines all actions in advance, closed-loop control continuously incorporates real-time feedback to adjust control decisions dynamically, making it particularly effective for complex and evolving systems where evolution of states can only be observed through measurement of the state. To simplify notation, we introduce a variable z, = [w,, 1r] to represent the concatenation of w, and u,, The transition probabilities in the training trajectories collected offline are assumed to satisfy the Markov property:
p(z,+1 11 [ng 21:7) p(2r+1/uu (u.) p( /u, (u [rr Wr+1).
(2)
3

---

Published as a conference paper at ICLR 2025
Initialization (T = 1)
Closed loop Control (r 2 1)
R-1
1-11
Myrs
M2(9)
System dynamics
System dynamics
System dynamics
0, 14, [9]
Sampling process
Start sampling
End sampling
Control signal
System State
Figure 2: CL-DiffPhy Con for closed-loop control. First, it uses the synchronous diffusion model for initialization. Then, uses the asynchronous diffusion model for iterative control. Sampling of each control signal is based on the latest state feedback from the system dynamics.
3.2 PRELIMINARY: DIFFUSION CONTROL MODELS
DiffPhyCon (Wei et al. 2024) is a recent diffusion generative method to solve the problem Eq.
T for small N in the open-loop manner. In this section, briefly review this framework of its light version. Suppose that we have a training set Dusin containing M trajectories {up (5) (t)
21:N li=1
collected offline. We define the following forward diffusion SDE (Song al. (02I) on the Durann
dz-(1) = f())z=(t)d + g(t)dw, (t), T TE =E(1 [1: N), 1 E [0,7],
where 2-( 2,, ft) and g(t) scalar functions, and (t) is Wiener process Through Eq.
we augment the distribution of Durin denoted as Purain (ug Z1:N) ) = Puain (ug Purain Z1.N [u)) to the distribution of the diffusion process Pe
Pe (U,, Z1:N (t) Prrain (o))p((Zis))
[u)). We have termi-
nal conditions, po (2 (0) un) = Punin I (0) uo) and pr (Z (AL:(() (T) (uo) Rs (() I).
The reverse-time SDE (the denoising/sampling process) Eq. has the following formula:
(t).N(t) = (f()) - V (1) P: (z (t) + g(t)du).N((), t O]. (4)
Once we have learned diffusion model еs which approximates the score function V log (t): we can sample (0: ~ Puain (Z1:N (0) [b)) through Eq. 4 by sustracting
predieded noise from (t: gradually from T (where 21:N(7) ())
The
control objective I (Z):((0) is optimized by the following denoising process (Song et al. 2021 Chung et al. 2023);
dz).N(t) = =[f(t)z(N(() g(t)²V (t) log (2).(
denoise by the score function
(5)
g(1)2> V (21.N dt + g(t)dwr,n(t), t [T.O].
guided sumpling by the contrul objective J
Here, 21-N (0) is the approximate noise-free Z (0) from Zi.N (t) given by Tweedie's estimate. The implementation is specialized by VP SDE (Song et al., 2021 1. i.e. DDPM (Ho et al.) 2020). and the diffusion model is implemented by a parameterized denoising network eg of horizon H, which equals N. Please refer to Appendices A and C.I for more details.
4 METHOD
In this section, we detail our method CL-DiffPhyCon. In Section 4.1 we illustrate our idea and derive the two distributions need learn. To sample from them, present the synchronous
In z, (t). the subscript denotes physical time step, and (f) the parentheses indicates SDE step.
4

---

Published as a conference paper at ICLR 2025
and asynchronous diffusion models in Section 4.2 and Section 4.3
respectively. In Section 4.4
introduce closed-loop control, which is illustrated in Figure The efficiency of CL-DiftPhyCon is analyzed in Section 4.5 For case of theoretical analysis, we adopt the SDE formulation of our method. For the DDPM implementation of its sampling process, please refer to Appendix C
4.1 ASYNCHRONOUSLY DENOISING FRAMEWORK
Recall that DiffPhyCon (Wei et al. 2024) requires latent variables, e.g., system state and control signal, during the reverse diffusion process of horizon H (which is typically much shorter than N) being denoised synchronously. As result, to sample control signal, full denoising process of length 7 over the whole horizon is performed, introducing a large amount of computation over all the latent variables within this horizon. To address this issue, we propose CL-DifiPhyCon, an asynchronous denoising process scheme such that the latent variables of the early physical time step are denoised in advance of the latter ones. At each physical time step, the sampled control signal is input to the system dynamics and the output state serves as the initial condition for the following denoising process. Thus, the control signal planned based on the current system state and closed- loop control is achieved. Meanwhile, the computational cost between two successive times steps is significantly reduced due to the parallel denoising nature, compared with synchronous sampling.
Formally, aim to model the joint distribution p( z (0), (2) Zn Zx(()) (0)Juo) in Markov com- plex physical system, where each (0) [w,, u-] is pair of noise-free free control signal and system #++#1((t)) as the sequence of hidden variables with synchromous noise levels in the horizontal interval [T + - and ër. H - chronous noise levels. Let's consider the augmented joint distribution
state, and ug is the initial state. Denote Z -- (t)
[z, (t), (t))
(t), 1 + H T).... 2
[T)] as its counterpart with asyn-
1 p( ( (0), Z1. | T),
H ... 2N+LNNH - ( (m)))
Through sampling from this augmented joint distribution, and states sequence Z1:N (0) ~ Puzan (Z1:N (1))(())un) (0))uo). By conditioning on previous variables sequen- tially, obtain the following decomposition theorem:
obtain the desired control signals
2xx has Markov prop- erty. For any > we assume z, (T)) independently normally distributed with density (Z (T)) I). The augmented distribution can be decomposed as:
Theorem 1. Assume that the joint distribution p(z, (0),
22) (2),
((Z)N(()) ZI. TT),.. ZN+1IN+H( 1 T))u))
1 11
H
-p(zi.u( 1 (1)))
N
(T)) N(2.+ (7)
II
p( -
2
initializing distribution
transttion distribution
The proofs of this theorem and the subsequent propositions are provided in the Appendix D Note 1 that 21.11 H T) serves as condition for p( ( (0)
T)) when 7 1.
This theorem implies that to sample from the augmented joint distribution, we only need to specify two kinds of distributions for ancestral sampling: the initializing distribution p pZ H (TT) Tu), and
the transition distribution ( (0) (1)) ā 111 (++() for 7 > O.
4.2 SAMPLING FROM INITIALIZING DISTRIBUTION
To sample from p( (H # TT)uu), we train synchronous diffusion model es by the following DDPM loss (Ho et al. 2020
Lynchh Ex, Ille (2(2.. uo. 113)
Here, 21:8 (t) = s(t)zı.H + +(t) P involves the same level of noise for all t E [1 H], and the expectation is about (0, T). (ug, ZI.H) ~ Dirain and Ao I). Intuitively, this model learns to predict the noise in its input (l. After training, we can compute the score function by (ey.e(t) log Pe ((z.:(()/uuu ~ Ec (Z (t), uo. t). Then we apply Eq. 4 to sample a sequence of (t)) where & goes from T to TT, from which we select the diagonal latent variables an ( ) [z) (t).).. zn (T)) following the desired distribution p( J)))).
5

---

Published as a conference paper at ICLR 2025
4.3 SAMPLING FROM TRANSITION DISTRIBUTION
The transition distribution p(2r:r+Ht-100 u, (0), 2z: H- (T converts Zr.r+ H- (+)) to
-r: by denoising over t where goes from T/H to 0, given the state condition -1 To learn such denoising process, first define component-wise asynchronous forward diffusion SDE, which characterizes the diffusion dynamics of #r: with an increasingly higher level of noise as increases from to T/H:
+ f H (t) [f(t), IT) f(t + T) and - lg(t),g(t +
d (t) -1
where
1 T) gt + TT) are vectors of scalar functions that are component-wisely applied. Similar H
to Eq. the reverse-tine time SDE (the denoising/sampling process) of Eq. Haas the following formula:
dă dă 1- (1) = =[- + 11 (1) +
- 2,, 1 1 (1)
log Pe
(10)
dennise by the function
+9r (00)
Thus, to sample 2r:++H11(0) from the transition distribution via Eq. 10 the key prob- lem to estimate the score function (2..)+8-1())
(0))- We in- troduce novel asynchronous diffusion model Co such that c0(2:r+H--100 -n-1 -V=-.---1(()) (/r.:///..t.t./uu-m--m
x This en could also be interpreted as predicting the noise in each component of its input H-:++H11(h To train Eø, need training data of the form H- The following proposition presents a way to sample 2- from the training trajec- tories given the initial state condition (0).
Proposition 1. Assume that the joint distribution p (z) (0), z2 (0) (0))uu) has Markov prop-
erty. For any E [0, # 7) we have:
H-1
-(t ut) = Ex,
(0)
II
(z 4i + H +)),+
(1I)
==
where the expectation is about Zy:r+ 11 -1 (0) Parin (Z (0) u, (0)). and each
1 p(zr+:(t+ // (T)) = (1 + T); s(t T)z++;(0)) s(t + H T)? at r))1)
From this proposition, sample 2T: first sample subsequence 1r: of trajec- tory following the state u0), and then independently sample each component (t " T) of H-1 by adding noise to the corresponding Z, (0)) Then, we train Eg by the following loss:
Lasyach Br,t,(u. 1
( =Ill
Eo (Zr:r+H-1(t)). 1.)
(12)
where the expectation is about UUI 1), U(0, + (u,-1 2 H (t)
Puain (1)-((0)) (2rr+ (f)
0). (0), After training, we can use Eq. where goes from 1 to O, to sample 2. - (0) -
To sample 2 -1 (0) that optimizes the control objective I (Z):N(0),, the denoising process can be performed with extra term representing the guidance of the control objective J:
dž - (1) = (11) (0 (2))
- - 2, 4 log Pt
+ 91 (eI+ (2
(13)
guided sampling by the control objective J
+ Tr: (0)
Here, 2 H (0) 2 given by an asynchronous Tweedie's estimate (implemented in Eq.
El
the noise-free approximation of
(0) s(t+ i
s(++ s(t+ H (t+ Ty² log P. (ž, (t))u
T 1 T)
H
Oz, TT)
(14)

---

through Published as a conference paper at ICLR 2025
Algorithm Closed-loop Control of CL-DiffPhyCon
1: Require synchronous model es. asynchronous model cs. control objective J(-). initial state Umnv, episode length N, model horizon H, full denoising steps 7, hyperparameters A.
2: Initialize 21:11 ( using Denv Eg, and Z1:11 (T) ~ (0, of by Eq.
3: Tort= 1, 1, N do
4: for -
1,... do
update Zr.r+ H-1 using Law,r-1 (0), es, and 2 H by Eq. 6: end for
7: ur (0), w-[(]] (0)
8:
input 10)) and (0) into the system dynamics G, which outputs Uem,r // closed-loop feedback
9:
sample (T) ~ ~N of // append the end of the horizon with noise
10:
4.51 (H ( T), T. 2 (T)]
11:
end for
4.4 CLOSED-LOOP CONTROL
Based on the two learned diffusion models, which together realize the distribution π in Eq. [T] under the guidance of J. We first use the syn- chronous model 6. produce the initial asynchronous variable Z1.H1 ( TT) conditioned
introduce the closed-loop control procedure,
the ini- tial state Uens, Bb by applying Eq. 5 in the horizon [1, H]. Then, the control process starts. At each physical time step 2 1, we sample Hr-: (0) using Uenv,--1 and 2 H1- 1 (+
Eq. 13 We extract the control signal w-(0) from the sampled = and input the pair (Meav, --1 (0)) to the system dynamies G, which outputs the next state uenvr Then we sample noise ( NN, I), and append it to the last H components of #-:++#11(0)
com-
[1 (T). Now take Menv,r-1. instead of the sampled -1 (0), and 2,+1:+4B( T) to the next loop. The whole procedure is presented in Algorithm and illustrated in Figure The closed-loop property of this inference process is analyzed in Appendix B
pose 2.+1:++H H
4.5 EFFICIENCY OF CL-DUPPHYCCO
is clear that CL-DifPhyyon H/h times faster than DiftPhyCon-h, the control method that conducts full sampling process of DiffPhyCon with horizon H every h physical time steps, as illustrated in Figure I Even compared with the adaptive replaning diffusion method (Zhou et al., 2024.. CI-DiftPhyCon is still more efficient because each latent variable is sampled only once and it does not involve extra computation such as likelihood estimation.
Additionally, although some fast sampling methods were proposed recently, such as DDIM Song iet al. (2020). to reduce the sampling cost of diffusion models, our CL-DitPPhyCn still has an independent acceleration effect beyond them. The key insight is that CL-DiffPhyCon adopts the same number of sampling steps T in each physical time step compared to DiftPhyCon (Wei et al., 2024], which brings the opportunity to incorporate fast sampling methods. Hence, by applying them to each physical time step separately in both the sampling processes of the synchronous and asynchronous models, the control efficiency of CL-DifiPhyCon could further enhanced.
5 EXPERIMENT
In the experiments, we aim to answer three questions: (1) Can CL-DiffPhyCon outperform the classical and state-of-the-art baselines? (2) Can CL-DiffPhyCon achieve the desired acceleration
of inference as we analyzed in Section 4.5 and obtain further acceleration by involving DDIM (Song al. 2020 (3) Can CL-DifiPhyCon address the challenges of noise, partial observation, partial/boundary control, and high dimensional indirect control? To answer these questions, we conduct experiments two control tasks: ID Burgers' equation control and 2D incompressible fluid control, both of which have important applications in science and engineering.
In Section 4.4 and Algorithm [] the subscript "env" in Uem.r denotes the state feedback from the system, distinguish De from the sampled state u- (0) by diffusion models.
7

---

Published as a conference paper at ICLR 2025
Table I: Comparison results ID Burgers' equation control. The average control objective (J)
and inference time (averaged across all settings) are reported, using a single NVIDIA A100 80GB GPU with 16 CPU cores. Bold font is the best model and the runner-up is underlined.
noise-free +
physicul constraint
system noise 4
measure noise 4
FOPC
POPC
average
time (8)
BC
0.4708
0.4704
0.4138
0.3981
0.1093
0.0587
0.8356
BPPO
0,4686
0.4507
0.4088
0.3979
0,1079
0.0984
0.8231
PID
0,3250
0.3585
0.2323
0.2911
0.0827
0.7717
DoniPhyCon-1
0.0210
0.0214
0.0222
0.0232
0.0330
0.0332
49.2347
DinfPhyCon-5
0,0252
0.0257
0.0271
0.0272
0,0482
0.0484
10.8308
DiffPhyCon-15
0,0361
0.0365
0.0382
0.0377
0.1128
0.1132
4,9833
RDM
0.0296
0.0296
0.0310
0.0336
0,1124
0.1121
6,9130
CL-DitIPhy Con (ours)
0.0096
0.0110
0.0095
0.0127
0.0291
0.0295
4,5474
EL-DiffPhyCon
0.0112
0.0123
0.0114
0.0146
0.0311
0.0313
0.8257
(DDIM, ours)
5.1 BASELINES
The following classical, imitation learning, reinforcement learning, and diffusion control methods are selected as baselines: the classical control algorithm PID (Li et al., 2006); an imitation learning method Behaviour Cloning (BC) (Pomerleau 1988); recent reinforcement learning method Be- havior Proximal Policy Optimization (BPPO) (Zuang et al. 2023); two diffusion control methods, including RDM (Zhou al., 2024). which adaptively decides when to conduct full sampling pro- cess of control sequence, and DiffPhyCon (Wei et al., 2024). whose original version plans the con- trol sequences of the whole trajectories in one sampling process. Since the trajectory length is much longer than the diffusion model horizon H (H = 16 and H = 15 in 1D and 2D tasks, respectively), we extend DiffPhyCon to its three variants DiffPhyCon-h based on the predefined interval / of physical time steps to conduct full sampling process, All the diffusion control baselines use the trained synchronous diffusion model of EL-DifiPhyCon for sam- pling, with steps T = 900 and T 600 on the 1D and 2D tasks, respectively. PID is inapplicable to the complex 2D task (Aström & Hägglund 2000p. RDM is reproduced following the official code However, the default values of two thresholding hyperparameters in RDM do not perform well on our tasks. Therefore, we select a pair of values that perform best (see Appendix K for details). For
{1, 5, H - 1}) as our baselines,
other basclines, we follow the implementations in DiftPhyCon Wei et al. 2024
5.2 1D BURGERS' EQUATION CONTROL
Experiment settings. The Burgers' equation is widely used equation to describe a variety of phys- ical systems. We follow the works in [Hwang al. (2022); Mowlavi Mowlavii & Nabi (2023) and consider the ID Burgers' equation with the Dirichlet boundary condition and external force wi [T, .) as follows:
Uu u Un dr u(T,x)) 0
in [0, T] x Q w((T
in [0, T an,
(15)
w(0,
x)- (x)
in {- 0} x 2
where v is the viscosity, and un(x) is the initial condition. Given a target state ua(T, a) defined in range [0, T] x Ω the control objective J is to minimize the error between u(T, x) and ua(T, z):
J= J= (- - ua(T, (2 dzdr,
(16)
subject to Eq. 15 We explore four kinds of settings from real-world considerations: (1) noise- free control; (2) control under physical constraint with limited range of allowance for control actions; (3) control under random system and measurement noise, respectively; (4) partial control (PC) where actuators are limited to control approximately 1/8 of the full spatial domain, which is further divided to full observation (FOPC) and partial observation (POPC) with sensors in 1/8 of the full spatial domain. Details of these settings are provided in Appendix F22
Results. In Table T present the results of our proposed CL-DifiPhyCon and baselines. Note that the reported metrics in different settings are not directly comparable. It can be seen that CL- DoffPhyCon delivers the best results compared to all baselines in all settings. BC and BPPO perform
8

---

Published as a conference paper at ICLR 2025
Table 2:
Comparison results 2D incompressible fluid control. The average control objective (
J ) and inference time (averaged across all settings) are reported, using a single NVIDIA A6000 48GB GPU with 16 CPU cores. Bold font the best model and the runner-up underlined.
Large domain control
Boundary control
Average
Fixed map
Random mup.
Fixed mapl
Random map!
time (8)
BC BPPO DiniPhyCon-I
0.6722
0.7046
0.8861
0.8871
4.67
0.6343
0.6524
0,8830
0.8844
4.65
0,5454
0.3754
0.7517
0.7955
1666.50
DinlPhCCon-5
0,5051
0.5458
0,6703
0,7451
357.66
DinPhyCon-14
0.5823
0.5621
0.6498
0.7221
141.88
RDM
0.4074
0.4356
0.6553
0,7087
238.43
CI-DirTPhyCon (ours)
0.3371
0.3485
0.6169
0,7003
144.04
CI-DDiPPhyCon (DDIM, ours)
0,4100
0,4254
0.6671
0.7109
26.01
poorly on this task as they rely heavily on the quality of training control sequences, which are far from optimal solutions (see Appendix F.I).. The diffusion control baselines perform better, com- pared with BC and BPPO, because diffusion models conduct global optimization over each horizon through the conditional generation and excel in sampling from high dimensional space (Wei et al.) 2024). Specifically, CL-DiffPhyCon decreases J by 54.3% and 48.6% compared with the best base- lines (DiffPhCCnn-l in the noise-free and physical constraint settings, respectively. Furthermore, when the system or measurement is perturbed by noise, our method achieves improvements compa- rable to those in the noise-free case, which decreases J by 48.6% and 57.2%, respectively. More- over, despite the reduced controllable range in FOPC and POPC settings typically diminishes the effectiveness of diffusion-based control methods. CL-Difthyyon maintains superior performance over all baselines with at least 11.8% and 11.1% decreasing on J. BC, BPPO, and PID demonstrate improved performance because only a small part of actions need to optimized, making it easier to fit compared to high-dimensional actions, while they still show significantly lower performance compared to diffusion-based methods. Visualizations are presented in Appendix [] they show that CL-DiffPhy achieves lower error regarding the target state compared with baselines. In terms of inference time besides the enhanced control performance, CL-DifiPhyCon brings significant ac- celeration of the sampling process. about H/h times faster than DiffPhyCon-A and two times faster than RDM. Furthermore, by combining with DDIM of 30 sampling steps, CL-DDifPhyCon achieves an additional 5x speedup, demonstrating the independent acceleration effect of CL-DifiPhy from existing fast sampling methods of diffusion models, As a result, the reduced time cost of CL- DiffPhyCov is comparable with those non-diffusion control baselines BC, BPPO, and PID, while the performance is much superior.
5.3 2D INCOMPRESSIBLE FLUID CONTROL
Experiment settings. Our experimental setup is based on previous studies of Holl et al. (2020); Wei et al. (2024). Given an initial cloud of smoke in a 64 x 64 incompressible fluid field, this task aims to minimize the volume of smoke failing to pass through the top middle exit (seven exits in total) over N = 64 physical time steps, by applying sequence of 2D forces outside the outermost obstacles. This task represents a simplified scenario of real-life applications such as indoor air quality control (Nair al. 2022). We consider two settings from real-world consideration: large domain control, where control signals are applied to all peripheral regions consisting of 1,792 cells outside the outermost obstacles, and boundary control, where control signals restricted to only the 4x8 cells inside the four exits. is very challenging since the control forces can only be exerted indirectly in the peripheral regions, which necessitates the model to plan ahead prevent the smoke from entering the wrong exits getting stuck in corners. The boundary control setting is even more challenging due to the significantly reduced range of influence of the controllable cells. During inference, for both settings, we follow Zhou et al. (2024) and add = 0.1 probability of random control the execution of control signals. Random controls cause unexpected changes in the system state, which may render previously planned control sequences no longer applicable, thus necessitating re- re-planning and adding additional challenge to the task. For both settings, besides fixed map (FM) evaluation mode where test samples use the same obstacles' configuration with training trajectories, we also introduce a random map (RM) mode where the obstacles' configuration varies. For details of experimental settings and implementation, please refer to Appendix

---

Published as a conference paper at ICLR 2025
Fixed map
CL-Dirihyyoon (ours) (J 0.135)
(J= 0.565
Random map
CL-DirPhyyon (ours) (J (J= 0.0727)
DilPhyCon-1 (J 0.2484)
t-10
t-40
t=50
Figure 3:
Visual comparisons between CL-DifiPhCCn and best baselines on 2D fluid control under fixed map (top) and random map (bottom) evaluation modes, respectively.
Results. Table shows the control performance of CL- CL-DifiPhyCon and baselines. The results in- dicate that CL-DifiPhyCon outperforms the baselines in both large domain control and boundary control settings, under both fixed map and random map evaluation modes. This validates that asynchronous diffusion model could effectively sample appropriate subsequent control sequences by conditioning on the changed system state closed-loop manner, showcasing strong generaliz- ability. The results also demonstrate the advantage of diffusion- control methods over BC and BPPO. Figure 3 we illustrate randomly selected test samples controlled by CL-DifiPhyCon and the best baselines in the large domain control setting. CL-DifiPhyCon exhibits a stronger capability of adapting to changed fluid and maps. More visualization results are presented Appendix J] We also present the comparison of average inference time in Table 2
Aligning with our observation on the ID task, CL-DDifPhyyoon achieves approximately H/h times speedup compared to DiffPhyCon- h. Besides, CL-DifIPhyCon is much more efficient than RDM by avoiding frequent replanning. By adopting DDIM, the inference is further accelerated, over faster than strong diffusion-basee control baseline RDM, with comparable control performance. For detailed study of the effect of the model horizon H, please refer to Appendix G.3
CONCLUSION AND LIMITATION
In this paper, we propose CL-DifiPhyCon. a novel diffusion-based method for closed-loop control of complex physical systems, grounded theoretical analysis of the target distribution. Experiments on two physical control tasks demonstrate its superior performance and efficiency. Still, it has sev- eral limitations that offer opportunities for future work. First, CL-DiifPhyCon is currently trained offline without interacting with the system dynamics. Incorporating real-time feedback during train- ing could enable dynamic adaptation and the discovery new strategies. Second, while the two diffusion models in CL-DifIPhyCon are theoretically derived, formal bound on its optimization performance under guidance sampling remains an open question, providing a promising direction for further theoretical research. Finally, the effectiveness of CL-DifiPhyCon in more domains is worth exploring. Although our method is designed for the control of complex physical systems, holds promise for broader applications, such robot control and drone control.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We gratefully acknowledge the support of Westlake University Research Center for Industries of the Future and Westlake University Center for High-performance Computing. This research funded by the Outstanding Postdoctoral Program Funding Project of Zhejiang Province. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding entities.
REFERENCES
Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B Tenenbaum, Tommi S Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision making? In The Eleventh Interna- tional Conference on Learning Representations, 2022.
Karl Johan Aström and Tore Hägglund. The future of pid control. In Digital Control Past, present, and future of PID Control, United States, 2000. Elsevier.
Gerben Beintema, Alessandro Corbetta, Luca Biferale, and Federico Toschi. Controlling rayleigh- bénard convection via reinforcement learning. Journal of Turbulence, 21(9-10):585-603 2020.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shu- ran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04133 2023.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference on Learning Representations, 2023. URL httttss:/epeereeiieeeeee
Forum?1d-OnD92GAGC
Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de Las Casas, et al. Mag- netic control of tokamak plasmas through deep reinforcement learning. Nature, 602(7897):414- 419, 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8777 2021.
MA Elhawary. Deep reinforcement learning for active flow control around a circular cylinder using unsteady-mode plasma actuators. arXiv preprint arXiv:2012.10165, 2020.
Haodong Feng, Yue Wang, Hui Xiang, Zhiyang Jin, and Dixia Fan. How to control hydrodynamic force fluidic pinball via deep reinforcement learning. Physics of Fluids, 35(4), 2023.
Haodong Feng, Dehan Yuan, Jiale Miao, Jie You, Yue Wang, Yi Zhu, and Dixia Fan. Efficient navi- gation of robotic fish swimming across the vortical flow field. arXiv preprint arXiv:2405.14251. 2024.
Elie Hachem, Hassan Ghraieb, Jonathan Viquerat, Aurélien Larcher, and P Meliga. Deep reinforce- ment learning for the control of conjugate heat transfer. Journal of Computational Physics, 436: 110317, 2021.
Michael Hesse, Julia Timmermann, Eyke Hüllermeier, and Ansgar Trächtler. A reinforcement learn- ing strategy for the swing-up of the double pendulum cart. Procedia Manufacturing, 24: 15-20, 2018. ISSN 23519789. doi: 10. 1.1100/-rrooff.e. (atmff.2011806.0000 20.6.00.004.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi. and David J Fleet. Video diffusion models. Advances Neural Information Processing Systems, 35:8633- 8646, 2022.
11

---

Published as a conference paper at ICLR 2025
Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning control pdes with differentiable physics. arXiv preprint arXiv:2001.074577 2020.
Rakhoon Hwang, Jae Yong Lee, Jin Young Shin, and Hyung Ju Hwang. Solving PDE-Constrained Control Problems Using Operator Learning, AAAI, 36(4):4504 4512, June 2022. ISSN 2374- 3468, 2159-5399, doi: 10.1609/aai.. v3614.20373 URL https: aaai orggindexx
/ /ttp//AA/aaaaacleeeee/ee///////a
Liviu Gr. Ixaru and Guido Vanden Berghe. Runge-Kuta Solvers for Ordinary Differential Equations, pp. 223-304. Springer Netherlands, Dordrecht, 2004. ISBN 978- 978-1-4020-2100-8. doi: 10.10 978-1-4020-2100-8..1 8.6.
0077 http://00000..11100000001-11--10
Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. Proceedings of Machine Learning Research, 162:9902-9915, 17-23 Jul 2022a.
Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162, pp. 9902-9915. PMLR, 17-23 Jul 2022b.
Leslie Pack Kaelbling and Tomás Lozano-Péree Pérez. Hierarchical task and motion planning the now. In 2011 IEEE International Conference on Robotics and Automation, pp. 1470-1477. IEEE, 2011.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 11-26565-26577, 2022.
Yun Li, Kiam Heong Ang, and G.C.Y. Chong. Pid control system analysis and design. IEEE Control Systems Magazine, 26(1):32-41, 2006. doi: 10. 10000//M
Saviz Mowlavi and Saleh Nabi. Optimal control of pdes using physics-informed neural networks. Journal of Computational Physics, 473:111731, 2023,
Ajith N Nair, Prashant Anand, Abraham George, and Nilabhra Mondal. A review of strategies and their effectiveness in reducing indoor airborne transmission and improving indoor air quality. Environmental Research, 213: 13579, 2022.
Guido Novati, Siddhartha Verma, Dmitry Alexeev, Diego Rossinelli, Wim M Van Rees, and Petros Koumoutsakos. Synchronisation through learning for two self-propelle swimmers. Bioinspira- fion & biomimetics, 12(3):030011, 2017.
Dean A Pomerleau. Alvinn: An autonomous land vehicle in neural network. Advances in neural information processing systems, 1, 1988.
Ilan Price, Alvaro Lanchez-Gonzaler Ferran Alet, Timo Ewalds, Andrew El-Kadi, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Gencast: Diffusion-basse ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312. 15796, 2023.
Jean Rabault, Miroslav Kuchta, Atle Jensen, Ulysse Réglade, and Nicolas Cerardi. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of fluid mechanics, 865:281-3022 2019.
Ricardo Reyes Garza, Nikos Kyriakopoulos, Zoran M Cenev, Carlo Rigoni, and Jaakko VI Timonen. Magnetic quincke rollers with tunable particle dynamics and collective states. Science Advances, 9(26)eadh2522, 2023.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
12

---

Published as a conference paper at ICLR 2025
David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion mod- els. In Forty-first International Conference on Machine Learning, 2024. URL https:
Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
Max Schwenzer, Muzaffer Ay. Thomas Bergs, and Dirk Abel. Review model predictive control: An engineering perspective. The International Journal of Advanced Manufacturing Technology, 117(5-6):1327-1344) 2021.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv:2010.02502,, October 2020.
0250
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learming Representations, 2021. URL
forum?1@=PXTIG12RRB
Siddhartha Verma, Guido Novati, and Petros Koumoutsakos. Efficient collective swimming by harnessing vortices through deep reinforcement learning. Proceedings the National Academy of Sciences, 115(2)):589-5554, 2018.
ZP Wang, RJ Lin, ZY Zhao, X Chen, PM Guo, N Yang, ZC Wang, and DX Fan. Learn to flap: foil non-parametric path planning via deep reinforcement learning. Journal of Fluid Mechanics, 984: A9, 2024.
Long Wei, Peiyan Hu, Ruiqi Feng, Haodong Feng, Yixuan Du, Tao Zhang, Rui Wang, Yue Wang, Zhi-Ming Ma, and Tailin Wu. A generative approach to control complex physical systems. arXiv preprint arXiv:2407.0644 06494, 2024.
Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. Integrating physics-based modeling with machine learning: A survey. arXiv preprint arXiv:2003.0499 1(1):1-34, 2020.
Tailin Wu, Takashi Maruyama, Long Wei, Tao Zhang, Yilun Du, Gianluca laccarino, and Jure Leskovec. Compositional generative inverse design. In The Twelfth International Confer- ence on Learning Representations, 2024. URL /ttpp::/openreviev. wet.:iewwwweeeeooooeeeeee wwt./oruuffiii..
wmX0CqFSd7
Tong Wu, Zhihao Fan, Xiao Liu, Hai-Tao Zheng, Yeyun Gong, Jian Jiao, Juntao Li, Jian Guo, Nan Duan, Weizhu Chen, et al. Ar-diffusion: Auto-regressive diffusion model for text generation. Advances in Neural Information Processing Systems, 36:39957-39974, 2023.
Masaki Yamakita, Kenichi Nonaka, and Katsuhisa Furuta. Swing up control of double pendu- lum. In 1993 American Control Conference, pp. 2229-2233, San Francisco, CA, USA, June 1993. IEEE. ISBN 978-0-7803-0860-2. doi: 10.2991/AACC.19934799779 // //ieeexplore leee. hwwp:///cuuettto /ttp://rooe.ooooooee//////222220 /7933779
URL https:
Cagatay Yildiz, Markus Heinonen, and Harri Lähdesmäki. Continuous-time model-based reinforce- ment learning. International Conference on Machine Learning, pp. 12009-12018, PMLR, 2021.
Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit- Van Yeung, and Chuang Gan. Adaptive online replanning with diffusson models. Advances in Neural Information Processing Systems, 36, 2024.
Zifeng Zhuang, Kun LEI, Jinxin Liu, Donglin Wang, and Yilang Guo. Behavior proximal policy optimization. In The Eleventh International Conference on Learning Representations, 2023. URL https:://penrewwee net/ nett/forummideee133lootppp
13

---

Published as a conference paper at ICLR 2025
A BASIC PROPERTIES OF STOCHASTIC DIFFERENTIAL EQUATION (SDE)
In this work, we use the formulas of stochastic differential equation (SDE) to express diffusion models. Specifically, the SDEs that
be written the following unified form:
dx(t) ((t)x(t)dt g(t)dw(tt)
(17)
The reverse-time SDE of Eq. 17 is:
dx(() (t) V x() log Pa (x(t)) Jtt (())
(18)
In this section, summarize some basic properties of Eq. which also be found in existing
SDE works Song et al. (2021); Chhng et al. (0233: ; Karras et al. (2022).
First, such linear SDE has the closed solution Särkkä & Solin (2019)
x(t) = x(0) expl / f(n)dn) + L." exp( L. (()))))s((()dd((()
(19)
where the second term is Itô integral. And the mean m(l) and covariance V(I) satisfies the following ordinary differential equations:
dm(t)
dt = f(t)m(t),
(20)
dV(I)
2((()((()) + 9()))
dt
Solving these equations, get:
m(t) s(t)m(0), (I) =(1) =())((())++++())
(21)
where stt) exp (So (()) and (t) = 1o gin)² dn. a(n)>
Tweedie's estimate. From Eq. 19 and Eq. 21 we know
x(t) x(0) ~ // (s(t)x(0),
(22)
According to the density function of normal distribution, we have:
V Vx(x) (og)(x(())(())) = s(t) ((t)) - -()))
(23)
Take the expectation over x0) conditional on x(t) on both sides:
(())
Ex(o) 1og
/ VX(1 (x0pbor/kk((0))0)0000000000000))
(24)
/
V x()) log (x(t)).
Rearranging the equation, we get Tweedie's estimate:
x0) A {[x(0))x()) s(t) -x(() + ((t) x(e) logp(x(t)).
(25)
Intuitively, Eq. 25 estimate the noise-free Xg given X,-
Approximate score function by diffusion models. Now we consider the following loss function of DDPM (Ho et al. 2020 train the diffusion model es::
Ex,( es (=())x()) (,
x(t) (())s(t)
(26)
=E, =E =0)
(())2 es (x(t), 13]
14

---

Published as a conference paper at ICLR 2025
The best prediction Es (s, in this loss is the following conditional expectation:
co- (x(1),t) -EI x(1 - x(0)s(()
s(t))o(t)2 - cx(x(t),))x(())
(27)
x(t) ((((()((()s(() (1)
Plugging Eq. 25 in it, we get the score function:
Vx(() logp(x(t)) cs- (x(t),
(28)
B VARIANCE PRESERVING (VP) SDE
In the practical training and inference, we use Variance Preserving (VP) SDE Song et al. (202
which is actually the continous version of DDPM et al. 2020).
VP SDE specializes Eq. 17 to
1 dx(() = - 3(t)((t)dt + BB()dw(tt)
(29)
where ((1) > for E [0, T.. And its reverse-time SDE
dx(t) = 1 1- 8()x((() 3(t) (xg)(( + 33(t)du(t),
(30)
In algorithm implementation, we use K time steps t, = & for = 0,....,, 1, to discretize
Eq. 29 Using At
K T, we have:
x(4: Al) x(4) 1 (())AI((() VB(G)AES
(31)
111-8(110 ((t)((() + VB(C)AEE.
where { (O,
Let 8 At, = x(t:). We get the discrete forward iterative equation, which is exact DDPM:
x++1 = - Bix, + VB.g.
(32)
Similarly, Eq. 30 can be discretize to
x(4 A1) = x (t.) + ((4))I((()) + ((4))/ x(t.) log log.r,(((11))) + /B(())
(33)
B(L)A x (t) + +((t))tV ((.)) log (x((,)) + AB(t.)AtE.
So, we get the discrete backward iterative equation:
X
V1 3 x; + 3Vvx logp(xi) + VBgg.
(34)
II this section, time step means diffusion step, denoted by the subscript rather than physical time step in the main text.
15

---

Published as a conference paper at ICLR 2025
Additionally, we have:
s(t,)² = exp
-3(n)dn)
exp
-(())a1)
II exp (- 8(())A)) 2=0
(35)
II ((t))
(1 33)-
j=0
And, we also have:
fettr -3(n)dn)
3(1) (." s(()d))
(tt))o(tt)) exp
to
B(7))
ex))ex((
-(()dc)dn
-3(1)) ( (f, P4-21
2
exp
- -3(()dc)dn)
20
(36)
exp
-3(1)dnn)
80
1 - stt))
1 II (1 -3)- 1=0
We define a; =1 Bg and a; I[ ag, Then, we get s(t,) ā, s(t:) 1 a; and xi/Xo- N(Vaxxo, (1 I). Therefore, Tweedie's estimate becomes
1 (x; (1 a.. Vx log pepp(xx (x):).
x0))
(37)
v
0
According Eq. 21 the covariance x; is (1 - a;) + Vo- Assuming the clear data has covariance I, we can start from Xx ~ (0, I) during inference.
C DDPM IMPLEMENTATION OF CL-DIFFPHYCON
DDPM (o et al. 2020 a widely adopted implementation of diffusion models. Based on
general DDPM derivations in Appendix B and SDE derivations of CL-DDffPhyCon in Section here
we also present the DDPM implementation of our CL-DiffPhyCon. By choosing f(t) / (1),
g(() /B(L) and following the values of ar 's and 3 's specified in Appendix
B we have the
following implementations.
C.I DDPM SAMPLING FROM THE SYNCHRONOUS DIFFUSION MODEL
By using the approximation V V (1) log Pt (ZI.N (t) un) z -Es (ZI. ug t) and Eq. 34 the
DDPM version implementation of the sampling process in Eq. 5 (from DinfPhyCon(Wer et al.
16

---

Published as a conference paper at ICLR 2025
2024 is
Z1:N (t -
21:N (t) + 3e V.Vn ( log Pa (21.N (t)(ua)
VT B3
-B - V ( J (21.N + BBC
(38)
2IN (t) - (2
(2 (t), uo,
V1 B
3,A V V (0)) + B.E.
where (0)
A (0) can be computed by the Tweedie's estimate according to Eq.
37 as follows:
(0: = Var
(t):N(t) - (1 @.)ev(Z)N(()), uo, t))
(39)
implementation, to sample Z ( T) as in Line Algorithm we run Eq. I t= to / T/H and use H to replace N.
38
iteratively from
C.2 DDPM SAMPLING FROM THE ASYNCHRONOUS DIFFUSION MODEL
Similarly, the DDPM version implementation of sampling process Eq. is
2 H 1tt---11111111----1- 1) = 2r: + 8r Vä,.. 2.. 18 (()) log Pa
V1 Be
3, V
2nrr 1 (2 (2 (0)) + BiE
(40)
VI Bi 2 2 - BrEa 12 uo.
3,A V
Vz..r-n-100) (2r:T+ (0)) / Brc.
where each component 2,( of 2 و (00) = (z, (0), 2, (0), ... z (0)) be computed by the Tweedie's estimate according to Eq. 14 and Eq. 37 as follows:
2 = /ā
(2:(1) (1 d(+4g)6))
(41)
for i- 0, 11 1. Here e; denotes the i-th component of the predicted sequence of noises 6o(2r:++ (t), u,-1, [eg. = by the asynchronous diffusion model Eg. The reason why each component of (0) of H- is estimated separately is that different components of 2r:++H-1(t) have different levels of noise, each corresponding to different scalar coefficient 0.144.
17

---

Published as a conference paper at ICLR 2025
D DERIVATIONS
Proof of Theorem
We first conduct the following decomposition:
p(Z1:N(0) 21.11(
|
1
T).
ZN+IIN+H 11
1 ( T))o))
H
-p(2
T). 21 (0), 22
H ..)......N 2N+IN+H -
H
H
=p(21.1( T)u0))
N
II ==1
p z, 20), T) un. ZI.H( T), 210), 22,, (T),
T),
2, ..
10))
-1 (1)
=p( T1/n)
H
N
1
II T=1
ZZ.++ (()) 21 ZI.H 1), 22,01 (T)) 2
=p(2i.1( 11
(u)) II N (T)) r=1
N
1 T). (T)22(())) 22.1111 1 T).... II p(2 2ns H
1
( T)).
1
Zr Zr: (),(),,))))))))))))
H
(42)
The third equation holds since [z, 2 [T] and -2::)+ 2, (T) are two different arrangements of a same vector [2,(0), 2++1 (T ........ -- H- T), 2 The last equation holds since (T) is independently normally distributed with den- sity (7), o2 for any т. Then, we analyze the conditional probability p(2r:++t=1 21-H (),, (0), 22.1+1
... (0) 21 TT) in detail.
Due to Markov property of physical systems, when u, - (0) appears, other variables with temporal subscripts less than can not affect those with subscripts greater than or equal to Therefore, we can remove those variables with subscripts less than v, except for (0). Thus, we have:
p( - up: 2I. T), (0), 22.H+1 (1 T) ... (0), 2 - (T T))
(43)
(00 1
where c denotes the subset of latent variables (T),,, 2r:r+1 ( -Z:r+H-22 T))
extracted from 1 (T), (0), 20),2r)))+111 2- Further, by adding new variables -1) to this probability, we have
(00 T))
= - (, Zr.r 2re4
(44)
p( 210 -
18

---

Published as a conference paper at ICLR 2025
The first distribution inside the integral in Eq. 44 can be simplified follows:
-1)))
c, |, (0)-
Cc 10)
2, 10 p -
(45)
( -
-00 1, (1 -)2)+++)10(0)))))
In the second equation, we use the Markov property of the diffusion process of zz( over and skip step including the following two equations
1 1 T)) p(cl (T)
Z.. (,,
1 ( = p(c/ (T)
1
2,
in the numerator and denominator respectively of the right side of the second equation. Similarly, we show that we can also simplify the other distribution inside the integral in Eq. 44
1
2 2r: T))
p(z H-1 2
- p(c, 2c,
17 T') u, -1
plz (f u, (0)
(( (0)
(0)
(2. 11 (0) p 8 ( (0))
(0000-111100000--0000 20r
T))-
(46)
Combining Eq. 43 to Eq.
46 we get the following result:
H T), 21 (0), 2 H T). ... "., ,2--1(0) 2 - T))
= / pl (0) 20) T): 2
1
(Z)))zzzrr 11 -)0
(47)
H
= (00 2 -
100
Plugging this to Eq. 42 we prove the conclusion.
19

---

Published as a conference paper at ICLR 2025
Proof of Proposition Adding variables Hr:++H-100), we have:
(1)
(48)
=
Notice that the distributions of {2r+1(t + H [=) are conditional independent. Therefore, we reformulate the above equation to:
Ps - (1):----1())
p(zr+o(t+ uT)
p (trtsltt
H-1
i T) [))=+:(0)))+ p( + H
E
II 2=0
(49)
Here, the second equation is because (t+i(t= # T) only depends on 2++:(0) with the distribution Eq. [2 described in Section A And we have proved the desired conclusion.
CLOSE-LOOP PROPERTY
The joint distribution of {wr(0) Uemv W. Algorithm satisfies the following proposition. Proposition 2. Using inference described in Algorithm p( ((), 1e. (w( Menv, N (env.))
the following holds:
= - Poal 21.11 {
11 Uem.o))
N
(50)
II Ped (2)) 2 H 7") pa (Ueme,/uun,,r-11 Hon,, w.(,),)))))))) T=1
(T M, T),... (.n.,,uuN()) 2N))) T)),
where Pyd denotes the transition distribution of the guided sampling (Eq. 13 and pG denotes the transition distribution of the system dynamics G.
According to Eq. 50 for every T, the control signal (0) conditional on the states Uenv,0:r- instead of the predicted (0)r Therefore, our method achieves closed-loop control.
Proof of Proposition This a direct conclusion from the process of Algorithm Among vari- ables in the history, only Lemv,--H:--1 +1- T)) {z--1( T) y=o involve the generation of w, (0). So, get the conclusion.
F DETAILS OF ID BURGERS' EQUATION CONTROL
F.1 DATASET
follow instructions in Wei et al. (2024) to generate a ID Burgers' equation dataset. Specifically, we use the finite difference method (FDM) to generate trajectories in domain of space range I E [0, 1] and time range 7 E [0. 1], with random initial states and control sequences following certain distributions. The space is discretized into 128 cells and time into 10000 steps. We generated 90000 rajectories for the training set and for the testing set. For each training sample, its target state (=(7 x) is randomly selected from other training samples, which means that almost all the
20

---

Published as a conference paper at ICLR 2025
samples in the training set are unsuccessful as they could hardly achieve the target states under random controls. Thus it is challenging to generate control sequences with performance beyond the training dataset.
F.2
EXPERIMENTAL SETTINGS
Similar to Wei et al. 2024). we design different settings of ID Burgers' equation control in Section 5.2
This is a scenario where all states u(T, 2), E [0, 1] for r E [o, 1] of the system can be observed. The system, control, and measurement do not have the noise.
Noise-free:
Physical constraint: In real-world scenarios, the actuator often has the upper and lower limits due the physical constraints. We limit the control with bounds -2 to 2 in this setting, while unconstrained control ranges from -5 5.
For complex engineering systems, the practical physical systems often have noise issues in the system (plant) and actuator (control). We consider such real-world scenarios and perturb the system by Gaussian noise with standard deviation o 0.025 following the work (Yildiz al., 2021). It can also simulate the control noise setting as the noise added to the external force can be decomposed and considered the system noise for the ID Burgers' equation in Eq. [5| Specifically, for a deterministic system: Burgers' equation, Go denotes the state variable and represents the control. To account for system noise, we augment the
System noise:
du 8'u 8h = -u 3e + 3z + w(T,x), where u Ø7
th dynamics with additive Gaussian noise: G du u de + v Dz + w(T x) + Er ~ NN0 o). 07
During numerical simulation, the perturbed system (Gi) supersedes the nominal system (Go) when implementing the control input w. Consequently, the feedback state u is obtained as the solution to the stochastic PDE (G1).
The measurement noise is also a common phenomenon in practical physical systems. We consider the measurement with Gaussian noise. The feedback state is characterized by the superposition the nominal solution u to system (Go) and a Gaussian noise term & ~ [( o) (a 0.025 in the experiments), thereby incorporating measurement noise into the feedback loop while preserving the dynamics of the nominal system.
Measurement noise:
In general, the sensors for observation and actuators for control are located small part of the spatial domain, unlike the full observation and full control (FOFC) woise-free setting. In this setting, we consider two observation cases: full observation and 16 sensors observation (1/8 of the spatial domain). Both cases are controlled by 16 evenly placed actuators. Although only partial observations are available, the reported results are calculated for an entire spatial domain, including unobservable parts. (As a common single-input single-output (SISO) controller, the PID controller is difficult to directly apply to multiple-input multiple-output (MIMO) systems. Its application to the MIMO system often requires additional decoupling and tar- get planning modules, and the control effect of PID is greatly influenced by decouplers and planners, making it difficult compare fairly. Therefore, we only apply the PID in 16 sensors for observation and 16 actuators for control.) The placement of sensors and actuators illustrated in Figure
Partial observations and partial control:
Half domain: In this setting, hide some parts of and measure the T of model control as the white area with oblique lines shown in Figure 11 Specifically, u(T, 2), x E [1- 31 is set to zero in the dataset during training and un(x), x E i | is also set to zero during testing, Only Ω = 10, 31 Ul 12, 1] is observed, controlled and evaluated.
F.3 IMPLEMENTATION
We use U-Net (Ronneberger et al. 2015) as architectures for the models eg and cg. Two models are separately trained using the same training dataset. Note that in the partial observation settings, the unobserved data is invisible to the model during both training and testing as introduced in Appendix We simply pad zero in the corresponding locations of the model input and conditions, and also exclude these locations in the training loss. Therefore, the model only learns the correlation between the observed states and control sequences. We use the MSE loss train the models. Both models have T = 900 diffusion steps. The DDIM (Song et al. 2020) sampling we use only has 30 diffusion steps with the hyperparameter 1 1. Hyperparameters of models and training are listed in Table
F22
21

---

Published as a conference paper at ICLR 2025
noise-free (FOFC)
FOPC
POPC
Observation sensors
Control actuators
X
Figure 4: Illustration of partial observation and control. Blue represents the position observed by sensors, and red represents the position that actuators are able to control. The region filled entirely with blue and red indicates that all spatial regions can be observed and controlled, respectively. The vertical line indicates where there are sensors and actuators in that spatial position.
Table 3: Hyperparameters of models and training for 1D Burgers' equation control.
Hyperparameter name
Value
U-Net (w(w)
Model horizon H
16
Initial dimension
64
pransampling/pssapphnn layers
4
Convolution kernel size
3
Dimension multiplier
2,8)
Attention hidden dimension
32
Attention heads
4
U-Net cu(u, w)
Model horizon H Initial dimension
16
64
Downsampling/ U Upsampling layers
4
Convolution kernel size
3
Dimension multiplier
4,88
Attention hidden dimension
32
Attention heads
4
Training
Training batch size
16
Optimizer Learning rate Training steps Learning rate scheduler
Adam
le-4
190000
cosine annealing
Inference
Synchronously sampling steps
900
Each asynchronously sampling step
60
F.4 RESULTS OF HALF DOMAIN
In this subsection, we consider another noise-free setting that only observe, control, and evaluate on half of the spatial domain. From the results in Table 4 our conclusion is consistent with the noise-frec scenario, and CL-DiftPhyCon shows significant improvements both performance and efficiency.
22

---

Published as a conference paper at ICLR 2025
Table 4: The average control objective I of 1D Burgers' equation control in the half domain, and inference time on single NVIDIA A100 80GB GPU with 16 CPU cores are reported. Bold font is the best model and the runner-up underlined.
J 4
time (8) 4
BC
0,2558
0.7150
ВPPO
0.2033
0.7342
PID
0.2212
0.7236
DinlPhyCon-I
0.0196
39.9720
DillPhyCon-5
0.0184
8.4481
DifiPhyCon-15
0,0192
3.8935
RDM
0.0196
9.8153
CL-DifPhyCon (ours)
0.0090
9.7516
CL- D-DDitPPyCoo (DDIM, ours)
0.0104
0.7628
64
MA
Edit
Target eott
Controllable Area
Controllable Area
Flxed locations and obstadle
controllable
Locations cantrolublle
large boundary settings
large
boundary setting
Fixed map (training data and in-distribution test)
64
5A
Edit
Target exit
Controllable Area
Controllable Area
Fixed locations of evits random locations obstacles large boundary settings
controllatle
Locatiors control boundary setting
large control setting
Random map (out-of-distribution test)
Figure 5: Illustration of the 2D incompressible fluid control task settings. In the large domain control setting (middle), control signals are applied to the peripheral green regions surrounding the obstacles. In the boundary control setting (right), control signals are limited to the green cells within the four exits. Two evaluation modes are used: fixed map (top) for in-distribution testing and random map (bottom) for out-of-distribution testing.
23

---

Published as a conference paper at ICLR 2025
Table 5: Hyperparameters of 2D experiments.
Hyperparameter Name
Value
Number of attention heads
4
Kernel size of conv3d
(3.3,3)
Padding of conv3d
(1,1,1)
Stride of conv3d
(1,1,1)
Kernel size of downsampling
(1,4,4)
Padding of downsampling
(1,
Stride of downsampling
1,
Kernel size of upsampling
(1,4.4)
Padding of upsampling
21,
Stride of upsampling
(0, 1)
G DETAILS OF 2D INCOMPRESSIBLE FLUID CONTROL
G.1 EXPERIMENTAL SETTING
Dynamics of 2D incompressible fluid follows the Navier-Stokes equations:
dv + Vv -V + Vp
vv= v(,, x) vo(x),
(51)
where denotes the external force, p denotes pressure, denotes the viscosity coefficient and denotes velocity. (x) is the initial condition. We follow the setup of the 2D incompressible fluid control task described in (Wei et al. 2024). using the Phiflow solver Holl et (2020) simulate fluid dynamics. The resolution of the 2D flow field is set to 64 x 64, and the flow field is unbounded. We consider two settings: large domain control and boundary control. In the large domain control setting, control signals are applied to all peripheral regions outside the obstacles, consisting of 1,792 cells, highlighted in green the middle subfigures of Figure [5] This setting consistent with (Wei et al., 2024). In the boundary control setting, control signals are restricted to only the 4 x 8 cells inside the four exits, as shown green the right subfigures Figure This setting is newly designed in this paper. Both configurations represent indirect control, the smoke primarily moves within the gray obstacles. The boundary control setting is more challenging due to the significantly reduced number of controllable cells. We also follow (Wei al. 2024) to generate the training dataset. For both settings, the obstacle locations (gray cells) are shown in the top row of Figure [ and are consistent across all training trajectories. Each trajectory contains N = 64 physical time steps and includes features such as horizontal and vertical velocities, smoke density, and horizontal and vertical control forces. We generated 40,000 training trajectories for the large domain control setting, and 30,000 for the boundary control setting.
During inference, following RDM (Zhou et al. 2024), we add small level of control noise make the task more challenging. Specifically, for a trajectory of length N = 64, the control signal in each physical time step is executed with probability p 0.1 as a random control, where the horizontal and vertical components following the uniform distributions bounded by the minimal and maximal values of horizontal and vertical control signals in the training dataset, respectively. On each setting of large domain control and boundary control, we design two evaluation modes, fixed map (FM) and random map (RM), to test the generalization capability of each method:
Fixed map (FM): In this mode, all 50 test samples use the same obstacle configuration with training trajectories. However, the initial locations of test samples are different. this mode, the testing and training initial conditions follow the same distribution (in-distribution test).
Random map (RM): In this mode, the 50 test samples use random obstacles' configuration in the fluid field. Specifically, the movement ranges for the five internal obstacles, whose default locations are shown the top two rows in Figure 5] are as follows:
The two upper obstacles move downward, left, right by more than 3 grid spaces.
24

---

Published as a conference paper at ICLR 2025
Table 6: Effect of the model horizon H 2D incompressible fluid control under the large domain control setting and fixed map (FM) evaluation mode. The average control objective J and inference time on single NVIDIA A6000 48GB GPU with 16 CPU cores are reported. Bold font is the best model and the runner-up is underlined. The results of H = 15 are copied from Table for comparison.
H = 6
H 10
H 15
J J4
time (s)
J 4
time (s) 4
J
time (s) 4
DiflPhyCon-I
0.8986
1022
0,5140
1216
0.5454
1677
CL-DiffPhyCon (ours)
0.6012
143.32
0.3367
136.81
0.3371
140.83
The two middle obstacles can move upward, downward, left, or right by no more than grid spaces.
The one lower obstacle can move upward, left, right by no more than 3 grid spaces.
In this mode, the testing and training initial conditions follow different distributions (out-of- distribution test). Note that although the obstacles' configuration varies across different test samples, for each test sample, its obstacles' configuration remains unchanged during the control process.
G.2 IMPLEMENTATION
use 3D U-net Ho et al. (002) as architectures for the models Es and Eg. Two models are sep- arately trained using the same training dataset. We use the MSE loss train the models. Both models have fusion steps with the hyperparameter v 0.3 for the large domain control setting and 120 diffusion steps with the hyperparameter = 0.3 for the boundary control setting. Hyperparameters of models and training are listed in Table 5
600 diffusion steps. The DDIM (Song et al. 2020) sampling we use has 75 dif-
G.3 EFFECT OF MODEL HORIZON
The choice the model horizon H is determined by balancing efficiency and effectiveness. We conducted experiments the 2D incompressible fluid control task under the fixed map (FM) setting with H = and = 10. The results are shown Table 6 where we also copy the results of H 15 from 2 together for comparison.
The results indicate that both our CL-DDffPhyCon and DiftPhyCon-1 yield similar performance when H 10, compared H 15 reported in Table 2 However, performance significantly deteriorates when H decreases to 6, which is likely due to the shorter observation window leading to inaccurate future control objective estimation and suboptimal guidance sampling (Line 5 in Algo- rithm D. On the other hand, increasing H beyond 15 significantly raises GPU memory costs during inference, thus not recommended. Across different values of H (i.e., H=6, 10, and 15), our method consistently outperforms DiffPhyCon-1. Therefore, for this task, horizon between 10 and 15 is ap- propriate. practical applications, the optimal model horizon can be determined through multiple trials to balance performance and efficiency, similar to the approach used in diffusion policies (Chi ct al. 2023) (see Figure (left) in the referenced paper).
H DOUBLE PENDULUM CONTROL EXPERIMENT
To investigate the performance of our CL-DitIPhCon lower-dimensional control problems, performed experiments on controlling the system of a cart-inverted double pendulum system. The goal is to keep the double pendulum from falling down by exerting a foree the cart.
H. SIMULATION ENVIRONMENT SETUP
The system consists two point masses fixed on the end of two massless rigid rods, which are mutually connected and connected cart with frictionless hinges, shown in Figure 6 The
25

---

Published as a conference paper at ICLR 2025
3
2-
0-
-2
Figure 6: Illustration of the cart-double pendulum system. The angles are measured with respect to the vertical direction, and the angular velocities are defined as the time derivative of the angles.
cart-double pendulum system is simulated with the 4th order Runge- Kutta method (Ixaru & Van- den Berghe 2004). In our experiment, the simulation time step interval is set to 5e-3 to ensure the accuracy of the simulation.
The physical quantities (unitless) are follows: both rods have a length of 0.2, both point masses have a mass of 0.1, the mass of the cart is 1, and the gravitational acceleration is 9.8. The control force is restricted in [-10, 10]. Although the control signal and system states are relatively low- dimensional, we note the control task is non-trivial. Since no torque can be exerted on the hinges, the system can exhibit limited controllability. Besides, the rods deviate from the equilibrium point, and the non-linear has been studied in the control research community (Hesse et al. 2018 Yamakita et al. 1993).
chaotic dynamics impose significant challenges its control. This system
We would emphasize that this experiment provides more easy-to-run verification of our method, as opposed to our 1D Burgers' equation experiment and 2D Navier-Stode Stokes equation control exper- iments. Our cart-double pendulum environment Intel Xeon Platinum 8358 CPU, which allows real- time simulation (200 Hz refreshing rate in our setting) without high-end GPUs need.
at refreshing rate of over 300 Hz an
H.2 TRAINING AND E EVALUATION DETAILS
In this experiment, we focus offline learning setting, where training dataset containing expert action-state sequences is provided. The training dataset consists of 1e5 trajectories generated by an expert policy pretrained with PPO. Our CL-DiffPhyCon and baseline methods are trained with shared hyperparameters. In all diffusion models, VP-SDE training and ODE sampling are adopted (Song 2021) All methods are trained for 2.9e5 steps, and 16-step Midpoint ODE solver Karras et al. 2022) is used. The backbone is kept to an identically configured Transformer network. Other hyperparameters are also kept the same.
Our control task is defined on finite-length episodes, with total of second (200 steps) environment time. We use the metric success rate to reflect control performance, which defined as the ratio of not-falling pendulums when the episode terminates. The not-falling criterion is defined as the upper point height larger than 0.9 maximum height. An ensemble of 100 cart-double pendulum systems independently randomly initialized and controlled to fairly evaluate the performance of cach task. The average and standard deviation are reported for five differently seeded runs to demonstrate the uncertainty of the evaluation.
To mimic realistic systems where the system state may be randomly perturbated, we introduce stochasticity into cart-double pendulum system. At each step, the environment will perturb the system state (including the angle and angular velocities of two rods, the position, and the ve- locity of the cart) with probability. Once added, the perturbation will be sampled from uniform
26

---

Published as a conference paper at ICLR 2025
distribution in the range |-0.02, 0.02). To make comprehensive comparison, report the results with three different rates: 0, 0.1, and 0.3.
Without Noise
Noise Probability 0.1
Noise Probability 0.3
1.0
0.8
0.8
0.6
0.6
0.6
Cor-1
0.4
DiflhhCCan4
DifiihyCon-
Con-
DiffhhCCnn
DiffPhyoon& 8
DifPhh Con
0.2
Con-12
0.2
DitrhhyCon-12
0.2
DifiPhy
DimPhyCon-16
DifPht Con-16
CL-OirPhhyoon
CL-DriPhyCon
OL-DiMPhyCon
0.0
0.0
0.0
0.1
0.4
0.2
0.5
0.2
0.5
Efficiency: control step NFE
Figure 7: Pareto plot comparing our CL-DiffPhyCon and baselines. The error bar shows stan- dard deviation across seeded runs.
H.3 RESULTS
As shown in Figure] CL-DifiPhyCon achieves a remarkable balance between the control algorithm runtime and the control performance. Compared the baselines of DiffPhyCon-Iss DiffPhyCon 12, and DifiPhyCon-8, our CL-DDifPhyCon provides much higher control performance while only requiring similar neural function evaluations (NFE, as in [Song al.) (202T); Karras al. (2022) environment step. Although DiftPhyCon-4 and DiftPhyCon- show marginal performance im- provement over our CL-DifPhyCon, they are nearly one order of magnitude slower.
For more intuitive demonstration of the performance comparison, we include a comprehensive vi- sualization where our method compared with two selected baselines under three different random perturbation settings, as shown in Figure g Compared to DiffPhyCon-15. Our CL-DiffPhyCon is able to stabilize the inverted double pendulum for longer time (subfigure (a)), and the final upper mass state (02,w2) is closer to the equilibrium point (0, 0) (subfigure (b)). Additionally, although the performance of our CL.-DifiPhyCor similar to that of DiffPhyCon-1, CL-DDifPhyCon costs about only 1/15 computation time of it.
I ID VISUALIZATION RESULTS
We present the visualization results of method and baselines under noise-free and half domain settings in Figure 10 11 and 12 respectively. Under each setting, we present the results of four randomly selected samples from the test set.
2D VISUALIZATION RESULTS
More visualization results of our method and compared baselines are presented. For both fixed map (Figure 13 and Figure 14) and random map (Figure [5] and Figure 16) settings, we present two randomly selected test samples. Our method outperforms baselines on average. Note that the fluid dynamics is very sensitive to the generated control signals. Thus different methods may perform dramatically differently on the same test sample. However, we find that each method performs stably each test sample through multiple evaluations.
K BASELINES
For RDM Zhou al. 2024). it involves two hyperparameters determine when to update the previously planned control sequences or replan from scratch. Their default values do not work well ID and 2D tasks. On the 1D task, they small, which results in very expensive
27

---

Published as a conference paper at ICLR 2025
(a)
0.0.0
T-0.15 T- 0-022
T-0.35
T-045
T-0.5s
T-0.65
T- 0.75
T- 0.85
T-
0.95
r- 1.05
D.DDirPhyCan
(ours)
V4tthou noise
DirPhyCon-1
DIrPhyCon-15
0.035
T-5.55
0.66 T-
0.9s
CL-DirPPhyCon (ours)
Noise probability D.1
DifPhyCon-1
DirPhyCon-15
Z
IL-DifPhyCon (ours)
Noise probabliity:
DirPhyCon-1
DifPhyCon-15
V
(b)
Probabiliy:
GuBrThyCor DiPPyCee-1 DiPPrycee-15
Satieyy vill W0--ty----155
AdA
AdA Stutes
Figure 8: Visualization of and instance of the cart-double pendulum under control. The goal is to keep the double pendulum from falling down by exerting a force on the cart over second (200 control steps) sequentially. Three control methods are applied under three different perturba- tion rates. In each setting, 10 snapshots of the pendulum are shown in subfigure (a), whereas the corresponding trajectories in the phase space are shown in subfigure (b).
28

---

Published as a conference paper at ICLR 2025
replanning from scratch almost every physical time, although its performance is still unsatisfactory. On the 2D task, they are too large, leading to few replannings and inferior results. Therefore, we extensive evaluations to search for their best values. On the ID task, we find that the best pair of values for the two hyperparameters is (0.06, 0.1) in the noise-free setting. On the 2D task, we find that the best pair of values is (0.0005, 0.002).
29

---

Published as a conference paper at ICLR 2025
Tarree State
1.0
# Targut State
1,0
0.5
60
0.5
-0
0.0
+.40
0.0
20
-05
20
-0.5
%
50
9
100
-100
v:
Emor: BC1/00
U: BC
1.0
Emor: BC (1+0 (u-0.2276)
1.0
0.5
0.5
0.5
0.5
40
0.0 440
0.
40
0.0
-40
0.0
20
-0.5 20
-0.5
20
-05 20
-05
0
50
100
%
50
102
50
100
-1.0
1:0
-10
Eroo:: BIDD (-2.2188)
BIPG (p0.8857))
0.5
0.0 +.0
0.0
400
0.0 -40
0.0
20
-05 20
-05
20
-0.5 20
-05
50
1.0
-1.0 %
50
100
-1.0
50
1.00
-1.0
50
100
-10
(-0.2782)
(=0.3915))
0.5
0.0 400
0.0
4.5
0.5
20
0.5 20
0.5
50
100
-1.0
%
50
-1.0
150
-1.5
1:0
-10
Dell
1.0
-0.
-0.5
-0.
%
50
-1.0
%
50
-1.
150
-1.0
100
-.0
DemPhyCon
1.0
Emar DffRhycon
1.0
DiffPhyCon 5
1.0
DifPhyyon (1-
1.0
0.5
0.5
-05
-05
0
%
-1.0
-.0
6: CimPlyCon 15
1.0
Erron: Difihyyoon 15 ((-5.0023)
1.0
DimPhhyCon-15
1.0
Emao:: DIfPlyCon-15 ((=0.1528)
1.0
60
60
60
0.5 60
0.5
4-40
+.40
0.0 4-40
0.0
0
%
RDM
1.0
Erron: RDM ((=0.0196)
1.
u: RDM
1.0
Emor: RDM (=0.1009)
1.9
60
0.5
60
05
60
0.5 60
0.5
4. +:40
+44
0.9
+:40
6.0
-0.5
-05
50
%
50
150
-1.0
100
6n Ciitiilhyooonn lours]
1.0
Errop Cl-billityCom (oun) (100,000))
12 Cl-bdllihyCan boun)
1.0
Erie: CL-DitPtyyEon (ouns) (-0.012))
60
0.5 60
0.5
60
0.5 60
0.5
40
0.0 +00
400
0.0 +40
0.0
-0.5
-0.5
20
-0. 20
-05
%
100
1.00
100
Cl-D (DDMM ourd
[rror Dorthyyoo (DDM) numt (=0.0010)
E1 CL-DIIPhyy (DDIM curs
J.Emrr: a Ooo (DDIM) nand) 0110)
6.5
0.5
0.5
6.5
40
6.0 440
440
0.0 +40
0.9
20
0.5 20
0.5
20
0.5 20
05
50
100
%
50
100
50
100
-1.
30
160
-10
Figure 9: Two visualizations results of 1D Burgers' equation control under the noise-free set- ting. The first line is the target ua(t, x) and the error u(t, x) uz(t, 2) measures the gap between the state under control and target. The horizontal axis is the time coordinate and the vertical axis is the state.
30

---

Published as a conference paper at ICLR 2025
Target State
1.0
e Target State
1,0
60
0.5
60
0.5
-0
0.0
+.40
0.0
20
-05
20
-0.5
%
50
%
100
-1.0
v:
1.0
Emor: (/0.33119) 3519)
U: BC
1.0
Emor: BC (1=0 9615)
1.0
0.5
0.5
0.5
0.5
40
0.0 440
0.0
40
0.0
-40
0.0
20
-0.5 20
-0.5
20
-0. 20
-05
0
50
100
%
50
100
%
50
100
-1.0
1:0
-10
Eroon: (12.1199))
BPPG y-o 45500
0.5
0.5
0.0 +.0
0.0
400
0.0 -44
0.0
20
-05 20
-05
20
-0.5 20
-05
50
1.0
-1.0 %
50
100
-1.0
50
1.00
-1.0
50
100
-10
(-03)
(-0.9797)
0.5
0.0 400
0.0
0.5
0.5
20
0.5 20
0.5
%
50
100
-1.0
%
50
-1.0
150
-1.5
100
-10
Dell HyyC-1
1.0
1.0
0.0
0.0
-0.
-0.5
-05
%
50
-1.0
%
50
-1.9
150
-1.9
100
-10
DemPhyCon
1.0
Emar DffRhxcon (j-0.0115)
1.0
DiffPhyCon 5
1.0
Enor: DMPhhyon (1000377)
1.0
0.5
0.5
-05
-05
0
%
150
-1.0
-10
6: CIMPeyCon 15
1.0
Erron: DifPhhyoon 15 ((b.0155))
1.0
DiMPhyyConn1
1.0
Error: DIfPlyCon (-20.0222)
1.0
60
60
60
0.5
60
0.5
4-40
+.40
0.0
+-40
0.0
0
%
RDM
1.0
Erron: RDM ((=0.0136)
1.
U: RDM
1.0
Emor: RDM Uuo 04188
1.9
60
0.5 60
0.
60
0.5 60
0.5
4. +:40
+44
0.9
+:40
0.0
20
-0.5
-05
50
%
50
150
-1.0
100
6: Ciitiiihyoonn [ours]
1.0
Errop Cl-billityCmm (oun) (m0.0000.)
15 Cl-bbilthyCann bous)
1.0
Erie: CL-DitPtyyEon (ours) 9-0.01
60
0.5 60
0.5
60
0.5 60
0.5
40
0.0 +00
9.
400
0.0 +40
0.0
-0.5
-0.5
20
-0.5 20
-05
50
%
100
-1.0
100
Cl-D (DDMM ourd
[rror: Dorthyyoo (DDM) numt (-0.0003)
C1 CL-DIIPhyyC (DDIM cun)
_.Errr O Ooo (DDIM, nard) 0199)
6.5
0.5
0.5
0.5
40
6.0 440
4.0
0.0 440
0.9
20
0.5 20
0.5
20
0.5 20
05
%
50
100
-1.0 %
50
100
11.0
50
100
-1.0
30
160
-10
Figure 10: Two visualizations results of 1D Burgers' equation control under the noise-free setting. The first line is the target ua(t, x) and the error u(t, x) uz(t, 2) measures the gap between the state under control and target. The horizontal axis is the time coordinate and the vertical axis is the state.
31

---

Published as a conference paper at ICLR 2025
Target State
Target
0.5
0.0
0.0
20
-05
20
-0.5
%
50
1.00
-1.0
55
1.00
-1.0
L.0
BC--002000)
0.9
0.0 440
0.0
0.5
-05
20
-05 20
005
%
50
1.00
-1.0
%
-1.9
%
100
-1.0
100
-1.
BPPO
Erron SPPO (-0.1652)
BFPO
1.0
BPPO V=c 1403)
1.0
60
0.5
0.0
0.0
-0.5
-05
0
30
-.0
%
33
-1.0
150
-1.0
100
1.0
1.0
Frar 1057)
1.0
1.0
Emor PID [ts 3030)
1.0
0.5
60
0.5 60
0.5
+:40
0.0 4:40
0.0
0
%
-1.0
1:0
DemPhhyonn
1.0
Error: DIPPhyoon1 ((-0.0155)
1.0
D.MPPhyCon 3
1.0
Enor: DinPayCon 1 (-0 0283)
1.0
60
0.5
60
0.5
60
0.5
60
0.5
440
4.40
0.0
4:40
0.0
20
0
1.0
%
50
150
-1.0
150
DerthhyCon-A
1.0
Erroe: Dirhheend (I-D.115)
2.0
u: DetihhyConn
1.0
Froor: ConPA 1-0 02700)
1.0
6.
0.5 80
0.5
60
0.5 60
0.5
-0
0. 440
0.0
+.40
0.0 +:40
0.0
20
-0.5 20
-05
50
100
%
53
%
1.00
100
cr DemPhyCon-15
Erron: DiPPhyCon-155 (1a0.0154)
DiMPhyGon-15
1.0
Eror: DiPhhyCon155 (Ju 0320)
1.0
0.5
0.5
80
0.5
60
0.5
40
6.0 440
+40
0.0 <40
6.0
20
-0. 20
-0.5
20
-0.5 20
-05
0
50
100
-1.0 %
50
105
%
50
100
-1.0
1:0
-10
Erroe: (1000152)
RDM
1.0
Emar: RDM U-o 0214)
1.0
0.5
0.5
0.5
6.5
0.0 4.0
0.0
4.0
0.0 +40
0.0
20
-05 20
-0.
20
-0.5 20
-05
%
50
100
-1.0 %
55
100
-19
15
1.0
-1.
100
-10
CemPlyCon (ours)
CL Dittioooon lours) (1-0 00555)
DiMRtyyoon tours)
Brnon: DomPhyCConnnooooooooooooooo (ours) 0008)
0.5
0.5
4.0
4.0
0.9 440
0.0
20
.05 20
-0.5
20
-0.5 20
-05
0
50
100
1.. %
50
150
-1.0
30
1V0
-1.0
100
1.0
DVnPhyyCCo (DDIM,
1. Error: Com (00R) (1-b (ogg)
Can (DDIM,
Emerr OLotiiityiann (D0 (-m 003g)
60
0.9
0.0
-0.5
-0.
-03
0
30
100
1.0
%
33
-.0
130
1.0
100
1.0
Figure 11: Two visualizations results of 1D Burgers' equation control under the half domain setting. The first line is the target ua(t, r) and the error u(t, x) uz(t, x) measures the gap between the state under control and target. The horizontal axis is the time coordinate and the vertical axis is the state. The white area with oblique lines in the middle represents an unobservable area.
32

---

Published as a conference paper at ICLR 2025
Tanget State
Target
0.5
0.0
0.0
20
-0.5
20
-0.5
%
50
1.00
-1.0
5.
1.00
-1.0
(C
1.0
(-2.0750)
0.0 <40
0.0
0.5
0.5
20
-0. 20
005
%
50
1.00
-1.0
%
-1.9
%
100
-1.0
100
-1.
BPPO
Erron, 31R0 (-0.25811
BFPO
1.0
BPPO (46.1733)
1.0
60
0.5
0.0
0.0
-0.5
-0.
0
30
-1.
%
33
-1.0
150
-1.0
100
1.0
1.0
Frar: 5003)
1.0
1.0
Freer PID (ta0 0507)
1.0
0.5
60
0.5
60
0.5
+.40
0.0 4:40
0.0
%
-1.0
100
DemPhhyoon
1.0
Error: DIPPhyCon3 (j-0.0326)
1.0
D.MPPhyCon 1
1.0
Emor: DinPayCon 1 (-O 0095)
1.0
60
0.5
60
0.5
60
0.5
60
0.5
4:40
+.40
0.0
4:40
0.0
20
0
%
50
%
150
-1.0
150
DetthhyCon-A
1.0
Erro:: DePhheend (I-D.111))
3.0
u: DitihhyConn
1.0
Froor: ConPA I-0 0D95)
1.0
6.
0.5 80
0.5
60
0.5 60
0.5
-0
0. 4:4
+.40
0.0 +:40
0.0
20
-0.5 20
-05
50
100
%
53
%
100
100
or DemPhyCon-15
Erron: DiPPhyCoo-155 (1a0.0374)
DeMPhyGon-15
1.0
Eror: DiPhhyCon115 (Ju. 0071)
1.0
0.5
0.5
80
0.5
60
0.5
-0
6.0 440
+40
0.0 <40
6.0
20
-0.5 20
-0.5
20
-0.5 20
-05
0
50
100
-1.0 %
50
100
%
50
100
-1.0
1:0
-10
Erroe: (3-0.339))
RDM
1.0
Emer:: RDM U-0 0p63)
1.0
0.5
0.5
0.5
6.5
6.0 4.0
0.0
4.0
0.0 +40
0.0
20
605 20
-0.
20
-0.5 20
-05
%
50
100
-1.0 %
55
100
15
1.0
-1.
100
-10
CemPlyCoo (ours)
CL Dittiyooon lours) (1-0 0103) g
DitRtyCoon tours)
Brnon: DwmPhyCConnoooooooooooooooo (ours) 0033)
0.5
0.5
4.0
+.0
0.9 <40
0.0
20
.0. 20
-0.5
20
-0.5 20
-05
0
50
100
1..0 %
50
190
-1.0
30
100
-1.0
100
1.0
1. Error: Com (00R), (10.0105)
Can (DDIM.
Emeer OLotiiiyy (D0 (-m 0053)
60
0.9
0.0
-0.5
-0.
-0.
0
30
100
1.0
%
33
-10
130
1.0
100
1.0
Figure 12: Two visualizations results of 1D Burgers' equation control under the half domain setting. The first line is the target ua(t, r) and the error u(t, x) uz(t, x) measures the gap between the state under control and target. The horizontal axis is the time coordinate and the vertical axis is the state. The white area with oblique lines in the middle represents an unobservable area.
33

---

Published as a conference paper at ICLR 2025
BC 0.5770
1.1
BPPO (J= 0.5197)
DifiPhyCon-1 (J= 0.0508)
DiffPhyCon-5 (J= 0.2269)
3.30
(J
335
3.00
RDM (J= 0.9949)
CL-DifIPhConn (ours) (J 0.0274)
3.0
122
320
035
330
CL-DifPPhyCon (ours, DDIM) (J 0.1124)
1.0
0.30
t-0
t=10
t=20
t=30
t=40
t=50
Figure 13: Visualizations results of 2D fluid control by our CL-DiffPhy Con method and base- lines for same test sample. This is example of the fixed map setting. Each row shows six frames of smoke density. The smaller the value of J, the better the performance.
34

---

Published as a conference paper at ICLR 2025
BC (J= 0.9251)
BPPO (J=0.9393)
3.3
325
320
DifiPhyCon-1 (J 0.6761)
DiffPhyCon-5 (J= 0.8716)
3.30
3.5
320
335
330
DiffPhyCon-1e (J= 0.9956)
3.00
RDM (J 0.9908)
3.30
CL-DifIPhyCon (ours) (J= 0.7463)
335
CL-DiffPhyCon (ours, DDIM) (J= 1.000)
3.00
0.30
025
0.0
4015
t=0
t=10
t=20
t=30
t=40
t=50
Figure 14: Visualizations results fluid control by our CL-DiffPhyCom method and base- lines for the same test sample. This is an example of the fixed map setting. Each row shows six frames smoke density. The smaller the value J, the better the performance.
35

---

Published as a conference paper at ICLR 2025
BC (J= 0.6535)
BPPO (J= 0.3156)
DiffPhyCon-1 (J= 0.9994)
DiffPhyCon-5 (J= 1.000)
3.30
DifiPhyCon- 14 (J=0.0447)
335
3.00
RDM (J= 1.000)
CL-DifIPhConn (ours) (J 0.0062)
1.2
320
035
330
CL-DifiPhyCon (ours, DDIM) (J= 0.3139)
1.0
0.30
t-0
t=10
t=20
t=30
t=40
t=50
Figure 15: Visualizations results of 2D fluid control by our CL-DiffPhy Con method and base- lines for the same test sample. This is example of the random map setting. Each row shows six frames of smoke density. The smaller the value of J, the better the performance.
36

---

Published as a conference paper at ICLR 2025
BC (J=0.9141)
BPPO 0.9008)
DiffPhyCon-1 (J= 0.0823)
DiffPhyCon-5 (J= 0.9858)
3.30
DifiPhyCon- (J= 0.9923)
335
3.00
RDM (J= 0.0850)
CL-DifIPhyCon (ours) (J 0.0184)
3.0
1.2
320
035
330
CL-DifPPhyCon (ours, DDIM) (J 0.1394)
1.0
0.30
t-0
t=10
t=20
t=30
t=40
t=50
Figure 16: Visualizations results of 2D fluid control by our CL-DiffPhy Con method and base- lines for the same test sample. This is example of the random map setting. Each row shows six frames of smoke density. The smaller the value of J, the better the performance.
37


Reviews:
[{"review_id": "MT0BJJr4k1", "reviewer": "Reviewer_GNga", "summary": "This manuscript addresses a closed-loop diffusion control problem of complex physical systems. The proposed CL-DiffPhyCon (Wei et al., 2024) extends the work of DiffPhyCon (Wei et al., 2024).", "questions": "Detailed comments are summarized below:\n\n1. How restrictive is the consideration of noise-free control? There are many noise issues arising from complex engineering systems and applications. Can the proposed method be extended to account for plant, control, and measurement noise? \n\n2. Another important aspect associated with complex and practical physical systems is physical constraints. For example, physical control actions are implemented via actuators which often have upper and lower limits. That is one major reason model predictive control is useful. Can the proposed method handle physical constraints associated with control actions? \n\n3.  From Figure 2 and the limitation mentioned in the appendix, the algorithm is trained offline. In that case, it is not clear how the control is designed and employed in a real environment, as controllers are typically designed and used in practice in an interactive manner. In Figure 2, the ending time instance is T, can it be changed to T+1, as a bigger loop, to make it useful in an online/interactive manner? \n\n4. In both examples, the controls are defined as a spatiotemporal function in the PDE model, which is too ideal to be implemented in a realistic system. For PDE system control, typically, the control is implemented as a temporal function. The corresponding actuator is located in the boundary or a small part of the entire spatial domain. It would be interesting to see the results in these more realistic settings by using the proposed method. \n\n5. In the Burgers’ equation example, there is no difference between the full observation and partial observation cases, because the latter case only observes and controls two parts of the entire domain. In the case considered in the manuscript, one can simply treat each observed sub-domain as a total entire domain and apply the algorithm on the PDE with new spatial coordinates/ranges/boundary conditions. Partial observation typically refers to the case using partial observation to control the entire domain. The control performance results of the unobserved state should be added to fully demonstrate the effectiveness of the proposed algorithm. Similarly. practical applications often have sensors installed in the boundary of PDE systems. It would be good to also add results by using boundary observation data. \n\n6. The limitations of the proposed method should be moved to the conclusion section for better readability.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "One advantage is to improve the computation efficiency, which is achieved by an asynchronous denoising framework. This aspect is validated using numerical simulations. Overall, this manuscript is well-written and organized.", "weaknesses": "There are some technical limitations, including noise-free inputs, input constraints, and numerical validations by using full in-domain control and observation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4hpuZKwzjL", "reviewer": "Reviewer_sLTC", "summary": "The paper proposes a novel method for using diffusion policies for closed loop control that speeds up the generation of a control sequence of a length up to a chosen horizon by avoiding full denoising of the latter steps in the sequence, as they will likely not be executed by a receding horizon controller. The proposed method demonstrates superior performance in terms of achieving control objectives, due to closed loop control using a diffusion policy, with computation time much shorter than other diffusion policy baseline controllers.", "questions": "I am not quite sure why the proposed method achieves a better control objective than DiffPhyCon-1 - isn't the latter replanning at each control step by means of full denoising, thus as closed-loop as the proposed method? Can the authors clarify this? \n\nThe planning horizon for the two control tasks is fixed. Why exactly to these lengths? What happens if full denoising is done over shorter horizons, replanning at each time step, using DiffPhyCon-1?", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The main strength of the proposed method is the idea to save computation by not fully denoising the latter steps of the control sequence, with the understanding that in receding horizon control, the controls for those latter stages will likely not be used, as new state observations will appear in the meantime and the control sequence will be replanned. This idea addresses one of the main shortcomings of controllers based on diffusion policies - the very long process of denoising during sampling, which is currently a major obstacle to the wider adoption of diffusion policies that have shown considerable advantages with respect to more traditional behavior cloning schemes.", "weaknesses": "I believe the proposed method is excellent and clearly applicable to very difficult control problems involving PDEs. However, most control problems are not of this kind. Why not show the performance on a simpler control problem with a handful of variables, hopefully not requiring high-end GPUs? This could potentially increase the applicability and appeal of the method significantly.\n\nSome minor typos:\n\nP. 3, L. 112: \"It\" -> \"They\"\nP. 3, L. 140: \"has\" -> \"have\"\nP. 7, L. 376: \"well well\" -> \"perform well\"?\nP. 10, L. 539: \"done\" -> \"drone\"?\nP. 19, L. 1009: \"128 grids\" -> \"128 cells\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9qERoRvF5v", "reviewer": "Reviewer_pd6k", "summary": "The paper studies the problem of designing feedback control algorithms using diffusion techniques. Differently from open-loop control strategies, which compute a pre-determined control sequence that is applied to a system over time, closed-loop methods use information about the state of the system to tune the control inputs to compensate for deviations of the system trajectories from the predicted one, due to unmodelled dynamics or noise. The paper focuses on the computational complexity of the methods, and it proposes a technique to render closed-loop diffusion-based control computationally efficient.", "questions": "A careful rewriting of the manuscript seems necessary to clarify the problem setting and contribution, and to appreciate the differences between the propose method and the existing ones. A characterization of the performance of the proposed method is necessary, otherwise the  method takes the form of a heuristic that may not be desirable in controls applications.\n\nAs a side note, the problem described in equation (1) is inherently an open loop problem, since no structure is imposed on the input w. In fact, the set of open-loop sequences include the set of closed-loop sequences, but the converse may not be true. Since the system dynamics are deterministic, it is also unclear why using a closed-loop strategy would be necessary. Finally, the used notation (w for control inputs and u for the system state) is very unusual and, to the best of my knowledge, never been used in any controls paper.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The overall objective of the paper is interesting and timely: to design closed-loop diffusion-based control algorithms. The issue of computational complexity is important, since diffusion-based methods may work well but require extensive computations, making them difficult to use in feedback settings.", "weaknesses": "The paper is poorly presented, to an extent that makes the contributions unclear and difficult to appreciate. This start with the definition of the problem (where what is known/unknown/available to the designer is unclear, where there is confusion between system and environment, where   the stated problem is inherently open-loop and the feedback structure is not evident), and continues into the description of the proposed method (the synchronous and asynchronous parts are vague and difficult to grasp). Additionally, and importantly, the paper doesn't seem to offer performance guarantees for the proposed method, which makes it difficult to advocate for the proposed method in any practical use.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)