Action Matching: A Variational Method for Learning Stochastic Dynamics from Samples
Authors: Kirill Neklyudov
Daniel Severo
Alireza Makhzani
Venue: Submitted to ICLR 2023 (2023)

Abstract: Stochastic dynamics are ubiquitous in many fields of science, from the evolution of quantum systems in physics to diffusion-based models in machine learning. Existing methods such as score matching can be used to simulate these physical processes by assuming that the dynamics is a diffusion, which is not always the case. In this work, we propose a method called "Action Matching" that enables us to learn a much broader family of stochastic dynamics. Our method requires access only to samples from different time-steps, makes no explicit assumptions about the underlying dynamics, and can be applied even when samples are uncorrelated (i.e., are not part of a trajectory). Action Matching directly learns an underlying mechanism to move samples in time without modeling the distributions at each time-step. In this work, we showcase how Action Matching can be used for several computer vision tasks such as generative modeling, super-resolution, colorization, and inpainting; and further discuss potential applications in other areas of science.

Under review as a conference paper at ICLR 2023
ACTION MATCHING: A VARIATIONAL METHOD FOR LEARNING STOCHASTIC DYNAMICS FROM SAMPLES
Anonymous authors Paper under double-blind review
ABSTRACT
Stochastic dynamics are ubiquitous in many fields science, from the evolution of quantum systems in physies to diffusion-based models in machine learning. Existing methods such as score matching can be used to simulate these physical processes by assuming that the dynamics is diffusion, which is not always the case. In this work, we propose a method called "Action Matching" that enables us to learn much broader family of stochastic dynamics. Our method requires access only to samples from different time-steps, makes no explicit assumptions about the underlying dynamics, and can be applied even when samples are uncorrelated (i.e., are not part of a trajectory). Action Matching directly learns an underlying mechanism to move samples in time without modeling the distributions at each time-step. In this work, we showcase how Action Matching can be used for several computer vision tasks such as generative modeling, super-resolution, colorization, and inpainting; and further discuss potential applications in other areas of science.
INTRODUCTION
The problem of learning stochastic dynamics is one of the most fundamental problems in many different fields of science. In physics, porous medium equations (Vázquez, 2007) describe many natural phenomena from this perspective, such Fokker Planck equation in statistical mechanics, Vlasov equation for plasma, and Nonlinear heat equation. Another prominent example is from Quantum Mechanics where the state of physical systems is distribution whose evolution described by the Schrödinger equation. Recently, stochastic dynamics have achieved very promising results in machine learning applications. The most promising examples of this approach are the diffusion- based generative models (Song et al. 2020b; Ho et al. 2020).
Informal Problem Setup In this paper we approach the problem of Learning Stochastic Dynamics from their samples. Suppose we observe the time evolution of some random variable X, with the density gt, from to to &1. Having access to samples from the density Se at different points in time t e [&o, ti want to build model of the dynamics by learning how to move samples in time such that they respect the marginals gt. In this work, we propose a method called "Action Matching" as a solution to this problem.
Learning Stochastic Dynamics vs. Time-Series There is an important distinction between the problem learning stochastic dynamics and time-series modeling (c.g., language, speech or video modeling). In time-series, the samples come in trajectories, where the samples in each trajectory usually highly correlated. However, in learning stochastic dynamics, we only have access to independent samples at any given time-step (i.e., uncorrelated samples through time). This degree of freedom allows us solve different types of problems that can not be approached by time- mene-sriess modeling. We provide several examples in our experiment section, but also point out that sometimes even physically impossible to obtain samples along trajectories. For example, in Quantum Mechanics, the act of measurement given point collapses the wave function which prevents us from obtaining further samples along that trajectory.
Generative Modeling with Action Matching From the Machine Learning perspective, the problem of learning stochastic dynamics is a generalization generative modeling. One way so solve generative modeling is first construct distributional path (stochastic dynamics) from the data

---

Under review as a conference paper at ICLR 2023
Score Matching
Action Matching
0q: 0t
v log q
V (2: Vs)
arbitrarily complicated
V8 coonst
Figure 1: Score Matching learns model for every distribution, while Action Matching learns the transition rule between distributions according to the continuity equation. Here, we illustrate that learning the dynamics might be much simpler task than learning all the distributions individually.
distribution to a tractable prior distribution (e.g., Gaussian or uniform), and then learn to move along this path to generate samples. The most prominent example of this approach is the recent developments in diffusion generative models (Song et al. 2020b; Ho al., 2020), where a stochastic differential equation (SDE) is constructed move the samples from the data distribution to the prior, and the reverse SDE is constructed by learning the score function of the intermediate distributions via Score Matching (Hyvärinen & Dayan 2005). Action Matching can be used for generative modeling in similar way, where we also construct a stochastic dynamics between the data distribution and the prior. However, the important distinction that this dynamics is constructed solely from samples of the intermediate distributions, rather than analytical SDEs used in diffusion. This heavily relaxes the constraint on the dynamics required in SDEs, and enables Action Matching to learn a much richer family of dynamics between the two distributions. For example, in both widely used VP-SDEs and VE-SDEs (Song et al., 2020b), the conditionals qe (2,(2z)) are tractable Gaussian distributions, while in Action Matching, the dynamics can have any arbitrary conditional gi (ax (zn). as long as it can be sampled from. We can also use Action Matching to learn the dynamics constructed by SDEs as SDEs can be sampled from. In Section 5. provide rich family of dynamics, that can be learned with Action Matching, without the knowledge of the underlying process.
Another important distinction between SDEs and Action Matching is that the Action Matching modeling capacity is spent only on learning how to move the samples (in a consistent way with the marginals), and does not make any attempt learn the marginals themselves. However, in diffusion models such as VP-SDEs or VE-SDEs, all the capacity of the model is spent on learning the score function of the individual densities V log gi (a:) for the backward diffusion. This is wasteful if the evolution of the density is simple, but the densities themselves are complicated. An illustrative toy example of this is provided in Fig. [ where complicated density is evolving with a constant velocity through time. In this case, Action Matching only needs to learn a constant velocity vector field, without learning anything about the individual marginals. As a practical example of this, we will consider the colorization task in the experiment section, and argue that moving directly from grayscale image to the colored image with action matching is much easier than moving from Gaussian noise to a colored image with a conditional diffusion that conditions on the grayscale image.
In short, compared to diffusion generative models, Action Matching has the following advantages:
1. Action Matching relies only on samples and does not require any knowledge of the underly- ing stochastic dynamics, which is essential when only have access to samples.
2. Action Matching is designed to learn only the dynamics, rather than the individual distribu- tions qt. which is useful when a complicated distribution has simple dynamics.
3. Action Matching's applicability extends beyond that of diffusion models, it can learn much richer class of stochastic dynamics (see Theorem T
Our contribution is two-fold: 1) In Section 2 we discuss a mathematically rigorous problem formula- tion for learning stochastic dynamics, why this problem is well-defined, and what types of dynamics we aim to learn. 2) In Section 3 we discuss Action Matching as a variational framework for learning these dynamics. Finally, some of the possible applications of Action Matching, we discuss several
2

---

Under review as a conference paper at ICLR 2023
computer vision tasks, such generative modeling, super-resolution, colorization, and inpainting; and provide experiments.
PROBLEM FORMULATION OF LEARNING CONTINUOUS DYNAMICS
Continuity Equation Suppose we have a set of particles in space X RR, initially distributed as Qta- Let each particle follow a time-dependent ODE (continuous flow) with the velocity field v to, [1] x + R as follows
r() = (x(()))
x(to) =
(1)
From fluid mechanics, we know that the density the particles at time t, denoted by qt, evolves according to the continuity equation
a
Ōt 9t (q:
which holds in the distributional sense, where V. denotes the divergence operator.
Note that even though we arrived the continuity equation using ODEs, the continuity equation can describe rich family of density evolutions in wide range of stochastic processes, including those of SDEs (see Equation 37 of [Song et al. (2020b)), or even those of the porous medium equation (Otto) 2001) that are more general than SDEs. This is intuitively because in modeling the density evolution, we only care about respecting the marginals gt, and not the underlying stochastic process that resulted in those marginals. This motivates to restrict ourselves to ODEs of the form Eq. ( and the continuity equation, without losing any modelling capacity. In fact, as the following theorem shows, under mild conditions, any continnous dynamics be modeled by the continuity equation, and moreover any continuity equation results a continuous dynamics.
Theorem 1 (Adapted from Theorem 8.3.1 of Ambrosio al. (2008)). Consider a continuous dynamic with the density evolution of qt. which satisfies mild conditions (absolute continuity in the Wasserstein space of distributions P2 (2)).. Then, there exists a unique (up to constant) function 8 (I),, called "action", such that vector field 27 (z) = Vss (x) and qu satisfies the continuity equation
Ot 9: - ( (s)
In other words, the ODE a x()) Vs, (z) can be used to move samples in time such that the marginals are qr-
Furthermore, we know that 5 (x), defined in Eq. defined below, along de (Ambrosio et al. 2008)
3 minimizes the kinetic energy functional K(
1 K(c.)) 2
f" E (e
V arg min K(or))
θ - -V-(9v)) ōt
(4)
where the optimization is over all i satisfying the continuity equation with gt- We can use the optimal value of the optimization in Eq. (4) to attribute unique kinetic energy value K. to any stochastic dynamics gr as follows:
x= 1 Vls
(5)
Using Theorem the problem of learning the dynamics be boiled down learning the unique vector field Vsy only using samples from дt- Motivated by this, we restrict our search space of velocity vectors to the family of curl-free vector fields
S = (Vss +
We use a neural network parameterize the set of functions (x)), and propose Action Matching for learning the neural network such that (x) approximates sf (æ). Once we have learned the vector field Vsf, we move samples forward or backward in time by simulating the ODE in Eq. [ with the velocity Vsg. The continuity equation ensures that samples at any given time t E [to. E are distributed according q:-
3

---

Under review as a conference paper at ICLR 2023
3 ACTION MATCHING
The main development of this paper the Action Matching method, which allows to recover the rree action st of a continuous dynamic and thereby simulate it, while having access only to samples from gt- In order to so, we define the variational action (x(),, parameterized by neural network, that approximates of (a)), by minimizing the "ACTION-GAP" objective
ACTION-GAP(s, 8*) = E (s)
Note that this objective is intractable, we do not have access to Vs*. We now propose action matching as variational framework for optimizing this objective.
We first show that the problem of minimizing the intractable Eq. ( is tightly related to estimating another intractable quantity: the kinetic energy of a continuous dynamics. As discussed in Theorem [ we can attribute a kinetic energy K quantity to any absolutely continuous dynamics qr. using the true action s* (see Eq. (4).. In order to estimate the intractable K, we define tractable variational kinetic energy lower bound (KIIBBO) as a functional of an arbitrary variational action as follows:
/ (gy())
[
[ -
Ose
(ILBO ge, (5) (x)] ge,, (t))
(x) dt (8)
Ol
action-increment
smoothness (regularization)
The following proposition establishes that Eq. is the gap between KILBO and the true kinetic energy.
Proposition 1. For an arbitrary variational action KILBO(s) is a lower bound on the true kinetic K., and the gap can be characterized with
== KILBO((s ACTION-GAP(S,
Thus, since K. is not function of s, the following optimization problems are equivalent
arg max ( (x)) = arg a
(10)
where the equality is up to an additive constant. The KILBO gap is tight iff Vs(()) = (s)
See Appendix for the proof. [ that maximizing the KILBO results in estimating the true kinetic energy, as well matching the variational action the true action. Note that unlike the intractable K. maximizing KILBO is tractable, as we can use the samples of qe to obtain an unbiased low variance estimate of KILBO.
KILBO can be decomposed into an action-increnee and a smoothness term. If we only optimize the action-incremenn term, we learn large values for st (x t and small values for sx(x) at to. this case, (a:) tends to learn a degenerate function with sharp transitions in both æ and / directions. The smoothness term acts as regularization term by penalizing large gradients with respect to both direction {x) and direction
(s((z)
8t
4 GENERATIVE MODELING USING ACTION MATCHING
While Action Matching has wide range of applications in learning the continuous dynamics, in this work, we focus on the applications of Action Matching in generative modeling task. In Action Matching generative models, we first have to define a dynamics (i.e., noising process) that transforms samples from the data distribution go Gaussian). Action Matching is then used to learn the vector field Vs* of the chosen dynamics. Once Vs* is learned, moving the samples using a reverse ODE with the velocity V Finally, Action Matching enables compute the exact likeelooo of the data.
to samples of prior distribution 4I (e.g., standard
sample from the target distribution by first sampling from the prior, and then
4

---

Under review as a conference paper at ICLR 2023
Sampling Process
diffusion
di a,30 + 0rE
super-resolatin
3i mask 20 (1 mask) (aran + ote)
colorization
di auto (gray(a)) 10 7.
inpainting
mask + mask) (arzo + otE)
torus
2x (ag + 035) mod 1
Figure 2: Examples of different noising processes used for different vision tasks. / from the data distribution. Depending the task, the noising process gradually destroys all partial information of data, and replace it with prior noise.
= 0, we start
4.1 NOISING PROCESSES IN ACTION MATCHING GENERATIVE MODELS
To learn the vector field Vs*, Action Matching only requires samples from the intermediate distri- butions qr. [0, 1], that define the noising process. We now provide broad family of noising processes that can be used for generative modeling tasks. Consider the process
32 f(()) oeE, To π (x), E p(s),
(11)
where f(( is some transformation of the data, which could be nonlinear. At 0, fo the identity function, and O. Thus, is distributed according to the data distribution, i.e., (u()))) n (zo).
The noising process then gradually eliminates information from the samples using ft, and increases the variance of noise o, At t 1, ft would become the zero function and we have 0i = 1. Thus, 21 would be distributed as qi (x1) = p(x1).
We now demonstrate that the same general idea can be used to construct different noising processes for solving different vision tasks, such as diffusion image generation, super-resolution, colorization, inpainting, and torus image generation. See Fig, 2 for the examples of these sampling processes. We will demonstrate Action Matching learning these dynamics the experiment section.
Action Matching for Learning the Diffusion Dynamics Diffusion processes be viewed special case of the process in Eq. I1I. when f((z)) is linear transformation f((o)) = ay
St. === a050 + ore, 6 ~ (O,
(12)
where a₂ and or we can be chosen such that the marginals of Eq. (12) corresponds to the marginals of VP-SDE and VE-SDE (Equation 29 from [Song et al. (2020b)) This sampling process corresponds the unconditional image generation task since this dynamics transforms all the information the image into the Gaussian noise. We can Action Matching to learn the dynamics of Eq. (12). solely from samples of Eq. without any knowledge of the underlying diffusion process.
The Equations II and 12 showcase that Action Matching generalizes diffusion, by allowing fa to be any non-linear function, E ~ p be any noise model. In contrast, Denoising Score Matching uses linear function for VP-SDE and VE-SDE, with the tractable Gaussian conditional.
Super-Resolution and Inpainting Action Matching provides a lot freedom in the choice of the sampling process, which demonstrate in the conditional generation tasks. Consider the following sampling processes
2G = mask Zo + (1 mask) (ay20 + o.E),
(13)
where the mask variable has the same dimensions as wo and every coordinate the mask vector is in {0, 1}. Thus, the noising process is only applied to the subset of pixels, which can be used to learn the inpainting and super-resolution tasks. the inpainting task, the mask itself is a Bernoulli random variable that decides either the tol-half or the Nottom-half of the image is destroyed. In the super-resolution case, the mask is fixed, and keeps one pixel in each x 2 block, while the remaining pixels are transformed to noise.
5

---

Under review as a conference paper at ICLR 2023
Algorithm I Generative Modeling using Action Matching
Require:
dataset [=') of
x(x)= (0())
Require:
parametric model st(2, 0)
for learning iterations do sample batch of data {zt ~ xx) (o
sample batch of noise {E*} ~ (1 (x1) ) - p(=)
sample times { ~ Uniform[0, 1]
sample two batches {ai]r faf 17 using 2 = fr (2t))
+ OnE
80 (0(x))) - s1(x) + 1 =|[ (2i)) update the model <- Optimizer(O,, VoLa) end for return trained model sy (2 0*)
I. =
Colorization Another option for the conditional generation the interpolation between the original datapoint and its nonlinearly transformed version with some noise added. For instance, we do image colorization using this approach
0270 + 0, + gray(ro)),
(14)
where the function gray(2o) returns the grayscale version of image zo. Note that function gray (zo) not injective, since it maps several different colorizations to the same grayscale image. As we show further, Action Matching sampling process is bijection; hence, the addition of noise is crucial for sampling from the data distribution images given the same conditioning. At the same time, the added noise partially destroys the information. To avoid this corruption of the conditional image, we concatenate the original grayscale image with the input.
Torus Finally, we consider the problem of learning stochastic dynamics on a manifold. Here we consider a distribution on torus, where every coordinate of the data vector to be in [0, 1] with periodic boundary conditions. Then, the sampling process interpolating between the data distribution on the noise distribution is
Generative Modeling
24 (za + ØイE) mod 1, E~ E~N(0,
(15)
Note that qı converges to the uniform distribution on the torus when o; + DC.
4.2 LEARNING, SAMPLING, AND LIKELIHOOD EVALUATION OF ACTION MATCHING GENERATIVE MODELS
Learning Once we define the noising process for 4t, E [0, 1], we apply Action Matching as described in Algorithm I It samples points with diffferent stee-steps and then minimizes the objective ( w.r.t. the parameters of sz(z, 0). In practice, we found that the performance of Algorithm [ might be hindered by high variance of the objective estimate. To reduce the variance of the objective (7). we propose to weight it over time and also adaptively select the distribution of sampled time-steps.
We derive the weighted KILBO objective in Appendix [A] and further discuss the details of training in
Appendix
B
Sampling We sample from the target distribution via the trained function (x((),, 0*) by solving the following ODE backward time:
i = V (z))) x(t = =s, E~ p(E).
(16)
Dt
Recall that this sampling process is justified by Eq. B where st(x(t), 8*) approximates s;.
Evaluating the Log-likelihood for the generation tasks can be done by integrating the same ODE forward, i.c.,
log loggo((1() (x)))) log qu (z(1)) + /' dt v*)(((()),
@ - === a0t =) = (17)
at
where we approximate st by s, (x(()), and assume the density qı (x) to be a known analytic distribution.

---

Under review as a conference paper at ICLR 2023
0
Score Matching
Action Matching
Figure 3: Illustration of the difference between Score Matching and Action Matching noising processes the colorization task. We argue that Action Matching provides more efficient way to learn the colorization model since the process requires less changes between the input and the resulting images. The additional channels are used condition all the inputs on the grayscale image.
4.3 ACTION MATCHING VS. SCORE MATCHING GENERATIVE MODELS
In this section, we give more insights Action Matching by drawing connections and highlighting differences to Score Matching (Hyvärinen & Dayan 2005) and recently introduced generative models relying on Score Matching (Song al. 2020bb Ho al. 2020).
The most fundamental difference between Action Matching and Score Matching is that they approach completely different estimation problems. Indeed, Score Matching estimates the gradient of the log-density from samples of the distribution, while Action Matching cannot be applied for a single distribution. Instead, Action Matching learns the underlying mechanism of a stochastic dynamics, i.c., learns how distributions change in time. We schematically depict this difference in Fig. !
Despite the fundamental difference in the problem setup both methods can be applied for generative modeling. For generative modeling, we have the freedom to choose the stochastic dynamics between the target distribution and the prior. Hence, by choosing the dynamics be diffusion (Fokker-Planck equation) with known drift and diffusion coefficients. one can learn the score of every marginal and then sample using the corresponding ODE or SDE (Song & Ermon 2019). Furthermore, the drift term is affine then Denoising Score Matching |innent] 2011) can be applied to learn the score model. More precisely, the model from (Song et al. (020bb requires samples from the noising process and the analytic formulas for the drift and diffusion coefficient, where the drift term has to be affine.
Action Matching requires only samples from the process to learn the dynamics. Hence, it includes the case of diffusion even without the knowledge of the drift and diffusion coefficient. Moreover, it can learn a much broader family of generative processes, which can have better properties for different applications. In Fig. 3 we give an example of such a process for the colorization task. Since Score Matching can be defined only for the diffusion, its forward process removes all the information about the image resulting in pure noise. In contrast, for Action Matching, we can remove just the information about the color of image while adding some low- noise along the way. For both models, we concatenate the grayscale image with the input. However, we argue that, for Action Matching, less computational efforts are needed since we have to apply less modifications to the original image to color it. We discuss and provide evidence for this in Section 5.1]
5
EXPERIMENTS
5.1 GENERATIVE MODELING
Action Matching has wide range of applications in modeling density evolutions. In this section, we showease the applications of Action Matching in generative modeling tasks, since they are among the most challenging high-dimensional stochastic dynamics. Action Matching generative models should not be directly compared with diffusion models, as they make different assumptions, and have access to different information. A score matching diffusion model such as VE-SDE and VP-SDE explicitly relies on the analytic forms of drift and the diffusion coefficients the SDE. In contrast, Action Matching infers the underlying vector field of any arbitrary continuous stochastic dynamics, solely from the samples. For this reason, we expect Action Matching generative models to under-perfone in this setting.
7

---

we report FID evaluated between gen- Under review as a conference paper at ICLR 2023
BPD
FID4
Table 1: Experimental results for Ac- tion Matching (AM) and Score Match- ing (SM) on computer vision tasks. Diffusion and Torus map images to
Dataset
Task
SM
AM
SM
AM
CelebA
Diffusion
2.56
3.78
4.60
18.07
CelebA
Superres
1.22
4.92
known distributions; hence, for them, report negative log-likelihood in bits per dimension (BPD). For all tasks,
CelebA
Inpainting
2.02
10.71
CelebA
Torus
3.90
18.09
CIFAR-10 10
Diffusion
3.19
4.31
12.05
53.86
CIFAR-10 10
Superres
5.94
26.42
erated images and the test data. For
CIFAR-10 R-
Colorization
5.35
7.91
CelebA, we use 20k images. For
CIFAR-10 10
Torus
6.42
39.42
CIFAR-10, we use 10k images.
Figure 4: Faster conver- gence of Action Match- ing (AM) compared to Score Matching (SM) in FID values and gen- erated samples qual- ity for the colorization task on CIFAR-10.
120
Action Matching Score Matching
100
80
FD
20
20000
40000
60000
steps
We apply Action Matching to MNIST, CelebA (Liu et al. 2013) and CIFAR-10 datasets for a variety of computer vision tasks. Namely, perform unconditional image generation via diffusion well as conditional generation for super-resolution, in- painting, and colorization tasks. addition to these settings, also learn unconditional image generation torus, where Denoising Score Matching can not be applied in the original formulation.
For the baseline in unconditional image generation tasks, which the diffusion-based generative process trained with Denoising Score Matching. For the baseline in conditional image generation tasks, follow (Saharia al. (022), and condition the model by concatenating the conditioning image as an additional channel with the main input. We refer all baselines as Score Matching (SM in Table I and Fig. We discuss further implementation
the model from (Song al. 2020b),
details in Appendix D.I
We train all models for 300k iterations and report the negative log-likelihood in bits per dimension (BPD) and FID scores (Heusel et al. (017)) in Table We demonstrate generated images by Action Matching in Appendix and provide animations the generation in geithub.co//eeeee wowataaiiioomaaaiiiinnggggggg
We observe that Denoising Score Matching performs better than Action Matching on all tasks, which was expected due to the additional information that the Denoising Score Matching objective uses about the underlying process. However, we discussed in Section 4.3 we expect Action Matching to converge faster on the conditional image generation tasks, as only needs learn cross-domain transformation, rather than learning the conditional generation from the Gaussian noise. experimentally verified this hypothesis by evaluating the FID throughout the training process, on the colorization task, shown in Fig.
5.2 SCHRÓDINGER EQUATION SIMULATION
In this section, we demonstrate that Action Matching can learn a wide range of stochastic dynamics by applying it to the dynamics of quantum system evolving according to the Schrödinger equation. The Schrödinger equation describes the evolution of many quantum systems, and in particular, it describes the physics of molecular systems. Here, for the ground truth dynamics, we take the dynamics of an excited state of the hydrogen atom, which is described by the following equation
① 0t (xx
1
(x, t)
2 (xx
(18)
xx]
8

---

Under review as a conference paper at ICLR 2023
0.90
00000
Matching
Anneeleed Langevin SM
SSM
Langevin SSM
Scote Squared Emar
0.00
Average KND
0.3
00015
035
0.0
208
LO0E
00000
008
130k
train iterationn
train iteracions
Figure 5: On the left, demonstrate performance of compared algorithms in terms of average MMD over the time of dynamics. The MMD is measured between generated samples and the training data. On the right, we report squared error of the score estimation for the score-based methods.
Table 2:
Performance of Action Matching and the Annealed Langevin Dynamics (ALD) for the Schrödinger equation simulation. For ALD, we esti- mate the scores in two ways: Score Matching (SM) and Sliced Score Matching (SSM). We also demon- strate that even using true scores does not allow for the precise simulation.
Method
Average MMD]
AM (ours)
5.7 1.7.10-44..1 10 10
ALD SM
4.8 188-102244.-00 10-3
ALD + SSM
1.7.10-2
ALD True Scores
3.6 36.110221411100
The function (x t) R x R is called wavefunction and it completely describes the state of the quantum system. particular, defines the distribution of coordinates æ by defining its density as gx(x) = (xx, 1) which dynamics is defined by the dynamics of (1, () in Eq. (8).
For the baseline, we take Annealed Langevin Dynamics as considered (Song Ermon 2019). It approximates the ground truth dynamics using only scores of the distributions by running the approximate MCMC method (which does not have access to the densities) targeting the intermediate distributions of the dynamics (see Algorithm 3). For the estimation of scores, consider Score Matching (SM) (Hyvärinen & Dayan 2005). Sliced Score Matching (SSM) (Song et al., (020a)), and additionally evaluate the baseline using the ground truth scores. For further details, we refer the reader to Appendix D.2 and the code wiwggg...toomiiee cww//action-matchiinii
Action Matching outperforms both Score Matching and Sliced Score Matching, precisely simulating the true dynamics (see Fig. 5 and T Despite that both SM and SSM accurately recover the ground truth scores for the marginal distributions (see the right plot in Fig. 5) one cannot efficiently use them for the sampling from ground truth dynamics. Note, that even using the ground truth scores in Annealed Langevin Dynamics does not match the performance of Action Matching (see Table since it is itself an approximation to the Metropolis-AAjusted Langevin Algorithm. Finally, we provide animations of the learned dynamics for different methods (see to illustrate the performance difference.
CONCLUSION
In this work, we discussed how any continuous dynamics (under mild conditions) can represented by unique continuous vector field minimizing the kinetic energy. This representation provides a rigorous mathematical formulation for the problem of learning stochastic dynamics. We then presented Action Matching, as a variational framework for learning this unique vector field solely from samples of the dynamics. We further demonstrated that Action Matching can learn a wide range of continuous dynamics, including those of diffusion. We believe the flexibility that Action Matching introduces will be useful applications in natural sciences, where stochastic dynamics appear, but the underlying mechanisms not controlled, and thus only make observations.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
David Griffiths and Darrell F Schroeter. Introduction quantum mechanics. Cambridge university press, 2018.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by time-scale update rule converge a local nash equílibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Aapo Hyvärinen and Peter Dayan. Estimation of nor-normalized statistical models by score matching.
Journal of Machine Learning Research, 6(4), 2005.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Felix Otto. The geometry of dissipative evolution equations: the porous medium equation. 2001.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234 241. Springer, 2015.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022.
Tim Salimans and Jonathan Ho. Should ebms model the energy or the score? Energy Based Models Workshop-ICL. 2021, 2021.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 574-584. PMLR, 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.1344 13456, 2020b.
Juan Luis Vázquez. The porous medium equation: mathematical theory. Oxford University Press Demand, 2007.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computa- tion, 23(7):1661-1677 2011.
10

---

Under review as a conference paper at ICLR 2023
A ACTION MATCHING
Proposition. For an arbitrary variational action s, KILBO(s) is a lower bound on the true kinetic K(s*), and the gap can be characterize with
I wrEar(s))
[ [Voi(a))
@st(x) at
dlogwt
(s)BO(s) we, By,, (4) se, (x)] wao Bee (2) (0)1 (x)]
st(x)
dt
dl
= =K- - AC-ACTION-GAP(S.s")
Thus, since K. is not a function of s, the following optimizatiion problems are equivalent arg max [KILBO(s)) arg (e 8)),
(19)
where the equality is up to an additive constant. The KILBO gap is tight iff (s) = (z).
Proof.
(CTION-GA(S,SS)
LEEiia Vs Vs
wi(l V-- 1| dxdd
dxdt
We q (=)((Vs)())) Vst (1))
wiqn(x) V>| duull
(v))
(1)
We <<(=]V (g: Vx) ( (rge(() V dadtt
(x)) 1
V+| dxdd
de de +
1 [ dt (= Bgelte) si(x)) dure
0sc(z)]
Wi B (.r
2
dt + we ot dt K
(z)
dloowe
[+)11
Ø1 + sx
wi Bae, (4)) 180 (x)] win Ban, (0) [x)]
KILBO(()) KK,
where in (1), we have used Ju (Vg. f) = fav (fg. ds) - Sv g(V f)dr, and in (2) we have used integration by parts.
B GENERATIVE MODELING IN PRACTICE
In practice we found that the naive application of Action Matching (Algorithm for complicated dynamics such image generation might exhibit poor convergence due to the large variance of objective estimate, Moreover, the optimization problem
min 1
2 d: (z) Vx. Vsj (x)))
(20)
might be ill posed due the singularity of the ground truth vector field Vsğ. Indeed, consider the sampling process
x = fe(ao) ote, xo ~ T(x), e~ Nxx 1),
(21)
where the target distribution is mixture of belta-functions
1 Txx) N Σ 8(x x²).
(22)

---

Under review as a conference paper at ICLR 2023
Algorithm 2 Generative Modeling using Action Matching (In Practice)
Require:
dataset (=1, of n (xx) (0(2)
Require:
parameteric model (a, 0), weight schedule w())
for learning iterations do sample batch of data {<t In ~ (x) (0(x)
sample batch of noise {e" J ~ Q1 (21)
sample times { p(t)
sample two batches {af }. faf | using ot. = fr + onE
L=
8o 50 (() - (<f (1) (1)+ V<r((t (l (++)+ (o.
08 ((t)) + 8 (
awir") oe
P(P
update the model < Optimizer(O, VoLş) end for
return trained model sz(z
Denoting the distribution of x, as qe (z), we can solve the continuity equation
dqe
(ge Vs)
(23)
Ōt
analytically (see Appendix
C..
The ground truth vector field is
log ae + af((x')]
Σ
98 ( fe(x*)) ot
= N(x
(24)
Σ.
Öt
For generative modeling, it's essential that qu (x)); hence, lime-+o o, = 0 and lim; f(( Assuming that of is continuous and differentiable at 0, in the limit, we have
1 2
lim || V (x))l² x lim o and t
lim t-0
at (v) x lim 200 of
(25)
Thus, the loss can be properly defined only on the interval & E (6, 1], where > In practice, we want to set 8 as small as possible; hence, ideally, we want to learn on the whole interval i E [0, 1]. can get rid of the singularity just by reweighting the objective time, i.e.,
1 at (=)))Vss())) - (x] drdt = 2/2 ((t))) (=)) - Vs; (z)] dzdt.
(26)
To give example, we can take o, Vt and fc(x) -VI-t, then w()) (1 t)18/2 cancels out the singularities at t = 0 and f = 1.
The second modification of the original Algorithm iis the sampling time-steps for the estimation of the time integral. Namely, the optimization of (26 is equivalent to the minimization of the following objective
L(s) = w(ti)st, (x)9 (x)dx= - ((to)sro (2)gt (x) +
(27)
boundary part
+ ((²) I (z) (w((t + w(t)
Os((x)
Ow(()
+ sx(x)
drdt
(28)
Øt
Øt
middle part
which consists of two parts. Estimation of the boundary part involves only sampling from 4i, and 4;, while the middle part estimate depends on the distribution of time samples, i.e..
p(t)
2ミヒ (0)0 + (1)) ds((z)
Ow(t) Ot
gi
(x)
+ (x)
drdt ~
(29)
ptt)
0t
1 l p(t)
@s((x)
se(x)) Ow(l)
Eroplty Bangi (x)
( V+)
w Ot
(30)
0t
12

---

Under review as a conference paper at ICLR 2023
Note that for every choice of p(t) we get unbiased estimate of the original objective function. Thus, we can design p(t) to reduce the variance of the middle part of the objective. our experiments, observed that simply taking p(t) proportionally to the standard deviation of the corresponding integrand significantly reduces the variance, i.e.,
p(t) & v Beang (Se - E Se 2 -0 + w(t) Ūs{(z)
Ow(t) Ot
+ s((x)
(31)
Ot
We implement sampling from this distribution by aggregating the estimated variances throughout the training with exponential moving average and then follow by linear interpolation between the estimates.
C "SPARSE DATA" REGIME
We start with the case where the dataset consists only of a single point zo E IRd
2a (x))) = 8(x mn), ke( = Na (x(x),
(32)
Then the distribution at time is
(x(x) = - f dr'
= f((x)) of).
(33)
The ground truth vector field comes from the continuity equation
dq:
(V,
log -V log ue v) - o).
(34)
at
Öt
For our dynamics, we have
log =
0 d 0t
log
(gg())o))
1
|| [2(2))11
(35)
ot
2
20
1
1
af((zo)
- d 0l
log
|| fz00)
log oe
x fc(xo),
(36)
Øl
01
d ①
log 0f(()o))
log o
x fe(wo), fe(wo))
(37)
0l
o7
01
1
log 9t- =
xx f((z)));
(38)
o,
0
( (Vlogg: qu (x f((2)) log o, + Of((xo)
log 9t =
log ae
(39)
0t
0t
Ot
Ot
Matching the corresponding terms in the continuity equation, we get
logo, Of((zo) Øt
U (a - f(())) Øt
(40)
For the set of delta deltt-func we denote
(z) = Σ 5(= (x(x) = Σ (x),, = d<x
(41)
Due to the linearity of the continuity equation w.r.t. we have
ΣdE
Σ Ogi. Σ q)))))
Σ ai
(
o Ot
logg qं + (Vlogg 0) + (V, 0.
(42)
0t
0/ We first solve the equation for 0/. = 0, then for at loga, and join the solutions. For we look for the solution in the following form
0.
0t
ot
A
Σ Val. xi(x) Aix f((=))))''')))
(43)
Egg
gt
13

---

Under review as a conference paper at ICLR 2023
Then we have
A
A E V2gt
(V,
v
EVgl
Vgl.
+
(44)
Σ, q
Σ, q
Σ Val
+ A
Σ 21
v
log
qi
||
d
(45)
(
E.
o2
[V) =
A
Σ v
+ Σ di
log
d
(46)
Σ
02
and from 42) we have
ΣΦ d Ot
log logo, +
log ghg 1o + o² θ
log or V log gh
+ (V.
= 0.
(47)
at
From these two equations we have
Σ VV vo) - o6
A Σn
Σ V
+ A Σ
V
log
||
2-
d
(48)
o2 = o 0
Σ ge d 0t
log lo
A
Σ V
log oi Σ 9/
V log gf |?
Σ, o
Ot
(49)
Thus, we have
A A= -=
0 log dz-
(50)
Ot
For 0 ae log oe = 0, we simply check that the solution is
1 Σ Of((xt)
rf
% ot
(51)
Σ. q:
Indeed, the continuity equation turns into
Cai((vlog
logg of vy
@f((x")
+ (V. (V, = 0.
(52)
Ot
From the solution and the continuity equation we write E, (e (,, vy) in two different ways.
ΣadV.on) =
1
Σ
Vgi
Vg..
Σ
ge
0f((z')
+ Σ
Vgl.
0f((2*)
(53)
Σ
Ot
Øt
0f((x')
Σ Vgl. Ug
+
Σ VgL.
(54)
ōt
Thus, we see that 51 is indeed a solution.
Finally, unifying vo and vy. we have the full solution
== - ( ② logar 0e
of
Σ Vol
E. Σ de
1
0f((z')
of), (55)
Ot
Σ qi
Øt
log ot + 0f((z')
0 Σ, ot
Σ di
(x - fe(x'))
(56)
0t
Øt
14

---

Under review as a conference paper at ICLR 2023
input
colorizations
Figure 6: Illustration that Action Matching can learn one to many relations using low variance noise added to the image. Here, we sample different col- orizations starting from the same grayscale input adding different samples noise.
D IMPLEMENTATION DETAILS
D.1 DETAILS OF ACTION MATCHING GENERATIVE MODELS
For the architecture of the neural network parameterizing 8t, we follow (Salimans Ho 202I In more details, we parameterize se (x) as ||unet((, x) - ==Il where unet(t, x) is the U-net architecture (Ronneberger et al. 2015). For the U-net architecture, we follow (Song al. 2020b) with the only difference is that we set the channel multiplier parameter to 64 instead of 128, thus, narrowing down the architecture. We have to narrow down the architecture since Action Matching requires taking the derivative w.r.t. the inputs at each iteration, which is a downside compared to Denoising Score Matching. Otherwise the training of one model takes week on 4 gpus. We consider the same U-net architecture for the baseline to parameterize V log 4t-
For diffusion, we take VP-SDE from (Song et al. 2020b which corresponds to a exp(-2 B (s))ds) and 0; /1 c1-(xppp_o (s)) where (s)) 0.1 + 19.9t. For other tasks we take og and a, on the torus, we first normalize the data such that every pixel in [0.25, 0.75). Thus we make sure that the shortest distance between the lowest and the largest pixel values is maximal on the circle [0, 1].
t. All images are normalized to the interval [-1, 1]. For image generation
Although Action Matching learns deterministic mappings, it is possible learn one-to-many map- pings by adding small amount noise to the data. For example, each row of Fig. shows that Action Matching has learned generate different colorizations from a single grayscale CIFAR-10 image. using different noise samples added to the grayscale image in Eq.
D.2 DETAILS ON THE SCHRÖDINGER EQUATION SIMULATION
For the initial state of the dynamics
i
1 (xx 1 (1,(1,))
(z,
(57)
01
we take the following wavefunction
(xx t = 0) x (x) + v221(()), and (=o(z) t
(58)
where nt, I, m are quantum numbers and yntm is the eigenstate of the corresponding Hamiltonian (see Griffthss Schroeter (2018)). For the details on sampling and the exact formulas for the initial state, we refer the reader to the code = 14 1 time units in the system h 1, m. = I, = I, Eo = 1 collecting the dataset of samples from qi- For the time discretization,
We evolve the initial state for
take 10x steps: hence, sample every 14 time units.
To evaluate each method, we collect all the generated samples from the distributions qt, t [0, T] comparing them with the samples from the training data. For the distance metric, we measure the MMD distance (Gretton et al. 2012) between the generated samples and the training data at 10 different timesteps 16
,,..., 10 and average the distance the timesteps. For the
15

---

Under review as a conference paper at ICLR 2023
Annealed Langevin Dynamics, set the number of intermediate steps for M 5, and select the step size dt by minimizing MMD using the exact scores log (z)).
For all methods, hidden units each. The architecture z(( takes æ E I and IR and outputs 3-d vector, i.c. 2(t,z)) R IR3 - R For the score-based models it already defines the score, while for action matching we use (x) [ x) =|| the model and the vector field is defined as (s)
the same architecture, which is multilayer perceptron with 5 layers 256
Algorithm 3 Annealed Langevin Dynamics
Require: score model 8y (xr),, step size dt, number of intermediate steps M Require:
initial samples 2/ E Rd
for time steps [0. do set the target distribution dt, such that (x) R V log (xr) for intermediate steps E l..... M do
s¹ ~ ~NO0 1) di
2 + 2 stat) dtt end for save samples s end for
return samples {=1]==
16

---

Under review as a conference paper at ICLR 2023
E
IMAGE EXAMPLES FOR ACTION MATCHING
G
6
0
3
8
2
9
2
7
4
3
4
3
7
6
6
4
4
6
7
9
5
9
2
G
3
4
8
3
8
5 8 4 9
2
9
7
6
3
9
4 7 9 8
5
8
9
1
0
3
5 3 6 9 J 0
8
2
0
4
3
9
5
3
9
4
3 5 3
9
3
5
9
8
4
4
2
9
0
0
2
5
7
4 4
5
2
0
5
0
6
5 3 6
8
9
7
6
6 5
6
2 3
4
8
7
0
7
8
2
7
7
9
8
8
2
6
44
Figure Action Matching on MNIST diffusion (on the left), torus (on the right).
diffusion
Figure 8: Action Matching on CelebA for diffusion.
superres
Figure 9: Action Matching on CelebA for superres.
17

---

Under review as a conference paper at ICLR 2023
inpaint
Figure 10: Action Matching on CelebA for inpaint.
torus
Figure 11: Action Matching on CelebA for torus.
diffusion
Figure 12: Action Matching on CIFAR-IO for diffusion.
18

---

Under review as a conference paper at ICLR 2023
superres
Figure 13: Action Matching on CIFAR-10 for super-resolution.
color
Figure 14: Action Matching on CIFAR- 10 for colorization.
torus
Figure 15: Action Matching on CIFAR-10 for torus.
19


Reviews:
[{"review_id": "vAe5XNU1cV", "reviewer": "Reviewer_Ln22", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper an action matching method to learn broader family of stochastic dynamics. Instead of modeling the distributions at each time-step, the action matching directly learns an underlying mechanism to move samples in time. Experimental results show that the proposed method can be applied to image super-resolution, colorization, and so on. \n", "summary_of_the_review": "Lacking analysis and comparisons with the related methods. ", "strength_and_weaknesses": "Strength: \n\nUsing action matching to avoid modeling the distributions at each time-step is interesting. Experimental results are able to demonstrate the effect of the proposed method.\n\nWeaknesses: \n\nAction matching is based on the diffusion model. However, it is not clear whether the proposed action matching performs better or not. \n\nIn addition, it lacks comparisons the with the related methods. Only showing the results of the proposed method is not enough. \n\nIt is not clear whether the proposed action matching has a better convergence property or not. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_WVrF5ilXB", "reviewer": "Reviewer_dWGD", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors show that many continuous dynamics can be represented as a continuous vector field and propose a method called action matching that can learn the vector field. One key idea is that instead of learning the distributions themselves, the method aims to learn only the dynamics. This can be effective when dynamics are simple but distributions are complex. On multiple image generation tasks, the method outperforms score matching by a clear margin.\n\n\n", "summary_of_the_review": "Overall the study presented in the paper is sound and promising. Both the theoretical and empirical results are convincing.", "strength_and_weaknesses": "Strengths:\n+ The proposed method is theoretically sound and general. It can potentially have a wide range of impact beyond image generation.\n+ Multiple vision tasks are evaluated, including diffusion, colorization, and impainting. Furthermore, results show consistent improvement on multiple image datasets. I find the results convincing.\n\nWeaknesses:\n- While the proposed method is general, the experiments are limited to low-resolution image generation tasks. Experiments on tasks beyond image generation (e.g. natural sciences ones), even just toy examples, can be interesting.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "M4iJjaepGS", "reviewer": "Reviewer_Sbd8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a method that enables us to learn a broader family of stochastic dynamics by directly learning an underlying mechanism to move samples in time without modeling the distributions at each time-step. The contributions are as follows:\n\n1. Action Matching relies only on samples and does not require any knowledge of the underlying stochastic dynamics by learning only the dynamics, which is based on a mathematically well-defined formulation and useful when a complicated distribution has simple dynamics. \n2. They discussed Action Matching as a variational framework for learning these dynamics\n3. For applications of Action Matching, they performed computer vision tasks with several stochastic equations such as generative modeling, super-resolution, colorization, and inpainting. \n\n", "summary_of_the_review": "Based on the comment above, I consider that the weakness of this paper outperformed the strength at this stage, then I cannot give a higher rating.", "strength_and_weaknesses": "Strength: \n* The idea of the above first contribution is interesting and may have a novelty.\n* The experimental results clearly show the superiority of the proposed approach.\n\nWeakness:\n* The presentations (in particular the first half of the main text) and figures were unclear to me, which is described as the specific comments.\n* It may be a small weakness: in my understanding, the generative model of the proposed approach needs the definition of dynamics, but the previous action-matching methods also require it. \n\nSpecific comments\n* In the Introduction, q_t and “score matching” were not defined and then Fig. 1 was meaningless.\n* In my understanding, this method can apply to the dynamics which can be expressed as curl free vector fields. Is this correct? Are there any other dynamics than you analyzed? \n* Fig. 3: what are the two types of pictures for Score/Action matching?\n* Fig. 4: If they are generated, the information of input and the dynamics may be necessary for understanding. Moreover, MINST was not used in the main text.\n* FID may be not defined.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BL8CkbMYNl", "reviewer": "Reviewer_iYDX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method called Action Matching for learning generative models. In particular, this technique falls along the lines of score-based / diffusion probabilistic models, by learning about the 'derivative' of the probability distribution rather than directly modeling the density.\n\nThe paper claims that the main difference between action matching and score matching is that action matching learns about the derivative along the time axis. The argument is that a distribution can be complicated to model for each time slice, but actually has very little change w.r.t. time, so modeling this change w.r.t. time can make the learning problem easier.", "summary_of_the_review": "The paper proposes Action Matching to learn the change in distribution through time. The paper claims that Action Matching can handle more general forward processes (decolorization, masking...). \n\nHowever, it appears that the idea of learning \"time scores\" is not new, and previous diffusion based methods can already handle these more general forward processes. Furthermore, the writing is not clear and concise, and the experimental results are weak.", "strength_and_weaknesses": "I don't think the method proposed in this paper is new.\nIt seems to be the same as the idea of learning \"time scores\", e.g., change in distribution through time.\nhttps://arxiv.org/pdf/2111.11010.pdf\n\nFurthermore, the paper claims that Action Matching can be preferred over score matching since score matching can only handle the forward process of noising, and not of more general transformations like decolorization. I don't think this is true, e.g., this paper below.\nTheir experimental results are much better than the ones presented here.\nhttps://arxiv.org/pdf/2201.11793.pdf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject