Diffusion Models are Evolutionary Algorithms
Authors: Yanbo Zhang
Benedikt Hartl
Hananel Hazan
Michael Levin
Venue: ICLR 2025 Poster (2025)
Keywords: Machine learning
evolutionary computation
Evolutionary Algorithms
Diffusion Models
Optimization
Primary area: generative models

Abstract: In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.

Published as a conference paper at ICLR 2025
DIFFUSION MODELS ARE EVOLUTIONARY ALGO- RITHMS
Yanbo Zhang!* Benedikt Hartl Hananel Hazan I++ Michael Levin
1 Allen Discovery Center at Tufts University, Medford, MA, 02155, USA
2 Institute for Theoretical Physies, TU Wien, Austria
3 Wyss Institute for Biologically Inspired Engineering at Harvard University, Boston, MA, 02115, USA
ABSTRACT
In a convergence of machine learning and biology, we reveal that diffusion mod- els evolutionary algorithms. By considering evolution denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherentl perform evolutionary algorithms, naturally encompassing se- lection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iter- ative denoising as originally introduced in the context of diffusion models to heuristically refine solutions in parameter spaces. Unlike traditional approaches. Diffusion Evolution efficiently identifies multiple optimal solutions and outper- forms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and ac- celerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in tigh-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffu- sion and evolution not only bridges two different fields but also opens new av- enues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian discrete diffusion models in the context of Diffusion Evolution.
INTRODUCTION
At least two processes in the biosphere have been recognized as capable of generalizing and driving novelty: evolution, a slow variational process adapting organisms across generations to their envi- ronment through natural selection (Darwin 1959 Dawkins 2016); and learning, faster transforma- tional process allowing individuals to acquire knowledge and generalize from subjective experience during their lifetime (Kandel) 2013 Courville et al.) 2006;) [Holland| (2000) Dayan & Abboti 2001).
These processes are intensively studied in distinct domains within artificial intelligence. Relatively recent work has started drawing parallels between the seemingly unrelated processes of evolution
and learning (Watson & Levin 2003) Vanchurin et al., 2022) Levin 2022; Watson et al. 2022 Kou-
varis al. 2017 Watson & Szathmáry [2016] Watson et al. 2016 Power et al. 2015 Hinton al.
1987 Baldwin. 2018). We here argue that in particular diffussio models (Sohl Dickstein et al. 22155
Song et al. [220bb] Ho al., 2020 [Song et al., 2020a), where generative models trained to sam- ple data points through incremental stochastic denoising, can be understood through evolutionary processes, inherently performing natural selection, mutation, and reproductive isolation.
The evolutionary process is fundamental to biology, enabling species to adapt to changing envi- ronments through mechanisms like natural selection, genetic mutations, and hybridizations (Rosen) 1991 Wagner 2013 Dawkins 1996); this adaptive process introduces variations in organisms' ge- netic codes over time, leading to well-adapted and diverse individuals (Mitchell & Cheney| 2024] Evolu- Levin 2023 Gould 2002, Dennett 1995 Smith & Szathmary, 1997. S
tionary algorithms utilize such biologically inspired variational principles to iteratively refine sets of numerical parameters that encode potential solutions often rugged objective functions (Vikhar
"qqual contributions. Author of correspondence: Hananel @Hazan. org.11

---

Published as a conference paper at ICLR 2025
evolution
XT
XO
diffusion
low fitness
high fitness
Figure 1: Evolution processes can be viewed as the inverse process of diffusion, where higher fitness populations (red points) have higher final probability density. The initially unstructured parameters are iteratively refined towards high-fitness regions in parameter space.
2016 Golberg 1989
Notably, recent breakthroughs in deep learning have led to the development of diffusion models- sandellsgeneratiiv algorithms that iteratively re- fine data points to sample novel yet realistic data following complex target distributions: models like Stable Diffusion (Rombach et al. 2022) and Sora (Brooks et al. 2024) demonstrate remarkable real- ism and diversity in generating image and video. Notably, both evolutionary processes and diffusion models rely on iterative refinements that combine directed updates with undirected perturbations: in evolution, random genetic mutations introduce diversity while natural selection guides popula- tions toward greater fitness, and in diffusion models, random noise is progressively transformed into meaningful data through learned denoising steps that steer samples toward the target distribution. This parallel raises fundamental questions: Are the mechanisms underlying evolution and diffusion models fundamentally connected? Is this similarity merely an analogy, or does it reflect a deeper mathematical duality between biological evolution and generative modeling?
Grefenstette 1993 Holland 1992).
To answer these questions, we first examine evolution from the perspective of generative models. By considering populations of species in the biosphere, the variational evolution process can also be viewed transformation of distributions: the distributions of genotypes and phenotypes. Over evolutionary time scales, mutation and selection collectively alter the shape of these distributions. Similarly, many biologically inspired evolutionary algorithms can understood that way: they optimize an objective function by maintaining and iteratively changing a large population's 's distri- bution. This concept is central to most generative models: the transformation of distributions. Vari- ational Autoencoders (VAEs) (Kingma 2013). Generative Adversarial Networks (GANs) (Good- fellow et al. 2014), and diffusion models are all trained to transform simple distributions, typically standard Gaussian distributions, into complex distributions, where the samples represent meaningful images, videos, or audio, etc.
On the other hand, diffusion models also be viewed from evolutionary perspective. As gen- erative model, diffusion models transform Gaussian distributions an iterative manner into com- plex, structured data points that resemble the training data distribution. During the training phase, the data points are corrupted by adding noise, and the model is trained to predict this added noise reverse the process. In the sampling phase, starting with Gaussian-distributed data points, the model iteratively denoises to incrementally refine the data point samples. By considering noise- free samples as the desired outcome, such a directed denoising can be interpreted directed selection, with each step introducing slight noise, akin to mutations. Together, this resembles an evolutionary process (Fields & Levin 2020), where evolution is formulated as a combination of deterministic dy- namics (e.g., natural selection) and stochastic mutations (Ao)
|Ao| (2055] Ping et al., (2014). If one were to revert the time direction of an evolutionary process, the evolved population of potentially highly cor- related high-fitness solutions will dissolve gradually akin to the forward process in diffusion models, into the respectively chosen initial distribution, typically Gaussian noise, see Figure
Driven by this intuition, conduct thorough investigation into the connections between diffusion models and evolutionary algorithms, discovering that these seemingly disparate concepts share the same mathematical foundations. This insight leads novel approach, the Diffusion Evolution
2

---

Published as a conference paper at ICLR 2025
algorithm, which directly utilizes the framework of diffusion models to perform evolutionary opti- mization. This can be obtained by inverting the diffusion process with the Bayesian method. Our analytical study of Diffusion Evolution reveals promising parallels to biological evolution, naturally incorporating concepts such as mutation, hybridization, and even reproductive isolation.
This equivalence provides new way of improving evolutionary algorithms and has the potential to unify developments in both fields. Mimicking biological evolution, evolutionary algorithms have shown promising results in numerical optimization, particularly for tasks that cannot be effectively trained using gradient-based methods (Wang et al. 2024] Goodfellow et al. 2014). These algorithms
thus excel in exploring complex, rugged search spaces and finding globally optimal or near-optimal
solutions (Hansen, 2016] |Hansen & Ostermeier 2001 Sehnke et al.] 2010). While the biosphere
exhibits extreme diversity in lifeforms, many evolutionary strategies, such as CMA-ES (Hansen & Ostermeier (0001), and PEPG (Sehnke et al., 2010), struggle to find diverse solutions (Lehman & Stanley, 1) However, our Diffusion Evolution algorithm offers a new approach. By naturally incorporating mutation, hybridization, and reproductive isolation, our algorithm can discover diverse solutions, mirroring the diversity of the biosphere, rather than converging on a single solution as is often the case with traditional methods. Since this parallel between diffusion and evolution exists naturally and is not imposed by our design, the two fields - diffusion models and evolutionary computing - can mutually benefit from each other. For example, we demonstrate that the concepts of latent diffusion (Rombach et al. 2022) and accelerated sampling (Nichol & Dhariwal (021) can significantly improve the performance of our Diffusion Evolution algorithm. Here, we focus on the theoretical foundations. In a complementary contribution (Hartl et al. 2024). dcep-learning-bars -ased diffusion models are employed in an analogous evolutionary setting to leverage conditional sampling
for explicit guidance. This approach enables controlling desired outputs and conditioning on fitness,
potentially accelerating optimization without additional shaping.
In the following sections, we will first review evolutionary strategies and diffusion models, introduce the mathematical connection between diffusion and evolution, and propose the Diffusion Evolution algorithm. Then, we will quantitatively compare our algorithm to conventional evolutionary strate- gies, demonstrating its capability to find multiple solutions, solve complex evolutionary tasks, and incorporate developments from diffusion model literature. Lastly, we will discuss the emerging con- nections between the derived algorithm and evolution, along with the potential implications this finding and the limitations of our algorithm. Codes are available Githut
BACKGROUND
2.1 EVOLUTIONARY ALGORITTMM
The principles of evolution extend far beyond biology, offering exceptional utility in addressing complex systems across various domains. The key components of this process - imperfect replica- tion with heredity and fitness-based selection are sufficiently general to find applications in diverse fields. In computer and data science, for instance, evolutionary algorithms play a crucial role in opti- mization (Vikhar 2016; Grefenstette [993] Golberg 1989 Holland 1992). These heuristic numer- ical techniques, such as CMA-ES (Hansen & Ostermeier 2001) and PEPG (Sehnke et al.) 2010),
maintain and optimize a population of genotypic parameters over successive generations through operations inspired by biological evolution, such as selection of the fittest, reproduction, genetic crossover, and mutations. The goal is gradually adapt the parameters of the entire population in such a way that individual genotypic samples, or short individuals, perform well when evaluated against an objective- or fitness function. These algorithms harness the dynamics of evolutionary bi- ology to discover optimal near-optimal solutions within vast, complex, and otherwise intractable parameter spaces. The evaluated numerical fitness score of an individual correlates with its proba- bility of survival and reproduction, ensuring that individuals with advantageous traits have a greater chance of passing their genetic information to the next generation, thus driving the evolutionary process toward more optimal solutions. Such approaches are particularly valuable when heuristic solutions are needed to explore extensive combinatorial and permutation landscapes.
Some evolutionary algorithms operate with discrete, others with continuous sets of parameters. Here, we focus on the latter since discrete tasks can be seen as a subcategory of continuous tasks.
3

---

Published as a conference paper at ICLR 2025
Typically, the structure of the parameter space is priori unknown. Thus, the initial population often sampled from a standard normal distribution. As explained above, this initially random population is successively adapted and refined, generation by generation, to perform well an arbitrary objective function. Thus, initially randomized parameters are successively varied by evo- lutionary algorithms into sets of potentially highly structured parameters that perform well on the specified task, eventually (and hopefully) solving the designated problem by optimizing the ob- jective function. Thus, evolutionary algorithms can be understood as generative models that use heuristic information about already explored regions of the parameter space (at least from the pre- vious generation) to sample potentially better-adapted offspring individuals for the next generation (c.f., CMA-ES (Hansen et al. 2003). etc.)
2.2 DIFFUSION MODELS
Diffusion models, such as denoising diffusion probabilistic models (DDPM) (Ho et al.
2020) and
denoising diffusion implicit models (DDIM) (Song et al. (020a)), 20220), have shown promising generative
capabilities in image, video, and even neural network parameters (Wang et al. 2024). Similar to other generative approaches such GANs, VAEs, and flow-based models (Dinh et al. 2016 Chen ct al. 2019). diffusion models transform a simple distribution, often a Gaussian, into a more complex distribution that captures the characteristies of the training data. Diffusion models achieve this, in contrast to other techniques, via iterative denoising steps, progressively transforming noisy data into less noisy (Raya & Ambrogioni 2024). more coherent representations (Sohl-Dickstein al. 2015).
Diffusion models have two phases: diffusion and denoising. In the diffusion phase, we are blending original data points with some extent of Gaussian noise. Specifically, let æo be the original data point and 2y be the fully distorted data, then the process of diffusion can be represented as:
3t. var30g + /1 a
(1)
where the amount of total noise e (0, added to the data xy at time step t E 2'] is controlled by a that is monotonously decreasing from ag original data, Zy will consist entirely of Gaussian noise. To restore such diffused data, a predictive model, typically& neural network Eo with parameter 0, trained to predict the added total noise given Tn and time step f. Thus, diffusion models can be trained by minimizing the loss function:
to aT O. Thus, while Do represents the
[ Beuu-Puu.ee-
Va,a0 + v a t) c||2
where Pdata is the distribution of training data. So, conventionally, diffusion models are understood as predicting the added noise during the diffusion process.
In the denoising phase, starting with noisy pattern, the trained models are used to iteratively remove the predicted noise from current data: from æT ~ (O, I), iteratively refine to æT 1, 2T until restoring the data Zp. In the DDIM framework, this sampling process is given by:
I aa-1
Lt
VI azco( Va
+ V 11v-rrr a of (z t) + o.w,
(3)
where controls the amount of noise w ~ N/ I) added during the denoising phase. Notably, the schedule of a; and o, will both affect the denoising process and can be chosen based on our needs under the DDIM framework.
3 DIFFUSION MODELS ARE EVOLUTIONARY ALGORITHMS
Similar to the relationship between energy and probability in statistical physics, evolutionary search can be connected to generative tasks by mapping fitness to probability density: higher fitness corre- sponds to higher probability density. Thus, given a fitness function f 1²" + R. we can choose a mapping g transform f into a probability density function p(x) [=)] When aligning the denoising process in a diffusion model with evolution, we want 2y to follow this density function, ie., p(x.. œ) (=)) This requires alternative view of diffusion models (Song et al. 2020a): diffusion models are directly predicting the original data samples from noisy versions of those samples at each time step. Given the diffusion process s Voraan + vI are, we can easily express To in terms of the noise e, and vise versa:
Lt
VI ALE
Ft / /a
and E
(4)
Vat
VI 04
4

---

Published as a conference paper at ICLR 2025
In diffusion models, the error between Do and H, is estimated by neural network, i.e, = Eg (Tt, t). Thus, Equation
4 provides an estimation Ty for Dy when replacing e with é. Hence, the sampling process of DDIM (Song et al. 2020a) in Equation B can be written as:
100 + 0- - of. + o
(5)
/=
1=
30
(=5
(b) deasity evolunton
80
(a) Diffusion Evolution on a two-peak fitness landscape: Populations near the two black crosses have higher fitness. For each individual æ; (black star), its target á. (red dots) is estimated by weighted average of its neighbors (c.f., dots within the blue disks, respectively); larger dot- size indicates higher fitness. The individual then moves small step forward the next generation (orange star). As evolution proceeds, the neighbor range decreases, making the process increasingly sensitive to local neighbors, thereby enabling global competition originally, while "zooming in" eventually to balance between optimization and diversity. (b) By mapping the population to 1-D space (dashed lines in (a)), we track the progress of Diffusion Evolution. evolution progresses, both the individuals (gray) and their estimated origins (red) move closer to the targets (vertical dashed lines), with the estimated origins advancing faster.
Figure 2:
Since the denoising step in diffusion models requires an estimation of æo, need to derive from sample æ, and the corresponding fitness f(xx). The estimation of æo can be expressed as conditional probability p(a, 2 (2))). Using Bayes' theorem and p(zg x) = [=)] yields:
p(an- = x ar.) p(æ; (w0 x))((20 x) p(æ, (x) [[x)]
p(x:)
p(x:.)
Here, p(z:|z0 = x) can be computed easily by Nax Vaja, 1 ax) given the design of the diffusion process, i.e., x2 /araa + /1 - agc. Since deep-learning based diffusion models are trained using mean squared error loss, the 2e estimated by æ; should be the weighted average of the sample æ. Hence, the estimation function of Vy becomes:
Σ gf((z)) N(ari N(tri Vanæ, - ort). xr, p(x:)
do (2t,, oz = Σ
p(æo = x: x
(7)
(x)
(xpeea(
where Peval is the evaluation sample on which we compute the fitness score, here given by the current population X, (ar)" (1) _x,²) ..... at (N), of N individuals. Equation has three weight terms: The first term g (x)) assigns larger weights to high fitness samples. For each individual sample a,, the second Gaussian term N ((x;; Vayar, - ax) makes each individual only sensitive to local neighbors of evaluation samples. The third term p(æ:) is normalization term. Hence, to can be simplified to:
1
Lo a, t) =
glf())]W(az Vaxx ax)xx x
Σ
5

---

Published as a conference paper at ICLR 2025
where Z is the normalization term:
= p(z)) Σ x
gff())) vara, -a)).
(9)
When substituting Equation 8 into Equation we can express as:
3t Vor (2 ar,
é(wr, a, t)
(10)
vI a
g and T0 into Equation 5 we derive the Diffusion Evolution algo- rithm: an evolutionary optimization procedure based on iterative error correction akin diffusion models but without relying neural networks at all, unlike previous works (Krishnamoorthy et al.) 2023 Yan Jin 2024). See the psuedocode of Diffusion Evolution in Algorithm| [ When inversely denoising, i.e., evolving from time 7 to O, while increasing Ot, the Gaussian term will initially have a high variance, allowing global exploration at first. As the evolution progresses, the variance de- creases giving lower weight to distant populations, leads to local optimization (exploitation). This locality avoids global competition, akin to reproductive isolation in biology, enabling the algorithm to maintain multiple solutions and balance exploration and exploitation. The weighted average in Equation / then acts as a gene recombination operation. Hence, the denoising process of diffusion models can be understood in evolutionary manner: ätn represents estimated high fitness pa- rameter target. In contrast, #t
and by substituting Equations
can be considered as diffused from high-fitness points. The first two parts in the Equation 5 i.e., /aa-1 + / 1 0t-1 o,c, guide the individuals towards high fit- ness targets in small steps. The last part of Equation 5 o,w, an integral part of diffusion models, perturbing the parameters in our approach similarly to random mutations.
Algorithm Diffusion Evolution
Require: Population size N, parameter dimension D, fitness function
f. density mapping function 9. total evolution steps T, diffusion schedule a and noise schedule σ.
Ensure: 06 1, AT ~ 0, 0z > 01+1,0 0 < as < ai
(N) + N00,, (NDD)
1: FT xt
b Initialize population
2: for LE [T,
I,.... do
3: Vi E [1, N]: Q; gff(x
D Fitness are cached to avoid repeated evaluations
4:
for i [1, 2,, N] do
5:
+ Σ
Na
()) ar) (1) varat
1=1
6:
2o 7
Σ
(x)
(0) varal -0)
7:
w + 10)
(i)
v1 Va
8:
x f-1 + / 10- + V 01-1 - 0
w.
v1 az
9: end for
10: end for
Figure 2[a) demonstrates the detailed evolution process of a multi-target fitness landscape with two optimal points (see exact fitness function in Appendix/AN II.I). Each individual estimates high fitness parameter targets and moves toward the target along with random mutations. The high fitness pa- rameter targets 2o are estimated based on their neighbors' fitness scores (neighbors are shown in blue disks, with radius proportional to a ltr/vax /ac. see Appendix A.3).. The estimated targets Co typically moving toward targets faster than the individuals while the individuals are successively refined in small denoising steps in the direction the estimated target, see Figure Although de often have higher fitness, they exhibit lower diversity, hence they used goal of individuals instead of the final solutions. This difference also provides flexibility in balancing between more greedy and more diverse strategies.

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
We conduct two sets of experiments to study Diffussin Evolution in terms of diversity and solving complex reinforcement learning tasks. Moreover, we utilize techniques from the diffusion models literature to improve Diffusion Evolution. In the first experiment, we adopt an accelerated sam- pling method (Nichol & Dhariwal 202T) to significantly reduce the number of iterations. In the second experiment, we propose Latent Space Diffusion Evolution, inspired by latent space diffu- sion models (Rombach et al., 2022), allowing us to deploy our approach to complex problems with high-dimensional parameter spaces by exploring lower-dimensional latent space.
4.1 MULTI-TARGET EVOLUTION
Rosenbrock
Beale
Himmelblau
Ackee
Rastrigin
Figure 3: Benchmark experiments (columns) on various fitness functions with selected evolutionary algorithms (rows). Blue (high fitness) and white (low fitness) regions represent a two-dimensional parameter space, with fitness normalized to [o, 1] for comparability (see Appendix). The Diffusion Evolution algorithm identifies multiple optima in 2D benchmarks while preserving genetic diversity. Red dots mark the final population, and gray lines show trajectories for 64 individuals. In CMA-ES, gray ellipsoids represent covariance estimates, and gray lines trace the history of estimated averages.
To compare our method to selected mainstream evolutionary algorithms, we choose five different wwo-dimensional fitness landscapes as benchmarks: The Rosenbrock and Beale fitness functions have a single optimal point, while the Himmelblau, Ackley, and Rastrigin functions have multiple optimal solutions, see Appendix for more details; we compare our method to other evolutionary | 2003), OpenES (Salimans et al. 2017), PEPG (Schnke) al. 2010), and MAP-Elite (Mouret & Clune 2015). The experiments show that our Diffusion Evolution algorithm can find diverse solutions on the Himmelblau, Ackley, and Rastrigin functions. In contrast, CMA-ES and OpenES struggle, they either focus finding single solution or get distracted by multiple high-fitnes peaks, leading to sub-optimal results. While the MAP-Elite method demonstrates diverse solutions, the corresponding average fitness is lower than that of
strategies, including CMA-ES (Hansen et al.

---

Published as a conference paper at ICLR 2025
Table 1: Average entropy of the top-64 elite populations with 100 different evolutions, along with average fitness (0 to 1) separated by commas. Higher entropy indicates greater diversity, and higher fitness reflects better solutions. See Table in the Appendix for details.
Diffusion Evolution
Latent Diffusion Evolution
CMA-ES
PEPG
OpenES
MAP- Elite
Rosenbrock
5.86, 0.88
5.64, 0.92
0.00, 1.00
0.85, 1.00
1.96, 0.71
6.00, 0.77
Beale
5.50, 0.96
5.11, 0.93
0.00, 1.00
0.35, 1.00
1.03, 1.00
6.00, 0.37
Himmelblau
5.23, 0.96
4.95, 0.89
0.00, 1.00
0.00, 1.00
0.28, 1.00
6.00, 0.28
Ackley
5.67, 0.78
5.31, 0.73
5.34, 0.66
0.04, 0.96
0.18, 1.00
5.98, 0.50
Rastrigin²
5.79, 0.64
5.34, 0.62
5.70, 0.57
0.00, 0.57
0.01, 1.00
5,95, 0.61
Rastrigin
5.82, 0.18
5.99, 0.37
5.67, 0.17
0.00,0017
0.00, 0.25
5.95, 0.18
Rastrigin 32
5.84, 0.02
6.00, 0.19
4.54, 0.01
0.00, 0.02
0.00, 0.02
5,94, 0.02
Rastrigin 256
5.84, 0.00
6.00, 0.15
2.41, 0.00
0.00, 0.00
1.65, 0.00
5.95. 0.00
Notations: highest value, second highest value, third highest value. Superscript numbers on Rastrigin func- tions indicate the dimensions of the fitness function.
method. Our experiments demonstrate that the Diffusion Evolution algorithm can identify high- fitness and diverse solutions and adapt to various fitness landscapes (see Figure and and Table
The most time-consuming part of evolutionary algorithms is often the fitness evaluation. In this experiment, we adopt an accelerated sampling method from the diffusion models literature to reduce the number of iterations. As proposed by Nichol & Dhariwal (2021), instead of the default or scheduling of DDPM, a cosine scheduling a; = cos cTtTT// 1/2 leads to better performance when T is small (see Appendix A.2 for detailed comparison). With this, we can significantly reduce the number of fitness evaluations while maintaining sampling diversity and quality.
To systematically compare different methods, we repeated the evolution 100 times for each method. In all experiments, we rescaled the fitness functions to the range to I, with representing the highest fitness (see Appendix (A.4). Each experiment
conducted with population of 512 for 25 iterations, except for the OpenES method, which requires 1000 steps to converge. To quantify diversity, we then calculated the Shannon entropy of the final population by gridding the space and counting the individuals in different grid cells (we select the top-64 fitness individuals, focusing solely on elite individuals). The results in Table show that our method consistently finds diverse solutions without sacrificing fitness performance. While CMA-ES shows high entropy on the Ackley and Rastrigin functions, finds significantly lower fitness solutions compared to Diffusion Evolu- tion, suggesting CMA-ES is distracted by multiple solutions rather than finding diverse ones (see examples in Figure B) Similarly, the MAP-Elite algorithm shows the highest diversity but com- promises fitness, yielding the lowest fitness for most tasks. In contrast, our Diffusion Evolution algorithm neurally excels at balancing quality and diversity, despite not being explicitly designed for this purpose, detailed in Appendix A.4.3
4.2 LATENT SPACE DIFFUSION EVOLUTION
Here, we apply the Diffusion Evolution method to reinforcement learning tasks (Sutton & Barto 1998) to train neural networks for controlling the cart-pole system (Barto et al., 1983). among other tasks (see Appendix A.5).. This system has cart with hinged pole, and the objective is to keep the pole vertical long as possible by moving the cart sideways while not exceeding a certain range, see Figure d). The game is terminated if the pole angle exceeds 1 or the cart position exceeds +2.4. Thus, longer duration yields higher fitness. We use a two-layer neural network of 58 parameters to control the cart, with inputs being the current position, velocity, pole angle, and pole angular velocity. The output of the neural network determines whether to move left or right. See more details about the neural network in Appendix [.... The task considered solved when a fitness score (cumulative reward) of 500 is reached consistently over several episodes.
8

---

Published as a conference paper at ICLR 2025
(a) evolution process
10
/ V Y/ 11 VI IN III IN
In
generation
8
WWAN AN IN V V /1 1I
6
4-
V WZ
50
100
150
200
250
300
350
400
450
500
sleps
(b) reward comparison
(c) latent space comparison
(d) cart-pole system
500
DiflEvo
DifiFve
Istert Diftliv (high-d)
100
CMA BS
10
2
generation
Figure 4: (a) Evolution of cart-pole tasks: The horizontal axis shows survival time, and the ver- tical axis represents generations. Each point represents an individual's final state (pole angle, cart position). Over time, agents survive longer and achieve higher rewards. (b) Latent Diffusion Evo- lution methods (red) find effective solutions within ten steps, whereas CMA-ES (gray) fails within the given generations. This latent approach also works in high-dimensional spaces (orange) with up to 17,410 dimensions. Experiments are repeated 100 times, with medians (solid lines) and ranges (25%-75%) shown as shaded areas. (c) Projecting individuals' parameters into latent space visual- izes diversity. The same projection is used for all results except for the high-dimensiooal case. This highlights the increased solution diversity of the latent method. (d) The cart-pole system includes pole hinged to cart, where the controller balances the pole by moving the cart left or right.
Deploying our original Diffusion Evolution method to high-dimensional parameter spaces may re- sult in poor performance, see high-dimensional Rastrigin functions in Table To address this issue, we propose Latent Space Diffusion Evolution: inspired by the latent space diffusion model (Rom-) bach al.) 2022), we map individual parameters into lower-dimensional latent space in which perform the Diffusion Evolution algorithm. This approach significantly improves performance. This approach is motivated from analyzing the Gaussian term in Equation 8 which we use to estimate the original points dp. In higher dimensions, the increased pairwise distances between parameters cause the evolution process to become or genotype space may have dimensions that do not effectively impact fitness, known as sloppi- ness (Gutenkunst et al., 2007). Assigning random values these dimensions often does not affect fitness, similar to genetic drift (KIMURA 1991) or neutral genes (King & Jukes 1969), suggesting the true high-fitness genotype distribution lower-dimensional. The straightforward approach directly denoising in lower-dimensional latent space and estimating high qugh-tuality targets via:
localized and consequently slower. Moreover, the parameter
Eo(Zz, a,
Σ p(z|z))=
Σ (lf(())) Vaiz, -0)2 (mm
(11)
However, this approach requires a decoder and a new fitness function f' for which can be chal- lenging to obtain. To cireumvent this, we approximate latent diffusion by using the latent space only calculate the distance between individuals. Although we do not know the exact distribution of a priori, a random projection can often preserve distance relations between populations. suggested by the Johnson-Linneestren lemma [Johnson] 1984). To this end, we change Equation 11 to:
Co a)
Σ
Σ (x)
slf(=)) Vaiz,1 1 -aa)aa a.). (12)
Z

---

Published as a conference paper at ICLR 2025
where 2 = Ex, Egg (1..1) i the dimension of æ, and dl is the latent space dimen- sion Johnsoon 1984), see Algorithm iin in the Appendix. Here we choose = in our experiments for visualization purposes. The results show the effectiveness of Latent Diffusion Evolution the cart-pole task and significant improvements benchmarks and other reinforcement learning tasks (see Table | and (5) We found that latent evolution remains effective even within significantly higher-dime higher-dimensional parameter space. For example, using three-layer neural network with 17, 410 parameters, it still achieving strong performance, see Figure Ib). By combining this approach with an accelerated sampling method, the cart-pole task can be solved in just 10 generations, using population size of 512 and one fitness evaluation per individual. As shown in Appendix [A.5] our approach demonstrates effectiveness in a variety of reinforcement learning tasks. This highlights the potential of using tools and theories from the diffusion model domain in evolutionary optimization tasks and vice versa, opening up broad opportunities to improve and understand evolution from new perspective.
DISCUSSION
By aligning diffusion models with evolutionary processes, we demonstrate that diffusion models are evolutionary algorithms, and evolution can be viewed as generative process. The Diffusion Evolution process inherently includes mutation, selection, hybridization, and reproductive isolation, indicating that diffusion and evolution are two sides of the same coin. Our Diffusion Evolution algo- rithm leverages this theoretical connection to improve solution diversity without compromise quality too much compared to standard approaches. By integrating latent space diffusion and accelerated sampling, our method scales to high nigh-dimensional spaces, enabling the training of neural network agents in reinforcement learning environments with exceptionally short training time.
This equivalence between the two fields offers valuable insights from both deep learning and evolu- tionary computation. Through the lens of machine learning. the evolutionary process can be viewed as nature's way of learning and optimizing strategies for survival of species over generations. Sim- ilarly, Diffusion Evolution algorithm iteratively refines estimation of high-fitness parameters, continuously learning and adapting to the fitness landscape. This positions evolutionary algorithms not merely as optimization tools, but also as learning frameworks that enhance understanding and functionality through iterative refinement. Conversely, framing evolution as a diffusion process of- fers a concrete mathematical formulation. In contrast to previous work (Ao, 2005), we provide an explicit and implementable evolutionary framework.
The connection between diffusion and evolution enables mutual contributions between the two fields. Diffusion models are extensively studied in the contexts of controlling, optimization, and probability theory, offering robust tools to analyze and enhance evolutionary algorithms. In our ex- periments, leveraging concepts from diffusion models enabled flexible strategies while maintaining the effectiveness of evolutionary processes. For instance, accelerated sampling methods (Nichol & Dhariwal 202I) can be applied seamlessly to Diffusion Evolution to accelerate the optimization process. Latent diffusion models (Rombach et al. (2022) inspired our Latent Space Diffusion Evolu- tion, enabling evolution in digh-dimensiooal spaces and substantially improving performance. Other advancements in the diffusion model field hold the potential to enhance our understanding of evolu- tionary processes. For instance, Gaussian noise diffusion models (Bansal al., 2024), discrete denoising diffusion models, classifier free guidance (Ho & Salimans) 2022). and theoretical stud- ies, e.g., spontaneous symmetry breaking (Raya & Ambrogioni (0224) in the generative process of diffusion models unveil entirely new possibilities and perspectives for understanding and advancing evolutionary methods; c.f. our complimentary Conditional Diffusion Evolution (Hartl et al., (022). While diffusion models are inherently designed with a finite number of sampling steps, viewing them through the lens of non-equilibrium thermodynamics, such as Langevin dynamics (Song et al., 2020b), may allow for open-ended evolution, a characteristic inherent in natural evolution.
However, this parallel we draw here between evolution and diffusion models also gives rise to sev- eral challenges and open questions. Could other diffusion model implementations yield different evolutionary methods with diverse and unique features? Can advancements in diffusion models help introduce inductive biases into evolutionary algorithms? How do latent diffusion models correlate with neutral genes? Additionally, can insights from the field of evolution enhance diffusion models? These questions highlight the potential of this duality and synergy between diffusion and evolution.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors acknowledge the Tufts University High Performance Compute Cluster and the Vienna which
Scientific Cluster which have been utilized for the research reported in this paper. M.L. gratefully acknowledges support via grant 62212 from the John Templeton Foundation, via grant TWCF0606 from the Templeton World Charity Foundation, and via sponsored research agreement from As- tonishing Labs. BH acknowledges an APART-MINT stipend by the Austrian Academy of Sciences.
REFERENCES
Ping Ao. Laws in darwinian evolutionary theory. Physics of life Reviews, 2(2): 12-:111 2005.
James Mark Baldwin. A new factor in evolution. Diacronia, pp. 1-13, 2018.
Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise. Advances in Neural Information Processing Systems, 36, 2024.
Andrew G Barto, Richard S Sutton, and Charles W Anderson. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE transactions on systems, man, and cybernetics, pp. 834-846, 1983.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators, 2024.
Ricky TQ Chen, Jens Behrmann, David K Duvenaud, and Jörn-Henrik Jacobsen. Residual flows for invertible generative modeling. Advances in Neural Information Processing Systems, 32, 2019.
Aaron Courville, Nathaniel Daw, and David Š Touretzky. Bayesian theories of conditioning in changing world. Trends in cognitive sciences, 10(7):294-30,, 2006.
Charles Darwin. The origin of species, 1859-1959 Bios, 30(2))67-72, 1959.
Richard Dawkins. The blind watchmaker: Why the evidence of evolution reveals universe without design. WW Norton & Company, 1996.
Richard Dawkins. The selfish gene. Oxford university press, 2016.
Peter Dayan and L. F. Abbott. Theoretical neuroscience: computational and mathematical modeling of neural systems. Computational neuroscience. Massachusetts Institute of Technology Press, 2001. ISBN 978-0-262-04199-1
Daniel C Dennett. Darwin's 's dangerous idea. The Sciences, 35(3):34-40) 1995.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv: 08803, 2016.
Chris Fields and Michael Levin. Does evolution have a target morphology? Organisms. Journal of Biological Sciences, 4(1):57-76, 2020.
David E Golberg. Genetic algorithms in search, optimization, and machine learning. Addion wesley, 1989(102):36, 1989.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Stephen Jay Gould. The structure of evolutionary theory. Harvard university press, 2002.
John J Grefenstette. Genetic algorithms and machine learning. In Proceedings of the sixth annual conference on Computational learning theory, pp. 3-4, 1993.
https: www. vsc. ac

---

Published as a conference paper at ICLR 2025
Ryan N Gutenkunst, Joshua J Waterfall, Fergal P Casey, Kevin S Brown, Christopher R Myers, and James Sethna. Universally sloppy parameter sensitivities systems biology models. PLoS computational biology, 3(10);118, 2007.
Nikolaus Hansen. The ema evolution strategy: A tutorial. arXiv preprint
2016.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9(2):159-155, 2001.
Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (ema-es). Evolutionary computation, 11(1):1-18, 2003.
Benedikt Hartl, Yanbo Zhang, Hananel Hazan, and Michael Levin. Heuristically adaptive diffusion- model evolutionary strategy, 2024. URL https:: /atpse//aaxvvvvvia orgpvv..rrgraba. /as///b//// wttpgaa//g....aaaa//// 2411.
Geoffrey E Hinton, Steven Nowlan, al. How learning can guide evolution. Complex systems, (3):495-502, 1987.
Jonathan Ho and Tim Salimans. arXiv:2207.12500 12598, 2022.
Classifier-free diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
John H Holland. Adaptation in natural and artificial systems: introductory analysis with appli- cations to biology, control, and artificial intelligence. MIT press, 1992.
John Holland. Emergence: From chaos to order. OUP Oxford, 2000.
William B Johnson.
Extensions of lipshitz mapping into hilbert space. In Conference modern analysis and probability, 1984, pp. 189-206, 1984.
Eric R. Kandel. Principles of Neural Science, Fifth Edition. McGraw Hill Professional, 2013. ISBN 978-0-07-139011-8.
Motoo KIMURA. The neutral theory of molecular evolution: A review of recent evidence. The Japanese Journal of Genetics, 66(4):367-386, 1991. doi: 10.1266)jg.6663367.
Jack Lester King and Thomas H. Jukes. Non-darwinis evolution. Science, 164(3881):788-775
1969. doi: 10. 166s/cieneeeeee.cce.c 144.3881.177 URL
10. 126 scsciiii ence. 64 3881 788
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
Kostas Kouvaris, Jeff Clune, Loizos Kounios, Markus Brede, and Richard A Watson. How evolu- tion learns to generalise: Using the principles of learning theory to understand the evolution of developmental organisation. PLoS computational biology, 13(4)=e1005358, 2017.
Siddarth Krishnamoorthy, Satvik Mchul Mashkaria. and Aditya Grover. Diffusion models for black-box optimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara En- gelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 17842-17857. PMLR, 23-29 Jul 2023. URL https: httt::://proceedinggs.. .tt.pnssss.r...sss/./ep html
v202 w2222/krishmamoorre..
Joel Lehman and Kenneth O Stanley. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary computation, 19(2):189-223, 2011.
Michael Levin. Technological approach to mind everywhere: an experimentally- frame- work for understanding diverse bodies and minds. Frontiers in systems neuroscience, 16:768201, 2022.
Michael Levin. Darwin's 's agential materials: evolutionary implications multiscale competency in developmental biology. Cellular and Molecular Life Sciences, 80(6):122, 2023.
12

---

Published as a conference paper at ICLR 2025
Kevin Mitchell and Nick Cheney. The genomic code: The genome instantiates generative model of the organism. arXiv preprint arXiv22407.15906 2024.
Saan-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arXiv preprint arXiv: 04909, 2015.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162-8171. PMLR, 2021.
Ao Ping et al. Equivalent formulations of "the equation of life". Chinese Physics. B, 23, 2014.
Daniel A Power, Richard A Watson, Eörs Szathmáry, Rob Mills, Simon T Powers, C Patrick Don- caster, and BlaZej Czapp. What can ecosystems learn? expanding evolutionary ecology with learning theory. Biology direct, 10:1-24, 2015.
Justin K Pugh, Lisa B Soros, and Kenneth O Stanley, Quality diversity: A new frontier for evolu- tionary computation. Frontiers Robotics and AI, 3:202845, 2016.
Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion mod- els. Advances in Neural Information Processing Systems, 36, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Robert Rosen. Life itself: a comprehensive inquiry into the nature, origin, and fabrication of life.
Columbia University Press, 1991.
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies scalable alternative to reinforcement learning. arXiv preprint arXiv:1703. 038664 2017.
Frank Sehnke, Christian Osendorfer, Thomas Rückstieß, Alex Graves, Jan Peters, and Jürgen Schmidhuber. Parameter-exploring policy gradients. Neural Networks, 23(4))551-555 2010.
John Maynard Smith and Eors Szathmary. The major transitions in evolution. OUP Oxford, 1997.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256-2265. PMLR, 2015.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500: 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
Richard S Sutton and Andrew G Barto. Reinforcement learning: introduction mit press. Cam- bridge, MA, 22447:10, 1998.
Eörs Szathmáry. Toward major evolutionary transitions theory 2.0. Proceedings of the National Academy of Sciences, 100033)10104-10111,, 2015.
Vitaly Vanchurin, Yuri Wolf, Mikhail Katsnelson, and Eugene V Koonin. Toward theory of evolution as multilevel learning. Proceedings of the National Academy of Sciences, 119(6): e2120037119, 2022.
Pradnya A Vikhar. Evolutionary algorithms: critical review and its future prospects. 2016 International conference on global trends in signal processing, information computing and com- munication (ICGTSPICC), pp. 261-265. IEEE, 2016.
Andreas Wagner. Arrival of the fittest: How nature innovates. Current, 2015.
Kai Wang. Zhaopan Xu, Yukun Zhou Zelin Zang, Trevor Darrell, Zhuang Liu, and Yang You. Neural network diffusion. arXiv preprint arXiv:2402. 13 1314 2024.
13

---

Published as a conference paper at ICLR 2025
Richard Watson and Michael Levin. The collective intelligence of evolution and development. Col- lective Intelligence, 222):2633913721 168355, 2023.
Richard A Watson and Eörs Szathmáry. How can evolution learn? Trends in ecology & evolution, 31(2):47-1577, 2016.
Richard A Watson, Rob Mills, CL Buckley, Kostas Kouvaris, Adam Jackson, Simon T Powers, Chris Cox, Simon Tudge, Adam Davies, Loizos Kounios, et al. Evolutionary connectionism: algorithmic principles underlying the evolution of biological organisation in evo-devo, evo-eco and evolutionary transitions. Evolutionary biology, 43:553-581, 2016.
Richard A Watson, Michael Levin, and Christopher L Buckley. Design for individual: nectionist approaches to the evolutionary transitions in individuality. Frontiers in Ecology and Evolution, 10:823588, 2022.
Xueming Yan and Yaochu Jin. Emodm: A diffusion model for evolutionary multi-objective opti- mization. arXiv preprint arXiv2240.119311 2024.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 TWO-PEAKS MODELS
We first apply our method simple two-dimensional fitness function, with two optimal points, to demonstrate its behavior and capability of finding multiple solutions. We choose continuous mixed Gaussian density function. The fitness function is a mixed Gaussian density function with means located at (1, 1) and (-1, -1). The fitness function is:
[e, v) [N((1, v): 11, o²) + (V v): 12, o2)] /2,
(13)
where 141 = 1) and 12 = (-1,-1). And the = 0.1.
A.2
ALPHA AND NOISE SCHEDULE
(a) a schedule
(b) compare performance
0.70
Linear
Average fitness (top 64 elites)
Cosine
0.65
0.8
DDPM
0.60
0.6
0.55
0.4
0.50
Linenr
0.2
0.45
Cosine
DDPM
1.
20
40
60
80
100
101
102
10a
Number of total steps
Figure 5: (a) Different scheduling methods used, here for total of 100 evolutionary steps. (b) Average final fitness and standard deviation of 100 experiments of 2-dimensional Rastrigin function with different numbers of total evolutionary steps analyzed under various a schedul- ing methods. In general, a larger number of total evolution steps lead to higher average fitness. Most scheduling methods result in fitness plateauing around achieves higher fitness when the total number of evolution steps is lower than 100, demonstrating its capability to accelerate sampling by reducing the total number of fitness evaluations.
100. However, cosine scheduling
our experiments, we tested three different schedules for de see Figure 5(a). The first a simple linear schedule, used in the two-peaks demonstration:
0t
T
(14)
The second is the schedule used in DDPM (Ho et al. 2020).
which can be approximated by:
7t2
02 exp
Bot
(15)
where So and are hyperparameters. These calculated by constraining do and ar E, with E = 10-4 as the default.
The third schedule is the cosine schedule proposed by Nichol & Dhariwal (2021) and is used in both Figures and
1
cos
(16)
2
The cosine scheduling demonstrates better performance when the total number of evolution steps are fewer than 100 (see Figure Fig)re(5lb)). aligning with the conclusions reported by Nichol & Dhariwal 2021). Hence, cosine scheduling by default unless otherwise specified.
15

---

Published as a conference paper at ICLR 2025
For de, we follow the DDIM setting with slight modification for better control:
1 at-1
Ot
ot Jm
(17)
at-1
where 0 om S hyperparameter control the magnitude of noise. We use dm for most experiments, except for the experiment demonstrating the process in Figure which requires lower noise magnitude Fm 0.1 for better visualization.
A.3 NEIGHBORHOOD INDIVIDUALS
In Figure 2 we use blue discs represent the neighborhood of each individual. The mean and stan- dard deviation of an individual's neighborhood can derived from the Gaussian term in Equation Transforming this term into equivalent form with the variable and D, the parameter yields:
N(z Vaia, a.)
N
E;
E2
04
(18)
Va
Var
C2
Hence, this Gaussian term can be transformed into form where the variable, which is more intuitive for identifying data points æ as the neighbors of æt /az. Thus, the neighborhood-disce are centered at μ 2t/ Var and have a squared radius ,22 (1 as)/ar.
A.4 FITNESS FUNCTIONS
Table 2: Fitness functions used in our experiments. The superscript Rastrigin indicates the dimension of the function.
Name
Formula
features
Rosenbrock
f(z, v) = 100(y =²)² (1 x²²²
The minimal value position is
(2,v) (L, where f(1, 0.
Beale
[x. = (1.5 - r + +zy)² (2.25 my?)
The minimal
value position (3. (3,0.5). where
(2.625 + +yy
(x. y)
f(3, 0.5) 0.
Himmelblau [(x,y) = (2" + 11) (x + 7)2
This function has minimal value points, they are:
f(3.0, 2.0) = 0.0,
f(-2.81,,.13) 0.0.
f(-3.78, -3.28) 0.0. f(3.58, 1.85) 0.0
Ackley
f(x, y)
exp -0.2 x²4y²
When restricting the range of x, y between -4 to 4, the maxi- mal points are located at the four corners.
exp
one 2mm-rrn 2m1)
+ 20
Rastrigin
f(x) An En I -Acos(2Tx.)
Here 10. and is the di- mension. Similar the Ack- Icy function above, when restrict- ing the range of æ, the maximal points are located al the four cor-
To benchmark the solution diversity and performance, we choose eight different fitness functions compare method with other evolutionary strategies. All the functions depend variable ac,
16

---

Published as a conference paper at ICLR 2025
with the objective being minimize maximize the function value. Specifically, constrain the range of z; to (-4, 4) and set the objective of the Rastrigin function to be maximization instead of minimization to benchmark the capability of finding multiple solutions. All the functions scaled by 4, i.e., ((= + f" (4(.),, to ensure that the standard Gaussian distribution can cover the parameter space. The details of these fitness functions are shown in Table 2
A.4.1
PROBABILITY MAPPING FUNCTION
(a) Final Fimess
0.75
Beals
Himmeblow
Ackley
0.50
00
Rastrigin
Rastrigin"
0.25
Rastigin
0.00
Rastrgin
7=0.1
0=0.5
T=100
7=2.0
T=5.0
7=10,0
40
QD-Score
30
(b) Emrropy
5
4-
10
10
10
10-1
10a
Temperature
Temperature
Figure 6: Effects of temperature T on the probability mapping function 9. (a) Average fitness and standard deviation of 100 run with functions under different temperature settings are shown. A value of one represents the highest fitness, while zero indicates the lowest fitness. The general trend reveals that as increases, Diffusion Evolution tends to sample fewer high-fitness populations. (b) The average entropy of sampled parameters increases temperature rises. plateauing rapidly around T' = 1. The shaded areas indicate the standard deviation of the entropy. (c) When measured by QD- score, many tasks achieve their maximum QD-score around 1. suggesting this the optimal setting for most tasks.
To standardize the comparison, apply the following fitness mapping function g convert target values to the highest fitness:
f)x -D(((t)), Imin
(19)
-min
Here, 7' is the temperature parameter used control the sharpness of the probability density func- tion, and f is the target fitness value, with as the scale factor to make different functions com- parable. The D) represents the distance function, defined D v) |x y). and Dmax is the maximum possible distance between the target fitness and the sampled fitness within the given pa- rameter region. The Jmun defined as exp( mexx/ (sT)), representing the minimal possible prob- ability value before rescaling. For each fitness function, we determine f as the minimal or maximal value, depending on the optimization objective. The scale factor is determined by the standard de- viation of fitness around their optimal points, with ranges adjusted for different functions. After this transformation, the lowest fitness is zero, and the highest fitness is one.
While we acknowledge that the choice of the probability mapping function can vary depending the task, we use Equation 19 with a T hyper-parameter to investigate how the probability mapping function affects the results and to determine the optimal g function. As illustrated in Figure 6 higher temperatures generally make fitness values less distinguishable, leading the algorithm to sample fewer high-fitness individuals while simultaneously increasing diversity. By evaluating the
17

---

Published as a conference paper at ICLR 2025
QD-score, as shown in Figure 6j),, recommend selecting T for most probability mapping functions when an appropriate scale factor s is applied. For this reason, we use T for all benchmark and reinforcement learning experiments, except in Figure where set T = 0.25 for better visualization.
Table 3: A detailed version of Table is presented, where each cell contains two lines. The first line indicates the entropy of the top-64 elite populations after evolution, while the second line represents the average fitness (ranging from 0 to 1). Standard deviations for both entropy and fitness are provided in parentheses. The superseript numbers on the Rastrigin functions indicate the dimension of the fitness function.
Diffusion Evolution
Latent Diffusion
CMA-ES
PEPG
OpenES
MAP-Elite
Evolution
Rosenbrock
5.86 (0.10)
5.64 (0.24)
0.00 (0.00)
0.85 (0.26)
1.96 (0.21)
6.00 (0.01)
0.88 (0.06)
0.92 (0.06)
1.00 (0.00)
1.00 (0.00)
0.71 (0.00)
0.77 (0.00)
Beale
5.50 (0.16)
5.11 (0.37)
0.00 (0.00)
0,35 (0.21)
1,03 (0.45)
6.00 (0.01)
0.96 (0.01)
0.93 (0.06)
1.00 (0.00)
1.00 (0.00)
1.00 (0.01)
0.37 (0.01)
Himmelblau
5.23 (0.20)
4.95 (0.39)
0.00 (0.00)
0.00 (0.00)
0.28 (0.46)
6.00 (0.01)
0.96 (0.01)
0.89 (0.11)
1.00 (0.00)
1.00 (0.00)
1.00 (0.00)
0.28 (0.01)
Ackley
5.67 (0.13)
5.31 (0.42)
5.34 (1.30)
0.04 (0.17)
0.18 (0.38)
5.98 (0.02)
0.78 (0.07)
0.73 (0.17)
0.66 (0.20)
0.96 (0.13)
1.00 (0.00)
0.50 (0.01)
Rastrigin
5.79 (0.10)
5.34 (0.43)
5.70 (0.38)
0.00 (0.00)
0.01 (0.04)
5.95 (0.04)
0.64 (0.04)
0.62 (0.08)
0.57 (0.11)
0.57 (0.16)
1.00 (0.00)
0.61 (0.01)
Rastrigin
5.82 (0.09)
5.99 (0.01)
5.67 (0.59)
0.00 (0.00)
0.00 (0.00)
5.95 (0.04)
0.18 (0.01)
0.37 (0.03)
0.17 (0.02)
0.17 (0.03)
0,25 (0.00)
0.18 (0.00)
Rastrigin 32
5.84 (0.07)
6.00 (0.00)
4.54 (1.13)
0.00 (0.00)
0,00 (0.00)
5.94 (0.04)
0.02 (0.00)
0.19 (0.01)
0.01 (0.00)
0.02 (0.00)
0.02 (0.00)
0.02 (0.00)
Rastrigin 256
5.84 (0.08)
6.00 (0.00)
2.41 (1.78)
0.00 (0.00)
1.65 (0.30)
5.95 (0.04)
0.00 (0.00)
0.15 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
Notations: highest value, second highest value, third highest value.
Superscript numbers on Rastrigin functions indicate the dimensions of the fitness function.
A.4.2 ESTIMATING ENTROPY TO QUANTIFY DIVERSITY
To quantify the diversity of the solutions, we divided the D-dimensional space into 80 grids and counted the frequencies of elite solutions within this space. We intentionally used this simple and coarse method to quantify entropy in order to eliminate the contribution of local diversity, focusing solely on the diversity of solutions across different basins. The entropy is calculated by:
Σ
P. loga P.
(20)
where P; is the probability of having sample in grid i.
A.4.3 QUALITY-DIVERSITY SEARCH
We use MAP-Elite (Mouret & Clune 2015) to perform quality-diversity search experiments, en- abling us to compare our method's ability to balance quality and diversity. Briefly, in addition to the fitness function, MAP-Elite requires an additional feature descriptor, c(x). Using this feature descriptor, MAP-Elite only compares individuals within the same feature class. If new individual introduces a novel feature, it is accepted and stored regardless of its fitness. Conversely, if the fea- ture is already present, the fitness of the new individual is compared to the existing best individual
18

---

Published as a conference paper at ICLR 2025
within that feature class. To ensure fair comparison between MAP-Elite and other evolutionary tasks, we use the same number of fitness evaluations. For instance, if Diffusion Evolution uses 512 populations and 25 iterations, then MAP-Elite is allocated 512 x 25 12, 800 iterations, with each iteration proposing one new individual and evaluating its fitness once.
To apply MAP-Elite benchmark functions, we manually designed feature descriptor by dividing the parameter space into segments of a specified unit length 1. Specifically, the feature descriptor is defined
c(x) = (L01/1), [x2/[], ....................... [zp/l]).
(21)
For simplicity, we set = in all experiments. We quantify the balance between quality and diversity using the quality-diversity score (QD-score) (Pugh et al., (016), calculated as the sum of the highest rewards in each feature class. For methods other than MAP-Elite, we apply the same classification approach to divide individuals into feature classes and compute the QD-score by summing the highest rewards within each class. The comparison of QD-scores presented in T
In two-dimensional fitness functions, Diffusion Evolution and Latent Diffusion Evolution achieve QD-scores comparable to MAP-Elite. However, in higher-dimensional tasks, the QD-score of MAP- Elite drops significantly, whereas our Latent Diffusion Evolution demonstrates substantial better performance. This result indicates that our method effectively balances quality and diversity, partic- ularly in high-dimensional settings, without being designed to do so.
Table 4:
The QD-scores of experiments on fitness functions with different algorithms are re- ported. All values averaged over 100 experiments, with the standard deviations of the QD- scores provided in brackets. Our Diffusion Evolution demonstrates competitive performance on QD-scores compared to MAP-Elite. In higher-dimensional experiments, our Latent Diffusion Evo- lution achieves the highest QD-score among all tested methods.
Diffusion
Latent
CMA-ES
PEPG
Open-ES
MAP-Elite
Evolution
Diffusion
Evolution
Rosenbrock
35.4 (1.82)
23.4 (9.78)
1.00 (0.00)
1.25 (0.43)
0.73 (0.12)
42.0 (0.36)
Beale
20.0 (0.94)
13.0 (4.50)
1.00 (0.00)
2.00 (0.00)
1.84 (0.72)
23.7 (0.48)
Himmelblau
15.8 (1.18)
11.4 (3.99)
1.00 (0.00)
1.00 (0.00)
1.00 (0.00)
18.1 (0.42)
Ackley
28.3 (1,35)
16.7 (9.07)
13.4 (4.53)
1.13 (0.34)
2.14 (0.63)
33.0 (0.53)
Rastrigin"
35.2 (2.18)
17.8 (9.36)
18.9 (6.24)
2.30 (0.64)
3.99 (0.00)
45.2 (0.93)
Rastrigin
10.6 (0.50)
33.1 (6.66)
6,03 (2.30)
0.67 (0.12)
1.01 (0.00)
13.5 (0.22)
Rastrigin
0.96 (0.04)
73.4 (2.10)
0.12 (0.03)
0.06 (0.01)
0.07 (0.01)
1.20 (0.02)
Rastrigin 256
0.11 (0.01)
70.2 (0.62)
0.01 (0.00)
0.01 (0.00)
0.00 (0.00)
0.14 (0.00)
Notations: highest value, second highest value, third highest value.
Superscript numbers on Rastrigin functions indicate the dimensions of the fitness function.
A.5 REINFORCEMENT LEARNING EXPERIMENTS
We applied our Diffusion Evolution and Latent Diffusion Evolution methods to various reinforce- ment learning tasks using neural network architectures described in Appendix A.5. including Ac- robot, Cart Pole, Mountain Car, Continuous Mountain Car, and Pendulum. Our approach consis- tently achieves higher final rewards compared to CMA-ES, with statistically significant differences.
To assess the robustness of our method, we introduced a new scaling hyper-parameter s, which transforms the original fitness function fxx into fa(x) fsx).. This enables evaluation of per- formance across diverse parameter spaces. The results, presented in Table and demonstrate that method is robust across different choices of the scaling parameter Furthermore, observe that Latent Diffusion Evolution is always outperforming the original Diffusion Evolution in terms of success rate, supporting the rationale for introducing the latent approach. The CMA-ES method,
19

---

Published as a conference paper at ICLR 2025
although successful with certain populations, does not achieve high success rate within the given steps. We hypothesize that the reason might that C.M--ES is distracted by multiple solutions, leading to significant proportion of unsuccessful outcomes.
Table 5: Grid search of reinforcement learning tasks with multiple environments, incorporating parameter scaling factor, was conducted. This scaling factor tests different landscapes of the parameter space and evaluates the robustness of evolutionary algorithms. Here, higher rewards indicate better performance. with the highest rewards per line shown in bold, and standard deviations (over 100 experiments) presented in brackets. The results demonstrate that Diffusion Evolution algorithms consistently achieve the highest rewards across different scaling factors. Furthermore, Latent Diffusion Evolution exhibits greater stability with respect to the scaling factor and often yields higher rewards.
Scaling
Environment
Diffusion Evolution
Latent
Large Latent Diffusion Evolution
CMA-ES
Diffusion Evolution
Acrobot
-279.5 (187.2)
-19.00 (115.0)
-15..6 (116.9)
-486.9 (56.7)
Cart Pole
447.4 (121.2)
447.0 (124.3)
407.6 (154.9)
32.64 (71.81)
0.1
Mountain Car
-16.22 (39.39)
-139.7 (35.63)
-138.1 (37.58)
-199.2 (7.29)
Mountain Car
-0.89 (1.19)
-0.02 (0.05)
-0.01 (0.05)
-0.14 (0.21)
Pendulum
-1211 (337.6)
-1224 (347.0)
-1227 (340.7)
-1257 (302.6)
Acrobot
199.9 (160.3)
127.00 (93.06)
147.6 (107.2)
-471.0 (81.48)
Cart Pole
482.9 (73.24)
491.6 (50.2)
445.0 (124.20)
77.67 (127.3)
Mountain Car
-134.7 (34.8)
-130.6 (33.3)
-13..8 (37.5)
-194.7 (18.3)
Mountain Cart
55.97 (47.9)
78.59 (39.05)
88.58 (21.66)
33.94 (63.75)
Pendulum
-262 (330.4)
-1187 (408.5)
-1094 (532.6)
-1397 (217.4)
Acrobot
-19.88 (156.3)
-12.00 (76.98)
-14988 (105.0)
-46.22 (83.56)
Cart Pole
478.7 (78.58)
488.6 (59.73)
428.9 (142.7)
79.47 (130.3)
10.0
Mountain Car
-13.22 (34.9)
-129,5 (32.78)
-133.9 (36.60)
-194.85 (17.89)
Mountain Cart
79.44 (37.46)
91.66 (1 (11,30)
83.41 (33.82)
10.90 (68.52)
Pendulum
-1132 (488.4)
-1077 (520.5)
-1101 (519.5)
-1368 (246.34)
Acrobot
-190.7 (156.28)
-120.6 (76.73)
-151.7 (110.32)
-469.6 (83.27)
Cart Pole
477.9 (82.10)
489.5 (57.86)
443.1 (128.48)
77.72 (127.89)
100.0
Mountain Car
-13..6 (34.99)
-130.6 (33.74)
-13..6 (37.45)
-194.6 (18.22)
Mountain Car
78.56 (39.49)
90.67 (16.02)
82.38 (35.57)
12.97 (69.27)
Pendulum
-1119 (494.4)
-1066 (535.6)
-1102 (521.7)
-1367 (243.0)
Mountain Cart: continuous version Mountain Car
Notations: highest value, second highest value, third highest value.
A.5.1 NEURAL NETWORK
The controller of the cart-pole system has four observational inputs: the current position, velocity, pole angle, and pole angular velocity. The system accepts two actions: move left or right. To model the controller, the four observations and an output layer of 2 neurons corresponding to the two actions. The action is determined by which output neuron has the higher value.
artificial neural networks with input layer of 4 neurons corresponding to
Our standard experiment uses a woe-hhdddn-llyyee neural network with the hidden layer of 8 neurons, resulting in (4x 8 + 8) + (8 2 +) = 58 parameters. We also use deeper neural network with two hidden layers (each has 128 neurons), totaling (4 x 128 + 128) + (128 x 128 + 128) + +128 x 2+2) = 17, 410 parameters. Both neural networks the ReLU activation function.
We use similar neural network configurations for other reinforcement learning tasks, with adapta- tions based the number of observations as well the size and type (continuous discrete) of
20

---

Published as a conference paper at ICLR 2025
Table 6: Success rate of selected environments with different scaling factors. The Pendulum exper- iment is not included as it does not have definition of success. Scaling
Environment
Diffusion Evolution
Latent
Large Latent
CMA-ES
Diffusion Evolution
Diffusion Evolution
Acrobot
59.3%
91.6%
91.1%
6.2%
Cart Pole
80.0%
80.8%
68.7%
1.5%
0.1
Mountain Car
53.8%
86.6%
88.6%
1.6%
Mountain Car
0.0%
0.0%
0.0%
0.0%
Acrobot
79.1%
95.2%
93.0%
13.5%
Cart Pole
92.5%
96.4%
79.4%
5.9%
1.0
Mountain Car
92.0%
97.1 %
91.9%
10.2%
Mountain Car
59.4%
80.7%
96.9%
56.7%
Acrobot
80.9%
97.3%
93.6%
14.4%
Cart Pole
90.4%
95.6%
76.4%
6,4%
10.0
Mountain Car
93.1%
98.2%
93.7%
10.2%
Mountain Car
92.0%
99.3%
94.0%
43.5%
Acrobot
81.0%
97.2% %
92.6%
14.1%
Cart Pole
90.9%
95.8% %
80.0%
6.0%
100.0
Mountain Car
93.5%
97.1 %
92.1%
10.5%
Mountain Carf
91.4%
98.7
93.3%
45.3%
Mountain Carl: continuous version Mountain Car
Notations: highest value, second highest value, third highest value.
operations. The design of neural networks generally follows similar principles in machine learning. Specifically, we rescaled the input to have a standard deviation of one to improve training.
A.6 LATENT SPACE DIFFUSION EVOLUTION
Following is the pseudocode for Latent Space Diffusion Evolution algorithm. The difference from the original Diffusion Evolution (Algorithm T is indicated in light blue.
21

---

Published as a conference paper at ICLR 2025
Algorithm Latent Space Diffusion Evolution
Require: Population size N, parameter dimension D, latent space dimension d, fitness function f, density mapping function g. total evolution steps T, diffusion schedule a and noise schedule or.
Ensure: 00 1, 0, 0441, < v1 a;
N 1: D(d.D) (0, 1/D)
Initialize the random mapping Initialize population
2: [x]]) Dt .....
+ VO, NND)
3: L [T,T ...., do
4:
Vi E
Q + gfx (0)
b Fitness are cached to avoid repeated evaluations Encode individual parameters into latent space
5:
Ei
[1 2t
(6)
Ex,
6:
for
E 21,2.... N] do
N
7:
++ Σ
Nz (9) Varzl (1) 1-
1=1
N
Σ
(= vorz(), - an)
8:
7
1=1
9:
w + 1P)
z 2 1a + V 04-1 - v; Vaf00
10:
w
v1 04
end for
12: end for
22


Reviews:
[{"review_id": "7xqaltkJqF", "reviewer": "Reviewer_jGHU", "summary": "The authors try to connect Diffusion Models and Evolutionary computation by arguing that both processes do iterative refinements through an update rule plus some perturbation. In evolutionary algorithms the update comes in the form of natural selection, in diffusion is the denoising phase. The perturbation corresponds to mutation for evolution, and the diffusion phase for diffusion models. \n\nFrom the above connection an evolutionary algorithm that performs in its iteration a diffusion process is proposed.  Instead of aiming to recover some distribution of the data, the goal is to turn the random initial population points towards a distribution centered around the optimized function optimum.", "questions": "In the text is not mention what g could be or some options. It could be like the utility function used in Natural Evolutionary Strategies? because it seems to assign more weight/importance to high ranking solutions.\n\nI think it would be very interesting in a  future work to explore more the latent diffusion version of the algorithm, and techniques to understand which parameters could be ignored or which parameters should be perturbed together.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "By connecting methods from different fields the work proposed an interesting new evolutionary algorithm that can be further improved using variations and techniques found in diffusion models.\n\nThe text is very clear and the math derivation from diffusion to an EA algorithm is easy to follow. Figure 1 really helps to visualize how the population of the diffusion EA spreads to the higher fitness regions.", "weaknesses": "Maybe I missed it but there is no discussion on how to choose the mapping to a density function g() or which g() was used for the experiments. It seems is important for the search to work properly to have a mapping that makes it clear which points should be paid more attention by assigning to them bigger weights. Maybe there is some connection to common selection strategies in evolutionary algorithms that could be discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lS8L2ti0aq", "reviewer": "Reviewer_DCF9", "summary": "This paper proposes a new evolutionary algorithm inspired by diffusion models. It views the evolution processes as the denoising process of diffusion models, and designs an evolutionary algorithm named Diffusion Evolution, which is based on the denoising framework of the famous diffusion model DDIM. During evolution, it takes each individual in the population as a solution in the denoising process, and updates it with similar updating rules in DDIM. To facilitate better performance, it further follows previous works to optimize in the latent space. Experiments on several benchmark functions and a simple cart-pole controlling problem show that compared with classic baseline methods include CMA-ES, OpenES and PEPG, the proposed Diffusion Evolution can obtain more diverse solutions with good fitness.", "questions": "As mentioned in the appendix, different Alpha and noise schedule settings are tested, what about the detailed experimental results? Is there any analysis about the results of different settings, which could be useful for the users?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-written and easy to follow. The proposed method Diffusion Evolution is well-motivated, with clear explanation and illustration.", "weaknesses": "As there has been a variety of evolutionary algorithms with various inspirations, the most important issue when proposing a new method should be clarifying its strength compared with previous methods, from the perspectives of theoretical analysis and extensive experiments. However, the proposed Diffusion Evolution in this paper is lack of theoretical analysis. Meanwhile, the experiments are much too simple. For example, only five synthetic functions and a simple cart-pole controlling problem are included. As the diversity of solutions seems to be the strength of the proposed Diffusion Evolution, except for methods like CMA-ES, which are not specified for diversity, comparison with other kinds of evolutionary algorithms like Quality-Diversity (QD) are necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EAOBbTJO79", "reviewer": "Reviewer_HsP7", "summary": "This manuscript introduces an innovative perspective by interpreting the diffusion model as an evolutionary algorithm, highlighting the mathematical similarities between the diffusion and evolutionary processes. It proposes a Diffusion Evolution Method that employs iterative denoising to heuristically optimize solutions within the parameter space, drawing an analogy between the generative process of the diffusion model and the selection and mutation mechanisms in biological evolution.", "questions": "In the experiments section, the authors select three different evolutionary strategies for performance comparison. However, the rationale for choosing these specific methods is not clearly articulated. I would appreciate more insight into this selection process, particularly since each of these methods was introduced over five years ago.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of regrading the diffusion model as an evolutionary algorithm is interesting.", "weaknesses": "1. The mathematical demonstration process in this manuscript is insufficient. The authors conceptualize the evolution process as a transformation of the probability density function, deriving the diffusion evolution algorithm from the Bayesian formula. However, this derivation overlooks key complexities inherent in evolutionary dynamics, such as genetic drift and gene recombination. Additionally, the application of the Bayesian formula assumes conditional independence, which may not be held in the context of evolutionary processes due to the interactions and competition among individuals. Furthermore, the manuscript does not provide a clear explanation of how to effectively map a complex fitness function into a probability density function, which is essential for the practical implementation of the proposed model.\n2. Since the author claims that the diffusion model functions as an evolutionary algorithm, the experimental section must reflect this comparison appropriately. Specifically, in scenarios involving multiple tasks and comparisons, aspects such as speed, performance metrics (including best, worst, median, and average results), and the stability of performance should be examined. Additionally, the model's excellence should be illustrated through the mean variance of the results. Currently, the scope of experiments conducted is insufficient and does not adequately support the claims made in the manuscript.\n3. There are several grammatical errors and unclear expressions throughout the article. For example, the definitions of certain terms lack clarity, the derivation process for the formulas is inadequately explained, and the labeling of the charts is inaccurate.\n4. The authors categorize diffusion models as evolutionary algorithms on the basis that both methods perform distribution transformation. However, this classification may be overly broad. For instance, semantic segmentation models also involve transforming distributions—from real images to pixel-level segmentation results. Should we, therefore, classify these segmentation models as evolutionary algorithms as well? This comparison seems inaccurate. A more robust argument would establish that the Markov process within a non-equilibrium thermodynamics framework (diffusion) function as an unconstrained parameter optimization technique (evolution). If this argument cannot be substantiated, I recommend revising the title to better reflect the content.\n5. The manuscript presents two primary sets of experiments: multi-objective evolution and latent space diffusion evolution. However, the multi-objective evolution experiments are limited to a two-dimensional parameter space and a simplistic fitness function, which fails to demonstrate the proposed approach's efficacy in high-dimensional parameter spaces or with more complex fitness functions. Similarly, the latent space diffusion evolution experiments focus solely on the CartPole task, lacking validation across a broader range of reinforcement learning tasks. Furthermore, the experimental results do not include any statistical significance tests, making it challenging to determine whether the observed improvements in algorithm performance are statistically significant. This omission significantly undermines the credibility of the findings presented.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FE8QUnUKX8", "reviewer": "Reviewer_tnPL", "summary": "This paper explores the theoretical and practical parallels between diffusion models and evolutionary algorithms, proposing the \"Diffusion Evolution\" approach, which adapts diffusion models for evolutionary tasks. This new method, Latent Space Diffusion Evolution, enhances the ability to find diverse and optimal solutions in high-dimensional parameter spaces, particularly within reinforcement learning.", "questions": "The work potentially contributes new insights and methods for both diffusion model and evolutionary algorithm communities, even if the practical impact in specific real-world applications remains to be validated. Can the authors address this?\n\nCan statistical tests be added?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "he core idea of treating diffusion models as evolutionary algorithms is innovative. It extends diffusion models to broader applications by framing them as tools for evolutionary tasks, potentially contributing new knowledge to both evolutionary biology and AI communities.\n\nTechnically: The paper provides thorough mathematical grounding for the equivalence between diffusion and evolution, specifically explaining diffusion as a probabilistic denoising process analogous to evolutionary mechanisms like mutation and selection.\n\nExperimentals: Comprehensive experiments benchmark Diffusion Evolution against traditional algorithms (e.g., CMA-ES, PEPG) ascross various fitness landscapes, demonstrating its strength in maintaining diversity and achieving multiple optima.\n\nApplications in High-Dimensional Problems: The adaptation of Diffusion Evolution to high-dimensional tasks via latent space diffusion showcases its practical viability in reinforcement learning contexts, with empirical results suggesting its potential in complex environments.", "weaknesses": "Complexity of Explanation: The theoretical connections between diffusion and evolution, while compelling, are highly complex, and the paper occasionally sacrifices clarity for depth. This might limit accessibility to a broader audience.\n\nAlthough promising, the methodology may have limitations in scenarios requiring open-ended evolution, a challenge acknowledged briefly. More thorough discussion could help set realistic expectations for potential users.\n\nWhile the Diffusion Evolution algorithm demonstrates superiority in finding diverse solutions, certain comparisons (e.g., high-fitness solutions) against traditional evolutionary algorithms lack statistical significance or detailed analysis of computational cost versus benefit, particularly in high-dimensional settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)