Diffusion Models are Evolutionary Algorithms

## Abstract

Abstract In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising – as originally introduced in the context of diffusion models – to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.

## 1 Introduction

At least two processes in the biosphere have been recognized as capable of generalizing and driving novelty: evolution, a slow variational process adapting organisms across generations to their environment through natural selection (Darwin, ; Dawkins, ); and learning, a faster transformational process allowing individuals to acquire knowledge and generalize from subjective experience during their lifetime (Kandel, ; Courville et al., ; Holland, ; Dayan & Abbott, ). These processes are intensively studied in distinct domains within artificial intelligence. Relatively recent work has started drawing parallels between the seemingly unrelated processes of evolution and learning (Watson & Levin, ; Vanchurin et al., ; Levin, ; Watson et al., ; Kouvaris et al., ; Watson & Szathmáry, ; Watson et al., ; Power et al., ; Hinton et al., ; Baldwin, ). We here argue that in particular diffusion models (Sohl-Dickstein et al., ; Song et al., ; Ho et al., ; Song et al., ), where generative models trained to sample data points through incremental stochastic denoising, can be understood through evolutionary processes, inherently performing natural selection, mutation, and reproductive isolation.

The evolutionary process is fundamental to biology, enabling species to adapt to changing environments through mechanisms like natural selection, genetic mutations, and hybridizations (Rosen, ; Wagner, ; Dawkins, ); this adaptive process introduces variations in organisms’ genetic codes over time, leading to well-adapted and diverse individuals (Mitchell & Cheney, ; Levin, ; Gould, ; Dennett, ; Smith & Szathmary, ; Szathmáry, ). Evolutionary algorithms utilize such biologically inspired variational principles to iteratively refine sets of numerical parameters that encode potential solutions to often rugged objective functions (Vikhar, ; Golberg, ; Grefenstette, ; Holland, ). In another side, recent breakthroughs in deep learning have led to the development of diffusion models–generative algorithms that iteratively refine data points to sample novel yet realistic data following complex target distributions: models like Stable Diffusion (Rombach et al., ) and Sora (Brooks et al., ) demonstrate remarkable realism and diversity in generating image and video. Notably, both evolutionary processes and diffusion models rely on iterative refinements that combine directed updates with undirected perturbations: in evolution, random genetic mutations introduce diversity while natural selection guides populations toward greater fitness, and in diffusion models, random noise is progressively transformed into meaningful data through learned denoising steps that steer samples toward the target distribution. This parallel raises fundamental questions: Are the mechanisms underlying evolution and diffusion models fundamentally connected? Is this similarity merely an analogy, or does it reflect a deeper mathematical duality between biological evolution and generative modeling?

To answer these questions, we first examine evolution from the perspective of generative models. By considering populations of species in the biosphere, the variational evolution process can also be viewed as a transformation of distributions: the distributions of genotypes and phenotypes. Over evolutionary time scales, mutation and selection collectively alter the shape of these distributions. Similarly, many biologically inspired evolutionary algorithms can be understood in the same way: they optimize an objective function by maintaining- and iteratively changing a large population’s distribution. In fact, this concept is central to most generative models: the transformation of distributions. Variational Autoencoders (VAEs) (Kingma, ), Generative Adversarial Networks (GANs) (Goodfellow et al., ), and diffusion models are all trained to transform simple distributions, typically standard Gaussian distributions, into complex distributions, where the samples represent meaningful images, videos, or audio, etc.

On the other hand, diffusion models can also be viewed from an evolutionary perspective. As a generative model, diffusion models transform Gaussian distributions in an iterative manner into complex, structured data-points that resemble the training data distribution. During the training phase, the data points are corrupted by adding noise, and the model is trained to predict this added noise to reverse the process. In the sampling phase, starting with Gaussian-distributed data points, the model iteratively denoises to incrementally refine the data point samples. By considering noise-free samples as the desired outcome, such a directed denoising can be interpreted as directed selection, with each step introducing slight noise, akin to mutations. Together, this resembles an evolutionary process (Fields & Levin, ), where evolution is formulated as a combination of deterministic dynamics and stochastic mutations within the framework of non-equilibrium thermodynamics (Ao, ). This aligns with recent ideas that interpret the genome as a latent space parameterization of a multi-scale generative morphogenetic process, rather than a direct blueprint of an organism (Mitchell & Cheney, ; Hartl et al., ; Levin, ; Gould, ). If one were to revert the time direction of an evolutionary process, the evolved population of potentially highly correlated high-fitness solutions will dissolve gradually, i.e., step by step and thus akin to the forward process in diffusion models, into the respectively chosen initial distribution, typically Gaussian noise, see Figure 1.

Figure: Figure 1: Evolution processes can be viewed as the inverse process of diffusion, where higher fitness populations (red points) have higher final probability density. The initially unstructured parameters are iteratively refined towards high-fitness regions in parameter space.
Refer to caption: x1.png

Driven by this intuition, we conduct a thorough investigation into the connections between diffusion models and evolutionary algorithms, discovering that these seemingly disparate concepts share the same mathematical foundations. This insight leads to a novel approach, the Diffusion Evolution algorithm, which directly utilizes the framework of diffusion models to perform evolutionary optimization. This can be obtained by inverting the diffusion process with the Bayesian method. Our analytical study of Diffusion Evolution reveals promising parallels to biological evolution, naturally incorporating concepts such as mutation, hybridization, and even reproductive isolation.

This equivalence provides a new way of improving evolutionary algorithms and has the potential to unify developments in both fields. By mimicking biological evolution, evolutionary algorithms have shown promising results in numerical optimization, particularly for tasks that cannot be effectively trained using gradient-based methods (Wang et al., ; Goodfellow et al., ). These algorithms thus excel in exploring complex, rugged search spaces and finding globally optimal or near-optimal solutions (Ho & Salimans, ; Hansen, ; Hansen & Ostermeier, ; Sehnke et al., ). While the biosphere exhibits extreme diversity in lifeforms of life, many evolutionary strategies, such as CMA-ES (Hansen & Ostermeier, ), and PEPG (Sehnke et al., ), struggle to find diverse solutions (Lehman & Stanley, ). However, our Diffusion Evolution Algorithm offers a new approach. By naturally incorporating mutation, hybridization, and reproductive isolation, our algorithm can discover diverse solutions, mirroring the diversity of the biosphere, rather than converging on a single solution as is often the case with traditional methods. Since this parallel between diffusion and evolution exists naturally and not imposed by our design, the two fields – diffusion models and evolutionary computing – can mutually benefit from each other. For example, we demonstrate that the concept of latent diffusion (Rombach et al., ) and accelerated sampling (Nichol & Dhariwal, ) can significantly improve the performance of our Diffusion Evolution algorithm.

In the following sections, we will first review evolutionary strategies and diffusion models, introduce the mathematical connection between diffusion and evolution, and propose the Diffusion Evolution algorithm. Then, we will quantitatively compare our algorithm to conventional evolutionary strategies, demonstrating its capability to find multiple solutions, solve complex evolutionary tasks, and incorporate developments from diffusion model literature. Finally, the emerging connections between the derived algorithm and evolution will be discussed, along with the potentials of this finding and the limitations of our algorithm. Codes are available on Github(^1^11[https://github.com/Zhangyanbo/diffusion-evolution](https://github.com/Zhangyanbo/diffusion-evolution)) and the package can be installed via pip install diffevo.

## 2 Background

### 2.1 Evolutionary Algorithms

The principles of evolution extend far beyond biology, offering exceptional utility in addressing complex systems across various domains. The key components of this process – imperfect replication with heredity and fitness-based selection – are sufficiently general to find applications in diverse fields. In computer and data science, for instance, evolutionary algorithms play a crucial role in optimization (Vikhar, ; Grefenstette, ; Golberg, ; Holland, ). These heuristic numerical techniques, such as CMA-ES (Hansen & Ostermeier, ) and PEPG (Sehnke et al., ), maintain and optimize a population of genotypic parameters over successive generations through operations inspired by biological evolution, such as selection of the fittest, reproduction, genetic crossover, and mutations. The goal is to gradually adapt the parameters of the entire population so individual genotypic samples, or short individuals, perform well when evaluated against an objective- or fitness function. These algorithms harness the dynamics of evolutionary biology to discover optimal or near-optimal solutions within vast, complex, and otherwise intractable parameter spaces. The evaluated numerical fitness score of an individual correlates with its probability of survival and reproduction, ensuring that individuals with superior traits have a greater chance of passing their genetic information to the next generation, thus driving the evolutionary process toward more optimal solutions. Such approaches are particularly valuable when heuristic solutions are needed to explore extensive combinatorial and permutation landscapes.

Some evolutionary algorithms operate with discrete, others with continuous sets of parameters. Here, we focus on the latter since discrete tasks can be seen as a subcategory of continuous tasks. Typically, the structure of the parameter space is apriori unknown. Thus, the initial population is often sampled from a standard normal distribution. As explained above, this initially random population is successively adapted and refined, generation by generation, to perform well on an arbitrary objective function. Thus, initially randomized parameters are successively varied by evolutionary algorithms into sets of potentially highly structured parameters that perform well on the specified task, eventually (and hopefully) solving the designated problem by optimizing the objective function.
Thus, evolutionary algorithms can be understood as generative models that use heuristic information about already explored regions of the parameter space (at least from the previous generation) to sample potentially better-adapted offspring individuals for the next generation (c.f., CMA-ES (Hansen et al., ), etc.).

### 2.2 Diffusion Models

Diffusion models, such as denoising diffusion probabilistic models (DDPM) (Ho et al., ) and denoising diffusion implicit models (DDIM) (Song et al., ), have shown promising generative capabilities in image, video, and even neural network parameters (Wang et al., ). Similar to other generative approaches such as GANs, VAEs, and flow-based models (Dinh et al., ; Chen et al., ), diffusion models transform a simple distribution, often a Gaussian, into a more complex distribution that captures the characteristics of the training data. Diffusion models achieve this, in contrast to other techniques, via iterative denoising steps, progressively transforming noisy data into less noisy (Raya & Ambrogioni, ), more coherent representations (Sohl-Dickstein et al., ).

Diffusion models have two phases: diffusion and denoising. In the diffusion phase, we are blending original data points with some extent of Gaussian noise. Specifically, let ${\bm{x}}_{0}$ be the original data point and ${\bm{x}}_{T}$ be the fully distorted data, then the process of diffusion can be represented as:

$$ ${\bm{x}}_{t}=\sqrt{\alpha_{t}}{\bm{x}}_{0}+\sqrt{1-\alpha_{t}}{\bm{\epsilon}},$ (1) $$

where the amount of total noise ${\bm{\epsilon}}\sim\mathcal{N}(0,I)$ added to the data ${\bm{x}}_{0}$ at time step $t\in[0,T]$ is controlled by $\alpha_{t}$ that is monotonously decreasing from $\alpha_{0}=1$ to $\alpha_{T}\sim 0$. Thus, while ${\bm{x}}_{0}$ represents the original data, ${\bm{x}}_{T}$ will consist entirely of Gaussian noise. To restore such diffused data, a predictive model, typically a neural network ${\bm{\epsilon}}_{\bm{\theta}}$ with parameter $\bm{\theta}$, is trained to predict the added total noise given ${\bm{x}}_{t}$ and time step $t$. Thus, diffusion models can be trained by minimizing the loss function:

$$ $\mathcal{L}=\mathbb{E}_{{\bm{x}}_{0}\sim p_{\text{data}},{\bm{\epsilon}}\sim \mathcal{N}(0,I)}\|{\bm{\epsilon}}_{\bm{\theta}}(\sqrt{\alpha_{t}}{\bm{x}}_{0} +\sqrt{1-\alpha_{t}}{\bm{\epsilon}},t)-{\bm{\epsilon}}\|^{2},$ (2) $$

where $p_{\text{data}}$ is the distribution of training data. So, conventionally, diffusion models are understood as predicting the added noise during the diffusion process.

In the denoising phase, starting with a noisy pattern, the trained models are used to iteratively remove the predicted noise from current data: from ${\bm{x}}_{T}\sim\mathcal{N}(0,I)$, iteratively refine to ${\bm{x}}_{T-1},{\bm{x}}_{T-2}$, $\dotsc$, until ${\bm{x}}_{0}$. In the DDIM framework, this sampling process is given by:

$$ ${\bm{x}}_{t-1}=\sqrt{\alpha_{t-1}}\left(\frac{{\bm{x}}_{t}-\sqrt{1-\alpha_{t}} {\bm{\epsilon}}_{\theta}({\bm{x}}_{t},t)}{\sqrt{\alpha_{t}}}\right)+\sqrt{1- \alpha_{t-1}-\sigma_{t}^{2}}{\bm{\epsilon}}_{\theta}({\bm{x}}_{t},t)+\sigma_{t }{\bm{w}},$ (3) $$

where $\sigma_{t}$ controls the amount of noise ${\bm{w}}\sim\mathcal{N}(0,I)$ added during the denoising phase. Notably, the schedule of $\alpha_{t}$ and $\sigma_{t}$ will both affect the denoising process and can be chosen based on our needs under the DDIM framework.

## 3 Diffusion Models are Evolutionary Algorithms

Similar to the relationship between energy and probability in statistical physics, evolutionary tasks can be connected to generative tasks by mapping fitness to probability density: higher fitness correspond to higher probability density. Thus, given a fitness function $f:\mathbb{R}^{n}\to\mathbb{R}$, we can choose a mapping $g$ to transform $f$ into a probability density function $p({\bm{x}})=g[f({\bm{x}})]$. When aligning the denoising process in a diffusion model with evolution, we want ${\bm{x}}_{0}$ to follow this density function, i.e., $p({\bm{x}}_{0}={\bm{x}})=g[f({\bm{x}})]$. This requires an alternative view of diffusion models (Song et al., ): diffusion models are directly predicting the original data samples from noisy versions of those samples at each time step. Given the diffusion process ${\bm{x}}_{t}=\sqrt{\alpha_{t}}{\bm{x}}_{0}+\sqrt{1-\alpha_{t}}{\bm{\epsilon}}$, we can easily express ${\bm{x}}_{0}$ in terms of the noise ${\bm{\epsilon}}$, and vise versa:

$$ ${{\bm{x}}}_{0}=\frac{{\bm{x}}_{t}-\sqrt{1-\alpha_{t}}{\bm{\epsilon}}}{\sqrt{ \alpha_{t}}}\text{, and }{\bm{\epsilon}}=\frac{{\bm{x}}_{t}-\sqrt{\alpha_{t}}{ {\bm{x}}}_{0}}{\sqrt{1-\alpha_{t}}}.$ (4) $$

In diffusion models, the error ${\bm{\epsilon}}$ between ${\bm{x}}_{0}$ and ${\bm{x}}_{t}$ is estimated by a neural network, i.e., ${\hat{\bm{\epsilon}}={\bm{\epsilon}}_{\theta}({\bm{x}}_{t},t)}$. Thus, Equation 4 provides an estimation $\hat{\bm{x}}_{0}$ for ${\bm{x}}_{0}$ when replacing ${\bm{\epsilon}}$ with $\hat{\bm{\epsilon}}$. Hence, the sampling process of DDIM (Song et al., ) in Equation 3 can be written as:

$$ ${\bm{x}}_{t-1}=\sqrt{\alpha_{t-1}}{\hat{\bm{x}}}_{0}+\sqrt{1-\alpha_{t-1}- \sigma_{t}^{2}}\hat{\bm{\epsilon}}+\sigma_{t}{\bm{w}}.$ (5) $$

Figure: Figure 2: (a) Diffusion Evolution on a two-peak fitness landscape: Populations near the two black crosses have higher fitness. For each individual ${\bm{x}}_{t}$ (black star), its target ${\hat{{\bm{x}}}}_{0}$ (red dots) is estimated by a weighted average of its neighbors (c.f., dots within the blue disks, respectively); larger dot-size indicates higher fitness. The individual then moves a small step forward to the next generation (orange star). As evolution proceeds, the neighbor range decreases, making the process increasingly sensitive to local neighbors, thereby enabling global competition originally, while “zooming in” eventually to balance between optimization and diversity. (b) By mapping the population to a 1-D space (dashed lines in (a)), we track the progress of Diffusion Evolution. As evolution progresses, both the individuals (gray) and their estimated origins (red) move closer to the targets (vertical dashed lines), with the estimated origins advancing faster.
Refer to caption: x2.png

Since the denoising step in diffusion models requires an estimation of ${\bm{x}}_{0}$, we need to derive it from sample ${\bm{x}}_{t}$ and the corresponding fitness $f({\bm{x}}_{t})$. The estimation of ${\bm{x}}_{0}$ can be expressed as a conditional probability $p({\bm{x}}_{0}={\bm{x}}|{\bm{x}}_{t})$. Using Bayes’ theorem and $p({\bm{x}}_{0}={\bm{x}})=g[f({\bm{x}})]$ yields:

$$ $p({\bm{x}}_{0}={\bm{x}}|{\bm{x}}_{t})=\frac{p({\bm{x}}_{t}|{\bm{x}}_{0}={\bm{x }})p({\bm{x}}_{0}={\bm{x}})}{p({\bm{x}}_{t})}=\frac{p({\bm{x}}_{t}|{\bm{x}})g[ f({\bm{x}})]}{p({\bm{x}}_{t})}.$ (6) $$

Here, $p({\bm{x}}_{t}|{\bm{x}}_{0}={\bm{x}})$ can be computed easily by $\mathcal{N}({\bm{x}}_{t};\sqrt{\alpha_{t}}{\bm{x}},1-\alpha_{t})$ given the design of the diffusion process, i.e., ${\bm{x}}_{t}=\sqrt{\alpha_{t}}{\bm{x}}_{0}+\sqrt{1-\alpha_{t}}{{\bm{\epsilon}}}$. Since deep-learning-based diffusion models are trained using mean squared error loss, the ${\bm{x}}_{0}$ estimated by ${\bm{x}}_{t}$ should be the weighted average of the sample ${\bm{x}}$. Hence, the estimation function of ${\bm{x}}_{0}$ becomes:

$$ $\hat{\bm{x}}_{0}({\bm{x}}_{t},\bm{\alpha},t)=\sum_{{\bm{x}}\sim p_{\text{eval} }({\bm{x}})}p({\bm{x}}_{0}={\bm{x}}|{\bm{x}}_{t}){\bm{x}}=\sum_{{\bm{x}}\sim p _{\text{eval}}({\bm{x}})}g[f({\bm{x}})]\frac{\mathcal{N}({\bm{x}}_{t};\sqrt{ \alpha_{t}}{\bm{x}},1-\alpha_{t})}{p({\bm{x}}_{t})}{\bm{x}},$ (7) $$

where $p_{\text{eval}}$ is the evaluation sample on which we compute the fitness score, here given by the current population ${\bm{X}}_{t}=({\bm{x}}_{t}^{(1)},{\bm{x}}_{t}^{(2)},...,{\bm{x}}_{t}^{(N)})$ of $N$ individuals. Equation 7 has three weight terms: The first term $g[f({\bm{x}})]$ assigns larger weights to high fitness samples. For each individual sample ${\bm{x}}_{t}$, the second Gaussian term $\mathcal{N}({\bm{x}}_{t};\sqrt{\alpha_{t}}{\bm{x}},1-\alpha_{t})$ makes each individual only sensitive to local neighbors of evaluation samples. The third term $p({\bm{x}}_{t})$ is a normalization term. Hence, $\hat{\bm{x}}_{0}$ can be simplified to:

$$ $\hat{\bm{x}}_{0}({\bm{x}}_{t},\bm{\alpha},t)=\frac{1}{Z}\sum_{{\bm{x}}\in{\bm{ X}}_{t}}g[f({\bm{x}})]\mathcal{N}({\bm{x}}_{t};\sqrt{\alpha_{t}}{\bm{x}},1- \alpha_{t}){\bm{x}},$ (8) $$

where $Z$ is the normalization term:

$$ $Z=p({\bm{x}}_{t})=\sum_{{\bm{x}}\in{\bm{X}}_{t}}g[f({\bm{x}})]\mathcal{N}({\bm {x}}_{t};\sqrt{\alpha_{t}}{\bm{x}},1-\alpha_{t}).$ (9) $$

When substituting Equation 8 into Equation 4 we can express $\hat{\bm{\epsilon}}$ as:

$$ $\hat{\bm{\epsilon}}({\bm{x}}_{t},\bm{\alpha},t)=\cfrac{{\bm{x}}_{t}-\sqrt{ \alpha_{t}}\,\hat{\bm{x}}_{0}({\bm{x}}_{t},\bm{\alpha},t)}{\sqrt{1-\alpha_{t}}},$ (10) $$

and by substituting Equations 8 and 10 into Equation 5, we derive the Diffusion Evolution algorithm: an evolutionary optimization procedure based on iterative error correction akin to diffusion models but without relying on neural networks at all, see psuedocode in Algorithm 1. When inversely denoising, i.e., evolving from time $T$ to 0 0, while increasing $\alpha_{t}$, the Gaussian term will initially have a high variance, allowing global exploration at first. As the evolution progresses, the variance decreases giving lower weight to distant populations, leads to local optimization (exploitation). This locality avoids global competition and thus allows the algorithm to maintain multiple solutions and balance exploration and exploitation. Hence, the denoising process of diffusion models can be understood in an evolutionary manner: ${\hat{{\bm{x}}}}_{0}$ represents an estimated high fitness parameter target. In contrast, ${\bm{x}}_{t}$ can be considered as diffused from high-fitness points. The first two parts in the Equation 5, i.e., $\sqrt{\alpha_{t-1}}{\hat{{\bm{x}}}}_{0}+\sqrt{1-\alpha_{t-1}-\sigma_{t}^{2}}
\hat{{\bm{\epsilon}}},$ guide the individuals towards high fitness targets in small steps. The last part of Equation 5, $\sigma_{t}{\bm{w}}$, is an integral part of diffusion models, perturbing the parameters in our approach similarly to random mutations.

Figure: Algorithm 1 Diffusion Evolution

Figure 2(a) demonstrates the detailed evolution process of a multi-target fitness landscape with two optimal points (see exact fitness function in Appendix A.1). Each individual estimates high fitness parameter targets and moves toward the target along with random mutations. The high fitness parameter targets ${\hat{{\bm{x}}}}_{0}$ are estimated based on their neighbors’ fitness scores (neighbors are shown in blue disks, with radius proportional to $\sqrt{1-\alpha_{t}}/\sqrt{\alpha_{t}}$). The estimated targets ${\hat{{\bm{x}}}}_{0}$ typically move faster than the individuals while the individuals are successively refined in small denoising steps in the direction of the estimated target, see Figure 2(b). Although ${\hat{{\bm{x}}}}_{0}$ often have higher fitness, they exhibit lower diversity, hence they are used as a goal of individuals instead of the final solutions. This difference also provides flexibility in balancing between more greedy and more diverse strategies.

## 4 Experiments

We conduct two sets of experiments to study Diffusion Evolution in terms of diversity and solving complex reinforcement learning tasks. Moreover, we utilize techniques from the diffusion models literature to improve Diffusion Evolution. In the first experiment, we adopt an accelerated sampling method (Nichol & Dhariwal, ) to significantly reduce the number of iterations. In the second experiment, we propose Latent Space Diffusion Evolution, inspired by latent space diffusion models (Rombach et al., ), allowing us to deploy our approach to complex problems with high-dimensional parameter spaces through exploring a lower-dimensional latent space.

### 4.1 Multi-target Evolution

Figure: Figure 3: One of the benchmark experiments (columns) on different fitness functions with selected evolutionary algorithms (rows). The blue regions represent high fitness, while white indicates low fitness in a two-dimensional parameter space. All fitness functions are mapped to a 0 to 1 range to make them comparable, as detailed in the Appendix. The Diffusion Evolution algorithm finds multiple optimal solutions in the 2D benchmarks while maintaining genetic diversity. Red dots indicating the final population and gray lines show the trajectories of populations; for simplicity, only the trajectories of 64 individuals are plotted here. In the CMAES and PEPG experiments, the gray ellipsoids represent the estimated covariances at each step, and the gray lines represent the history of estimated averages. The red dots indicate the final population. In the OpenES experiments, the red dots indicate the final population, and the gray line represents the parameter trajectory.
Refer to caption: x3.png

To compare our method to selected mainstream evolutionary algorithms, we choose five different two-dimensional fitness landscapes as benchmarks: The Rosenbrock and Beale fitness functions have a single optimal point, while the Himmelblau, Ackley, and Rastrigin functions have multiple optimal solutions, see Appendix A.4 for more details; we compare our method to other evolutionary strategies, including CMA-ES (Hansen et al., ), OpenES (Salimans et al., ), and PEPG (Sehnke et al., ). The experiments show that our Diffusion Evolution algorithm can find diverse solutions on the Himmelblau, Ackley, and Rastrigin functions, while other methods struggle (see Figure 3 and Table 1). CMA-ES, OpenES, and PEPG either focus on finding a single solution or get distracted by multiple high-fitness peaks, leading to sub-optimal results. Our experiments demonstrate that the Diffusion Evolution algorithm can identify diverse solutions and adapt to various fitness landscapes.

The most time-consuming part of evolutionary algorithms is often the fitness evaluation. In this experiment, we adopt an accelerated sampling method from the diffusion models literature to reduce the number of iteration. As proposed by Nichol & Dhariwal (), instead of the default $\alpha_{t}$ scheduling in DDPM, a cosine scheduling $\alpha_{t}=\cos(\pi t/T)/2+1/2$ leads to better performance when $T$ is small. With this, we can significantly reduce the number of fitness evaluations while maintaining sampling diversity and quality.

**Table 1: Average entropy of the top-64 elite populations after evolution, with average fitness (0 to 1) in brackets. Higher entropy indicates greater diversity, and higher fitness reflects better solutions. Bold numbers are the highest value per line, and underlined numbers are the second-highest.**
| Task | Diffusion Evolution | CMA-ES | OpenES | PEPG |
| --- | --- | --- | --- | --- |
| Rosenbrock | 4.93 (1.00) | 0.00 (1.00) | 1.02 (1.00) | 0.86 (1.00) |
| Beale | 4.21 (1.00) | 0.00 (1.00) | 1.06 (1.00) | 0.32 (1.00) |
| Himmelblau | 2.58 (1.00) | 0.07 (1.00) | 0.05 (1.00) | 0.00 (1.00) |
| Ackley | 2.49 (1.00) | 3.96 (0.52) | 0.00 (1.00) | 0.02 (0.96) |
| Rastrigin | 3.29 (0.82) | 5.77 (0.18) | 0.00 (1.00) | 0.00 (0.11) |

To systematically compare different methods, we repeated the evolution 100 times for each method. In all experiments, the fitness functions were rescaled from 0 to 1, with 1 representing the highest fitness (see Appendix A.4). Each experiment was conducted with a population of 512 and 25 iterations, except for the OpenES method, which requires 1000 steps to converge. To quantify diversity, we then calculated Shannon entropy of the final population by gridding the space and counting the individuals in different grid cells (we select the top-64 fitness individuals, focusing solely on elite individuals). The results in Table 1 show that our method consistently finds more diverse solutions without sacrificing fitness performance. While CMA-ES shows higher entropy on the Ackley and Rastrigin functions, it finds significantly lower fitness solutions compared to Diffusion Evolution, suggesting it is distracted by multiple solutions rather than finding diverse ones (see examples in Figure 3).

### 4.2 Latent Space Diffusion Evolution

Figure: Figure 4: (a) Evolution process of cart-pole tasks: The horizontal axis shows survival time, and the vertical axis represents generations. Each point indicates an individual’s state (pole angle, cart shift) at their final survival. As the evolution progresses, more systems survive longer and achieve higher rewards. (b) Compared to the original Diffusion Evolution (blue), the Latent Space Diffusion Evolution method (red) significantly improves performance, while the CMA-ES method (gray) fails to find any solutions in given generations. This latent method can even be applied to high-dimensional spaces (orange), with dimensions as high as 17,410. Each experiment is repeated 100 times, with medians (solid lines) and ranges (25% to 75% quantile) shown as shaded areas. (c) Projecting the parameters of individuals into a latent space visualize their diversity. The same projection is used for all results (except for the high-dimensional experiment, which has a different original dimension). This indicates enhanced diversity with the latent method. (d) The cart-pole system consists of a pole hinged to the cart. And the controller balances the pole by moving the cart left or right.
Refer to caption: x4.png

Here, we apply the Diffusion Evolution method to reinforcement learning tasks (Sutton & Barto, ) to train neural networks for controlling the cart-pole system (Barto et al., ). This system has a cart with a hinged pole, and the objective is to keep the pole vertical as long as possible by moving the cart sideways while not exceeding a certain range, see Figure 4(d). The game is terminated if the pole-angle exceeds $\pm 12^{\circ}$ or the cart position exceeds $\pm 2.4$. Thus, longer duration yield higher fitness. We use a two-layer neural network of 58 parameters to control the cart, with inputs being the current position, velocity, pole angle, and pole angular velocity. The output of the neural network determines whether to move left or right. See more details about the neural network in Appendix A.5.1. The task is considered solved when a fitness score (cumulative reward) of 500 is reached consistently over several episodes.

Deploying our original Diffusion Evolution method to this problem results in poor performance and lack of diversity, see Figure 4(b-c).
To address this issue, we propose Latent Space Diffusion Evolution: inspired by the latent space diffusion model (Rombach et al., ), we map individual parameters into a lower-dimensional latent space in which we perform the Diffusion Evolution Algorithm. This approach significantly improves performance and restores diversity. The key insight comes from the Gaussian term in Equation 8 for estimating the original points $\hat{\bm{x}}_{0}$: the distance between parameters increases with higher dimensions, making the evolution more local and slower. Moreover, the parameter or genotype space may have dimensions that don’t effectively impact fitness, known as sloppiness (Gutenkunst et al., ). Assigning random values to these dimensions often doesn’t affect fitness, similar to genetic drift or neutral genes, suggesting the true high-fitness genotype distribution is lower-dimensional. The straightforward approach is directly denoising in a lower-dimensional latent space ${\bm{z}}$ and estimating high-quality targets ${\bm{z}}_{0}$ via:

$$ $\hat{\bm{z}}_{0}({\bm{z}}_{t},\bm{\alpha},t)=\sum_{\bm{z}}p({\bm{z}}|{\bm{z}}_ {t}){\bm{z}}=\frac{1}{Z}\sum_{{\bm{x}}\sim p_{\text{eval}}({\bm{x}})}g[f^{ \prime}({\bm{z}})]\mathcal{N}({\bm{z}}_{t};\sqrt{\alpha_{t}}{\bm{z}},1-\alpha_ {t}){\bm{z}}.$ (11) $$

However, this approach requires a decoder and a new fitness function $f^{\prime}$ for ${\bm{z}}$, which can be challenging to obtain. To circumvent this, we approximate the latent diffusion by using the latent space only to calculate the distance between individuals. While we don’t know the exact distribution of ${\bm{x}}$ apriori, a random projection can often preserve the distance relationships between populations, as suggested by the Johnson-Lindenstrauss lemma (Johnson, ). To do this, we change Equation 11 to:

$$ $\hat{\bm{x}}_{0}({\bm{x}}_{t},\bm{\alpha},t)=\sum_{{\bm{x}}\sim p_{\text{eval} }({\bm{x}})}p({\bm{x}}|{\bm{x}}_{t}){\bm{x}}=\frac{1}{Z}\sum_{{\bm{x}}\sim p_{ \text{eval}}({\bm{x}})}g[f({\bm{x}})]\mathcal{N}({\bm{z}}_{t};\sqrt{\alpha_{t} }{\bm{z}},1-\alpha_{t}){\bm{x}},$ (12) $$

where ${\bm{z}}={\bm{E}}{\bm{x}}$, ${\bm{E}}_{ij}\sim\mathcal{N}^{(d,D)}(0,1/D)$, $D$ is the dimension of ${\bm{x}}$, and $d$ is the dimension of latent space (Johnson, ), see Algorithm 2 in Appendix. Here we choose $d=2$ in our experiments for better visualization. The results show a significant improvement in both fitness and diversity, see Figure 4(b-c). We also found that this latent evolution can still operate in a much larger dimensional parameter space, utilizing a three-layer neural network with $17,410$ parameters, while still achieving strong performance. Combined with accelerated sampling method, we can solve the cart pole task in only $10$ generations, with $512$ population size, one fitness evaluation per individual. These highlights the potential of using tools and theories from the diffusion model domain in evolutionary optimization tasks and vice versa, opening up broad opportunities to improve and understand evolution from a new perspective.

## 5 Discussion

By aligning diffusion models with evolutionary processes, we demonstrate that diffusion models are evolutionary algorithms, and evolution can be viewed as a generative process. The Diffusion Evolution process inherently includes mutation, selection, hybridization, and reproductive isolation, indicating that diffusion and evolution are two sides of the same coin. Our Diffusion Evolution algorithm leverages this theoretical connection to improve solution diversity without compromise quality too much compared to standard approaches. By integrating latent space diffusion and accelerated sampling, our method scales to high-dimensional spaces, enabling the training of neural network agents in reinforcement learning environments with exceptionally short training time.

This equivalence between the two fields offers valuable insights from both deep learning and evolutionary computation. Through the lens of machine learning, the evolutionary process can be viewed as nature’s way of learning and optimizing strategies for survival of species over generations. Similarly, our Diffusion Evolution algorithm iteratively refines estimation of high-fitness parameters, continuously learning and adapting to the fitness landscape.
This positions evolutionary algorithms and diffusion models not merely as optimization tools, but also as learning frameworks that enhance understanding and functionality through iterative refinement. Conversely, framing evolution as a diffusion process offers a concrete mathematical formulation. In contrast to previous approaches (Ao, ), we provide an explicit and implementable evolutionary framework.

The connection between diffusion and evolution enables mutual contributions between the two fields. Diffusion models are extensively studied in the contexts of controlling, optimization, and probability theory, offering robust tools to analyze and enhance evolutionary algorithms. In our experiments, leveraging concepts from diffusion models enabled flexible strategies while maintaining the effectiveness of evolutionary processes. For instance, accelerated sampling methods (Nichol & Dhariwal, ) can be applied seamlessly to Diffusion Evolution to accelerate the optimization process. Latent diffusion models (Rombach et al., ) inspired our Latent Space Diffusion Evolution, enabling evolution in high-dimensional spaces and substantially improving performance. Other advancements in the diffusion model field hold the potential to enhance our understanding of evolutionary processes. For instance, non-Gaussian noise diffusion models (Bansal et al., ), discrete denoising diffusion models, and theoretical studies of diffusion models through the lens of non-equilibrium thermodynamics, such as Langevin dynamics (Song et al., ) or spontaneous symmetry breaking (Raya & Ambrogioni, ), unveil entirely new possibilities and perspectives for understanding and advancing evolutionary methods.

However, this parallel we draw here between evolution and diffusion models also gives rise to several challenges and open questions. While diffusion models, by design, have a finite number of sampling steps, evolution is inherently open-ended. How can Diffusion Evolution be adapted to support open-ended evolution? Could other diffusion model implementations yield different evolutionary methods with diverse and unique features? Can advancements in diffusion models help introduce inductive biases into evolutionary algorithms? How do latent diffusion models correlate with neutral genes? Additionally, can insights from the field of evolution enhance diffusion models? These questions highlight the potential of this duality and synergy between diffusion and evolution.

## Acknowledgments

The authors acknowledge the Tufts University High Performance Compute Cluster (^2^22[https://it.tufts.edu/high-performance-computing](https://it.tufts.edu/high-performance-computing)) and the Vienna Scientific Cluster (^3^33[https://www.vsc.ac.at](https://www.vsc.ac.at)) which have been utilized for the research reported in this paper. M.L. gratefully acknowledges support via grant 62212 from the John Templeton Foundation and via grant TWCF0606 from the Templeton World Charity Foundation. BH acknowledges an APART-MINT stipend by the Austrian Academy of Sciences.

## Appendix A Appendix

### A.1 Two-peaks models

We first apply our method to a simple two-dimensional fitness function, with two optimal points, to demonstrate its behavior and capability of finding multiple solutions. We choose a continuous mixed Gaussian density function. The fitness function is a mixed Gaussian density function with means located at $(1,1)$ and $(-1,-1)$. The fitness function is:

$$ $f(x,y)=\left[\mathcal{N}((x,y);\bm{\mu}_{1},\sigma^{2})+\mathcal{N}((x,y);\bm{ \mu}_{2},\sigma^{2})\right]/2,$ (13) $$

where $\bm{\mu}_{1}=(1,1)$ and $\bm{\mu}_{2}=(-1,-1)$. And the $\sigma=0.1$.

### A.2 Alpha and Noise Schedule

In our experiments, we tested three different noise schedules for $\alpha_{t}$. The first is a simple linear schedule, used in Figure 2:

$$ $\alpha_{t}=1-\frac{1}{T}.$ (14) $$

The second is the schedule used in DDPM, which can be approximated by:

$$ $\alpha_{t}=\exp\left(-\beta_{0}t-\frac{\gamma t^{2}}{T}\right),$ (15) $$

where $\beta_{0}$ and $\gamma$ are hyperparameters. These are calculated by constraining $\alpha_{0}=1-\varepsilon$ and $\alpha_{T}=\varepsilon$, with $\varepsilon=10^{-4}$ as the default.

The third schedule is the cosine schedule proposed by Nichol & Dhariwal () and is used in both Figures 3 and 4:

$$ $\alpha_{t}=\frac{1}{2}\cos\left(\frac{\pi t}{T}\right)+\frac{1}{2}.$ (16) $$

For $\sigma_{t}$, we follow the DDIM setting with a slight modification for better control:

$$ $\sigma_{t}=\sigma_{m}\sqrt{\frac{1-\alpha_{t-1}}{1-\alpha_{t}}}\sqrt{1-\frac{ \alpha_{t}}{\alpha_{t-1}}},$ (17) $$

where $0\leq\sigma_{m}\leq 1$ is a hyperparameter to control the magnitude of noise. We use $\sigma_{m}=1$ for most experiments, except for the experiment demonstrating the process in Figure 2, which requires a lower noise magnitude $\sigma_{m}=0.1$ for better visualization.

### A.3 Neighbor of Individuals

In Figure 2, we use blue discs to represent the neighbors of each individual. Its mean and standard deviation can be derived from Equation 8. By transforming it into an equivalent form with ${\bm{x}}$ as the variable and ${\bm{x}}_{t}$ as the parameter, we have:

$$ $\mathcal{N}({\bm{x}}_{t};\sqrt{\alpha_{t}}{\bm{x}},1-\alpha_{t})=\frac{1}{ \sqrt{\alpha_{t}}}\mathcal{N}\left({\bm{x}};\frac{{\bm{x}}_{t}}{\sqrt{\alpha_{ t}}},\sqrt{\frac{1-\alpha_{t}}{\alpha_{t}}}\right).$ (18) $$

Hence, this Gaussian term can be transformed into a form where ${\bm{x}}$ is the variable, which is more intuitive for describing ${\bm{x}}$ as the neighbors of ${\bm{x}}_{t}/\sqrt{\alpha_{t}}$. Thus, the discs have $\bm{\mu}={\bm{x}}_{t}/\sqrt{\alpha_{t}}$ and $r^{2}=(1-\alpha_{t})/\alpha_{t}$.

### A.4 Two-dimensional Fitness Function

To benchmark the solution diversity and performance, we choose five different fitness functions to compare our method with other evolutionary strategies. All the functions depend on variables $x$ and $y$, with the objective being to minimize or maximize the function value. Specifically, we constrain the range of $x$ and $y$ to $(-4,4)$ and set the objective of the Rastrigin function to be maximization instead of minimization to benchmark the capability of finding multiple solutions. To standardize the comparison, we apply the following transformation to convert target values to the highest fitness:

$$ $F(x,y)=\frac{\epsilon}{\epsilon+\|f(x,y)-f^{*}\|^{2}/s^{2}}$ (19) $$

Here, $\epsilon=10^{-3}$ is used to avoid singular values, and $f^{*}$ is the target fitness value, with $s$ as the scale factor to make different functions comparable. For each fitness function, we determine $f^{*}$ as the minimal or maximal value, depending on the optimization objective. The scale factor is determined by the standard deviation of fitness around their optimal points, with ranges adjusted for different functions. After this transformation, the lowest fitness is near zero, and the highest fitness is one.

**Table 2: Fitness functions used in our experiments.**
| Name | Formula | features |
| --- | --- | --- |
| Rosenbrock | $f(x,y)=100(y-x^{2})^{2}+(1-x)^{2}$ | The minimal value position is $(x,y)=(1,1)$, where $f(1,1)=0$. |
| Beale | $f(x,y)=(1.5-x+xy)^{2}+(2.25-x+xy^{2})^{2}+(2.625-x+xy^{3})^{2}$ | The minimal value position is $(x,y)=(3,0.5)$, where $f(3,0.5)=0$. |
| Himmelblau | $f(x,y)=(x^{2}+y-11)^{2}+(x+y^{2}-7)^{2}$ | This function has four minimal value points, they are:<br>$f(3.0,\ 2.0)=0.0$,<br>$f(-2.81,3.13)=0.0$,<br>$f(-3.78,-3.28)=0.0$,<br>$f(3.58,-1.85)=0.0$ |
| Ackley | $f(x,y)=-20\exp\left({-0.2\sqrt{\frac{x^{2}+y^{2}}{2}}}\right)-\exp\left(\frac{<br>\cos 2\pi x+\cos 2\pi y}{2}\right)+e+20$ | When restricting the range of $x,y$ between -4 to 4, the maximal points are located at the four corners. |
| Rastrigin | $f(x)=An+\sum_{i=1}^{2}[x_{i}^{2}-A\cos(2\pi x_{i})]$ | Here $A=10$. Similar as the Ackley function above, when restricting the range of $x,y$, the maximal points are located at the four corners. |

#### A.4.1 Estimating Entropy to Quantify Diversity

To quantify the diversity of the solutions, we divided the 2-D space into $80\times 80$ grids and counted the frequencies of elite solutions within this space. We intentionally used this simple and coarse method to quantify entropy in order to eliminate the contribution of local diversity, focusing solely on the diversity of solutions across different basins. The entropy is calculated by:

$$ $H=\sum_{i=1}^{N}P_{i}\log_{2}P_{i},$ (20) $$

where $P_{i}$ is the probability of having a sample in grid $i$.

### A.5 Cart-Pole Experiment

#### A.5.1 Neural Network

The controller of the cart-pole system has four observational inputs: the current position, velocity, pole angle, and pole angular velocity. The system accepts two actions: move left or right. To model the controller, we use artificial neural networks with an input layer of 4 neurons corresponding to the four observations and an output layer of 2 neurons corresponding to the two actions. The action is determined by which output neuron has the higher value.

Our standard experiment uses a one-hidden-layer neural network with the hidden layer of 8 neurons, resulting in $(4\times 8+8)+(8\times 2+2)=58$ parameters. We also use a deeper neural network with two hidden layers (each has 128 neurons), totaling $(4\times 128+128)+(128\times 128+128)+(128\times 2+2)=17410$ parameters. Both neural networks use the ReLU activation function.

### A.6 Latent Space Diffusion Evolution

Following is the pseudocode for Latent Space Diffusion Evolution algorithm. The difference from the original Diffusion Evolution (Algorithm 1) is indicated in blue.

Figure: Algorithm 2 Latent Space Diffusion Evolution

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: In the text is not mention what g could be or some options. It could be like the utility function used in Natural Evolutionary Strategies? because it seems to assign more weight/importance to high ranking solutions.

I think it would be very interesting in a  future work to explore more the latent diffusion version of the algorithm, and techniques to understand which parameters could be ignored or which parameters should be perturbed together.
soundness: 3
strengths: By connecting methods from different fields the work proposed an interesting new evolutionary algorithm that can be further improved using variations and techniques found in diffusion models.

The text is very clear and the math derivation from diffusion to an EA algorithm is easy to follow. Figure 1 really helps to visualize how the population of the diffusion EA spreads to the higher fitness regions.
summary: The authors try to connect Diffusion Models and Evolutionary computation by arguing that both processes do iterative refinements through an update rule plus some perturbation. In evolutionary algorithms the update comes in the form of natural selection, in diffusion is the denoising phase. The perturbation corresponds to mutation for evolution, and the diffusion phase for diffusion models. 

From the above connection an evolutionary algorithm that performs in its iteration a diffusion process is proposed.  Instead of aiming to recover some distribution of the data, the goal is to turn the random initial population points towards a distribution centered around the optimized function optimum.
weaknesses: Maybe I missed it but there is no discussion on how to choose the mapping to a density function g() or which g() was used for the experiments. It seems is important for the search to work properly to have a mapping that makes it clear which points should be paid more attention by assigning to them bigger weights. Maybe there is some connection to common selection strategies in evolutionary algorithms that could be discussed.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: As mentioned in the appendix, different Alpha and noise schedule settings are tested, what about the detailed experimental results? Is there any analysis about the results of different settings, which could be useful for the users?
soundness: 2
strengths: The paper is well-written and easy to follow. The proposed method Diffusion Evolution is well-motivated, with clear explanation and illustration.
summary: This paper proposes a new evolutionary algorithm inspired by diffusion models. It views the evolution processes as the denoising process of diffusion models, and designs an evolutionary algorithm named Diffusion Evolution, which is based on the denoising framework of the famous diffusion model DDIM. During evolution, it takes each individual in the population as a solution in the denoising process, and updates it with similar updating rules in DDIM. To facilitate better performance, it further follows previous works to optimize in the latent space. Experiments on several benchmark functions and a simple cart-pole controlling problem show that compared with classic baseline methods include CMA-ES, OpenES and PEPG, the proposed Diffusion Evolution can obtain more diverse solutions with good fitness.
weaknesses: As there has been a variety of evolutionary algorithms with various inspirations, the most important issue when proposing a new method should be clarifying its strength compared with previous methods, from the perspectives of theoretical analysis and extensive experiments. However, the proposed Diffusion Evolution in this paper is lack of theoretical analysis. Meanwhile, the experiments are much too simple. For example, only five synthetic functions and a simple cart-pole controlling problem are included. As the diversity of solutions seems to be the strength of the proposed Diffusion Evolution, except for methods like CMA-ES, which are not specified for diversity, comparison with other kinds of evolutionary algorithms like Quality-Diversity (QD) are necessary.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: In the experiments section, the authors select three different evolutionary strategies for performance comparison. However, the rationale for choosing these specific methods is not clearly articulated. I would appreciate more insight into this selection process, particularly since each of these methods was introduced over five years ago.
soundness: 3
strengths: The idea of regrading the diffusion model as an evolutionary algorithm is interesting.
summary: This manuscript introduces an innovative perspective by interpreting the diffusion model as an evolutionary algorithm, highlighting the mathematical similarities between the diffusion and evolutionary processes. It proposes a Diffusion Evolution Method that employs iterative denoising to heuristically optimize solutions within the parameter space, drawing an analogy between the generative process of the diffusion model and the selection and mutation mechanisms in biological evolution.
weaknesses: 1. The mathematical demonstration process in this manuscript is insufficient. The authors conceptualize the evolution process as a transformation of the probability density function, deriving the diffusion evolution algorithm from the Bayesian formula. However, this derivation overlooks key complexities inherent in evolutionary dynamics, such as genetic drift and gene recombination. Additionally, the application of the Bayesian formula assumes conditional independence, which may not be held in the context of evolutionary processes due to the interactions and competition among individuals. Furthermore, the manuscript does not provide a clear explanation of how to effectively map a complex fitness function into a probability density function, which is essential for the practical implementation of the proposed model.
2. Since the author claims that the diffusion model functions as an evolutionary algorithm, the experimental section must reflect this comparison appropriately. Specifically, in scenarios involving multiple tasks and comparisons, aspects such as speed, performance metrics (including best, worst, median, and average results), and the stability of performance should be examined. Additionally, the model's excellence should be illustrated through the mean variance of the results. Currently, the scope of experiments conducted is insufficient and does not adequately support the claims made in the manuscript.
3. There are several grammatical errors and unclear expressions throughout the article. For example, the definitions of certain terms lack clarity, the derivation process for the formulas is inadequately explained, and the labeling of the charts is inaccurate.
4. The authors categorize diffusion models as evolutionary algorithms on the basis that both methods perform distribution transformation. However, this classification may be overly broad. For instance, semantic segmentation models also involve transforming distributions—from real images to pixel-level segmentation results. Should we, therefore, classify these segmentation models as evolutionary algorithms as well? This comparison seems inaccurate. A more robust argument would establish that the Markov process within a non-equilibrium thermodynamics framework (diffusion) function as an unconstrained parameter optimization technique (evolution). If this argument cannot be substantiated, I recommend revising the title to better reflect the content.
5. The manuscript presents two primary sets of experiments: multi-objective evolution and latent space diffusion evolution. However, the multi-objective evolution experiments are limited to a two-dimensional parameter space and a simplistic fitness function, which fails to demonstrate the proposed approach's efficacy in high-dimensional parameter spaces or with more complex fitness functions. Similarly, the latent space diffusion evolution experiments focus solely on the CartPole task, lacking validation across a broader range of reinforcement learning tasks. Furthermore, the experimental results do not include any statistical significance tests, making it challenging to determine whether the observed improvements in algorithm performance are statistically significant. This omission significantly undermines the credibility of the findings presented.

---

confidence: 4
contribution: 2
initial_rating: 8
presentation: 3
questions: The work potentially contributes new insights and methods for both diffusion model and evolutionary algorithm communities, even if the practical impact in specific real-world applications remains to be validated. Can the authors address this?

Can statistical tests be added?
soundness: 3
strengths: he core idea of treating diffusion models as evolutionary algorithms is innovative. It extends diffusion models to broader applications by framing them as tools for evolutionary tasks, potentially contributing new knowledge to both evolutionary biology and AI communities.

Technically: The paper provides thorough mathematical grounding for the equivalence between diffusion and evolution, specifically explaining diffusion as a probabilistic denoising process analogous to evolutionary mechanisms like mutation and selection.

Experimentals: Comprehensive experiments benchmark Diffusion Evolution against traditional algorithms (e.g., CMA-ES, PEPG) ascross various fitness landscapes, demonstrating its strength in maintaining diversity and achieving multiple optima.

Applications in High-Dimensional Problems: The adaptation of Diffusion Evolution to high-dimensional tasks via latent space diffusion showcases its practical viability in reinforcement learning contexts, with empirical results suggesting its potential in complex environments.
summary: This paper explores the theoretical and practical parallels between diffusion models and evolutionary algorithms, proposing the "Diffusion Evolution" approach, which adapts diffusion models for evolutionary tasks. This new method, Latent Space Diffusion Evolution, enhances the ability to find diverse and optimal solutions in high-dimensional parameter spaces, particularly within reinforcement learning.
weaknesses: Complexity of Explanation: The theoretical connections between diffusion and evolution, while compelling, are highly complex, and the paper occasionally sacrifices clarity for depth. This might limit accessibility to a broader audience.

Although promising, the methodology may have limitations in scenarios requiring open-ended evolution, a challenge acknowledged briefly. More thorough discussion could help set realistic expectations for potential users.

While the Diffusion Evolution algorithm demonstrates superiority in finding diverse solutions, certain comparisons (e.g., high-fitness solutions) against traditional evolutionary algorithms lack statistical significance or detailed analysis of computational cost versus benefit, particularly in high-dimensional settings.

Meta-review: This paper bridges the fields of machine learning and evolutionary computation by revealing a mathematical equivalence between diffusion models and evolutionary algorithms. Diffusion models, initially designed for generative tasks, are reinterpreted as performing evolutionary algorithms through iterative denoising. The denoising process in diffusion models is analogous to natural selection in evolutionary algorithms, while the perturbation step corresponds to mutation or diffusion. Based on this insight, the authors introduce a novel evolutionary algorithm called Diffusion Evolution.

Diffusion Evolution uses the iterative denoising process from diffusion models to refine solutions in a parameter space. Instead of recovering a data distribution, the algorithm shifts an initial random population toward an optimized solution distribution. By leveraging latent space diffusion and accelerated sampling techniques, the authors further propose Latent Space Diffusion Evolution, which addresses high-dimensional optimization problems with reduced computational overhead. This method enables the efficient discovery of diverse and high-quality solutions.

Experimental results on benchmark optimization functions and a cart-pole control task demonstrate that Diffusion Evolution outperforms traditional evolutionary algorithms like CMA-ES, OpenES, and PEPG. It achieves improved solution diversity and fitness, showcasing its effectiveness in complex optimization scenarios.

The reviewers have generally found the contribution sound, novel, and worthy of publication and I highly recommend this paper for being accepted.

Decision: Accept (Poster)