Soft Quality-Diversity Optimization
Authors: Saeed Hedayatian
Stefanos Nikolaidis
Venue: ICLR 2026 Poster (2026)
Keywords: Quality Diversity Optimization
Primary area: optimization
TL;DR: We introduce Soft QD, a new formulation of quality-diversity optimization and introduce SQUAD, a derived algorithm that achieves state of the art performance with improved scalability to high-dimensional spaces.

Abstract: Quality-Diversity (QD) algorithms constitute a branch of optimization that is concerned with discovering a diverse and high-quality set of solutions to an optimization problem.
Current QD methods commonly maintain diversity by dividing the behavior space into discrete regions, ensuring that solutions are distributed across different parts of the space.
The QD problem is then solved by searching for the best solution in each region.
This approach to QD optimization poses challenges in large solution spaces, where storing many solutions is impractical, and in high-dimensional behavior spaces, where discretization becomes ineffective due to the curse of dimensionality.
We present an alternative framing of the QD problem, called \emph{Soft QD}, that sidesteps the need for discretizations.
We validate this formulation by demonstrating its desirable properties, such as monotonicity, and by relating its limiting behavior to the widely used QD Score metric.
Furthermore, we leverage it to derive a novel differentiable QD algorithm, \emph{Soft QD Using Approximated Diversity (SQUAD)}, and demonstrate empirically that it is competitive with current state of the art methods on standard benchmarks while offering better scalability to higher dimensional problems.

Published as a conference paper at ICLR 2026
SOFT QUALITY-DIVERSIT OPTIMIZATION
Saeed Hedayatian' & Stefanos Nikolaidis
' 'University of Southern California
2 Arrhhmedes AI ss (seedhed,, n nikoolidd nikolaidjeusc.edi
ABSTRACT
Quality-Diversine (QD) algorithms constitute branch of optimization that is con- cerned with discovering a diverse and high-quality set of solutions to an optimiza- tion problem. Current QD methods commonly maintain diversity by dividing the behavior space into discrete regions, ensuring that solutions are distributed across different parts of the space. The QD problem is then solved by searching for the best solution in each region. This approach to QD optimization poses challenges in large solution spaces, where storing many solutions impractical, and in high- dimensional behavior spaces, where discretization becomes ineffective due to the curse of dimensionality. We present an alternative framing of the QD problem, called Soft QD, that sidesteps the need for discretizations. We validate this for- mulation by demonstrating its desirable properties, such as monotonicity, and by relating its limiting behavior the widely used QD Score metric. Furthermore, we leverage it to derive a novel differentiable QD algorithm, Soft QD Using Ap- proximated Diversity (SQUAD), and demonstrate empirically that it is competi- tive with current state of the art methods on standard benchmarks while offering better scalability higher dimensional problems. Source code is available https:: htt//gtttuu.../ com conf wtt.://///nteeeeeeeee//////////e soff qd
INTRODUCTION
Optimization in machine learning typically cast as the search for a single solution that maximizes performance with respect to some objective. Quality-diversity (QD) optimization (Pugh et al.) 2016) Chatzilygeroudis al. 202T) challenges this paradigm by instead discovering collection of solu- tions that are both high- performing and behaviorally diverse. This perspective is especially powerful in domains with multiple useful optima, or where robustness and user choice matter as much as raw performance, To illustrate, consider the task of painting a portrait. A traditional optimizer might aim for the single image that most similar to the subject. Conversely, a QD optimizer not only aims for high fidelity, but also explores behavior space that captures stylistic dimensions like color palette, brushstroke texture, and degree of abstraction, yielding a set of portraits that all resemble the subject while spanning spectrum of artistic expressions. Such diversity is valuable for human selection and for escaping the limitations of optimizers (Qian et al. 2024) and imperfect objectives.
In recent years, QD optimization has grown from its roots in evolutionary computation into a broadly applicable machine learning paradigm. In reinforcement learning, QD has generated diverse policies that facilitate exploration and improve robustness, both single (Pierrot et al.) 2022, Faldor al.
Batra et al. 2024) and multi-agent (Ingvarsson et al. 2023) settings. In the context of large foundation models, QD has been adopted for red-teaming and safety analysis (Sanvelyan et al.)
2023
2024
Wang et al. 2025) as well as diverse content generation (Bradley et al., [024|] Ding et al.
2024 Beyond these, QD has found applications in scenario generation (Bhatt et al. Fontaine
& Nikolaidis 2022 Zhang al. 22224), creative design (McCormack & Gambardella] 2022 Zammit
ct al.
2024). engineering (Stikas et al. 2023 Hagg et al. 2025), robotics (Huber et al. 2023 Zhong
al.
2023). and scientific discovery (Boige al. 2023
Janmohamed et al. 2024).
QD algorithms typically operate by partitioning the behavior space into discrete cells and seeking
the best solution in each cell (a tessellation, together with the stored solutions, is often referred to as the archive). Progress on this objective is commonly measured by the QD Score (Pugh et al.
2016), which sums the performance of the best solutions across all occupied cells, thus capturing

---

Published as a conference paper at ICLR 2026
both quality and coverage. This approach has fueled much of the progress in QD, including recent advances that incorporate surrogate models (Gaier et al. 2018 Zhang et al. 2022). gradient informa- tion (Nilsson & Cully] [202T] Fontaine & Nikolaidis 2021 2023). and alternative archive structures (Vassiliades et al., 20188] [Fontaine & Nikolaidis] [023] Mouret 2023). Yet, the formulation also presents two fundamental limitations. First, the non-differentiable nature of tessellations precludes direct optimization using gradient- based optimizers that dominate modern machine learning, except through heuristics (Fontaine & Nikolaidis 2021] Nilsson & Cully 2021) Second, discretizing the behavior space suffers from the curse of dimensionality in high-dimensiona, setting, as either the number or the volume of cells will grow exponentially, forcing methods to rely on dimensionality reduction techniques such as PCA or autoencoders (Paolo et al.) 2020; Grillotti & Cully 202 He-
dayatian Nikolaidis, 2025). As a result, QD optimization remains hindered in high-dimensional, gradient-rich machine learning domains, despite its promise.
We address these challenges by rethinking the formulation of QD optimization itself. We introduce Soft QD Score as a new objective for QD optimization that measures how well collection of solu- tions cover the behavior space with high-quality solutions, without discretizing the behavior space. Building it, derive SQUAD, a novel algorithm that leverages tractable lower bound of the Soft QD score to enable end-to-end differentiable optimization. This approach has an intuitive in- terpretation finding an equilibrium between attractive forces, which drive solutions toward higher quality, and repulsive forces, which spread them across the behavior space. Through experiments on both established and newly designed QD benchmarks, we demonstrate that SQUAD broadens the applicability of QD and provide further insights into its properties.
Our contributions are threefold: 1. We introduce Soft QD, a new formulation of QD optimization and analyze its theoretical properties. 2. We develop SQUAD, a differentiable QD algorithm derived from the aforementioned formulation. 3. We evaluate SQUAD on multiple benchmarks, showing its effectiveness in high- nigh-dimensional and complex optimization settings.
BACKGROUND
The quality diversity (QD) problem aims to find collection of high-quality solutions that are diverse in their behavior. A QD problem is defined by solution space θ an objective quality function f: + IR which measures a solution's ' quality, and a behavior descriptor function desc : θ + B that maps each solution in B a high-quality solution that exhibits that specific behavior. We can formalize this objective as finding a set of solutions O = {0a he that maximizes the integral of their quality over the behavior space. [g f(0)) db. The problem is referred to as Differentiable Quality-Diversine (DQD) (Fontaine & Nikolaidis 202T when both the objective and descriptor functions differentiable.
point in the behavior space B. The goal is to discover for each point
Since the behavior space B is continuous, QD algorithms often partition it into n cells, A == (c,,.... J, known as an archive or lessellation. The QD objective is then framed as finding a high-quality solution for each cell. This is captured by the QD Score, which is the sum of the maximum quality found within each cell:
max QD Score (0)
Σ ma0 :0 e 0, desc(O) e c). cEAA
Discretizing the behavior space introduces fundamental challenge in high dimensions due to the curse of dimensionality. Grid archives (e.g.. gulll et al. (2015) divide the space evenly along each dimension, which makes the number of cells grow exponentially with the dimensionality of B. This makes it infeasible maintain fine- graineg grid when B is nigh-dimensionall Centroidal Voronoi Tessellation (CVT) archives ( (Vassiliades et al. 2018) address this by fixing the number of cells and defining them around a set of centroids. Each cell contains all points that are closer to its assigned centroid than to any other centroid, creating an almost uniform partitioning of the space. While this avoids the exponential increase of the number of cells, the volume of each CVT cell still grows exponentially with dimensionality. Large cells make it difficult to explore new regions by building on existing solutions (which is commonly done in QD methods), since reaching a different cell would likely require substantial changes to current solution. Consequently, both discretization strategies face practical limitations in high-dimensiona behavior spaces, either requiring an infeasibly large number of cells forcing exploration across cells that too large navigate effectively.
2

---

Published as a conference paper at ICLR 2026
3 SOFT QUALITY-DIVERSITY
To overcome the challenges of discretizing the behavior space, we introduce Soft QD Score, an objective for quality-diversity that forgoes tessellations. Conceptually, our approach builds on the view of QD algorithms as a form of "illumination" (Cully et al. 2013). We treat each solution as a light source that illuminates the behavior space, where brightness is proportional to its quality. A set of solutions is then evaluated based on how well they illuminate the entire behavior space. This contrasts with traditional approaches, where a discrete cell considered fully illuminated by its single best occupant. In Soft QD, solutions contribute to illuminating multiple regions, with their influence decaying smoothly as function of distance. Figure T Tillustrates this difference.
K
50
(
Figure 1: Left: In a discrete archive, each cell is fully illuminated by its highest-quality occupant. Right: In Soft QD, each solution illuminates the area around with an intensity proportional to its quality. The smooth scalar field defined by the behavior value vo (b) is independent of discretization. Formally, to assess population of solutions O = (f..... vN we first define the behavior value vo (bb that induces at any point b in the behavior space as
(b)= = max fa exp (- |lb b IENEN
(-1lb-ball*
(2)
20
where fn = f(0x) and b, desc(0x) are the quality and behavior descriptors of solution On. respectively, and is a kernel width parameter. We use the Gaussian kernel, in line with its standard application in methods like density estimation (Bishop) 2007). to model a smooth, localized field of influence for each solution. Intuitively, vo (b) measures the quality of the best available solution for a target behavior b, discounted exponentially by its distance in behavior space. If the population contains a solution whose behavior b, perfectly matches b, the behavior value up will be at least as large its quality fn. Conversely, (bb approaches zero when there are no high-quality solutions near b. Segureejilussteee move around in the behavior space.
how the scalar field of behavior values changes as the solutions
The total behavior value that population 0 induces over the entire behavior space measures its combined quality and diversity. We call this quantity Soft QD Score and formally define as:
S( = / ue (b) db.
(3)
The term "Soft" highlights a key difference from the traditional QD Score. Instead hard assign- ment of solutions discrete cells, here each solution continuously contributes to the illumination of the behavior space. Soft QD Score captures our expectations of a QD solution. To obtain a high value, the population must contain high-quality solutions spread across the behavior space. More- over, adding new solutions to a population will only ever increase the value of the population, and so does increasing the quality of existing solutions. The following theorem, formally stated and proven in Appendix B. establishes some of these properties. Furthermore, this theorem provides additional grounding for Soft QD Score by connecting its limiting behavior to the conventional QD Score.
Theorem 1 (Informally stated). The Soft QD Score, defined in Eq. satisfies the following properties:
Monotonicity. The value is non-decreasing with respect to the addition of new solutions and the improvement of existing solution qualities. Submodularity. The value is submodular set function.
3

---

Published as a conference paper at ICLR 2026
2.8
Delantar rable ((b)
10
Behavior space B
Figure 2: Optimizing Soft QD Score with SQUAD. The plots visualize the behavior value func- tion, vo (b), induced by population of five solutions. The bottom plane represents the behavior space B, the height corresponding to solution quality, f. and the colored surface shows the induced behavior value ve. Initially (left), cluster of low-quality solutions induces a low behavior value. As SQUAD improves the population (right), the induced behavior value increases in both magnitude and coverage, leading higher Soft QD Score.
Limiting Equivalence. In the limit - O. the Soft QD Score converges (up constant factor) to the traditional QD score on a fine-grained archive.
4 SQUAD: SOFT QD USING APPROXIMATED DIVERSITY
Directly maximizing the Soft QD Score of population, S(0), is challenging it involves integral over the behavior space. We can, however, maximize a tractable lower bound. Theorem 2 establishes such bound, which forms the basis of our algorithm.
Theorem 2. Given a population O {0m) I with qualities (fr and behavior descriptor
vectors {b,)] N n= in behavior space R2 its Soft QD Score S() can be approximated by a lower bound (()) defined as:
N
b, (_Ilb: 1ibb-b,ll''
5(0) = (2TO?)
En fm
Σ IEiJEN
Vifa
(4)
SØ
proof is provided in Appendix A.I
Our algorithm, Soft QD Using Approximated Diversity (SQUAD), iteratively improves randomly initialized population of N solutions by updating its constituent solutions to maximize this lower bound. For brevity, drop the leading constant (2 it does not affect the optima. Fur- thermore, we rename S02 as x2 which, as we shall see in Section 5.3) controls the quality-diversity trade-off. Therefore, with some slight abuse of notation, we define the SQUAD objective (()) as:
5(0) Σ J Σ Vill exp (-Ib:- [[bbbbb]] byll3
(5)
LEIGIEN
Assuming that the quality and behavior descriptor functions (f and desc) are differentiable, this objective will also be fully differentiable with respect to the solutions' parameters. Hence, use modern optimizer like Adam (Kingma & Bal 2015) to iteratively update a population to im- prove its combined quality and diversity. This objective also has a remarkably simple interpretation. Essentially, it is composed of two summation terms:
A quality term Σnn which encourages all solutions to have higher qualities.
A diversity term that acts as pairwise repulsion, penalizing solutions that are behaviorally close.
The diversity term penalizes behavioral similarity through sum over solution pairs. Each pair's penalty is the product of two components: the geometric mean of their qualities, / Vfff, and an exponential term that increases with their proximity. The combination of these two terms heavily penalizes high-quality solutions that are behaviorally similar. Notably, the geometric mean term

---

Published as a conference paper at ICLR 2026
Algorithm Soft QD Using Approximated Diversity (SQUAD)
Require: Optimizer O, learning rate 7) population size N, batch size M, neighbors K, epochs
Tmxx kernel bandwidth 7².
Require: Differentiable evaluation function Eval returning quality and descriptor b.
1: Initialize:
2: Population 0 == J
3: Evaluations (F, B) < Ev)l(
4: Optimizer state S + O.init(0)
5: for = 1 to Tax do
6: for each batch of indices S (I,..., N] do
7: For each i E I, find neighbor indices N; + K K-Neares--Neighboss/h B) Compute objective function for batch:
8:
(0) EiEI Si EIEIJEN, v VTsy exp (- [b; bbbb,,ll"))
9:
G (x)
10:
Update parameters: (0z, Sx) O.update Oz, Gz, 11) Update evaluations for the batch: (Fz, BT) + (v))
11:
12:
end for
13:
end for
14: return Final population
discounts the similarity penalty for low-quality solutions, allowing them to first prioritize quality optimization before gradually shifting optimize for behavioral diversity qualities increase.
The presence of pairwise interactions in the diversity term is a direct consequence of the second- order approximation used to derive the bound in Theorem Although higher-order interactions between triplets and larger groups of solutions also exist in the full integral, our approximation considers only the most significant pairwise terms. As our analysis in Appendix A.. shows, the component of approximation error originating from ignoring the higher-order interactions decreases as the solutions spread out. Therefore, the solutions spread during optimization, ignoring higher-order interactions becomes less detrimental to the accuracy of the approximation.
Building on this objective, we next describe two additional components needed for an efficient algorithm.
Efficient computation with batches and nearest neighbors. A naive implementation of the SQUAD objective in Eq. requires computing and applying O(N2) pairwise repulsions, which computationally prohibitive for large populations. To overcome this, we only compute the repul- sion for each solution from its k-nearest neighbors in the behavior space. This reduces the number of gradient updates that needs to be calculated at each iteration to (())). The omission of far- ther solutions from the caleulation is justified by the exponential decay of the repulsive force with distance, which quickly makes the contribution from distant solutions negligible. Additionally, to manage the memory and computational cost per gradient step, we update the population in mini- batches rather than all at once. In Appendix C.2 and C. we report the results of ablation studies on the choice of i and the batch size, which show the robustness of SQUAD to these hyperparameters.
Handling bounded behavior spaces. Our derivation of the SQUAD objective assumes bounded behavior space B = Rd_ However, many problems, including our experiments, have in- trinsically bounded behavior descriptors. While extending the derivation bounded domains is possible, it leads to a significantly more complex and potentially less stable final objective. We instead adopt simpler approach: we transform the bounded space into an unbounded one using the logit function. Specifically, we map each point in the bounded behavior space b E [0, 1]d to b' = log b-b E _d. We found this choice to be critical for the success of the algorithm, and ablated it in Appendix C.3 to confirm its importance.
b
Putting these pieces together, Algorithm 1 presents complete pseudocode for SQUAD.
5

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
Our experiments are designed to comprehensively evaluate SQUAD and answer three key ques- tions First, how does SQUAD scale with the dimensionality of the behavior space? Second, how does it navigate the fundamental trade-off between solution quality and population diversity? Lastly, how does SQUAD compare with state-of-thee tion problems? To answer these questions, evaluate SQUAD and several baselines on three benchmark domains, each selected to probe one of these specific aspects.
methods on complex, high- high-dimensional optimiza-
5.1 EXPERIMENTAL SETUP
Benchmark domains We evaluate different facets of QD optimization on three benchmarks, de- scribed below and in detail in Appendix [.I]
Linear Projection (LP): Following Fontaine et al. (2020), an algorithm must maximize an objective while maintaining diversity in a d-dimensional behavior space defined by a linear projection of the
solution vector. We use the multi-moda Rastrigin function (Rastrigin 1974). making this a simple yet challenging testbed for analyzing scalability with respect to d. Image Composition (IC): Inspired by computational creativity tasks Tian & Hap 2022 Ibarrola & Grace| 2023), this benchmark adjusts the parameters of set of circles (position, radius, color, transparency) to reconstruct a target image. A solution's 's quality is its similarity to the target image, while a 5-d behavior space encodes properties such color harmony. The moderately sized behav- ior space and challenging optimization make it ideal for analyzing the quality-diversity trade-off. Latent Space Illumination (LSI):: Based on /Fotaiiee & Nikolaidis (202T). algorithms search the
latent space of StyleGAN2 (Karras et al. 2020) to generate images matching a target text prompt. Following Fontaine & Nikolaidis (2023), we target images of "Tom Cruise" while diversifying in age and hair length. We also propose a harder version with a 7-d behavior space in which we target images of "a detective from a noir film". Both objective and behavior descriptors use CLIP (Radford iet al. 2021) embeddings to evaluate the similarity between generated images and given texts. This serves our most difficult domain for testing QD algorithms.
Baselines We compare SQUAD against state-of-the-an art algorithms for high high-dimensional and dif- ferentiable QD, using the open source pyribs (Tjanaka et al. 2023b) implementations: CMA- MAEGA (Fontaine & Nikolaidis 2023), CMA-MEGA (Fontaine & Nikolaidis 2021), and Sep- CMA-MAE (Tjanaka et al.) 2023a). We also include Gradient-Assisted MAP-Eliter (GA-ME), which adapts the pollic-gradient-ee based algorithm PGA-ME NNlssonn & Cully (202T) to the DQD setting by using direct gradients from the objective function (GA-ME is also similar to the OG- MAP-Elites (line) baseline in Fontaine & Nikolaidis (202T)). To ensure a fair comparison in high- dimensional behavior spaces, all baselines use Centroidal Voronoi Tesselation (CVT) to discretize the behavior space into a fixed-size archive Vassiliades et al. (2018). In addition to these methods, we also include DNS (Bahlous- Boolo al. 2025) which is modern variant of novelty search (Lehman & Stanley 2011) used for QD optimization in more complex domains. We also include an improved variant of it that uses gradient-based updates (similar to GA-ME) to complement its regular muta- tion operator. We denote this variant as DNS-G. Additional details about the hyperparameters of the algorithms are presented in Appendix D.3
Evaluation metrics To provide a thorough analysis of the trade-off's offered by each algorithm, we use several metrics measure the quality and diversity of the generated populations.
For quantifying diversity, we primarily use the Vendi Score (VS) (Friedman & Dieng 2023), which quantifies the effective number of distinct clusters in population. We supplement this with Cover- age which is the percentage of occupied cells in a fixed CVT archive. To understand the quality of the generated populations, we report the Maximum Quality to assess pure optimization performance and the Mean Quality to evaluate the overall quality of solutions across the entire population. Fi- nally, to compare the overall performances, we use QD Score (Pugh et al.) 2016 which measures the sum of qualities in a fixed CVT archive and weighty- Vendi Score (QVS) (Nguyen & Dieng 2024) which extends Vendi Score to account for the quality of a population as well. A full definition of all of the metrics and further discussion is provided in Appendix D.2
'The source code for all of our experiments is available here

---

Published as a conference paper at ICLR 2026
5.2 SCALABILITY TO HIGH-DIMENSIONAL BEHAVIOR SPACES
To evaluate SQUAD's scalability to high- nigh-dimensional behavior spaces, we conducted three sets of experiments on the LP benchmark, using 4, 8, and 16-dimensional behavior spaces. As shown in Figure methods that leverage gradient information from descriptors (SQUAD, CMA-M(AJEGA) perform much better than methods that do not (Sep-CMA-MAE, GA-ME, DNS) which highlights the difficulty of exploring high-dimensional behavior spaces without gradients. To assess statistical significance of the results in Figure 3 we evaluated algorithm performance per task and metric using Bonferroni-correet
Mann-V Hoann-hhtneyy U tests to compare the best algorithm against the others. All differences were significant (p < 0.001) except for CMA-MAEGA on medium (d 8) and hard (d 16) tasks for QD Scores, where was not significantly different from the top-pert nop-performing algorithms; in the hard domain, this is primarily due the high variance of CMA-MAEGA.
Fruskal-Wallle tests (all p 0.001) followed by
While CMA-MEGA and CMA-MAEGA have a slight edge in the 4-dimensional behavior space, SQUAD closes this gap and noticeably outperforms them in the more challenging versions of the task. We attribute the initial success of CMA-M(AJEGA to their large archives (10* cells), which are dense enough effectively cover the (ow-dimensional space. However, as the dimensionality increases, the density of their archives drops exponentially, making the feedback less informative. This limitation is a key reason for their performance decline in nigher-dimensional spaces. SQUAD, on the other hand, does not discretize the behavior space which enables it to maintain strong per- formance as dimensionality increases. Furthermore, SQUAD demonstrates greater stability across all three tasks, with the lowest variance across different evaluations. More detailed results of these experiments are presented in Appendix
Quality Weighted Vendi Score
QD Score
[x10')
[x100)
25
20
15
10
medium
hard
$)
Algorithm
SQUAD
CMA-MAEGA
CMA-MEGA
DNS
DNIS-G
Sep- Sep-CMA-MAE
Figure 3: QVS (left) and QD Score (right) on LP tasks with increasing behavior descriptor dimensionality (4, 8, 16). All results are averaged over 10 runs, with error bars depicting the standard errors. SQUAD's performance relative to the baselines improves with task complexity, with it outperforming all other methods on the most challenging 16-d task for both metrics.
5.3 ANALYSIS OF THE QUALITY-DIVERRE TRADE-OFFS
Obtaining diversity often comes the expense of quality. In this section, we shed some light on the quality-diversity tradeoff that SQUAD offers in comparison to baseline algorithms using the IC domain, which provides a realistic optimization challenge with moderately sized, 5-d behavior space. We also examined the bandwidth parameter, 72, and showed that it acts a as a knob, allowing SQUAD to effectively trade off quality for diversity.
As the results in Table | Indicatee SQUAD outperforms the baselines on most metrics. The average quality of the solutions found by SQUAD is significantly higher than those of the baselines. Further- more, the high quality of the best solution in SQUAD's population shows that it is highly capable in pure quality optimization. On diversity metrics, SQUAD maintains its noticeable lead on Vendi Score but falls short of CMA-MAEGA on Coverage by small margin. The discrepancy between the two diversity metrics can be explained by the fact that Vendi Score takes the shape of the archive into account. Therefore, while the larger number of solutions found by CMA-MAEGA helps it
7

---

Published as a conference paper at ICLR 2026
Table I: Performance the IC domain. Comparing SQUAD (7² = 1) with baselines in terms of the quality (Best Objective, Mean Objective) and diversity (Vendi Score, Coverage). Results are mean standard error averaged over 10 runs, with the best score for each metric shown bold.
Quality
Diversity
Algorithm
Mean Objective
Max Objective
Vendi Score
Coverage
SQUAD
83.37 0.02
93.58 + 0.10
5.49 + 0.00
5.68 0.06
CMA-MAEGA
74.83 0.20
88.79 + 0.92
3.93 + 0.03
5.85 0.05
CMA-MEGA
75.98 + 0.26
86.18 + 1.58
3.25 + 0.24
4.54 = 0.49
DNS
71.30 0.15
74.54 = 0.18
+
1.52 0.02
DNS-G
74.49 + 0.03
76.48 = 0.11
1.67
1.49 = 0.03
Sep-CMA-MAE
72.15 0.21
74.57 + 0.04
1.32
0.46 0.02
GA-ME
73.44 + 0.41
74.53 + 0.45
1.14 0.04
0.19 + 0.03
cover the behavior space more densely, this coverage is concentrated in a smaller region, leading its lower Vendi Score. This is further supported by the archive visualizations in Appendix
Having compared SQUAD with the base- lines, we examined how to control its quality- diversity tradeoff via the bandwidth parame- ter, 72 As Equation 5 Juggestt, increasing 22 boosts the contribution of the diversity term by increasing its effective range and intensity. Empirically, we confirmed this by evaluating SQUAD with different values of y² ranging from 10 to 50 in the IC domain. As Figure 4 shows, increasing ?2 indeed improves the di- versity of the solutions, albeit at the price of their quality. These results show how SQUAD lets the users trade off quality for diversity (and vice versa) by changing the value of single hyperparameter.
Qualtty (Mean Obpecive)
Uiserity Store)
(log morl)) Quality
Divenity
Figure 4: Controlling the quality-diversity trade- off with 72. The plot shows how varying 7² im- pacts solution quality, measured by the mean objec- tive (blue line), and diversity, measured by the Vendi Score (dashed red line).
5.4 PERFORMANCE ON CHALLENGING DQD PROBLEMS
In our last set of experiments, we compared SQUAD In line with prior work (Fontaine & Nikolaidis 2023) we searched the latent space of StyleGAN2 for latents that would generate images of Tom Cruise and are diverse with respect to age and hair length. Additionally, we set up more challenging version of the LSI benchmark with 7-d behavior space. In this more difficult task the goal is to generate "photos of a detective from noir film" and images have be diverse with respect to different attributes including facial expression, pose, and hair color. Additional details about both versions of LSII is presented in Appendix 1.1..3
with baselines on the challenging LSI domain.
Table 2: Performance in the Latent Space Illumination (LSI) domain. Results are averaged over runs and reported as mean + standard error, with the best scores highlighted bold. Algorithms that failed to achieve positive mean objective (hence, have a zero QVS) are shown with
LSI
LSI (Hard)
Algorithm
QD Score 10
QVS
QD Score (x109)
QVS
SQUAD
13.41 + 0.19
177.0 2.8
2.55 + 0.08
151.3 + 0.1
CMA-MAEGA
6.82+ 0.10
121.6
0.39 + 0.07
99.3 + 1.0
CMA-MEGA
8.70 0.07
140.1
0.27 0.05
92.8 1.5
DNS
-9.31
0.0
-11.27 1.46
0.0
DNS-G
8.53
0.0
-6.81 +: 0.47
0.0
Sep-CMA-MAE
-0.59
0.0
0.02 0.04
0.0
GA-ME
14.90
0.0
0.08 0.00
0.0
8

---

Published as a conference paper at ICLR 2026
The performance of different algorithms compared in terms of QD Score and Quality-weighted Vendi Score in Table 2 In both LSI tasks, SQUAD outperforms the baselines by a significant margin, showcasing its capability in challenging domains. A closer inspection of the data (available in Appendix [.] reveals that SQUAD is particularly adept in exploring the behavior space. Thus, despite performing similar to the baselines in terms of mean quality, SQUAD's ability to maintain a better coverage of the behavior space differentiates it from the baselines. Similar to our prior results, Sep-CMA-MAE and GA-ME perform poorly on both versions of the LSI task. Here, we attribute their poor performance to the high-dimensional and non-linear nature of the optimization landscape, which makes it difficult to navigate the behavior space. Interestingly, GA-ME, despite using gradient ascent updates, is unable to effectively escape local optima, highlighting the important role of modern optimizers in more challenging domains.
6 RELATED WORK
Quality-diversity (QD) optimization has recently become a topic of interest in machine learning, following the introduction of the NSLC (Lehman & Stanley 2011) and MAP-Elites (Cully al.) 2013) algorithms. MAP-Elites, as a canonical QD algorithm, uses random mutations to find solu- tions that occupy different behavioral niches in tessellated behavior space. This general recipe is improved by methods that have proposed better discretization schemes (Vassiliades et al., 2018 Mouret 2023), incorporated modern evolutionary optimizers (Fontaine et [0020] Tjanaka al., 2023a Batra et al. 2024; Choi & Togelius (021), and improved the search by using gradient- aware mutations (Nilsson & Cully Pieroo
Pierrot et al. 2022 Faldor et al., 2023) and crossovers (Ing-) varsson et al., [023). It is worth noting that these advancements rely on discrete archives, which are known to be sensitive to the archive resolution (Fontaine Nikolaidis, 2023). To the best of our knowledge, the only other work that proposes discretization-frii free formulation of QD is the continuous QD Score of Kent et al. (0022). Continuous QD Score is similar to SoftQD Score in that both compute an integral over the behavior space. However, continuous QD Score employs a non-smooth kernel, which makes it difficult to approximate analytically and restricts estimation to Monte Carlo sampling methods. Consequently, it is used exclusively as an evaluation metric rather than as an optimization objective. Another major contribution was the formulation of differ- entiable QD (Fontaine & Nikolaidis (02T) where the quality and behavior descriptor functions assumed to be differentiable. This, along with the introduction of gradient arborescence algorithms (Fontaine & Nikolaidis 2203) paved the way for QD algorithms to tackle large scale optimization problems (Y Yu et al. 2025] Wan et al.) 2025). Our work is a continuation of this trend that frames the whole QD problem as a unified optimization problem, enabling seamless integration with modern gradient-based optimizers.
The pairwise repulsive term in SQUAD's objective is reminiscent of the kernel-based repulsion used in particle variational inference methods such as Stein Variational Gradient Descent (SVGD) (Liu Wang 2016), where particles are simultaneously attracted toward regions of high target density and repelled from one another to prevent collapse. Unlike QD, these methods aim to approximate a probability distribution rather than optimize quality while diversifying in a behavior space, which does not exist in SVGD or its reinforcement learning variant, SVPG (Liu et al., 2017). Nonethe- less, recent SVGD advances for high-dimensional inference, including message passing (Zhuo et al.] 2018), matrix-valued kernels (Wang al., 2019), and Newton-like updates (Chen al., 2019), offer practical ideas for stabilizing and scaling repulsive forces that could be adapted to SQUAD's 's ob- jective. Lastly, a related example outside variational inference is DOMINO (Zahavy et al. 2023). which enforces diversity in reinforcement learning via repulsive forces and a Lagrangian formula- tion, showing the broader utility of such mechanisms for balancing quality with diversity.
CONCLUSION
In this work, we introduced Soft QD as new formulation of QD optimization that does not require discretizing the behavior space. Building on it, we proposed SQUAD, a novel QD algorithm tailored for differentiable domains. Our experiments across multiple benchmarks demonstrated that SQUAD achieves competitive performance with state-of-the art methods, and exhibits promising scalability to higher-dimensions behavior spaces. In addition, we highlighted how it enables a convenient off between quality and diversity, offering fresh perspective on the design of QD algorithms.

---

Published as a conference paper at ICLR 2026
Despite the encouraging results, several limitations remain. First, the current formulation of SQUAD assumes differentiable objectives and behavior descriptors, which may be costly to obtain or un- available in certain domains. Extending Soft QD to reinforcement learning settings with estimated gradients, or to fully non-differentiable domains via evolutionary strategies, represents a promising direction, where issues such navigating deceptive behavior landscapes may pose unique chal- lenges, Second, our analysis in Section 5.3 showed the critical role of the kernel bandwidth, y in shaping the quality-diversity trade-off of SQUAD. Future work could investigate adaptive or per- solution schedules for y, for example by adjusting it based on the distribution of solutions or by annealing it during training. Moreover, while SQUAD's second-order approximation of the Soft QD Score offers tractability, it also discards higher-order interactions. Alternatives such sparsi- fication the interactions (Spielman & Srivastava 2011), message-passing (Zhuo al. 2018). or Monte Carlo approaches may provide richer modeling of interactions, though potentially at higher computational cost. Lastly, while we use a logit transformation to handle bounded behavior spaces, exploring alternative transformations could be avenue for future work to further improve per- formance. Taken together, we introduce Soft QD as powerful conceptual tool and SQUAD as a practical algorithm, highlighting new path for quality-diversiee in complex, differentiable domains. We hope this work inspires the community to build upon this foundation, realizing the full promise of scalable and general purpose QD.
8 ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
We would like thank Varun Bhatt, Sophie Hsu, Bryon Tjanaka, and Shihan Zhao for their feedback on a preliminary version of this work. This work has been partially supported by the NSF CAREER #2145077, NSF NRI #2024949 and the DARPA EMHAT project.
REFERENCES
Ryan Mahlous-Boldi, Maxence Faldor, Luca Grillotti, Hannah Janmohamed, Lisa Coiffard, Lee Spector, and Antoine Cully, Dominated novelty search: Rethinking local competition in quality- diversity. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2025, NH Malaga Hotel, Malaga, Spain, July 14-18, 2025, pp. 104-112. ACM, 2025. URL wtt.e/...//////1111111 0ee.111113322555 3726310
Sumeet Batra, Bryon Tjanaka, Matthew Christopher Fontaine, Aleksei Petrenko, Stefanos Niko- laidis, and Gaurav S. Sukhatme. Proximal policy gradient arborescence for quality diver- sity reinforcement learning. In The Twelfth International Conference on Learning Representa- lions, ICLR 2024, Vienna, Austria, 2024. httpl:tps////m
hnttps://oppereviiiii //tp////peneiiieeee/e/ee net wtt.m/orrmmoooooooo
FPKTfhydm
Varun Bhatt, Bryon Tjanaka, Matthew C. Fontaine, and Stefanos Nikolaidis. Deep surrogate as- sisted generation of environments. In Advances in Neural Information Processing Systems 35; Neural Information Processing Systems 2022, NeurIPS 2022, New Or- leans. LA, USA, 2022. URL httt::://paaeeeet ctpps..eppppp..r.....c..pppppp.. http:a/eleeeeeeeeeeeeeeeeaaeeeee
Annual Conference
hash/ 1aaaT4/49911111133eeea0000333333
html
Christopher M. Bishop. Pattern recognition and machine learning, 5th Edition. Information science and statistics. Springer, 2007. URL https:///// : www. wor www.wwow......ooooooooo. wttp://occaa.toooo//////////0000
Raphaël Boige, Guillaume Richard, Jérémie Donå, Thomas Pierrot, and Antoine Cully. Gradient- informed quality diversity for the illumination of discrete spaces. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2023, Lisbon, Poruuaal, pp. 119-128. ACM, 2023. URL httts::///..tt. /ttp::://dol..rr///. http://10..1153351111111
3590407
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Ne Milne, and Qiao Zhang. JAX: composable transformations Python+NumPy programs, 2018. URL http:
7 weit..u..ccm com waccco/ojaaaaa http//xxxmmmm//m
10

---

Published as a conference paper at ICLR 2026
Herbie Bradley, Andrew Dai, Hannah Benita Teufel, Jenny Zhang, Koen Oostermeijer, Marco Bellagente, Jeff Clune, Kenneth O. Stanley, Grégory Schott, and Joel Lehman. Quality- diversity through AI feedback. In The Twelfth International Conference Learning Repre- sentations, ICLR. OpenReview.ner, 2024. URL htl.https:// http:///penneeeeeeeeeeeee ww..et......e.......e.. forum?
@wokkCrGYY
Konstantinos Chatzilygeroudis, Antoine Cully, Vassilis Vassiliades, and Jean-Baptiste Mouret. Quality-diversity optimization: novel branch of stochastic optimization. In Black box opti- mization, machine learning, and no-free lunch theorems. pp. 109-135. Springer, 2021.
Peng Chen, Keyi Wu, Joshua Chen, Tom O'Leary-Roseberyy and Omar Ghattas. Projected stein variational newton: fast and scalable bayesian inference method in high dimen- sions. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, Vancouver, BC, Canada, pp. 15104-15113, 2019. URL https: nncrriippp http./eppppcccpppe///0001100
neurips laa1//essdSsedeessrrddd5222222b Absstaaa Abstract. html
Tae Jong Choi and Julian Togelius. Self-referential quality diversity through differential map-elites. In GECCO '21: Genetic and Evolutionary Computation Conference, Lille, France, pp. 502-509. ACM, 2021.
145114445443344 3459383
Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. Robots that can adapt like animals. Nat., 52105553):503-507, 2015. URL http::::/ddi..rggg
1442)
Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, and Joel Lehman. Quality diversity through hu- man feedback: Towards open-ended diversity-driven optimization. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria. OpenReview.net, 2024. URL
net htre//ooummaa 2id= 11wz22bbuubB
Maxence Faldor, Félix Chalumeau, Manon Flageat, and Antoine Cully. Map-elites with descriptor- conditioned gradients and archive distillation into single policy. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2023, Lisbon, Portugal, pp. 138-146, ACM, 2023. URL /ttp:ttt////otorr///// http:////oo..orrr11.111//////53. 11e...1111/31311111 /tt/S/3333313333355003 3590503
Matthew C. Fontaine and Stefanos Nikolaidis. Differentiable quality diversity. Advances in Neu- ral Information Processing Systems 34: Anmual Conference on Neural Information Processing Systems, NeurlPS, pp. 10040 10052, 2021. URL
wwwrsss...e...pp...e.. cc/ http:://h.aaaaaa22......////0200
Matthew C. Fontaine and Stefanos Nikolaidis. Evaluating human-robot interaction algorithms in shared autonomy via quality diversity scenario generation. ACM Trans. Hum. Robot Interact., 11 (3):55:1-2::30, 2022.
Matthew C. Fontaine and Stefanos Nikolaidis. Covariance matrix adaptation map-annealing. Pro- ceedings the Genetic and Evolutionary Computation Conference, GECCO 2023, Lisbon, Portu- gal, pp. 456-465. ACM, 2023. URL http::://oo..org/0.. 11tp:r//0...11111331113131313331 3590388
Matthew C. Fontaine, Scott Lee, Lisa B. Soros, Fernando de Mesentier Silva, Julian Togelius, and Amy K. Hoover. Mapping hearthstone deck spaces through map-elites with sliding boundaries. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, pp. 161-169. ACM, 2019. URL hRl.hhtts::/ddd wtt.:ii.rgr/.......1 1ttp///...1//33/17777777
33217 794
Matthew C. Fontaine, Julian Togelius, Stefanos Nikolaidis, and Amy K. Hoover. Covariance matrix adaptation for the rapid illumination of behavior space. In GECCO '20: Genetic and Evolutionary Computation Conference, Cancún Mexico, pp. 94-102. ACM, 2020. URL httpps://ooi.
exq/10.11455 00544///3773.0. 3390222
Matthew C. Fontaine, Ruilin Liu, Ahmed Khalifa, Jignesh Modi, Julian Togelius, Amy K. Hoover, and Stefanos Nikolaidis. llluiiaaiing mario scenes in the latent space of a generative adversar- ial network. In Thirty-Fifih AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third
11

---

Published as a conference paper at ICLR 2026
Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Sym- posium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, pp. 5922- 5930. AAAI Press, 2021. URLhttps: /twi/iiiooooooo wtt./oogg//...//.. aaai. v3517.16740
Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. Trans. Mach. Learn. Res., 2023. URL BtRphtt:::/ htttss://openrevieww wwwpooppperr.e...ooe/.e www.rrewwwwwweeeeeeeeeeeeeeeee www.torumforr...o ftr/morrrmmm g970HbQyk1
Adam Gaier, Alexander Asteroth, and Jean- can--Bptiste Mouret. Data-efficient design exploration through surrogate-assisted illumination. Evol. Comput., 26(3), 2018. URL https://doi.
org/10. FTg/10..162/evcco. a_00231
Luca Grillotti and Antoine Cully. Unsupervised behaviour discovery with quality optimi- 2106. 05648
sation. CoRR, abs/2106.00664,, 2021.
Alexander Hagg, Adam Gaier, Dominik Wilde, Alexander Asteroth, Holger Foysi, and Dirk Reith. Full domain analysis in fluid dynamics. CoRR, abs/2505.22275. 2025. URL /ttht::t////ee
org/10. 48550/ arxiv. 2505 22275
Saeed Hedayatian and Stefanos Nikolaidis. Autoqd: Automatic discovery diverse behaviors with quality-diversity optimization. CoRR, abs/2506.05634, 2025. URL https: http::::/ddoi.eo/ orq/10.
48550/arXiv. 25633
Johann Huber, François Hélénon, Miranda Coninx, Faiz Ben Amar, and Stéphane Doncieux. Quality diversity under sparse reward and sparse interaction: Application to grasping in robotics. CoRR, abs/2308.05483, 2023. URL https: www.:............ http:/1110..55//a1111/ 2308. 05483
Francisco Ibarrola and Kazjon Grace. Differentiable quality-diversity for co-creative sketch- ing AI. In Proceedings of the 14th International Conference on Computational Creativ- ity, ICCC 2023, Ontario, Canada, pp. 84-88. Association for Computational Creativity (ACC), 2023. URL httpt:://///... htttps::/ccompuattttt ona creati wwt.y.net/cccc/////eeeeee (CCC-2023 paper 101.pdf
Gardar Ingvarsson, Mikayel Samvelyan, Bryan Lim, Manon Flageat, Antoine Cully, and Tim Rocktäschel. Mix-me: Quality-diversity for multi-agent learning. CoRR, Abs/2311.01122 2023.
URL https::/doo wrgg/00..00.. 10. 48550/arxi 2311 01829
Hannah Janmohamed, Marta Wolinska, Shikha Surana, Thomas Pierrot, Aron Walsh, and Antoine Cully. Multi-objective quality-diversity for crystal structure prediction. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2024, Melbourne, VIC, Australia. ACM, 2024. URL https:://ds/. wrw.:o...11... 11411163555333 1ttpe//.....1133335..2255333 3654048
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In 2020 IEEF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020. Seattle, WA, USA, pp. 8107-8116. Computer Vision Foundation / IEEE, 2020. URL https: /oppnaccesss... thecvf. wwwm///ooeett..ooo Seipeent__c_____eeee 2020 htt.0//tmm//aaaaaaa Aaaarriisnaaaainnnggaaa and Improving hae___tte_________eeeeee Saapeee_aaattt___seeeeeeee Sft__offfSSyeeCAN CVPR 2020 www.ppaappppaaa
Paul Kent, Jürgen Branke, Adam Gaier, and Jean-Baptiste Mouret. A discretization-Free metric for assessing quality diversity algorithms. In GECCO '22: Genetic and Evolutionary Computation Conference, Companion Volume, Boston, Massachuseuss, USA, pp. 2131-2135. ACM, 2022. URL
1145/ https: /tttps://doi.oor oroo..org/0,. wtt.r///....1..1... 1t.....111411310000100 33203042033333 001144433233443300000. 3534018
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Truck Proceedings, 2015. http:htttp///iiiieea
abw aawaaass1111 /122. 6980
Joel Lehman and Kenneth O. Stanley. Evolving a diversity of virtual creatures through novelty search and local competition. In 13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, pp. 21 211-218. ACM, 2011. URL https: //doi.
oorg/10. 01151/10.111//20115500 20000011666101000111111111111110 606
12

---

Published as a conference paper at ICLR 2026
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Advances in Neural Information Processing Systems 29: An- mual Conference Neural Information Processing Systems 2016. Barcelona, Spain, pp. 2370-2378, 2016. URL http:/t.ss//prcccciingg..eooe b3ba8f bee h15.
Yang Liu, Prajit Ramachandran, Qiang Liu, and Jian Peng. Stein variational policy gradient. In Pro- ceedings of the Thirty-Third Conference on Uncertainty Artificial Intelligence, UAI 2017, Syd- ney, Australia. AUAI Press, 2017. URL http:://aue http//aaw..ooooaaaaa/////rrrr/ae
papers/239.g pdf
Jon McCormack and Camilo Cruz Gambardella. Quality-diversity for aesthetic evolution. In Artificial Intelligence in Music, Sound, Art and Design 11th International Conference, Evo- MUSART 2022, Held as Part of EvoStar 2022, Madrid, Spain, April 20-22, 2022, Proceed- ings, volume 13221 of Lecture Notes in Computer Science, pp. 369-384. Springer, 2022. URL /tt.////....1100011111111111
Jean-Baptiste Mouret. Fast generation of centroids for map-elites. In Companion Proceedings of the Conference on Genetic and Evolutionary Computation, GECCO 2023, Companion Volume, Lis- bon, Portugal, pp. 155-158. ACM, 2023. URL https:///o// /tt.lo.oorg111..55355555
3590726
George L. Nemhauser, Laurence A. Wolsey, and Marshall L. Fisher. An analysis of approxima- tions for maximizing submodular set functions I. Math. Program., 14(1):265-294, 1978. URL,
https:://ol wrw/oo..o.....o..o..ooooo.......
Quan Nguyen and Adji Bousso Dieng.
vendi scores and their application di- verse experimental design. In Forty-first International Conference Machine Learning, ICML. OpenReview.net, 2024. URL BRLL/tttss///peeeeeeett wtwp://wwwww.eeetoommmeeeeeeee oog.
Olle Nilsson and Antoine Cully. Policy gradient assisted map-elites. In GECCO '21: Genetic and Evolutionary Computation Conference, Lille, France, pp. 866-875. ACM, 2021. URL httpst
//doi. oww.horrg11... 10.....11553449990 3459304
Giuseppe Paolo, Alban Laflaquière, Alexandre Coninx, and Stéphane Doncieux. Unsupervised learning and exploration of reachable outcome space. In 2020 IEEE International Conference on Robotics and Automation, ICRA, pp. 2375- IEEE, 2020. /lEtps://doi.czoe
1108/ICRA40945. 2020. 9196819
Thomas Pierrot, Valentin Macé, Felix Chalumeau, Arthur Flajolet, Geoffrey Cideron, Karim Be- guir, Antoine Cully, Olivier Sigaud, and Nicolas Perrin-Gilbert. Diversity policy gradient for sample efficient quality-diversin optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1075-1083, 2022.
Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. Quality diversity: A new frontier for lutionary computation. Frontiers Robotics AI, 3:40, 2016. URL https::///ot http::::///oii.or/1//.
3389/frobt. 2016.0006100
Chao Qian, Ke Xue, and Ren-Jian Wang. Quality-diversit algorithms can provably be helpful for optimization. In Proceedings of the Thirty-Third International Joint Conference on Artifi- cial Intelligence, IJCAL, pp. 6994-7002. ijcai.org, 2024. URL https: www. Ajcai.orgg proceedings/2024/77
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8748 $748-8763, PMLR, 2021. URL http: httt:::/pprooeedings..
m1 r. www.press/v33//rddoooso html
Leonard Andreeviž Rastrigin. Systems of extremal control. Nauka, 1974.
13

---

Published as a conference paper at ICLR 2026
Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao. Minqi Jiang, Jack Parker-Holder, Jakob N. Fo- erster, Tim Rocktäschel, and Roberta Raileanu. Rainbow teaming: Open-ended gen- eration of diverse adversarial prompts. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, UIrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neu- ral Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, 2024.
URL
www..
wwtps//pppeee//22/////////// html
Konstantinos Sfikas, Antonios Liapis, and Georgios N. Yannakakis. Controllable exploration of a design space via interactive quality diversity. In Companion Proceedings of the Conference Genetic and Evolutionary Computation, GECCO 2023, Companion Volume, Lisbon, Portugal, pp. 167-170. ACM, 2023. URL.ttts:://b...rrg// http:////o........11111110 114.0...11435551
3590616
Daniel Spielman and Nikhil Srivastava. Graph sparsificaation by effeettie resistances. SIAM J. Comput., 40(6):1913-))))) 1926, 2011. [tl.pptpp///bb..or/00111113////0
Yingtao Tian and David Ha. Modern evolution strategies for creativity: Fitting concrete images and abstract concepts. In Artificial Intelligence in Music, Sound, Art and Design 11th International Conference, EvoMUSART 2022, Held as Part of EvoStar 2022, Madrid, Spain, Proceedings, volume 13221 of Lecture Notes in Computer Science, pp. 275 275-2752299 291. Springer, 2022. URL StRhttps: / //dbl.org//1.10077/
Bryon Tjanaka, Matthew C. Fontaine, David H. Lee, Aniruddha Kalkar, and Stefanos Nikolaidis. Training diverse nigh-dimensional controllers by scaling covariance matrix adaptation map- annealing. IEEE Robotics Autom. Lett., 8(10):6771-6778, 2023a. URL http:httts//// //do: org/
10. 10.1109/IRA DRR/RRR...... 2.01//RRR.22233....... 3313012
Bryon Tjanaka, Matthew C. Fontaine, David H. Lee, Yulun Zhang, Nivedit Reddy Balam, Nathaniel Dennler, Sujay S. Garlanka, Nikitas Dimitri Klapsis, and Stefanos Nikolaidis. pyribs: bare- bones python library for quality diversity optimization. In Proceedings of the Genetic and Evo- Iutionary Computation Conference, GECCO 2023, Lisbon, Portugal, pp. 220-229, ACM, 2023b. 3590374
URL httpt:tp///mm /tttt::t////o//t http:::://oolloog/11111555333330 0tt.:oo00//1111133381111111
Vassilis Vassiliades, Konstantinos I. Chatzilygeroudis, and Jean- Gann-Bapisste Mouret. Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Trans. Evol. Comput., 22(4):623-630, 2018. URL httpt: http:://di...o////.///
TEVC. 2TCC..207.275520
Zhenglin Wan, Xingrui Yu, David Mark Bossens, Yueming Lyu, Qing Guo, Flint Xiaofeng Fan, Yew-Soon Ong, and Ivor Tsang. Diversifying robot locomotion behaviors with extrinsic behav- ioral curiosity. In Forty-econnd International Conference on Machine Learning. 2025. URL
net http://pppeeeeeeeeeeeeeeeeeeeeee
Dilin Wang, Ziyang Tang, Chandrajit Bajaj, and Qiang Liu. Stein variational gradient descent with matuix-valued kernels. In Advances in Neural Information Processing Systems 32: An- nual Conference Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada, pp. 7834-7844, 2019. UR//atpps://oooeeeinnaa hat.aaaaaace11111111101000001110
http://www..............cceaaaaa 201 p0t///aaaSSccddaaaaee000000ccc23 A htm
Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengeai Liu, and Chao Qian. Quality-diversity red-teaming: Automated generation of high-quality and diverse attackers for large language models. CoRR, abs/2506.07121, 2025. URL httpps://do org/10.
48550/arXiv 2....55066000
07121 2506.07121
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600-61) 2004.
htlLhtts::// /tttps:://oo.o .oriilorgg/ie /ttpg//...11///// TTP 2003. 81 9861
14

---

Published as a conference paper at ICLR 2026
Xingrui Yu, Zhenglin Wan, David Mark Bossens, Yueming Lyu. Qing Guo, and Ivor W. Tsang. Imitation from diverse behaviors: Wasserstein quality diversity imitation learning with single- step archive exploration. In Proceedings of the 24th International Conference Autonomous Agents and Multiagent Systems, AAMAS 2025, Detroit, MI, USA, pp. 2271-2280. International URL https://
Foundation for Autonomous Agents and Multiagent Systems ACM, 2025.
dl.acm. / 00t/000//0.5555/3770000
3743867)
Tom Zahavy, Yannick Schroecker, Feryal M. P. Behbahani, Kate Baumli, Sebastian Flennerhag, Shaobo Hou, and Satinder Singh. Discovering policies with domino: Diversity optimization maintaining near optimality. In The Eleventh International Conference on Learning Represen- tations, ICLR 2023, Kigali, Rwand. OpenReview.net, 2023. URL https::/opeereviise met/forun id kkd2B33b8p
Marvin Zammit, Antonios Liapis, and Georgios N. Yannakakis. Map-elites with transverse sessment for multimodal problems in creative domains. In Artificial Intelligence in Music, Sound, Art and Design 13th International Conference, EvoMUSART 2024, Held as Part of EvoStar 2024, Aberystwyth, UK, April 3-5, 2024, Proceedings, volume 14633 of Lecture Notes in Computer Science, pp. 401-417. Springer, 2024. URL https:://eo..rr//111
978-3-031-56992-3 26
Yulun Zhang, Matthew C. Fontaine, Amy K. Hoover, and Stefanos Nikolaidis. Deep surrogate assisted map-elites for automated hearthstone deckbuilding. In GECCO '22; Genetic and Evolu- tionary Computation Conference, Boston, Massachusetts, USA, pp. 158-167. ACM, 2022. URL
3528711
Yulun Zhang, Matthew C. Fontaine, Varun Bhatt, Stefanos Nikolaidis, and Jiaoyang Li. Arbitrar- ily scalable environment generators via neural cellular automata (extended abstract). In Sev- enteenth International Symposium on Combinatorial Search, SOCS 2024, Kananaskis, Alberta, Canada, pp. 307-308. AAAI Press, 2024. URL https://doi http:////ol..rr/00.1000///////// 11711.31594
Sheng Zhong, Dmitry Berenson, and Nima Fazeli. CHSEL: producing diverse plausible pose esti- mates from contact and free space data. In Robotics: Science and Systems XIX, Daegu, Republic of Korea, July 10-14, 2023, 2023. URL htttpa://ao..orr//////o 11..111.1110077811 202.ESSS.....2.22 XXIX.
077
Jingwei Zhuo, Chang Liu, Jiaxin Shi, Jun Zhu, Ning Chen, and Bo Zhang. Message passing stein variational gradient descent. In Proceedings of the 35th International Conference on Machine Learning. ICML 2018, Stockholmsmässsan, Stockholm, Sweden, volume 80 of Proceedings of Machine Learning Research, pp. 6013-6022, PMLR, 2018 URL htt:::/procceiiinnn//
mlr. wezess/v80.. zhuo 8a hamml
15

---

Published as a conference paper at ICLR 2026
A LOWER BOUNDING SOFT QD SCORE
A.1 APPROXIMATE SOFT QD SCORE
Here, we will provide a formal proof of Theorem
Similar to the main paper, let θ be parameter space and 0 (....... .o.......))... be set of N solutions where 6, O. Furthermore, let f è > [0, oc) be an objective function that assigns higher values to better solutions, and let desc @ R be the behavior descriptor function that quantifies the behavior of each solution with d-dimensional vector. Throughout the proofs, we use b, and f; as shorthands for desc(O.) and f(0,), respectively. The Soft QD Score of the population f is defined
as
[11 max fn exp [EnEN
| [|bbbb,ll bnll3) 20
S = (b) db
db.
(6)
Theorem 2. Given a population 0 {0m] with qualities (1 and behavior descriptor
vectors {ba) x=1 in behavior space B R its Soft QD Score S()) can be approximated by
lower bound S (0)) defined as:
N
(-Ilb; - by 113
(()) = (2TO?)$
Σ fn
E
Vifg exp
(7)
So
IEKJEN
Proof. To make the notation simpler, let us define the contribution of a single solution 0n at a point as
gn (b) - fn exp (- Ilb [[lbb-b) b
(8)
2a²
The behavior value (b) the maximum these individual contributions:
Ug (b) = max 9n (b)
(9)
Using Maximum eikimuum-iiniumms identity, we can rewrite this as Eg (b) max gr. (b) 22
(10)
- g((b) Σ min (b)) +
Σ
min (b) gx),
(11)
Therefore,
S(0) / max In (b) db
(12)
Σ) f 9b(b) db
(13)
Σ
f min (g. (b). db
(14)
Σ Kijk
min (g.(b). g0 (b))).,bb).gg)) (b),gx(b))
db
(15)
(16)
To get tractable lower bound on this, we truncate the series and only consider the first two sums. This discards the higher-order interactions involving three more solutions. Intuitively, if the
solutions are well-spread in the behavior space (i.e., |bb; - b, 12 > 2x²) this approximation is acceptable (detailed error analysis is provided in the next section). The second order-approximation, S(0), therefore obtained by only keeping the individual and pairwise effects:
((0) 5(0)
Σ/
(i( db
Σ i23
min (a.(b), (b)) db
(17)
16

---

Published as a conference paper at ICLR 2026
Next, we will derive closed-form solution for each of the two terms.
The Quality Term (Individual Contributions): The integrals in the first sum are standard Gaussian integrals and represent the contribution of each solution to the Score, irrespective of other solutions. We can evaluate them analytically:
Jec 9:(b) db = J Seexp
b,ll²) (-1lb-bull"" (-!lb- (lbbb,ll") 2x²
db = f2(2m02)
(18)
Hence, the first sum in (() evaluates
N Σ
N
gi(b) db = (2TO²) } Rd
Σ fr
(19)
The Diversity Term (Pairwise Overlaps): The integrals in the second term are more difficult to compute and require yet another approximation. Note that for any pair of non-negative numbers x, we have min(x, y) < vay (geometric mean). Using this, we have the following upper bound approximation:
/ (b) db ~ f (b) db.
(20)
Now note that
(-1lb -220 1:(12) (-llb-bill
(- | - b
v (bb]gg(bb)=
v fi feep
f, eexp
(21)
20²
20
lb - 12 + ||b blub-bbl'""
b
-VJs exp
(22)
402
The exponent is quadratic in b and can be simplified as
|b b,l 2 + lb - byll = |lb| 2b- b
+ ||b,| 2| + |bb| 2b b, + ||bb||
(23)
b|2 - 2b (b, + b,) + |bb]| + ||b,||
(24)
b, + by
2
b;
bj
+ ||b:|| + ||b,1
(25)
2
2
b, + bj
2
1
2
b
+
(-11b,12 -bb, b, - + 2||b,
+
2 2|bb)
2
2
(26)
b, + bj
1
= 2
+
( 2b, b, + +1,b)
(27)
2
2
b, + by
2
1
2
+
b, b
(28)
2
2
Substituting this back into the exponential, we get
exp
(-
1 2 2 b
b:
bj
1
b: b,11²
1) exp (-
(_llb;
lb;
b
exp
||
b.
||2
2
802
20
(29)
The overlap integral is therefore
Je V9gg db Virfg exp
[_lb, [b; [bbbb,,1' b,
by+b ||b
1|2
exp
2 db
(30)
80²
20
Rd
where the integral is now just regular Gaussian integral centered at butb, The whole integral thus evaluates to
b b
v (i exp
(31)
17

---

Published as a conference paper at ICLR 2026
Putting these all together, establish closed form approximation (lower bound) of the archive score:
( -
b 8
S(0) (2TO²)$
Σ
E IENJJEN
Vfige
exp
(32)
A.2 ANALYSIS APPROXIMATION ERROR
The error between the true Soft QD Score of population and the second order approximation de-
rived above stems from two sources: (1) the truncation of higher-order interaction in the maximum- minimums equality (Equation 17, and (2) the replacing of pairwise minimums with their geometric means in the integrals (Equation 20 We will analyze each of these errors and discuss how we can control them.
Truncation Error: The truncation error for the second order approximation is
E1
|S(0) ((0)
(33)
/
max gn (b) db Σ (b)
-E icj
min (a:(b),;;b)) db
(34)
n
min (9. (b). 9b (b)) db
(35)
i<jck"
Lemma 1 (Bonferroni Inequalities for the Maximum). Let fæ1,..., IN be a set of non-negative real numbers and let Sm Elr==m min 24. The partial sums of the maximum-minimmmn miiniumis equal- ity, PK- ΣK Em=1 1-1)mm1 Sm. provide alternating bounds on the maximum value:
If K odd, PK max
KfKK is even, Px < max(F),.....
Proof. For bounding the approximation error, only need to use this lemma with K = 2, which we shall prove. Without loss of generality, assume that the numbers are sorted such that
T1 > 12 2 7 ZN O. The contribution æ to the sum Sn, is =x since Ta the minimum subset of m variables iff the other m elements are chosen from the elements smaller than it.
The partial sums Pk can thus be written as
K PK
Σ m=1
(-1)m-15
S..
Σ
Σ (-1) m-1
Fk.
(36)
m
k=1
The inner sum is partial sum binomial expansion. Therefore, using the fact that
Σ ( - (-1)m ((-)))))--))
m
(-)
(37)
==
(which follows from induction on m) we can see that the coefficient of oa in Px is
min(k 1,K-)
(-1y t
( ) m-1
k- min(k - 1, K .
C(k, K)
Σ
(38)
J=D
Let now consider the cases of K and K 3 individually.
18

---

Published as a conference paper at ICLR 2026
Case K = 2: Consider P2 = 24 - min( 2T). For all 2, the coefficient of 24 in Pa is 2 - k and is only present once, with coefficient 1 in Sj. Therefore, have
P2 x1
Σ (2 k)m
(39)
Since z& 's are all non-negative and 2 k non non-iositive, the sum is non-positive. Therefore,
P2 x ma..(11.........................
(40)
Case K = 3: Consider P3 = P2 + Σ Kijck 2i, vx). For all >> 3. the coefficient of 24 in P3 is (k=²) (k- (k=2)(h -3) Furthermore, æ1 is only present once, with coefficient 1 in S1 and 22 is present only twice, once with a +1 coefficient in Si and once with coefficient -1 S2. Therefore, have
(k: 2) 3) Zk.
P3 2 +
(41)
2
Since 2g 's are all non-negative and so are all the coefficients, the sum is non-negative. Therefore,
P3 2 x1 max
(42)
Lemma 2 (Bounding the Truncation Error). The truncation error E1 is bounded by the of the integrals of the hider minimums:
/ ())
E1
Σ i<jCk
gx db
(43)
Proof. The Bonferroni inequality for = 2 states that for any point b
max gn (b)
Σ (b) min gy (b)).
(44)
Integrating this pointwise inequality over the domain of b yields:
S(0) = f max In (b)db db >
/ Σ (b)
Σ (in db -(())
(45)
Therefore, S(() is indeed lower bound on S(0) and can write the error as e S(0) 5()). Similarly, the Bonferroni inequality for K = 3 states that for any point
max (b) & Σ (b) Σ min(g.(b).g)
Σ (b)) (46)
<<jck
Integrating over the domain of b yields:
S(0) / max (b) db
(47)
/ Σ (b) Σ
min(g.(b).,, +
Σ (b) (b))
db (48)
ikjck
5( + / Σ min(g db
(49)
i<jck
((0) - 5(()
min(g: (bn(g((),,. (b), 9x (b)) db
(50)
t<jck
-1 61 < Σ /
min(g:(b), (b),, (b) db
(51)
Kjjkk
19

---

Published as a conference paper at ICLR 2026
Given the above, the same technique that used before and replace minimum with geo- metric mean to compute the integral. We can see that
f min(o:(b). en (b)) db < (a:(bb- gz(b) (b)) db
(52)
= (fefffa) / exp
b b
+ b by 1| +
| by ||
db
60
(53)
The integral the product of three Gaussians. By completing the square, similar to the pairwise case, we can rewrite as a Gaussian centered around b.+b,+be and a constant term. The result would be
3
i (ar(b) gj(b) gh (b)) db (f.f, exp
b by + bb; b& 112 + ||;; b& 11
1862
(54)
So, we have
( 2 2 sejck
|b; - b + |bb
b 2 + |b;; 1
E1
exp
(55)
1802
From this, we conclude that if the solutions well separated (i.e., the mean squared distance of any triplet is significantly larger than 6o2) the error becomes negligible.
Pairwise Integral Approximation Error The pairwise integral approximation error is due to the estimation of pairwise minimums using geometric means. For every pair of solutions (i,j) this error
G. = (V.(()),(()) min (g: 9,))) db.
(56)
follows from the arithmeic-geometree inequality that for every non-negative numbers x and y, Vzy- min(x, y) & flz - y/. Integrating over b preserves this inequality, yielding
C.j < [ [b) 9j (b) db.
(57)
Note that the right hand side just the distance between the functions 9t. 9j. This distance can be further simplified by breaking down 19:-
(9--)) タソ using the triangle inequality. Without loss of generality,
[bbbb, we get
assume that fj < fi, then add and subtract f, exp
2r²
(bb 1f--ffyl fyl exp (-1lb
|lb bill
|b
b.
bb b 11
+fz
exp
exp
20t
20
20
(58)
Now, if we integrate both sides, the first term in the right hand side is just multiple of a Gaussian integral. So we would get
/ (l:(bb - 9 (b)) & (2TO?) (1s-
(p((b) (b) db)
(59)
where (b) the pdf of Gaussian centered at b; with covariance I²I. The remaining integral twice the Total Variation distance drv (p.((p.)))) Pý). Using Pinsker's inequality, can transform
into the KI. divergence between two Gaussians, which does have closed form solution, 2. Replacing the integral and combining everything, we arrive at the following bound for the error:
bb,b,,
G & (2 (1fi- min(Ji, f3) |bb, -byll
(60)
Finally, using the triangle inequality, we see that
Σ ( fyl min(fe, f,) |lb; by
Eg & 2
(61)
KOEK
Putting both these bounds together, a final application of triangle inequality shows that SS(0) - _(() < E1 + E2,
(22)
where and E2 are themselves bounded by Equation 55 and Equation 61 respectively.
20

---

Published as a conference paper at ICLR 2026
B PROPERTIES OF SOFT QD SCORE
Here, we will further discuss some properties of the Soft QD Score that make it a suitable measure of quality and diversity. Theorems 4 the main paper and show multiple properties of the Soft QD Score.
5 and O formalize and prove the statement of Theorem |
Theorems 3 a show that SoftQD Score monotonic with respect to population size and qualities of members of the population. This means that adding new solutions to population and improving the quality of existing ones will never decrease the SoftQD Score. We note that this a desirable, yet non-trivial property of measure of quality and diversity. For instance, neither mean objective value (as a measure of quality) nor the mean behavior distance (as a measure of diversity) of a population solution can decrease them.
monotonic with respect the population size; that is, adding
Theorem 5 shows that SoftQD Score is also submodular. Being submodular is particularly conve- nient as it implies the existence of efficient approximate algorithms for maximizing it under certain conditions. For instance, it is well known that maximizing a submodular function subject to cardi- nality constraint admits when we want to select fixed-size subset of large population that preserves as much of the Soft QD Score as possible (e.g., for evaluation or compression).
approximation algorithm (Nemhauser et al. 1978). This is valuable
Lastly, we establish a connection between the traditional QD Score and the limiting behavior of the Soft QD Score through Theorem
Theorem (Monotonicity with respect population size). Let (#1..... ON be a population and let N+1 be any new solution. If - (0N+1), then S()) < S(et).
Proof. Let fN+1 = f(0++1) and bN+1 = desc(ON+1). The behavior value for the new population 0+ can be written
l bN+zll")
ni+t (b) = max (ob()), IN-I exp
2 (b)...
(63)
20
Integrating both sides of the above inequality over the behavior space B yields the result:
/ uo (b) db >
/
S(0+)
(b(b) db S(0).
(64)
Theorem 4 (Monotonicity with respect to quality). Let = { ON4 be a population and O' 0 U (0', (otm) be another population that is identical to 0 except that the n-th solution On is replaced by 6',, such that dcsc(f) = dcsc(6',) = bn and (( = fa 2 fn = ((,)- Then, S()) S(0').
Proof. Let the behavior values for and O' be (bb and ug (b), respectively. Since, S > fno for [b-b,,l"
every we have f/ exp(-
[b 20
> fn exp(
|b lbbb..l Therefore, we can write 20
l b ))
vor (b) max
(b))), Sn exp
uo (b().
(65)
2n²
Integrating both sides of the above inequality yields the result:
s(0') / (b) db / vb(b) db S(O).
(66)
Theorem (Submodularity). Let 0 (f.....,00) be population. The Soft QD Score S defined on subsets of is submodular. That is, for any U C V C and any new solution 0' $ with quality f' f00) and behavior vector b' desc((").
S(U U (0')) S(U) > S(V U (0')) S(V).
(67)
21

---

Published as a conference paper at ICLR 2026
Proof. For brevity, let us denote U U {8') U and V U {0'} V Similar the argument in Theorem we have
1-bb-b11 -
Utr (b) - (b) = max vo S exexp
-vr(b)
(68)
2x²
max
0, exp
(- lb
- b112
vb(
(69)
20
Similarly, we also have
uy. vv((b) max (o. 0,
||b [ b111 (-10-11---
exp
er or(b))
(70)
20²
Since C V, for every b we have un (b) & uv (b). Therefore,
by (v) - vv (b) > uy - (b)
(71)
Integrating both sides yields the result
S(U") - S(U) > S(V) - S(V).
(72)
Theorem 6. Let 0 {01..... .8.] be a population of N solutions with corresponding quality
values fi....,NN fN and distinct behaviors bi...., by in Rd Let S(0) be the Soft QD Score defines as
bn (-Ib-ball)
)
S(0) =
max Rd IEnEN
fn Snexp
db
(73)
20²
In the limit as the kernel width a approaches zero, the scaled Soft QD Score converges to the sum of the qualities of all solutions in the population.
5(0)
N
lim 7-20 (2TT²)
Σ
fm
(74)
This limit is equivalent to the traditional QD Score calculated on a grid fine enough to isolate each solution into its own cell.
Proof. Let the scaled Soft QD Score be denoted by L(o):
S())
(-llb- lb byr 11 }
L(g) = (2 (2mO²)²
max
Jn exp
db
(75)
Rd IEnEN
20
Since the sum of set of non-negative number is at least as large as their maximum, we can replace the max in the integral with summation and write
(-1lb- |b
b )
L(a)= &
Σ
fn exp
db
(76)
(2
Rd
20
N
l b db
fn
exp
(77)
(2
gd
202
n=1
N
fm.
(78)
n=1
Note that Eq. 77 holds due to the lincarity of integration and Eq. 78 is true since the Gaussian integrals over the entire domain evaluate to (
Next, let r = min b, byll Following this definition. we can see that the open balls B, = (b: lb- b,il <r) centered at each behavior point with radius are disjoint.
22

---

Published as a conference paper at ICLR 2026
The integrand in S()) is non-negative, the integral over IR is greater than equal to the integral over the union of these disjoint balls:
| by 11 (-1lb-b:ll') ]
S(0)
max
fiexp
db
(79)
J,
2
|lb
by 112
Σ
max
Si
exp
db
(80)
20²
Ba
For any point b inside a specific ball Bn, the maximum value is always greater than or equal to the term for nz
(- 1-1b-bbll
b
)
|lb (-1lb--)) b 112
max
St exp
2 fn exp
(81)
2n
20²
Substituting this, gives lower bound for S(Ø)
(-Ibb-ball
|lb
b
S(0) > Σ fn
exp
db
(82)
2x²
B.
Now, we can analyze the limit of the scaled version of this lower bound:
fn
|
b
lim L(o)
>
lim
exp
db
(83)
o
a-00
(2TO?)
Bn
20²
|lb b 12
Σ fn
lim o (2TO²)
exp
db
(84)
20
B,,
Similar to the integral of Eq. 777, The term in the parenthesis here is the integral of the PDF of A ((,,, o²1). Since the ball B, is a fixed region containing the mean b, all of the probability mass concentrates inside B, and the whole limit will evaluate to one and we have
0. Therefore, the limit of the integral approaches the denominator
lim L(a) Σ fn
(85)
Together, Eq. 85 and Eq. 76 sandwich [(o). Hence,
lim L()) = Σ fn. o o
(86)
n=1
Lastly, note that if the solutions in 0 all fall into distinct cells of archive, as is the case with the populations that conventional QD methods generate or in the limit of having very fine-grained archives, the sum of the objectives coincides with the QD Score and we have
S(0)
N
lim
fn QD Score(
(87)
(2TO2)²
n=1
C ADDITIONAL ABLATIONS
C.I EFFECT OF BATCH SIZE
SQUAD updates the solutions in its population one batch time (Section 4.. primarily reduce the computational cost of simultaneous updates. To examine the role of batch size M, we performed an ablation in the IC domain, evaluating SQUAD with M {4, 16, 32, 64). Each setting was repeated three times with different random seeds. The results are reported in Table
Overall, the performance of SQUAD is stable across tested batch sizes. Smaller batch sizes tend to yield slightly higher QD scores, whereas larger batch sizes achieve marginally better QVS. How- ever, these differences are minor compared to the variation observed between algorithms, indicating machine
that SQUAD is robust to the choice of batch size. All experiments were conducted equipped with an NVIDIA GeForce RTX 4090. The approximate runtimes for different batch sizes were as follows: batch size took 5 hours and 47 minutes, batch size 8 took 2 hours and 55 minutes, batch size 16 took hours and 42 minutes, batch size 32 took 3 hours and 10 minutes, batch size 64 took 3 hours and 10 minutes.
23

---

Published as a conference paper at ICLR 2026
Table 3: Effect of batch size on SQUAD. Results report the mean and standard error over three random seeds for five batch sizes. Experiment in the main paper use M = 64.
Algorithm
QD Score (x (x10))
QVS
SQUAD (M 4)
5.32 + 0.05
457.4 + 0.4
SQUAD (M 8)
5.12 + 0.04
457.7 + 0.4
SQUAD (M 16)
5.04 0.16
458.4 : 0.5
SQUAD (M- 32)
4.99 + 0.09
457.9 + 0.3
SQUAD (M 64)
5.07 = 0.05
457.4 + 0.1
C.2 EFFECT OF NUMBER OF NEIGHBORS
We ablate the sensitivity of SQUAD to the number of nearest neighbors i used in the diversity term of the objective. Experiments were run in the IC domain with k E {0, 4, 8, (6,32) and the results over three random seeds are reported Table[4] Note that the case k 0 removes the diversity term
entirely and therefore corresponds independently optimizing each solution for quality alone.
Table 4: Effect of nearest neighbors' count on SQUAD. Results report the mean and standard error over three random seeds. Experiment in the main paper use 16.
Algorithm
QD Score (x10*)
QVS
SQUAD
0.09 + 0.00
90.8 + 0.0
SQUAD (I: 4)
5.37 + 0.06
457.7 + 0.4
SQUAD =
5.02 0.03
459.7 + 0.5
SQUAD (A: 16)
5.07 = 0.05
457.4 = 0.1
SQUAD (A: = =32)
5.19 = 0.04
448.5 = 0.6
Two conclusions follow from these results. First, the diversity term is essential as the k: = 0 baseline achieves significantly worse performance in both QD Score and QVS, suggesting a collapse into set of nearly identical solutions. Second, for other values of k > 0, the performance is largely insensitive to the exact number of neighbors, since neither QD Score nor QVS vary largely across k 4, 8, 16, 32. In practice, this means that small k already provides the repulsive pressure needed to encourage diverse and high quality solutions.
Our hypothesis for this observed insensitivity is that solutions separated by distances larger than the kernel bandwidth y, which could happen early in the optimization, the kernel contribution from farther solutions rapidly decays towards zero, so only the closest neighbors exert meaningful repulsion. This hypothesis, combined with our results indicating the important role that y plays, also suggests several promising directions for future work: (1) per-solution or adaptive kernel widths based on the current spread of solutions in the behavior space to control the repulsive force each so- lution receives; possibly combined with (2) annealing schedules that dampen the diversity term over training so that exploration is encouraged early on and fine- fine-grained quality optimization dominates later.
C.3 EFFECT OF BEHAVIOR SPACE TRANSFORMATION
Recall that the derivation of SQUAD's objective (Equation 5) assumed that the behavior space is unbounded. To transform the bounded behavior space [0, 1] that is used in the tasks to RRd, SQUAD used the logit transformation:
b
b' log I b'
(88)
where all operations are performed element-wise. In our final ablation, we seek to understand the impact of this transformation on SQUAD's performance. To this end, we run SQUAD without using this transformation with three different random seeds and compare the results with the original ex- periments that were conducted in Section55.3 (conducted over ten random seeds). As the comparison of the results in Table shows, the logit transformation is critical to the successful performance of SQUAD.
24

---

Published as a conference paper at ICLR 2026
Table 5: Effect of behavior space transtormation SQUAD. Results report the mean and stan- dard error over three random seeds for the ablated version and over ten random seeds for the base version.
Algorithm
QD Score (x10*)
QVS
SQUAD
5.09 + 0.05
457.3 0.2
SQUAD (w/o behavior space transformation)
2.96 + 0.04
186.6 + 0.8
C.4 EFFECT OF POPULATION SIZE
We also study the impact that role that population size has on the performance of SQUAD. To this end, we compare the performance of SQUAD in the IC domain with 4 different population sizes N = 128, 256, 512, 1024. The results, reported in Table conform with our expectation that larger populations yield better performance. Furthermore, we observed that the runtime of SQUAD had linear relation with the population size, with the mean runtimes being 24, 48, 95, 190 minutes for population sizes of 128, 256, 512, and 1024, respectively (averaged over seeds, with variances less than 10 seconds).
Table 6: Effect of population size on SQUAD. Results report the mean and standard error over three random seeds. Experiment in the main paper N 1024.
Algorithm
QD Score (x10")
QVS
SQUAD (N 128)
3.24 + 0.43
390.6 + 1.6
SQUAD (N 256)
4.12 + 0.57
422.7 + 0.8
SQUAD (N =512)
4.26 = 0.60
445.2 0.3
SQUAD (N = 1024)
4.50 = 0.62
457.5 0.6
D IMPLEMENTATION DETAILS
D.1 BENCHMARK DOMAINS
D.1.1 LINEAR PROJECTION
We adopt the Linear Projection (LP) domain introduced by Fontaine et al. (2020), using the Rastrigin
objective function (Rastrigin 1974). The QD search is performed over solution vectors x E IR".
with dimensionality set 1024. The Rastrigin function is defined
|ustrrigin (x) 10m >
10 coscoossaaa (o0x;));
(89)
Following prior work (Fontaine et al. 2019 we restrict the search space to 5.12.5.122 and apply an offset so that the global optimum shifted from the origin to [2.048.... 2.048 To
transform the problem from minimization to maximization, normalize the objective via linear transformation
f(x) = 100 x - RRastrigii
(90)
M
where .M denotes the maximum value of the Rastrigin function in the search domain. This results in objective values scaled to the range [0, 100). A heatmap of the transformed Rastrigin function in dimensions depicted in Figure
The behavior space is defined by partitioning [æ 1 into d equal-sized chunks and com- puting the mean of clipped values within each chunk. More formally, the k-dimensional behavior descriptor is given by
2p
desc(x) = d
Σ clip(x:).
clip(..)....
clip(zi)
(91)
25

---

Published as a conference paper at ICLR 2026
where the clipping function is defined
clip(x;))
32
if
- S 5.12
S
(92)
5.12
otherwise
In our experiments, evaluated behavior spaces of dimensionality d E {4,8, 16). For additional details on this domain, including its challenges in high-dimensional settings, we refer the reader to Fontaine & Nikolaidis (2021 2023).
D.1.2 IMAGE COMPOSITION
40
(a) (Transformed) 2-d Rastrigin function
(b) Target image for the IC domain
Figure 5
Image Composition (IC) is new differentiable QD benchmark we introduce, inspired by related works (Tian & Ha 2022 Ibarrola Grace 2023). The IC task aims to reconstruct a target image by composing large number of simple primitives (circles) on canvas. The objective is match the target as closely as possible while exploring diverse visual effects, which are captured through five behavioral descriptors. In this task, solution is vector of size x where denotes the number of circles (here n = 1024). Each row parameterizes a circle with 7 values: the (I, y) center coordinates, radius, RGB color values, and an opacity coefficient. All parameters are represented as unconstrained logits, which are passed through sigmoidal transformations and rescaled to the appropriate range.
A differentiable renderer composites these circles in sequence onto a canvas of resolution 64 x 64. Each circle is drawn with smooth edges, controlled by a softness hyperparameter (here set 10.0). Rendering is performed by alpha-compositing onto a black canvas.
The objective is defined the similarity between the rendered image and fixed target image. We structural similarity (SSIM) (Wang et al. 2004), normalized to the range [0, 100) to measure this.
The behavioral descriptors are five statistics computed from the circles:
mean radius of circles,
variance of radii,
variety of RGB values in the palette (color spread).
coherence of circle hues in HSV space (color harmony),
degree of spatial clustering based on average 5-nearest-neighhor distances.
These descriptors are each normalized to lie in [0, 1]. with higher values representing larger radii, greater diversity, more harmony, or tighter clustering, respectively. Together with the objective, they define continuous and differentiable QD landscape. As the target image in experiments, Johannes Vermeer's 's painting Girl with a Pearl Earring (Figure 3) obtained from the freely available reproduction Wikimedia Commons.
26

---

Published as a conference paper at ICLR 2026
D.1.3 LATENT SPACE ILLUMINATION
Latent Space Illumination (LSI) (Fontaine et al. 2021 Fontaine Nikolaidis, 2021 challenging QD benchmark designed to illuminate the latent space of generative model by discovering diverse and high-quality solutions. Following the experimental setup of Fontaine & Nikolaidis (2023); Tjanaka et al. (2023b), we employ StyleGAN2 Karaas et al. (2020) as the generative model and use 2021) to define both the objective and behavior descriptor functions.
CLIP (Rafford et al.
Each solution in LSI is represented as 9216-dimeniional vector corresponding to point in the latent space of StyleGAN2. To evaluate solution quality, we pass the vector through StyleGAN2 to generate an image, which is then compared to target text prompt using CLIP embeddings. Behavior descriptors are similarly computed by comparing the generated image against pairs of descriptor sentences, one positive and one negative. Our implementation is based on JAX (Bradbury) et al. 2018). and we rely on publicly available JAX-based implementations of both StyleGAN2 and CLIP.
We define two task variants:
Base version. This setup follows Fontaine & Nikolaidis (2023) and uses the prompt "A photo of Tom Cruise" as the objective. Behavior descriptors are specified by two sentence pairs:
("Photo of Tom Cruise as small child", "Photo of Tom Cruise as an elderly person")
("Photo of Tom Cruise with long hair", "Photo of Tom Cruise with short hair")
Hard version. To increase task difficulty, we use the objective prompt "A photo of a detective from a noir film" and define seven behavior descriptor pairs:
P"hhooo of young kid", "Photo of an elderly person")
Photo of person with long hair", "Photo of person with short hair")
("Photo of person with dark hair", "Photo of person with white hair"
P"hoooo of person smiling". "Photo of a person frowning")
("Photo of person with round face", "Photo of a person with an oval face")
("Photo of person with thin, sparse hair", "Photo of person with thick, full hair")
("Photo of person looking directly into the camera", "Photo of a person looking sideways")
D.2 EVALUATION METRICS
We used multiple evaluation metrics in our experiments, which we shall explain here in more detail.
Vendi Score (Friedman & Dieng 2023) is widely applicable metric of diversity in machine learn- ing. Given set of samples and pairwise similarity function, Vendi Score be interpreted the effective number of unique elements in the set. Formally, it is defined as
VS(K)
exp (-tr(
1 log -k) exp Σ Ar log log 71
(93)
n
where K E TRxx is positive semi-definite similarity matrix and X the eigenvalues of 1 K. In our case, the similarity matrix is derived by applying the Gaussian kernel on the distance between the behavior descriptors of solutions. That is, for solutions and j with behavior descriptors b; b, we define: Kij = exp (- [lb. b,1l") - where o, kernel bandwidth. For d dimensional behavior space, we choose of = in our evaluations. This (heuristic) choice is motivated by the fact that the mean squared distance between two uniformly selected vectors in [0, 1]d is d g. Since in all of our experiments the behavior space is defined as [0, 1]d, this ensures that the similarity between two random vectors the behavior space will -1 Rs 0.37.
n
Quality Vendi Score (QVS) (Nguyen & Dieng 2024) extends Vendi Score to also incor- porate the quality of solutions. computed by multiplying the Vendi Score by the mean quality of the solutions:
QVS(K, (f1, (fi...,,,fl)) = Σ fa VS(K),
(94)
27

---

Published as a conference paper at ICLR 2026
where f; is the quality of the i-th solutions. We use QVS to capture the joint effect of quality and diversity in a population. Importantly, the mean quality of the solutions must be non-negative for the metric to be meaningful. While in domains the objectives solutions have objectives in the [0, 100] range, it is still possible for out-of-bound solutions in the LP and LSI domains to obtain negative values, since, theoretically, their objective functions unbounded from below. In cases where the mean objective of the solutions were negative (which only happened with Sep-CMA-MAE and GA-ME in LSI), we report the QVS as 0.0 and report the fine-grained statistics, including the mean objective and Vendi Score, in Appendix E
normalized such that sensible
We also leverage a discretization of the behavior space using CVT (Vassiliades et al. 2018) and report traditional QD metrics such as QD Score (Pugh et al. 2016 and Coverage. Even though the exponentially growing volume of the cells hinders the performance of optimization algorithms that leverage such archives (as noted in the paper), we can still use them for evaluation. To compute these metrics, we discretize the behavior space into fixed number of cells (1024 for IC and 512 for LP and LSI) and insert the solutions from population into the resulting archive, keeping only the best solution that lands in a cell. The Coverage is then defined as the fraction of cells that are filled with solutions and captures the diversity of the population. In a same manner QD Score is defined as the sum of the qualities (objectives) of all the solutions in the resulting archive, and captures both quality and diversity.
Lastly, we also use Mean Objective and Max Objective to measure the quality of populations. These are simply defined the mean (and respectively maximum) of the qualities (objectives) of the solutions in population.
D.3 HYPERPARAMETERS
The full set of hyperparameters that we used for the experiments can be found in the accompanying code. Here, we will go over the most important choices.
D.3.1 BASELINES
All baselines use CVT archive with 10 cells LP and IC domains and a finer archive with 4 x 10 cells in the LSI domain. In LP and IC domains we ran a grid search for each algorithm on the most important hyperparameters and selected the configuration that yielded the highest QD Score.
For CMA-MEGA, we searched over initial step size of the ES (op) and the optimizer learn- ing rate.
For CMA-MAEGA we searched over initial step size of the ES (ao). optimizer learning rate, and archive learning rate.
For Sep-CMA-MAE searched over initial step size of the ES (ag) and archive learning rate.
For GA-ME we tuned iso and line sigma parameters and the gradient step size.
For LSI, we used the default hyperparameters from Fontaine & Nikolaidis (003) used in the pyribs (Tjanaka et al.) 2023b) implementation, with the only difference being the batch size, where we use 16 instead of 32 due to computational constraints. Furthermore, for DNS, we ran a similar grid search over the iso and line sigma parameters, the number of nearest neighbors (k), as well as the learning rate (for the DNS-G variant) in the IC domain to determine appropriate hyperparameters. We also chose the number of iterations such that all algorithms use (roughly) the same number of solution evaluations.
D.3.2 SQUAD
By default, SQUAD uses the values presented in Table / for its hyperparameters and uses Adam (Kingma & Ba, 2015) optimize its objective. Below, will discuss the exceptions these default values.
1. LSI domain: Due to computational constraints, batch size of 8. We use x² = 0.01 for the base version of the task and x?2 = 0.1 for the hard version. We also use larger learning rate of 0.1 for the Adam optimizer.
population size of 256 and
28

---

Published as a conference paper at ICLR 2026
2. LP domain: We use y2 0.1 for the easy version, 72 0.5 for the medium version, and 222 1.0 for the hard version.
Table 7: Default SQUAD parameters
Parameter
Value
Population Size (N)
1024
Batch Size (M)
No. Neighbors (K)
16
Learning Rate
0.05
We train SQUAD for 1000 iterations in LP and IC and for 175 iterations in LSI. The number of training iterations of baselines were set such that they use at least as many evaluations as SQUAD in all domains.
E ADDITIONAL EXPERIMENTAL RESULTS
Here we provide more fine-grained statistics from main experiments in the paper. Table and report the mean and max objectives, Vendi Score, and Coverage statistics of each algorithm in the LP and LSI domains, respectively. Table[10]repot the QD Score and QVS from the IC experiments. As noted in the paper, LP and IC results averaged over 10 seeds and LSI results are averaged over 5 seeds.
We also provide hand-picked samples of the solutions found by SQUAD well the two best baselines, CMA-MEGA and CMA-MAEGA, both LSI tasks (Figure 6 and Figure ( and the IC domain (). Lastly, Tigure@Jcompant in the LSI domain when they are put in a traditional CVT archive. Since this domain has a 2-d behavior space, we can provide CVT archive visualizations for it. Table 8: Additional statisties from LP experiments.
the solutions found by SQUAD and CMA-M(AJEGA
Algorithm
Mean Objective
Max Objective
Vendi Score
Coverage
easy (d 4)
SQUAD
68.36 + 0.02
89.28 = 0.15
6.55
86.4 0.3
CMA-MAEGA
66.02 + 0.28
91.00 = 0.43
6.93 0.00
98.7
CMA-MEGA
66.40 + 0.26
94.54 = 0.38
7.61 +.10
9..
DNS
68.07 + 0.06
78.06 = 0.04
1.63
8.1
DNS-G
78.23 0.12
92.51 = 0.09
1.35
4.0
MeP-CMA-MAE
69.81 + 0.28
78.71 = 0.29
1.25
1.8
GA-ME
69.76 + 0.95
79.42 = 0.21
1.07
0.9
medium (d = 8)
SQUAD
69.41 + 0.02
87.70 = 0.36
9.17
98.1
CMA-MAEGA
62.12 + 0.20
84.41 = 0.33
9.27 +
1.0
CMA-MEGA
62.13 + 0.67
86.76 = 0.70
+.58
99.9
DNS
66.87 0.08
77.61 = 0.07
1.00
13.4+0.2
DNS-G
75.96 0.07
91.41 = 0.22
1.00
7.5
MAACCMA-MAE
66.49 + 1.42
77.04 = 0.13
1.25 0.002
1.1 0.15
GA-ME
69.48 + 1.08
78.99 0.16
1.07 - 0.01
0.6
hard (d 16)
SQUAD
72.86 0.01
83.92 = 0.21
6.61 0.01
91.1 0.2
CMA-MAEGA
64.76 + 2.04
81.27 0.66
159+
71.8 + 13.3
CMA-MEGA
58.11 + 0.39
76.19 0.38
3.73+
99.8
DNS
06.19
77.13 = 0.07
1.69
6.85
DNS-G
74.29 + 0.09
90.54 = 0.15
1.45
5..2
DeP-CMA-MAE
65.04 0.84
78.87 0.11
1.33 +.3
4.7
GA-ME
78.83 + 0.66
89.76 = 0.17
1.08
3.8
29

---

Published as a conference paper at ICLR 2026
Table 9: Additional statistics from LSI experiments.
Algorithm
Mean Objective
Max Objective
Vendi Score
Coverage
L.SI
SQUAD
79.56 0.45
83.81 0.09
2.22 0.03
32.4
CMA-MAEGA
77.18 3.96
86.98 0.45
1.57 +
2.0
CMA-MEGA
83.38 + 0.89
87.46 + 0.06
1.68
1..7
DNS
-221.24 28.01
#.11
1.39
1.3
DNS-G
288.33 + 26.12
85.22 + 0.05
1.36
9.0
Sep-CMA-MAE
476.62 241.08
139.05 135.77
1.01
0.1
GA-ME
358.40 + 68.50
83.72 + 0.10
1.25 0.01
6.8
LSI (hard)
SQUAD
0.01 82.51 0.01
84.26 0.09
+.00
6.0 0.2
CMA-MAEGA
81.55 + 1.25
$7.05+0.11
1.22
0.2
CMA-MEGA
84.24+0..
8.2
1.10
0.1
DNS
-222.68 + 12.24
84.02 1002:0000 0.04
1.38
1022
DNS-G
214.770 35.08
15.17
1.35
9.1
Sep-CMA-MAE
-37.6 + 94.16
16.74 + 42.55
1.00 - 0.00
0.2
GA-ME
1688.09 = 217.38
83.46 + 0.14
1.04 0.01
0.2
Table 10: Additional statistics from IC experiments.
Algorithm
QD Score
QVS
SQUAD
5086.2 54.7
457.35 0.23
CMA-MAEGA
4605.8 40.5
294.07 1.96
CMA-MEGA
3565.7 +:657
246.89 + 17.7
DNS
1128.4 15.7
115.17
DNS-G
1148.0 + 24.2
124.72 0.33
Sep-CMA-MAE
348.9 17.6
94.735 0.8
GA-ME
140.9 + 23.2
83.43 + 2.97
F STATEMENT ON GENERATIVE AI USAGE
Generative AI tools were used aid improve clarity and style in the writing of this paper.
G RUNTIME ANALYSIS
Table 11 summarizes the wall clock runtimes for SQUAD and all baselines the three domains we consider. The most influential factor in SQUAD's runtime is the cost of gradient computation. In the Rastrigin domains this cost is relatively small. Therefore, even in the hard setting where the behavior space has dimension 16, SQUAD completes all 1000 iterations in under one minute. In these domains the computational structure is simple and backpropagation inexpensive, which leads to very fast overall runtime.
In contrast, the and LSI domains require gradients that must be backpropagated through signif- icantly more complex computational pipelines. In IC, each gradient step involves differentiation through a differentiable renderer. In I.SI, gradients pass through both StyleGAN and CLIP, which large networks and therefore incur substantial computational overhead. As result, the wall clock times for SQUAD these two domains are noticeably higher.
It is important to emphasize that these higher runtimes do not indicate inefficiency of SQUAD. In fact. SQUAD converges very quickly to high-quality solutions. Figure shows the training curves of SQUAD together with the final performance of all basclines in terms of QD Score and QVS in the IC domain. SQUAD surpasses all baselines both metrics in fewer than 200 iterations. Nevertheless, SQUAD for 1000 iterations primarily to ensure equal evaluation budget
30

---

Published as a conference paper at ICLR 2026
Generated Samples
SQUAD
CMA-MEGA
CMA-MAEGA
Figure 6: Qualitative comparison of SQUAD against baselines in LSI. Each corresponds one algorithm, with five representative samples handpicked from the populations.
Generated Samples
SQUAD
CMA-MEGA
CMA-MAEGA
Figure 7: Qualitative comparison of SQUAD against two baselines LSI (hard). Each row corre- sponds to one algorithm, with five representative samples handpicked from the populations.
across algorithms. In practice one can use SQUAD with a far smaller number of iterations and still obtain superior results, which directly reduces the wall clock time below the values reported in the table above.
31

---

Published as a conference paper at ICLR 2026
Generated Samples
SQUAD
CMA-MAEGA CMA-MEGA
Figure 8: Qualitative comparison of SQUAD against two baselines in IC. Each row corresponds to one algorithm, with five representative samples handpicked from the populations.
Table 11: Wall clock runtime (in minutes) of SQUAD and all baselines across the three domains. For Rastrigin we report runtimes for the easy, medium, and hard settings. For LSI we report the base and hard settings.
Rastrigin
IC
LSI
Method
Easy
Medium
Hard
Base
Hard
SQUAD
<1
<1
<1
190
732
1316
GA-ME
17
403
707
CMA-MAEGA
45
78
66
8
222
339
CMA-MEGA
64
106
233
337
DNS
5
8
10
6
15
25
DNS-G
3
5
50
207
355
Sep-CMA-MAE
1
33
8
10
32

---

Published as a conference paper at ICLR 2026
0.6
60
0.6
60
0-4
114
0.2
0.2
0.
0.2
0.6
0.9
0.0
0.2
0.4
0.6
0.6
Age
A0e
(a) SQUAD
(b) CMA-MEGA
1.0
100
H42 Lenigh
0.00
(c) CMA-MAEGA
Figure 9: Final CVT archives of SQUAD, CMA-MEGA, and CMA-MEGA in the LSI domain.
QD Score Throughout Training
QVS Throughout Training
5000
400
4000
QD Score
CMA-MAEGA
9 300
CMA-MAFGA
CMA-MEGA
CMA-MEGA
2000
DNS
DNS
DNS-G
200
DNS-G G
1000
Sep: CMA-MAE
GA-ME
100
GA-ME
400
Training Iterations
Training Iterations
Figure 10: Training curves for SQUAD compared with the final QD Score and QVS values of all baselines in the IC domain. SQUAD exceeds all baselines on both metrics in fewer than 200 iterations.
33


Reviews:
[{"review_id": "tVYZAL4LEf", "reviewer": "Reviewer_8sBH", "summary": "Quality-Diversity (QD) algorithms typically discretize the behavior space into cells, find the best solution in each cell, and use the QD Score (i.e., the sum of the fitness value of the solution in each cell) as the objective. However, the discretizing mechanism makes it difficult to optimize end-to-end with gradient-based optimizers, and the number of cells grows exponentially when the dimension of the behavior space is large. For QD problems with differentiable fitness and behavior functions, this paper introduces a new objective, the Soft QD Score, to solve these issues. The behavior value is defined as the maximum fitness exponentially decreasing with distance among the population, and the Soft QD Score is defined as the integral of the behavior value over the behavior space. The Soft QD Score is then approximated by a computable and differentiable form, consisting of a quality term and a distance-based diversity term, which can be optimized directly with gradient-based optimizers. Experimental results on three differentiable QD (DQD) problems demonstrate that the proposed method outperforms other DQD methods significantly when the dimensions of behavior spaces are large.", "questions": "- How will the baselines perform if they use the Soft QD Score as the objective?\n- What hyperparameter values do the baselines use in the experiments? These values may affect the performance of the baselines.\n- Note that neighbor computing of batches might be time-consuming, since it applies $O(MNk)$ pairwise repulsions for each batch. Can you compare the running time of the algorithms?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The proposed new objective bypasses the issues arising from behavior space discretizing, and enables it to be optimized by gradient-based optimizers easily.\n- The proposed method performs well on DQD tasks with high-dimensional behavior spaces.\n- The paper is well-organized and easy to follow. The code is available, contributing to reproducibility.", "weaknesses": "- The experiments are limited. The proposed method is not evaluated on high-dimensional reinforcement learning tasks, which are a more important type of DQD problem. Consequently, it is not compared with state-of-the-art methods that utilize policy gradients effectively.\n- There are also other QD methods that do not discretize the behavior space (e.g., novelty search-based methods). They are not compared in the experiments.\n- It is unclear whether the outstanding performance comes from the definition of the Soft QD Score, the end-to-end gradient-based optimization, or the population management.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JrSxvV8YwU", "reviewer": "Reviewer_9uNt", "summary": "This paper identifies a key limitation in current Quality-Diversity (QD) optimization methods: reliance on discrete archives (tessellations) to maintain diversity, which suffers from the curse of dimensionality in high-dimensional behavior spaces. To address this, the authors propose \"Soft QD,\" a continuous relaxation of the QD problem. They define a new objective, the \"Soft QD Score,\" which models each solution as a light source with a Gaussian field of influence, integrating the maximum influence over the entire behavior space.\n\nBecause directly maximizing this integral is intractable, the authors derive a differentiable lower bound that simplifies to a standard quality maximization term minus a pairwise repulsion term (weighted by quality and proximity). This leads to a new algorithm, SQUAD (Soft QD Using Approximated Diversity). Empirical results on Linear Projection (up to 16D behavior space), Image Composition, and Latent Space Illumination benchmarks demonstrate that SQUAD scales significantly better to high-dimensional behavior spaces than state-of-the-art differentiable QD baselines (like CMA-MAEGA) and offers a controllable trade-off between quality and diversity via its kernel bandwidth hyperparameter.", "questions": "1. Adaptive Bandwidth: Given the strong influence of $\\gamma^2$ on the quality-diversity trade-off (Figure 4), did you explore any mechanisms for adapting $\\gamma^2$ online? For instance, starting with a larger $\\gamma$ for broad exploration and annealing it for fine-grained illumination later in training?\n\n2. Wall-clock time: Could you provide more details on the wall-clock time comparison between SQUAD (using its batch/KNN approximations) and the highly optimized pyribs baselines, particularly as the population size $N$ scales up?\n\n3. Deceptive Landscapes: How does SQUAD behave in highly deceptive behavior landscapes? Does the continuous repulsion field ever prevent it from traversing narrow \"corridors\" in behavior space that a discrete archive might serendipitously cross?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Novel Problem Formulation: Shifting the QD objective from a discrete, archive-based metric to a continuous, integral-based \"illumination\" field is a significant and original theoretical contribution. It elegantly sidesteps the arbitrary nature of defining grid resolutions or pre-computing tessellations.\n\n- Scalability to High-Dimensional Behavior Spaces: The empirical results strongly support the core claim that this method handles higher-dimensional behavior spaces better than archive-based counterparts. Figure 3 shows standard methods (CMA-MEGA/MAEGA) degrading at $d=16$ while SQUAD maintains performance. Besides, by framing the problem as a unified differentiable objective, SQUAD can seamlessly leverage modern optimizers like Adam, simplifying the typical QD loop.\n\n- Theoretical Grounding: The derivation of SQUAD from the Soft QD Score via Bonferroni inequalities and geometric mean approximations (Theorem 2 and Appendix A) provides a solid theoretical foundation for the proposed pairwise repulsion objective, rather than just proposing it as a heuristic.\n\n- Clarity and Presentation: The paper is exceptionally well-written. The conceptual difference between hard and soft archives is intuitively visualized in Figure 1.", "weaknesses": "- Hyperparameter Sensitivity ($\\gamma$): The method's performance and its trade-off between quality and diversity are heavily dependent on the kernel bandwidth $\\gamma^2$ (Section 5.3). While this provides controllability, it also introduces a critical hyperparameter that might be difficult to tune a priori for new domains compared to setting a grid resolution.\n\n- Bounded Space Reliance: Appendix C.3 highlights that the logit transformation for bounded spaces is \"critical\" for success. This suggests the method might struggle with intrinsically unbounded behavior spaces or spaces where appropriate transformations are unknown, limiting generality slightly compared to robust binning strategies.\n\n- Computational complexity of pairwise interactions: While standard QD archive insertions are typically $O(1)$ (grids) or roughly logarithmic (CVT trees) per solution, SQUAD's objective involves pairwise interactions. Although mitigated by KNN ($O(Nk)$), this could potentially become a bottleneck for very large populations compared to purely archive-based approaches, though it is likely an acceptable trade-off for high-dimensional capabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ywbugzHqXj", "reviewer": "Reviewer_RoFt", "summary": "This paper proposes a Soft Quality-Diversity (QD) optimization framework that avoids discretizing the behavior space. It defines the Soft QD Score, analyzes its properties (monotonicity, submodularity, limiting equivalence to traditional QD Score), and derives SQUAD, a differentiable algorithm based on a lower bound of the Soft QD Score. SQUAD optimizes via a quality term (maximizing solution quality) and a repulsive diversity term (spreading solutions). Experiments on Linear Projection, Image Composition, and Latent Space Illumination benchmarks show SQUAD outperforms baselines in high-dimensional scenarios.", "questions": "1. what is the technical difference with Kent et al. (2022)’s continuous QD Score design?\n2.  If generated \"diverse\" solutions differ in real-world attributes rather than just behavioral space coordinates?\n3.  Why the logit transformation is selected for SQUAD, instead of other transformations  (e.g., Box-Cox, standardization)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The idea of \"illumination\" for addressing the high-dimensionality is interesting. A differentiable method of SQUAD is appreciated.", "weaknesses": "1. The core idea of continuous behavior space “illumination” overlaps with Kent et al. (2022)’s continuous QD Score. However, the paper only notes Kent et al.’s work as an evaluation tool, a more indepth discussion should be provided to support its novelty.\n2. The paper claims SQUAD scales to high-dimensional spaces, but LP domain tests only go up to 16-dimensional behavior spaces. The impact of solution count  on computational efficiency and optimization performance is unexamined.\n3. For baselines like CMA-MAEGA, grid search is used for LP/IC domains, but default parameters are adopted for LSI (due to computational constraints). Additionally, SQUAD uses a smaller population size (256 vs. default 1024) in LSI. This inconsistent tuning may prevent baselines from performing optimally.\n4. QD often claims the diversity. In LSI, SQUAD outperforms baselines in QVS, but the paper does not validate if the “diversity” is semantically meaningful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)