Convergent Functions, Divergent Forms
Authors: Hyeonseong Jeon
Ainaz Eftekhar
Aaron Walsman
Kuo-Hao Zeng
Ali Farhadi
Ranjay Krishna
Venue: NeurIPS 2025 poster ()
Keywords: Morphology Evolution
Embodiment Co-Design
Quality-Diversity
Primary area: reinforcement_learning

Abstract: We introduce LOKI, a compute-efficient framework for co-designing morphologies and control policies that generalize across unseen tasks.  Inspired by biological adaptation—where animals quickly adjust to morphological changes—our method overcomes the inefficiencies of traditional evolutionary and quality-diversity algorithms. We propose learning convergent functions: shared control policies trained across clusters of morphologically similar designs in a learned latent space, drastically reducing the training cost per design. Simultaneously, we promote divergent forms by replacing mutation with dynamic local search, enabling broader exploration and preventing premature convergence. The policy reuse allows us to explore $\sim780\times$ more designs using 78\% fewer simulation steps and 40\% less compute per design. Local competition paired with a broader search results in a diverse set of high-performing final morphologies. Using the UNIMAL design space and a flat-terrain locomotion task, LOKI discovers a rich variety of designs—ranging from quadrupeds to crabs, bipedals, and spinners—far more diverse than those produced by prior work. These morphologies also transfer better to unseen downstream tasks in agility, stability, and manipulation domains (e.g. $2 \times$ higher reward on bump and push box incline tasks). Overall, our approach produces designs that are both diverse and adaptable, with substantially greater sample efficiency than existing co-design methods.

Convergent Functions, Divergent Forms
Hyeonseong Jeon 1,2
Ainaz Eftekhar 1,3
Aaron Walsman
Kuo-Hao Zeng"
Ali Farhadì
Ranjay Krishna 1.3
University of Washington 2 Seoul National University
Allen Institute for AI
Kempper Institute at Harvard University
loki-codesign.githubbe
(A) Clustering the space designs
(B) Multi Desiin Control Policies
Sampling Designs using Dynamic Local Search
(Divergare Farms)
(Conengent Furetlond)
Evaluate with A
Roward
Cluster Cluster-Specht fic Locomatiin Policy ott
Vp
top cesigns
Cluster 4
Dynamic pool of Elites
Ranclom designs sampled from
cluster 4
We introduce LOKI R. compute-efficient co-design framework that discovers diverse, high- performing robot morphologies (divergent forms) using shared control policies (convergent functions) and dynamic local search instead of mutation. (A) The design space is clustered in a learned latent space so that morphologies within each cluster share structural similarities and exhibit similar behaviors. policy is trained within each cluster dynamic pool of elite morphologies. (C) Morphologies co-evolve with the shared policy as elites iteratively refined through dynamic local search.
Figure 1:
(B) A shared control
Abstract
We introduce LOKI, a compute-efficien framework for co-designing morphologies and control policies that generalize across unseen tasks. Inspired by biological adaptation-whert animals quickly adjust to morphological changes-our method overcomes the inefficiencies of traditional evolutionary and quality-diversity algo- rithms. We propose learning convergent functions: shared control policies trained across clusters of morphologically similar designs in learned latent space, drasti- cally reducing the training cost per design. Simultaneously, we promote divergent forms by replacing mutation with dynamic local search, enabling broader explo- ration and preventing premature convergence. The policy reuse allows us to explore 780x more designs using 78% fewer simulation steps and 40% less compute per design. Local competition paired with a broader search results in a diverse set high-performing final morphologies. Using the UNIMAL design space and flat- terrain locomotion task, LOKI discovers a rich variety of designs-ranging from quadrupeds crabs, bipedals, and spinners-fal far more diverse than those produced by prior work. These morphologies also transfer better to unseen downstream tasks
Equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Locomotion Behavior
Downstream Task
Locomotion Behavior
Downstream Task
Spinner
Bump
Crawler er
Push box incline
Crab
Manipulate ball
Quadruped
Obstacle
v
Cheetah
Exploration
pedal
Incline
Figure LOKI discovers high performing morphologies with diverse locomotion behaviors that effectively generalize across various unseen tasks.
in agility, stability, and manipulation domains (e.g. 2x higher reward on bump and push box incline tasks). Overall, our approach produces designs that are both diverse and adaptable, with substantially greater sample efficiency than existing co-design methods.
Introduction
Embodied cognition asserts that intelligence emerges not just from the brain but is shaped by the physical embodiment [ This perspective acknowledges that "how creature behaves is influenced heavily by their body," which acts as an interface between the controller and the environment [2]. A four legged embodiment gallops while a two legged one simply ruis. This adaptation occurs across both evolutionary timescales-an in Darwin's finches, whose beak shapes evolved to exploit different food sources and individual lifetimes, through mechanisms like Wolff's Law, where bones strengthen in response to load [3]. Animals can seamlessly readapt their controllers to accommodate subtle changes in morphology 4
Designing or discovering optimal embodiments for robots is a difficult combinatorial challenge [5] 8 9
6, 7 typically framed as livel optimization: an outer loop searches for morphologies (often using evolutionary strategies), while inner loop trains control policy for each design to evaluate its fitness (performance on task). Due to the vastness of the design space, exhaustive search is infeasible D4 4. Worse, the fitness landscape is often deceptive [15), causing evolutionary strategies to converge prematurely to local optima. Escaping such traps requires maintaining diversity across the population [16] [7 18]. Quality- divverity algorithms address this by promoting exploration of high- high-performing yet diverse designs [19 20 21 22]. However, evaluating each candidate costly, and diversity comes the expense of the number of designs evaluated in the inner loop 23) Unlike animals- who quickly adapt controllers to new bodies prdies-robots still require retraining from scratch for each morphology, as policies transfer poorly across embodiments
I0
11 12
13
Co-design, the joint optimization of morphology and control, is
24
(Locally Optimized Kinematic Instantiations), an efficient framework for discovering diverse nigh-performing morphologies that effectively generalize across unseen tasks, Our framework is designed with the following principle: convergent functions and divergent forms. First, we mirror Wolff's Law of adapting policies to new morphologies. We train convergent functions, ie. a single policy function is trained across similarly structured morphologies. We learn latent space of possible morphologies. cluster this space and learn a shared policy that can adapt any morphology within that cluster. These cluster-specific policies exploit shared structural features, generalizing across similar designs and behaviors. and enabling the reuse of learned behaviors. With it, we can evaluate significantly more designs without incurring additional expensive training. Competition occurs locally within each cluster-instead of a global selection pressure-resulting in diverse high-performing final designs (similar to quality-diversy algorithms but much more efficient).
We introduce [.OKT

---

Second, we break away from how most algorithms sample new designs using mutation. Mutating high- performmng designs leads to limited exploration. Instead, we sample divergent forms by adopting dynamic local search (DLS) 25
We maintain dynamic elite pool per cluster to train shared policy for the cluster. During training. random designs are sampled from a cluster, evaluated by the current policy, and top ones replace the worst-performing elites in the pool (see Alg. D Compared to mutation, which typically yields small, incremental changes, DLS enables broader and more adaptive exploration, reducing the risk of getting stuck in local minima. Since evaluations are made inexpensive with shared policies, we can afford to use random search instead of local mutation, efficiently filtering out low-quality candidates.
26 27
We use UNIMAL 14, an expressive design space encompassing approximately 10 unique mor- phologies with fewer than 10 limbs. We evolve morphologies on a simple flat-terrain locomotion task. In nature, locomotion acts as a universal selection pressure across species. It aask-agnotic--heppeee to prevent overfitting to narrow objectives and is efficient to simulate and easy to reward.
Beyond being substantially more efficient, LOKI discovers both genetically and behaviorally diverse perh-performing morphologies, outperforming previous state-of-the-ar algorithms. Using signif- icantly less computational budget-approximatll 78% fewer simulation steps (20B + 4.6B) and 40% fewer training FLOPs per design-LOKI explores ~ 780x more designs than prior evolution- based approaches (4,000 + (Tbbla LOKI discovers varied locomotion strategies including quadrupedal, bipedal, ant-like, crab-like, cheetah, spinner, crawler, and rolling behaviors (Fig. 2 In contrast, prior global competition-baste evolutionary methods often converge to a single behavior type-for example, consistently producing only cheetah-like morphologies [4]
LOKI results in morphologies that transfer better new unseen tasks. The final set of evolved designs are trained on a suite of downstream tasks across three domains: agility (bump, patrol, obstacle, exploration), stability (incline), and manipulation (push box incline, manipulation ball). Compared to baseline methods, such as DERL [4] and Map-Elites (24). LOKI'S cluster-based co-evolution results in superior adaptability to these challenging unseen tasks (e.g., bump: 981 - 1908; push box incline: 1519- + 3148: see Fig. ( Our convergent functions, shared across morphologies, results in high- per-orerformnn divergent forms that generalize better to new tasks with significantly less compute.
Background and Related Work
Brain-body co-design. The automation of robot behavior is one of the early goals of artificial intelligence [8 (2). In settings where the robot design is fixed, reinforcement learning (RI.) [30] 3T
32 33 has emerged as a dominant method for optimizing control policies. However, the problem is more complicated when we wish to optimize not only the robot's policy E 11, but also the its physical morphology within some design space M M. This design space can be simple real-valued limb length parameters [13] more complex combinatorial structures consisting of link and joint primitives [34], or even the voxelized parameters of soft robots (35]. In these settings, the physical design of a robot and its policy are inherently coupled, as both the transition function
T (st+11 St, a, A and the total expected reward J J(T,, M) E,,M Σ/ 7'T; are now a function of both action and morphology. This means we must find ways to optimize morphology and behavior jointly: M = arg max (MTr .M) subject to iir- arg max, arg max. M).
Early foundational work optimized morphology and behavior jointly using genetic algorithms 14 36 A popular approach is to use bi-level optimization in which an inner RL loop finds the best policies for a given design, and an outer evolutionary loop selects morphologies that produce higher performing behaviors 44. While the bi-level optimization can be computationally expensive, some authors have proposed solutions such as policy sharing with network structures that can support a variety of morphologies (5) 9. Others discard the outer evolution loop entirely by incorporating morphology refinement 133 or compositional construction [T 37] into the RL policy. Between these extremes are approaches that use two reinforcement learning loops that update at different schedules [6]. Recently, researchers have also attempted to produce a diverse set of morphologies rather than a single high-performing design in order to accommodate multiple downstream tasks [38 4 Our approach is also in this category and demonstrates substantial improvements over these prior methods.
Optimization and Diversity. Algorithms designed to find multiple diverse solutions to an optimiza- tion problem have long and rich history. Island models [39 40 41). fitness sharing 42, tournament
selection 43 and niching [4] in genetic algorithms were proposed different ways to maintain the
3

---

diversity of a population and prevent premature collapse to single solution mode. Similar techniques
were developed for coevolutionary/mulii- -agent settings 45
46
attempt to illuminate the search space by discovering broad repertoire of high-performing yet behaviorally distinct solutions. An important component of algorithm design in this space is deciding how to measure diversity and how accumulate the resulting collection of solutions. Novelty Search [48] 20 measures diversity as the average distance to nearest neighbors, and adds solutions to an archive if their diversity score is higher than threshold. MAP-Elites [3 22 forms discretized grid of solutions according to user-defined feature space, and updates this map with the best solution found for each grid cell so far. This grid of
More recently, Quality Diversity (QD) algorithms 19
47
21
solutions has been extended to Voronoi tessellations for high- eighddimensional feature spaces [49] [50] and
combined with evolution strategies [5]] and reinforcement learning [52]. QD algorithms can generate a repertoire of diverse solutions, enabling adaptive selection based on changing task demands or
environmental conditions 53 54). In morphology optimization, evolving a diverse population on
single environment has been shown to yield better generalization and transfer to new tasks [55]
3 LOKI
We introduce LOKI, a compute-eficiiet co-design framework that discovers diverse, high-performing morphologies (divergent forms) using shared control policies (convergent fumctions). Leveraging structural commonalities the design space, we train multi- -esign policies on clusters of similarly structured morphologies. These shared policies exploit common features, enabling behavior reuse and allowing us to evaluate many more designs without retraining. Morphologies are co-evolved alongside the policies using Dynamic Local Search (DLS) [25] 26 27], which facilitates broader exploration within each cluster. By localizing the competition 20] within each cluster and having a broader exploration, LOKI evolves diverse high-performing solutions that effectively generalize across different unseen tasks. We describe our clustering approach in 3.1 the training of cluster-specific
multi-design policies in Sec 3.2
and our co-design framework in Sec. 3.3
LOKI offers three key benefits: Efficient Search. We explore a design space 780x larger while requiring significantly fewer simulation steps (20B - 4.6B) and less compute per evaluated design (160B - 100B) (Tab. []]. Robust Evaluation. dingle-design policies are narrowly specialized, whereas our multi-design policies are exposed to a broader range of morphologies and behav- leadnng to more robust fitness evaluations grounded in the behavioral landscape of previously encountered agents. Diverse Solutions. LOKI independently discovers a range of high-performing designs across clusters, without relying on explicit diversity objectives like novelty search 20).
3.1 Clustering the Space of Designs
Multi-design policies often struggle to generalize across mor- phologies with diverse behaviors. To address this, we cluster morphologies that are both structurally and behaviorally similar, and train separate policies for each cluster. Effective general- ization within a cluster requires feature space where distance meaningfully reflects similarity. Each morphology represented as a sequence of limbs, with a mix of continuous and categor- ical parameters. Due to the complexity of this representation, clustering in the raw feature space is suboptimal. As shown in Fig. 3 K-means clustering [56] with Euclidean distance over raw parameters groups topologically distinct diaiinn-difffringg overall structure-ing into the same cluster, despite similaritics in simpler attributes like limb length or radius.
3
2
4
3
5
Figure 3: Clustering using raw morphology parameters fails to capture the topological similarity. The 3 designs are from the same clus- ter. Indices show the DFS traversal.
Instead, we train Transformer-basee VAE 57 388 to encode morphologies into a structured latent space. Each morphology is represented as an L X matrix, where L is the maximum number of limbs and D 47 is the number of parameters per limb, with both continuous (e.g., limb orientation, length, joint gear) and categorical (e.g.. joint joint angle, depth) attributes. Limb vectors are ordered using depth-first search (DFS) traversal [59], starting from the head, which encodes torso type (horizontal/vertical),, head density, and radius. If morphology has fewer than limbs, the sequence is zero-padded. The transformer encoder [57] maps each morphology to (L x H) latent

---

Table 1: Efficiency and Coverage Comparison. LOKI explores much larger design space while requiring
significantly fewer simulation steps and less training FLOPs per evaluated design See Appendix [ for details.
# Interactions (B)
# Searchedd Morphologies 1
FLOPs per Morphology (B) 4
DERL LOKI
20
4000
160
4.62
3,120,000
100
space, where H < D. To support decoding, we append an EOS (end-of-sequence) token to mark the final limb and use depth tokens to indicate each limb's position in the depth-first traversal.
The loss over a batch is defined as Cream E C, where L 2i feon (e3 MM
E, May
Σ, fy is the reconstruction loss for the i-th limb. fcom and feat are the L.2 loss and Σ, Mg
cross-entropy loss for continuous or categorical attributes. M and M are binary masks showing existence of the j-th attribute in the i-th limb, and Yij the ground-truth value. Instead of the standard ELBO objective, we adopt an adaptive scheduling strategy for the KL. divergence scaling factor Our goal is learn a compact latent representation suitable for clustering. Therefore, we prioritize minimizing the reconstruction loss over enforcing strong regularization towards a standard
CO
normal prior. Following [61], we exponentially decay B within a predefined range [Smin: 3max]
We generate 500k unique random morphologies within the UNIMAL space 4 (see Appendix C for details). We train a transformer-basee
VAE (4 layers, 4 heads, latent dimension
32) on
these designs using a batch size of 4096, an initial learning rate of 10 and a single A40 GPU for 200 epochs. We then cluster the 500k morphologies into Ne 40 groups using the K-means algorithm [56] based on Euclidean distance of their corresponding VAE latent codes. A t-SNE 62 visualization of the clusters shown in Fig. A.
3.2 Convergent Functions: Multi-Design Control Policies
Most prior evolution-based co-design approaches train each morphology from scratch using reinforcement learning, without sharing experience across the popu- lation. In contrast, ities within the design space by training multi-design policies for clusters of morphologies that share struc- tural and behavioral similarities. These cluster-specific policies serve as surrogate scoring functions, enabling efficient evaluation of large number of designs within each cluster without retraining, thereby significantly improving search efficiency.
PIB Whot 200 DESRRNNOO
3000
leverage structural commonal-
2009
12
201
401
300
1000
200
400 EWM
RMM
100
Figure 4: Coevolution of morphologies with multi-design policies. Left: Training rewards for each policy. Right: Mean pairwise distance of morphologies in the elite pool of each cluster.
As shown in Fig. BB for each cluster, we initialize a training pool PfC for Cs with N., = 20 randomly sampled agents. We train a multi-design policy Tq Co from scratch using the dynamic agent pool within that cluster for Ni 1220 training iterations. Every fair 2 iterations, the pool is updated pCc - P.Cx to identify the cluster's clites via the stochastic search process described in Sec. [3.3 Each cluster-specife policy is trained on a large number of designs, enabling generalization across similar morphologies and behaviors.
We adopt the Transformer-basee policy architecture (5 layers/2 heads) from 63 for our multi-design policies. Each policy is trained for 100 million simulation steps using batch size of 5120 and an initial learning rate of 3 x 10-4 with cosine annealing. Training is distributed across six A40 GPUs, with each GPU handling 6-7 cluster-specific policies in parallel (more details in the Appendix H)
3.3 Divergent Forms: Co-evolving Elites using Dynamic Local Search within Clusters
Most evolution-based approaches generate new designs by mutating existing ones. However, such small incremental changes often restrict exploration to local neighborhoods, increasing the risk of premature convergence to local optima. Additionally, global selection pressure can cause the search to favor single dominant species or behavior-ofe often biasing toward short-term gains like faster
5

---

Algorithm 1: LOKI
1: Input:
Morphology clusters C (Cx H k= training iterations Nuar, initial pool size Nu. update frequency fam. sample size Nample. replace count Nanor for each cluster C2 E do
3: Initialize training pool pee with Nw random morphologies from C2
4: Initialize policy TT and value function V from scratch
5: for = 1 to Nax do
6: if i mod fair = 0 then
Sample Nample new morphologies from Cj evaluate using "t"
Select top Nemr sampled morphologies {ū: Mater
9:
Identify Natte lowast-reward agents {w, ( Nare pCc
10:
Replace: D.uK +- PC_, {we} U {m5z}
11:
else
12: pea
-
13: end if
14: PPO_step(ee Ct,
)
15: end for
16: end for
learning, while overlooking more complex unconventional strategies (e.g., spinning) that require longer-term training (Fig. 2).
To overcome these limitations, we sample divergentforms. using Dynamic Local Search (DLS) [25] 26) 27]. combined with localized competition within each cluster. This allows clusters to independently evolve high-performing and behaviorally diverse morphologies (see different locomotion behaviors Fig. 2). DLS promotes broader and more adaptive exploration within each cluster, reducing the risk of getting stuck in local minima. Our multi multi-design policy TT acts as dynamic evaluation function, continually updated via PPO (32 on dynamic pool of elite designs p., This enables efficient assessment of diverse candidates sampled broadly across the cluster, rather than relying narrowly mutated variants. Because evaluations are made efficient through policy sharing, we can use random to quickly filter out poor designs and retain promising ones.
search instead of mutation-allowiii
As shown in Fig. [ every fair = training iterations, Nsample = 128 new random designs are sampled from the cluster and evaluated using the current policy Cs. The top Nitter = 2 designs replace the lowest-perforning Nitter agents in the pool. Training then continues with the updated pool PC Fig. 4 shows the training rewards and pairwise distances of the designs within each training
pool Pc.. Over time, each pool converges to steady state, where the same Nitter 2 agents are consistently replaced each update cycle. The complete algorithm is shown Alg.
How to choose the number of clusters? Number of clusters plays a key role in balancing intra-cluster behavioral similarity with the breadth of localized competition. Within each cluster, morphologies should be behaviorally similar enough for the policy to generalize effectively, while the clustering should remain fine-grained enough capture morphological diversity. With N, 10, clusters roughly align with limb count, resulting in overly coarse groupings. We select N = 40 to better capture finer morphological distinctions while maintaining a feasible computational budget. Each
policy evaluates up to Nus Fan
Naamle + Nw 78,000 samples during training, allowing each
morphology to be evaluated approximately 6 times on average. These repeated evaluations are valuable, early policies may be unreliable; re-evaluation with improved policies uncover
better-performing behaviors (see Appendix for details).
4 Experiments
In all experiments, morphologies evolved for locomotion flat terrain (FT) using UNIMAL space. Locomotion is a universal and ubiquitous evolutionary pressure across species; it is aask-agnostic, avoids overfitting to narrow objectives, and is easy simulate and reward.
We evaluate both the performance and diversity of the final evolved morphologies, comparing them to other bolseioonbbssed co-design methods well Quality-Diversity approaches (Sec. LOKI

---

Table 2: LOKI maintains both quality and diversity. It obtains the
Spers Verssere
highest QD-Score, demonstrating its ability to find nigh-performing solutions across a wide range of niches. (= denotes standard error
across 4 training seeds)
Method
Max Fitness QD-score Coverage %
Eatee Srnp Fkow
LOKI
RANDOM
3418.9
32.1
90
k neerest neighbor(()
DERL
5760.8
248
26.2
37.5
Figure 5: LOKI evolves significantly more sparse solutions (measured by the average distance to k-nearest neighbors). This is due using Dynamic Local Search for sampling designs, rather than mutations. Higher sparseness indicates more diversity. (Statistical significance was assessed using independent samples t-tests; *p < 0.005; **P <
MAP-ELITES
5807.3 = 1965
43.5
65.0
LATENT-MAP-ELIE
5257.0
491.3
38.1
100
LoKI CLUSTER)
4825.8 76.1
40.0
75
LOKI (N, = 10)
4008.0 0 363.
21.6
47.5
LOKI (Ne 20)
5544.0
268.1
26.6
45.0
LOKI (N, 40)
5671.9 300.1
60.9
100
10 11, #: P 10 10016). 19).
promotes both genetic and behavioral diversity, discovering a wide range of locomotion strategies including quadrupedal, bipedal, crab-like, cheetah, spinner, crawler, and rolling behaviors, as shown in Fig. 2 We assess the transferability of the evolved morphologies to a suite of downstream tasks across three domains: agility (patrol, obstacle, bump, exploration), stability (incline), and manipulation (push box incline, manipulation ball). Compared to baselines, LOKI exhibits superior adaptability to these unseen challenges- at both the morphology and policy levels driven by the quality and diversity of the evolved designs (Sec. 4.2)
Baselines and Experiment Details. We compare our method with DERL [4] a prior evolution- based co-design approach, and CV Eap-iees [22], representative Quality-Diveer Diverrstty baseline. For fair comparison, we evaluate the top N = 100 final evolved morphologies (elites) from each algorithm. For methods using N. = clusters, we select the top 2-3 agents from the final training pool of each cluster to form the final population. We include the following baselines and ablations:
1. RANDOM: 100 morphologies are randomly sampled from the design space.
2. DERL 4
We the publicly released set of N 100 agents evolved flat terrain. DERL introduces a scalable framework for evolving morphologies the UNIMAL design space using bi-level optimization strategy: reinforcement learning loop trains a separate MI.P policy for each agent for over 5 million steps. DERL employs distributed, asynchronous evolutionary search parallelize this process, evolving and training 4,000 morphologies over an average of 10 generations to select the top 100.
outer evolutionary loop mutates designs, while an inner
A Quality-Diversity (QD) baseline where the morphology space is discretized into N₂ 40 niches using K-means clustering on raw morphology parameters. Each cluster serves as a cell in the MAP-Elites archive maintaining 3 elites throughout evolution. The algorithm explores the same number of morphologies as DERL (40 generations x 100 morphologies per generation=4000), training a separate MLP policy for each agent for 5M steps.
MAP-ELITES 23 49
4. LATENT-MAP-ELTEER Similar to MAP-ELITES but morphology space is discretized into No 40 niches using K-means clustering in the VAE latent space rather than raw parameters.
LOKI (w/o CLUSTER): An ablation of our method with no clustering. Morphologies are evolved over 40 independent runs, each initialized with pool of N,, agents sampled randomly from the full design space rather than within clusters.
LOKI: Our full co-design framework using N₂ = 40 clusters in the VAE latent space.
4.1 Performance and Diversity of the Evolved Designs
The design space includes both fast and slow learning morphologies, requiring varying amounts of interaction to master locomotion on flat terrain. Training single-design controllers from scratch is constrained by limited budgets-c.. e.g. DERL [4] trains each morphology using MLP policies
7

---

Obstacle (n-50)
Obstacle (n-150)
Bump
Patrol
0.5
32.8
Reward
2038
17.4
1506
Too Agent(s)
Top Agani(ts
Agent(()
Top Agent(s)
Incline
Push box incline
Manipulation bell
Exploration
3003
Reward
2004
2540
2003
200
Agent(s)
Ageniis)
Ageniss)
Ageniss)
Harsdam
DEAL
LOG Into cluster)
LOKI
Figure 6: Morphology-level task adaptability. We evolve diverse morphologies on flat terrain that generalize effectively to unseen tasks requiring varied skills. DERL morphologies overfitted flat terrain and perform best on obstacle (n=50) and incline, which are structurally similar. LOKI shows significantly better adaptability on bump (981 + 1908), push box incline (1519- + 3148), manipulation ball (142 + 172) enabled by its morphological diversity (c.g., crawlers, crabs) and the emergence of more complex behaviors (e.g., spinning, rolling).
Obstacle (n=150)
Incline
Bump
Push box incline
2000
5000
5000
800
Reward
1900
200
1000
100
600
1000
200
-00
-200
0
20M
K0M
Step
EDV
403
Step
S3M
BBV
100M
204
Step
62V
12M
LOKI (top clusters)
DERL
Figure 7: Policy-level task adaptability. Our co-evolution framework not only produces a diverse set of morphologies but also cluster-specific policies that generalize more effectively to unseen tasks. Each cluster captures distinct morphologies and behaviors, enabling its policy better adapt tasks aligned with those traits.
for 5 million steps, which is often insufficient to learn more complex behaviors, such as spinning (see Appendix BB. Global competition causes the algorithm to favor faster learners like cheetah- like morphologies, reducing the diversity of behaviors. In contrast, LOKI promotes both fast and slow leamnrrs by localizing competition within clusters, using powerful cluster-specific policies, and incorporating Dynamic Local Search (DLS) to encourage broader exploration within each cluster. This leads to greater behavioral and morphological diversity (Fig. 2
In Tab. 2. we compare LOKT for locomotion on flat terrain (FT) against baselines using three core metrics commonly used to evaluate Quality-Diversey (QD) algorithms [9]: Maximum Fitness (the highest reward achieved by any agent in the final population), Coverage (the number of clusters filled by the final population), and QD-Score (sum the highest fitness values discovered in each cluster, capturing both quality and diversity). For fair comparison of QD-score and coverage, assign top 100 agents from each baseline to No=40 latent clusters (details in Appendix [
While LOKI achieves slightly lower maximum fitness than DERL and MAP-ELITE it obtains the highest QD-Score, demonstrating ability to find high-performing solutions across wide range of ecological niches. Although MA-NT--MPEELLTTT the same latent clusters, it struggles to discover high-quality solutions many of them, resulting in significantly lower Qc-rcore. DERL, on the other hand, is optimized solely for fitness, not
achieves full coverage by evolving agents within
8

---

diversity. As a result, it overfits to the training task (flat terrain locomotion) and lacks behavioral variety- its low coverage score of 37.5%, with only 15 out of 40 clusters filled.
Our ablations further highlight the importance of clustering. When clusters are too large and contain morphologies with different behavior types, the shared policy struggles generalize effectively, which explains the performance drop at N = 10 and N. = 20.
Fig. 5 compares the sparseness 120] of solutions as novelty metric. A simple measure of sparseness p point is p(x:) & E dist (x, where μ4 is the ith-nearest neighbor of in the morphology latent space. LOKI achieves the highest sparseness among the three learning- methods, closely approaching RANDOM. This is largely due to LOKI'S use of Dynamic Local Search (DLS) to sample diverse designs within each cluster, as opposed to the incremental mutations used in DERL and LATENT-MAP-ELINEE which tend produce solutions in local neighborhoods.
4.2 Transitioning to New Tasks and Environments
Morphology-Level. LOKI produces a diverse set of solutions that are better suited for adaptation to various unseen tasks and environments, reducing the need to re-evolve agents for each setting, We evaluate generalization across eight test tasks, each requiring distinct set of skills. For each method, the final set of N = 100 evolved morphologies (elites) is independently trained from scratch on each test task using MIP--based policies, with 5 random seeds and training durations of 5, 15, or 20 million steps depending on task difficulty. Obstacle (n=150) is more challenging with three times more obstacles than obstacle (n=50). In the bump task, agents must learn behaviors like jumping or climbing without explicit information about the bump's ' location. All tasks except obstacle (n=150) and hump were originally introduced in [4 (more details in Appendix D).
Fig. ØJshows the cumulative mean reward of the top 10 agents from each method on each task. LOKI outperforms the DERL baseline on 6 out of 8 tasks, indicating that the diverse set of morphologies evolved for flat terrain transfers more effectively to new environments. In contrast, DERL agents exhibit limited behavioral diversity and are overfitted the training task-they perform best on obstacle (n=50) and incline, likely because these tasks are structurally similar to flat terrain. LOKT achieves large gains on tasks that differ more significantly from locomotion, such as bump (981 - 1908), push box incline (1519 + 3148), and manipulate ball (142 - 172). These improvements stem from the emergence of diverse morphologies (e.g., crawlers, crabs) and complex behaviors (c.g.. spinning, rolling) that DERL and MAP-ELITER fail to discover (see Fig. 2
Polecy-Level. Each multi-desige policy is trained cluster using a dynamic pool that is updated with new designs every few iterations. We evaluate the generalization capabilities of these policies on four new tasks. Since each cluster captures distinct subset of morphologies and behaviors, the corresponding policy is expected to adapt more effectively to tasks aligned with those traits.
large number of morphologies within its
We fine-tune the top 10 cluster-specific new task. As baseline, we pretrain the Transformer policy architecture on the top N, 20 morphologies evolved by DERL on flat terrain and fine-tune similarly. Fig. shows that for cach task, certain morphologies are inherently aligned with the skills required. This highlights the benefit of co-evolving diverse morphologies alongside specialized controllers within structurally coherent clusters. The intra-cluster similarity further aids generalization, enabling policies to learn transferable behaviors. Overall, framework not only produces diverse morphologies but also yields cluster-specific policies that generalize more effectively to unseen tasks.
on flat-terrain locomotion-og on each
policies adapt significantly better after fine-tuning, as their corresponding
Conclusion
We introduce LOKI, a compute-efficient co-design framework that discovers diverse, high-performing robot morphologies (divergent forms) using shared control policies (convergent functions). By clustering structurally similar morphologies, we train multi-design policies that enable behavior reuse and allow for the evaluation of significantly more designs without retraining. Morphologies are co- evolved with policies using Dynamic Local Search (DL.S) rather than incremental mutations, allowing broader exploration within each cluster. Localized competition combined with broader exploration

---

leads LOKI to outperform Quality Diversity (QD) algorithms in evolving diverse locomotion strategies. Our agents also generalize better to unseen tasks compared to prior evolution-base and QD methods.
Limitations
One limitation of method is that the number of clusters- consequently, the diversity of discovered solutions- be determined at the start of training. While this could be partially addressed by adding new morphology clusters later to better cover poorly performing regions, doing so may require costly retraining. Additionally, training costs scale linearly with the number of clusters. In future work, this limitation could be alleviated by introducing an adaptive mechanism that dynamically grows or shrinks the number of clusters based on their performance during training.
References
[1] Francisco J Varela, Evan Thompson, and Eleanor Rosch. The embodied mind, revised edition; Cognitive science and human experience. MIT press, 2017.
[2] Emma Hjellbrekke Stensby, Kai Olav Ellefsen, and Kyrre Glette. Co-optimising robot morphol- ogy and controller in simulated open-ended environment. In Applications of Evolutionary Computation: 24th International Conference, EvoApplications 2021, Held as Part of EvoStar 2021, Virtual Event, April 7-9, 2021, Proceedings 24, pages 44449. Springer, 2021.
[3] Julius Wolff. The law of bone remodelling. Springer Science & Business Media, 2012.
[4] Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1))5721, 2021.
[5] Tingwu Wang. Renjie Liao, Jimmy Ba, and Sanja Fidler. Nervenet: Learning structured policy with graph neural networks. In International conference on learning representations, 2018.
[6] Charles Schaff, David Yunis, Ayan Chakrabarti, and Matthew R Walter. Jointly learning to construct and control agents using deep reinforcement learning. 2019 international conference on robotics and automation (ICRA), pages 9798-98888 9805. IEEE, 2019.
[7] Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei Efros. Learning to control self-assembling morphologies: study of generalization via modularity. Advances in Neural Information Processing Systems, 32, 2019.
[8] Kevin Sebastian Luck, Heni Ben Amor, and Roberto Calandra. Data-efficient co-adaptation of morphology and behaviour with deep reinforcement learning. In Conference Robot Learning, pages 854-869, PMLR, 2020.
[9] Tingwu Wang, Yuhao Zhou, Sanja Fidler, and Jimmy Ba. Neural graph evolution: Towards efficient automatic robot design. arXiv preprint a9Xiv: 05370, 2019.
[10] Wenlong Huang, Igor Mordatch, and Deepak Pathak. One policy to control them all: Shared modular policies for agent-agnostic control. International Conference on Machine Learning, pages 4455-4464. PMLR, 2020.
[11] Ye Yuan, Yuda Song, Zhengyi Luo. Wen Sun, and Kris Kitani. Transform2act: Learning transform-and-contro policy for efficient agent design. arXiv preprint arXiv:2/10.0366 2021.
[12] Vitaly Kurin, Maximilian Igl, Tim Rocktäschel, Wendelin Boehmer, and Shimon Whiteson. My body is a cage: the role of morphology in graph-based incompatible control. arXiv preprint arXiv:2010001856. 2020.
[13] David Ha. Reinforcement learning for improving agent design. Artificial life, (5(4))352-36 2019.
[14] Karl Sims. Evolving virtual creatures. Proceedings of the 21 annual conference on Computer graphics and interactive techniques, pages 15-22, 1994.
10

---

[15] David E Goldberg. Genetic algorithms and walsh functions: Part 2, deception and its analysis. Complex systems, 3:153-171, 1989.
[16) Gregory S Hornby. Alps: the age-layered population structure for reducing the problem of premature convergence. In Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 815-822, 2006.
[17] Leonardo Trujillo, Gustavo Olague, Evelyne Lutton, Francisco Fernandez de Vega, León Dozal, and Eddic Clemente. Speciation in behavioral space for evolutionary robotics. Journal of Intelligent & Robotic Systems, 64:323-351, 2011.
[18] Leonardo Trujillo, Gustavo Olague, Evelyne Lutton, and Francisco Fernández de Vega. Discov- cring several robot behaviors through speciation. In Workshops on Applications of Evolutionary Computation, pages 164-174, Springer, 2008.
[19] Justin K Pugh, Lisa Soros, and Kenneth O Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016.
[20] Joel Lehman and Kenneth O. Stanley. Evolving a diversity of virtual creatures through novelty search and local competition. In Annual Conference on Genetic and Evolutionary Computation, 2011.
[21] Konstantinos Chatzilygeroudis, Antoine Cully, Vassilis Vassiliades, and Jean-Baptiste Mouret. Quaeity-diversity optimization: novel branch of stochastic optimization. In Black Box Opti- mization, Machine Learning, and No-Free Lunch Theorems, pages 109-135. Springer, 2021.
[22] Vassilis Vassiliades, Konstantinos Chatzilygeroudis, and Jean- SannBBapistte Mouret. Using centroidal voronoi tessellations scale up the multidimensional archive of phenotypic elites algorithm. IEEE Transactions on Evolutionarr Computation, 22:623-633 630, 2016.
[23] Jean-Bape Sann-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. ArXiv, Abs/1504.04999, 2015.
[24] Jean-Baptiste Mouret. Evolving the behavior of machines: from miero to macroevolution. Iscience, 23(11), 2020.
[25] Holger H. Hoos and Thomas Stützle, Stochastic Local Search Algorithms: An Overview, pages 1085-1105, Springer Berlin Heidelberg, Berlin, Heidelberg, 2015.
[26] Frank Hutter, Dave A. D. Tompkins, and Holger H. Hoos. Scaling and probabilistic smoothing: Efficient dynamic local search for sat. In International Conference Principles and Practice of Constraint Programming, 2002.
[27] W. Pullan and H. H. Hoos. Dynamic local search for the maximum clique problem. Journal of Artificial Intelligence Research, 25:159-185, February 2006.
[28] Hans Peter Moravec. Obstacle avoidance and navigation in the real world by seeing robot rover. Stanford University, 1980.
[29] Artificial Intellgence Center. Shakey the robot. 1984.
[30] Timothy P Lillicrap. Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint
2015.
[31] Nicolas Manfred Otto Heess, TB Dhruva, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyun Wang, S. M. Ali Eslami, Martin A. Riedmiller, and David Silver. Emergence of locomotion behaviours in rich environments. ArXiv, abs/1707. 707.02286. 2017.
[32] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.

---

[33] Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprini arXiv: 18/200590, 2018.
[34] Heng Dong, Junyu Zhang, Tonghan Wang, and Chongjie Zhang. Symmetry-awand robot design with structured subgroups. In International Conference on Machine Learning, pages 8334 8355, PMLR, 2023.
[35] Muhan Li, Lingji Kong, and Sam Kriegman. Generating freeform endoskeletal robots. In The Thirteenth International Conference on Learning Representations, 2025.
[36] Karl Sims. Evolving 3d morphology and behavior by competition. Artificial life, 144):353-372, 1994.
[37] Haofei Lu, Zhe Wu, Junliang Xing, Jianshu Li, Ruoyu Li, Zhe Li, and Yuanchun Shi. Bodygen: Advancing towards efficient embodiment co-design. In The Thirteenth International Conference on Learning Representations, 2025.
[38] Donald J Hejna, Pieter Abbeel, and Lerrel Pinto. Tank-agnostic morphology evolution. In 9th International Conference on Learning Representations, ICLR 2021, 2021.
[39] Reiko Tanese. Distributed genetic algorithms for function optimization. University of Michigan, 1989.
[40] Theodore Belding. org/9504007, 1995.
The distributed genetic algorithm revisited. arXiv preprint adap-
[41] Darrell Whitley, Soraya Rana, and Robert B Heckendorn. Island model genetic algorithms and linearly separable problems. In Evolutionary Computing: AISB International Workshop Manchester, UK, April 7-8, 1997 Selected Papers, pages 1255 Springer, 1997.
[42] David E Goldberg, Jon Richardson, et al. Genetic algorithms with sharing for multimodal function optimization. In Genetic algorithums and their applications: Proceedings of the Second International Conference on Genetic Algorithms, volume 4149, pages 425- Lawrence Erlbaum, Hillsdale, NJ, 1987.
[43] Christopher K Oci, David Goldberg, and Shau-Jin Chang. Tournament selection, niching, and the preservation of diversity. IlliGAL report, 91011, 1991.
[44] Bruno Sareni and Laurent Krahenbuhl. Fitness sharing and niching methods revisited. IEEE transactions on Evolutionary Computation, 2(3):97-106, 1998.
[45] John Cartlidge and Seth Bullock. Caring versus sharing: How maintain engagement and diversity in coevolving populations. In European Conference on Artificial Life, pages 299-308. Springer, 2003.
[46] Melanie Mitchell, Michael D Thomure, and Nathan L Williams. The role of space in the success of coevolutionary learning. In Artificial life X: proceedings of the Tenth International Conference the Simulation and Synthesis of Living Systems, pages 118-124. The MIT Press (Bradford Books) Cambridge, 2006.
[47] Antoine Cully and Y. Demiris. Quality and diversity optimization: A unifying modular frame- work. IEEE Transactions Evolutionary Computation, 22:245-2555 2017.
[48] Peter Krčah. Solving deceptive tasks in robot body-brain co-evolution by searching for be- havioral novelty. In 2010 10th International Conference on Intelligent Systems Design and Applications, pages
IEEE, 2010.
[49] Vassilis Vassiliades, Konstantinos Chatzily and Jean -aantttpisst Mouret. Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic clites algorithm. IEEE Transactions on Evolutionary Computation, 22(4):623-630, 2017.
[50] Vassiis Vassiliades and Jean-Baptiss Mouret. Discovering the elite hypervolume by lever- aging interspecies correlation. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 149-156, 2018.
12

---

[51] Cédric Colas, Vashisht Madhavan, Joost Huizinga, and Jeff Clune. Scaling map-elites to deep neuroevolution. Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pages 67-75, 2020.
[52] Thomas PIERROT and Arthur Flajolet. Evolving populations of diverse RL agents with MAP-elites. The Eleventh International Conference on Learning Representations. 2023.
[53] Adam Gaier, Alexander Asteroth, and lean-Baptiste Mouret. Data-efficient exploration, opti- mization, and modeling of diverse designs through surrogate-assisten illumination. In Proceed- ings of the Genetic and Evolutionary Computation Conference, pages 99-106, 2017.
[54] Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. Robots that can adapt like animals. Nature, 521(7553):503-507 2015.
[55] Jørgen Nordmoen, Frank Veenstra, Kai Olav Ellefsen, and Kyrre Glette. Map-elites enables powerful stepping stones and diversity for modular robotics. Frontiers in Robotics and AI, 8:639173, 2021.
[56] S. Lloyd. Least squares quantization in pem. IEEE Transactions on Information Theory. 28(2):129-137, 1982.
[57] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30:5998-6008, 2017.
[58] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1312v1131..61144 14, 2013.
[59] Thomas H Cormen, Charles E Leiserson, Ronald I. Rivest, and Clifford Stein. Introduction to algorithms. MIT press, 2022.
[60] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017.
[61] Hengrui Zhang. Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. In The Twelfth International Conference on Learning Representations, 2024.
[62] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using I-SNE. Journal of machine learning research, 9(11), 2008
[63] Agrim Gupta, Linxi Fan, Surya Ganguli, and Li Fei-Fei. Metamorph: Learning universal controllers with transformers. arXiv preprint arXiv:2203.1192: 2022.
[64] Manon Flageat, Bryan Lim, Luca Grillotti, Maxime Allard, Simón C Smith, and Antoine Cully. Benchmarking quality-diversity algorithms on neuroevolution for reinforcement learning. arXiv preprint arXiv:221 093, 2022.
[65] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf. Edward Yang. Zach De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library, 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction clearly state the claims and contributions that match our experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have explicit limitation section (Sec. 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not include theoretical results.
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
Code is available on the project website to reproduce the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open tions to faithfully reproduce the main experimental results, as described in supplemental material?
the data and code, with sufficient instruc-
Answer: [Yes]
15

---

Code with instructions to reproduce the main results are available on the project website.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines tttttp://niis.coo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (ht
//nips. httpc/cccccccggudeeeeeeeeeeeee/e
ssionsolicyyyyc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experiment details and hyperparameters are mostly provided in the Appendix. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Max fitness in Tab. is shown with the standard errors, and the results of Fig. is evaluated using the independent samples t-test.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources required for the experiments are provided Sec. 3.1 and
Sec. 3.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httppp////bbrrrsssccc///a///ttte
Answer: [Yes]
Justification: Research in this paper conform with the Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Societal impacts are discussed Appendix ! Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original papers that produced any code data used in our paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, |paper swi thcode. com/ conasaatsess has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy https////eeerroo.cccoooooeeeesss for what should or should not be described.
20

---

Appendices for Convergent Functions, Divergent Forms
The following items are provided in the Appendix:
Benefits of clustering the design space in the learned latent space instead of using raw
morphology parameters (ApplA)
Analysis of the locomotion learning speed of our evolved agents (fast and slow learners) (ApplB)
Procedure for sampling random morphologies within the UNIMAL space (App
Detuiled description of the different test tasks (App
Detailed description of the QD-score metric (App
Description of how efficiency metries Tab are calculated (App
Eyeudo-code for the Map-Elite Elites baseline G
Full hyperparameter details for the experiments (App. H
Broader Societal Impacts (App- !
On our website loki-codesign.githubb. we have
Videos showing the diverse locomotion behaviors of LOKI's evolved agents
Videos of the evolved agents transferred to new test tasks (Obstacle, Bump, Incline, Push box incline, Manipulation ball, Exploration)
A Benefits of Clustering in the Learned Latent Space
To investigate the role of latent-space clustering versus clustering based on morphology pa- rameters, we conduct an ablation study: LOKT www.wwww.A........... a variant of our method in which latent-space clusters replaced with clusters computed from raw morphology parameters, while keeping the number of clusters fixed at N, 40.
Fig overlays the cumulative mean reward of the top 10 agents from LOKI RAW-PARAM- and MAP-ELITES (w/ PARAM-UUSSSEE on top of Fig in the main paper. Notably, LOKI outperforms
LOKI (wl RAW--ARAM-CUUSIE across all tasks, indicating that co-evolution framework benefits from structuring the morphology space via a learned latent representation. In contrast, clustering in the raw parameter space fails to capture structural and behavioral similarities, leading to poor generalization of
multi-design policies within each cluster. Tab. 3 also compares LOKT against LOKI across the quality-diversity metrics.
(w// RAW-PAR AMTTUUSTR
B Learning Speed of the Evolved Morphologies (Fast & Slow Learners)
We examine the correlation between learning speed and the final performance of evolved agents. To evaluate performance across different training durations, we train single-agent MLP policies 4 for both 5M and 15M steps on flat terrain. This evaluation excludes agents from clusters with consistently low training rewards or predominantly simple morphologies. As shown in Fig. g the top-performing agents under short- and long-term training largely disjoint, revealing the presence of both fast- and slow-learning morphologics. Notably, high short-term performance does not necessarily indicate high long-term performance some slow learners achieve superior results after 15M steps despite underperforming at 5M steps.
Prior approaches such as DERL train single-agent policies for a fixed number of steps (e.g., 5M), which biases evolution toward fast learners-ofie resulting in morphologies dominated by cheetah- like forms. In contrast, LOKI does not rely short-term training. It leverages multi-design transformer policies trained over significantly longer durations, enabling the discovery of both fast and slow learners across the morphology clusters.
21

---

Table 3: LOKI benefits both quality and diversity from clustering a structured morphology latent space. One new baseline, LOKI (wi wWw.IAAM marked with (9), is added to Tab. 2 to assess the impact of clustering in raw parameter space. (= denotes standard error across training seeds.)
Method
Max Fitness
QD-score
Coverage %)
RANDOM
3418.9 +: 3908
32.1
90
DERL
5760.8 + 248.2
26.2
37.5
PLUSSR-
5807.3 + 1965
43.5
65.0
LATENT-MAP-ELITEE
5257.0 +: 491.3
38.1
100
LOKI CLUSIR))
4825.8 76.1
40.0
75
LOKI (N. 10)
4008.0 + 5544.0 +: 208.
21.6
47.5
LOKI (N: 20)
26.6
45.0
LOKI (w/.ENN-AAAAMCCUUSIEE)
4055.0 + 523.7
27.5
62.5
LOKI (N. 40)
5671.9 +: 36011
60.9
100
Obstacle (n=50)
Obstacle (n=150)
Bump
Patrol
Rewrrrd
275
2500
2234
2010
Agentts)
Agentisl
Agenttss
Too Agent(s)
Incline
Push box incline
Manipulation bell
Exploration
Reward
3003
Too Agentes)
Top N Agontisi
Top N Agentts)
Top Agandes)
Fandom
MAP- MAP-Eltex
DCTI
LOS (ats duster
LD4 pww.accctttttter
LOKI
Figure 8: Morphology-level task adaptability. Two baselines (MAP-ELITER PAA LOKI (W/ PARAA-C are added Fig. to assess the impact of clustering in raw parameter space.
C Sampling Random Morphologies within the UNIMAL Space
The process of sampling random morphologies follows the procedure introduced in prior work [4] During the population initialization phase, new morphology is created by first sampling the total number of limbs to grow, followed by a series of mutation operations until the desired number of limbs reached. These mutations include: growing or deleting limbs, mutating limb parameters, density, degrees of freedom (DoF), gear ratios, and joint angles. For each mutation, the parameters are uniformly sampled from predefined ranges specified in [4. The key difference is that DERL samples parameter values from discrete sets, while LOKI samples continuously within each range, enabling a denser and more comprehensive coverage of the morphology space. Table[4preeents the parameter ranges used to create our morphologies. The notation range(a, b) denotes continuous range from a to
D Test Task Descriptions
We describe the task specifications and required skills for the eight test tasks used in our morphology- level evaluation. All tasks, except for Bump and Obstacle (5=) were introduced in prior work We adopt the same task configurations and reward structures outlined therein.
22

---

Top 2 (15M)
Thp 19M) Top 20% 113M) median
6000
lop 2 15M0 (hrrh)
4000
3000
2000
1000
9.201112
1718 2021 232425 77282/000311
Cluster Indeo
Figure 9: Training rewards across short- and long-term learning. Final rewards for LOKI's agents after 5M and 15M training steps. The top-pen porprrrrnnnnn agents short and long training horizons are largely disjoint. High short-term performance does not guarantee long-term success-some slow learners significantly outperform fast learners after extended training.
Parameters
Sampling Range
Max limbs
11
Limb radius
range(0.00,, 0.06)
Limb height
range(0.2, 0.4)
Limb density
range(500,, 1000)
Limb orientation
[0, 45, 90, 135, 180, 225, 270, 315]
Limb orientation Ø
[90, 135, 180]
Head radius
0.10
Head density
range(500, 1000)
Joint axis
[x, y, xy] range( 150, 300)
Motor gear range
Joint limits
[(-30, 0), (0, 30), (-30, 30), (-45, 45), (-45, 0), (0, 45),
(0, -60), (0, 60), (-60, 60), (-90, 0), (0, 90), (-60, 30), (-30, 60)]
Table 4:
Design parameters used for sampling random morphologies in the UNIMAL space.
The agent must repeatedly traverse between two target points separated by 10m along the x-axis. High performance in this task requires rapid acceleration, short bursts of speed, and quick directional changes. (Training steps: 5 million)
Patrol.
Incline. The agent operates in 150 x 40m² rectangular arena inclined at 10-degree angle. The agent is rewarded for moving forward along the axis. (Training steps: 5 million)
Push Box Incline. The agent must push a box with a side length of 0.2m up an inclined plane. The environment is 80 40m² rectangular arena tilted 10-degree angle. The agent starts of the arena and tasked with propelling the box forward along the slope. (Training steps: 5 million)
end
Obstacle (n=50, 150). The agent must navigate through cluttered environment filled with static obstacles to reach the end of the arena. Each box-shaped obstacle has a width and length between 0.5m and 3 with fixed height of 2 m. denotes the number of randomly distributed obstacles across a flat 150 x 60m² terrain. (Training steps: million)
Bump. The agent must traverse arena filled with 250 low-profile obstacles randomly placed on 150 x 60m terrain. Each obstacle has width and length between 0.8m and 1.6m, and a height between 0.1m and 0.25m. As obstacle height is comparable to the agent's body, this task promotes behaviors such as jumping or climbing, adding complexity to locomotion and body coordination. (Training steps: 15 million)
23

---

Manipulate Ball. The agent must move a ball from random source location to fixed target. A ball of radius 0.2 placed at random location in a flat, square 30 30m² arena, with the agent initialized at the center. This task requires fine interplay of locomotion and object manipulation, as the agent must influence the ball's motion through contact while maintaining its own balance and stability. (Training steps: 20 million)
The agent begins at the center of flat 100 100m² arena divided into 1 1m² grid cells. The goal is to maximize the number of unique grid cells visited during an episode. Unlike previous tasks with dense locomotion rewards, this task provides sparse reward signal. (Training steps: 20 million)
Exploration.
E QD-Score (a quality-diversity metric)
QD-score 64
is more comprehensive metric than maximum fitness, as captures not only the performance of the single best agent but also the diversity of segshppeforming solutions across the search space. Given the vast combinatorial complexity the UNIMAL space, QD-score is particularly well-suited for evaluating the quality and spread of evolved morphologies.
19 55
In Tab. report the percentile QD-score computed N 40 latent morphology clusters, defined as:
1 No
N 1 M
fi - fmin
QD-score
fmax
Smin
1=1
Here, .M, denotes the set of evolved morphologies allocated to the i-th cluster, and fi is the mean fitness the best-pes pest-performing agent in that cluster. fmx and fmin represent the maximum and minimum mean fitness values across all final population clusters, respectively. This normalization ensures comparability across clusters with different fitness scales.
F Efficiency Metrics in Table
This section defines the efficiency metrics reported in Tab. 1
Number of interactions. The number interactions refers to the total number of environment steps taken by all agents throughout the entire evolutionary process.
In our multi-desige evolution framework, two types of interactions are counted: (1) trajectories collected from agents in the training pool, which are used to train the shared multi-design policy via PPO, and (2) evaluation episodes conducted during each drop-off round.
The total number of interactions is calculated as:
Total interactions
No (# interactions (per cluster) + # evaluated samples episode length)
Niser
No 100M +
NNsaplee
Leval
fai
4.62B
In contrast, DERL trains 4,000 agents independently using single-agent MLP policy, with each agent trained for 5M environment steps- resulting in a total of 20B interactions. Since DERL uses the training reward directly for agent selection, no separate evaluation phase is required.
Number of searched morphologies. The total number of morphologies explored across the mor- phology space is calculated as follows:
Nise
# of covered morphologies (per cluster)
<
NNsapll + No,, RJ 78,000
(3)
fain

---

# of searched morphologies # of covered morphologies (per cluster) Noc
FJ 78,000 40
3.1
Note that each morphology is evaluated approximately times on average. These repeated evaluations are valuable because early-stage policies may be unreliable. Re-evaluating morphologies with progressively improved policies allows for the discovery of higher-performing behaviors.
FLOPs per searched morphology. Given the MLP and Transformer policy model architectures used in prior work [ 63), we compute the training FL.OPs per searched morphology shown in Eq. 5 with the values Tab.
FLOPs (per step)
X FLOPs (per forward pass) Batch size
FLOPs model) FLOPs (per step) x PPO epochs (# of iterations)
FLOPs (per morphology)
FLOPs (per model)/ /(# searched morphologies)
(5)
DERL employs a single-agent MLP policy, resulting in a per-model training compute of x 31.9k x 512 x 4 x 1220 159B FLOPs. In contrast, LOKI uses a multi-agent Transformer policy, which incurs higher compute per forward pass, as well as larger batch sizes and more training epochs. However, this higher cost is offset by the fact that each trained model serves large number of morphologies. As a result, the total training compute per searched morphology amortized and given by 7x 2x79.5ff51200 x5120x8xx 1220 102B FLOPs. Despite the higher overall training cost of the 78,000
Transformer policy compared to the MLP, its shared usage across large number of morphologies leads to more sample-efficient training, resulting in approximately 40% lower compute cost per morphology.
Table 5: Comparison of total FLOPs required for MLP and Transformer policy architectures.
Model
FLOPs (per forward pass)
Batch size
PPO Epochs
evaluated morphologies (per model)
ML.P
31.9K
512 5120
Transformer
79.5M
8
78,000
G MAP-Elites
We provide the algorithm for the Map-Elites
23
49
baseline in Alg.
2
25

---

Algorithm 2 MAP-ELITES
1: Input:
M: MAP-Elites repertoire for agent design
W: UNIMAL morphology space
g: Cluster classifier based on morphology parameters
Nuain: Total number of agents to train
M: Number of offispring per generation
S: Number of interaction steps for training each MLP policy
2: // Initiall a ion
3: Randomly sample M initial morphologies {uf) Cu
4: Train single-agent MLP policies [T:] on their respective morphologies for S steps
5: Evaluate fitness {fP M
Ji=1 via one episode roll-out per trained policy
6: Insert {u]] MM
== into M using fitness ( [/J
and cluster assignments {g (g("))]
7: while [M| < Nirai do
8: // Reproduction via mutation
9: Sample M clite morphologies {u] from M without replacement
10: Apply random mutation to produce M offspring morphologies {ū² )
// Train and evaluate
morphologies
12:
Train MLP policies {T² ) for S steps
13:
Evaluate fitness {, LM via one episode per policy
Insert { into M using fitness {f: j and clusters {g(
14:
end while
26

---

H
Implementation Details
Detailed hyperparameters provided in Tab. 6 and Tab.
Table 6: Hyperparameters of LOKI.
Name
Value
# of samples for K-means
x 10
# of clusters No
10, 20, 40
Niter
1220
fan
Nw
20
Naterr
2
Stochastic Multi-Design Evolution
N sample
128
of parallel environments
32
Total interactions
10a
Timesteps per PPO rollout
2560
PPO epuchs
Training episode length Iaan
1000
Evaluation episode length leval
200
# of heads
# of layers
5
Batch size
5120
Feedforward dimension
1024
Multi-Design Transformer Policy
Dropout
0.0
Initialization range for embedding Initialization range for decoder Limb embedding size
[-0.1, 0.1]
[-0.01, 0.01]
128
Joint embedding size
128
Continuous feature embedding size
32
Depth feature embedding size
32
Latent dimension
32
# of heads
4
Feed-forward network hidden dimension
256
of layers
Opitmizer
Adam
Morphology VAE
Initial learning
10-4
Weight docay
10 10-5
Learning rate scheduler
RcdlceLLROnPlateau
65
Learning rate reduction factor
0.95
Learning rate reduction patience
10
of epochs
200
Batch size
4096
[Amin 3
[10 10
Table 7: Hyperparameters of MAP-ELITES.
Name
Value
of samples for K-means
5 x 10"
Clusters
40
Nuan
4000
100
S
5 x 10
I Societal Impacts
Positive impacts: Our work aims to challenge prevailing assumptions about optimal robot morphol- ogy, potentially reshaping how robotic design is approached. By promoting diversity and functionality
27

---

beyond conventional forms, LOKI encourages exploration of unconventional yet effective morpholo- gies. We believe this contributes toward a more inclusive and biologically inspired understanding of embodiment, potentially serving as a bridge between AI, robotics, and the natural sciences.
Negative impacts: A potential concern is that the ability to autonomously generate large number of novel and capable morphologies could be misused in contexts that prioritize performance over safety, For instance, this approach could enable rapid prototyping of morphologies for autonomous systems without adequate human oversight, increasing the risk of deploying untested designs in sensitive environments.
28


Reviews:
[{"review_id": "ghlMEapBkT", "reviewer": "Reviewer_uzg9", "summary": "This paper introduces a learning method to train a control policy for a cluster of morphologically similar designs, aiming to address the joint problem of co-designing morphologies and controllers.", "questions": "Please refer to the weaknesses section.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n+ The motivation for co-design of robot morphology and control is strong.\n\n+ The paper presents a working solution with promising results on a variety of tasks, including locomotion, crawling, and box pushing.\n\nWeaknesses:\n\nThe paper suffers from several critical issues:\n\n- The claim of co-design between morphology and control is questionable. The initial control policies are learned after the morphologies are clustered. The adapted policies also appear to be updated after the morphological evolution. Since the morphology is designed without considering the control, describing this as \"co-design\" is misleading.\n\n- The theoretical novelty and contribution are limited. The proposed approach sequentially applies a series of well-established techniques, such as VAEs, k-means clustering, and dynamic local search (DLS).\n\n- Since the VAE is not trained specifically for k-means, it is unclear why applying k-means clustering to the VAE embeddings can address the issue shown in Figure 3 (i.e., capturing topological similarity). What is the advantage of using VAE + k-means over more recent deep learning-based clustering methods?\n\n- The argument for diversity in addressing small local changes and avoiding local minima is unconvincing. The observed diversity mainly results from large-scale sampling of initial morphologies and DLS within each cluster. However, since DLS is applied within each cluster, it is unclear how this approach can theoretically overcome the limitations of small, incremental changes. Furthermore, although DLS may help avoid local minima within a cluster, the overall approach cannot escape local minima at the clustering level, as k-means is not guaranteed to do so.\n\n- The claim of efficiency is not well justified, especially when compared to co-design methods based on curriculum learning. The proposed method requires training a separate policy for each morphology cluster, with additional training for each update after DLS. While it may be efficient in terms of continued learning within a cluster, the lack of policy or knowledge sharing across clusters undermines its overall efficiency.", "was_revised": true, "final_justification": "I appreciate the authors’ feedback. While I recognize the potential of this work, I believe further effort is needed to strengthen its novelty and address several key aspects that remain unconvincing from a robotics perspective.", "rebuttal": "We thank Reviewer  **uzg9** for the constructive feedback. We appreciate your recognition of our paper’s strong motivation and promising results on a variety of tasks. Below, we address each concern you raised and clarify the perceived weaknesses.\n\n> ### 1. The claim of co-design between morphology and control is questionable.\nAlthough the reviewer is correct that our morphologies are predefined and we sample from this set during the evolutionary process, we argue that LOKI still qualifies as a co-design method. Co-design refers to the joint optimization of morphology and control for a given task, typically framed as a bi-level optimization problem: the outer loop searches over morphologies (often using evolutionary strategies), while the inner loop trains a control policy for each morphology to assess its performance [4]. Most existing methods alternate between these two loops. To simplify the optimization, some approaches integrate morphology refinement directly into the reinforcement learning process, enabling joint optimization in a single loop [11, 13].\nIn LOKI, we frame co-design as a **search problem** over a large, predefined space of morphologies. This space is generated through extensive random mutations to densely cover the design landscape. Instead of evolving new morphologies online, our design optimization loop samples from this rich, precomputed set. While the implementation differs from traditional co-design pipelines, it still falls under the broader definition of co-design. \n\n> ### 2. The theoretical novelty and contribution are limited.\nLOKI’s strength lies in introducing a compute-efficient framework for co-designing a diverse set of high-performing morphologies that generalize well to a variety of unseen downstream tasks. While the individual components are not novel, the overall framework effectively addresses key challenges in evolution-based co-design methods, such as **computational efficiency** and the **diversity** of final solutions.\n\n> ### 3. What is the advantage of using VAE + k-means over recent deep learning-based clustering methods?\nWe agree with the reviewer that the VAE is not explicitly trained for clustering. However, its latent space is designed to capture meaningful structure in the morphology space by learning a smooth, low-dimensional manifold. Applying k-means to the VAE embeddings offers a simple yet effective way to partition this space based on topological similarity.\nWhile more advanced deep clustering methods could potentially improve performance, their exploration is beyond the scope of this work. Our focus is on the overall framework rather than the specific choice of clustering algorithm. In fact, the effectiveness of LOKI despite using a simple clustering method underscores the robustness of our approach.\n\n> ### 4. The argument for diversity in addressing small local changes and avoiding local minima is unconvincing. \nThe key to LOKI’s broader exploration lies in how the initial set of morphologies is generated and utilized. Instead of relying on small, incremental mutations from current solutions, as is common in traditional evolutionary approaches, LOKI samples from a large, diverse pool of precomputed morphologies that densely cover the design space. This pool is created by repeatedly applying random mutations to produce a wide variety of designs. Dynamic Local Search (DLS) is then applied within each individual cluster to identify the best-performing morphologies. This sampling strategy allows for **non-local jumps across the morphology space**, helping to escape local minima that incremental mutations often fail to overcome.\nFurthermore, standard evolutionary methods with global competition tend to favor a single dominant behavior (e.g., faster learning), which can reduce behavioral diversity. Quality-Diversity (QD) methods address this by maintaining a repertoire of diverse solutions and localizing competition [20], allowing the algorithm to discover morphologies that are optimal within their own behavioral niches (e.g., spinning may lead to faster locomotion but requires more training to master). Similarly, LOKI restricts competition within clusters rather than across the entire population (e.g., spinners compete only with other spinners), preserving behavioral diversity across different regions of the morphology space.\n\n> ### 5. The claim of efficiency is not well justified. The proposed method requires training a separate policy for each morphology cluster.\nWhile LOKI requires training a separate policy for each cluster, it is still significantly more sample-efficient than prior evolution-based and Quality-Diversity approaches, which typically train a separate policy for **each individual morphology** to evaluate its fitness. In contrast, LOKI evaluates 500,000 morphologies using only 40 policies. Table 1 presents efficiency metrics compared to the most closely related baseline, DERL [4], highlighting LOKI’s advantage. Moreover, maintaining separate policies across clusters serves a purpose beyond efficiency: it localizes competition within each cluster, allowing the discovery of morphologies that are optimal within distinct behavioral niches.\nCo-design methods based on curriculum learning typically operate within more restricted and structured design spaces, such as modular voxel-based soft robots [A], and have been shown to converge to local morphological optima [B].\nIn contrast, LOKI is designed as an efficient evolutionary framework capable of navigating exponentially large morphology spaces (e.g., UNIMAL) while preserving diversity and avoiding premature convergence to local optima. Although LOKI trains a separate policy for each cluster, this approach enables scalable and diverse policy learning across a wide range of morphologies, without relying on restrictive assumptions or handcrafted curricula.\n\n- #### [A] Yuxing Wang et al. Curriculum-based Co-design of Morphology and Control of Voxel-based Soft Robots. In International conference on learning representations, 2023.\n- #### [B] Yuxing Wang et al. PreCo: Enhancing Generalization in Co-Design of Modular Soft Robots via Brain-Body Pre-Training. In the ​​proceedings of The 7th Conference on Robot Learning, PMLR 229:478-498, 2023.\n\n**We hope to address all the reviewer's concerns, and would be grateful if after reviewing our response you would consider raising your score.**"}, {"review_id": "2batSuuY2n", "reviewer": "Reviewer_T5wK", "summary": "This paper proposes LOKI, an efficient framework for co-designing robot morphologies and control policies that generalize across unseen tasks. Inspired by biological adaptation, LOKI clusters morphologically similar designs within a latent space learned by a transformer-based VAE. Within each cluster, a multi-design policy is trained to generalize across similar morphologies, allowing evaluation of many more morphologies without retraining. Additionally, LOKI leverages dynamic local search (DLS) instead of mutation, enabling broader exploration and preventing premature convergence. Experiments conducted in the UNIMAL design space demonstrate that LOKI significantly reduces computational cost while discovering diverse, high-performing morphologies that generalize better to unseen downstream tasks compared to previous methods.", "questions": "- Transformer-based VAE Loss:\nThe transformer-based VAE currently only uses a reconstruction loss (with an adaptive KL weight). However, two morphologies that are structurally very similar (e.g., a humanoid with a slightly shorter leg) may exhibit drastically different locomotion behaviors. Would incorporating a behavioral or performance-aware loss component into the VAE training improve the latent representation for clustering? Please clarify or justify the current loss choice.\n\n- Number of Clusters and Relation to Population-based Training:\nThe paper uses clustering to divide the morphology space into multiple clusters, each with its own policy. Could the authors clarify how the choice of cluster number relates conceptually and practically to population-based training methods? What are the trade-offs involved, and how sensitive is the performance of LOKI to the number of clusters chosen?\n\n- Baseline Selection:\nCould the authors justify the rationale for using QD-based methods (such as MAP-Elites) as primary baselines rather than recent Brain-body co-design approaches that share similar goals, claims, and technical frameworks (e.g., transformer-based morphology encoding, improved generalization, diverse solutions)? A careful justification or additional comparisons with recent Brain-body co-design approaches would significantly strengthen the manuscript.\n\n- Visualization of Morphological Diversity:\nCould the authors include additional qualitative visualizations (beyond numerical metrics) clearly showing the diversity of morphologies generated by LOKI for a particular task? Such visualizations would greatly enhance the clarity and impact of the paper.", "limitations": "See weaknesses", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n \n- The paper creatively integrates clustering techniques, transformer-based latent representations, and dynamic local search to improve efficiency and diversity in robot morphology optimization.\n- Computational Efficiency: LOKI demonstrates substantial computational efficiency gains (78% fewer simulation steps, 40% fewer compute per design) compared to previous evolutionary methods, allowing exploration of significantly larger design spaces.\n- Generalization and Transferability: The approach explicitly focuses on generalization across morphologies and demonstrates strong zero-shot transfer to new tasks, addressing a known limitation of previous co-design frameworks.\n\nWeaknesses\n \n\n- Insufficient Literature Survey and Novelty Clarification:\nThe literature survey in the current manuscript needs improvement, especially regarding recent Brain-body co-design research. In the past two years, multiple relevant papers have proposed frameworks that also claim efficient training, diverse solutions, and zero-shot transfer using transformer-based morphology encodings. Notable examples include:\n\n[1] \"Curriculum-based co-design of morphology and control of voxel-based soft robots\"\n\n[2] \"Accelerated co-design of robots through morphological pretraining\"\n\nThe authors should more clearly position their work relative to these recent contributions and explicitly clarify their specific novelty and contributions.\n\n- Baseline Comparisons:\nThe current baseline comparisons are somewhat problematic. The paper primarily compares against DERL and Quality-Diversity (QD)-based methods (such as MAP-Elites). However, QD and Brain-body co-design methods are orthogonal approaches. The authors should instead compare their method primarily against recent Brain-body co-design frameworks rather than directly comparing against QD-based methods. Specifically, it would be more meaningful to compare the quality and diversity of generated morphologies against recent Brain-body co-design approaches, rather than using QD approaches as the main baseline.\n\n- Visualization and Qualitative Results:\nThe manuscript would benefit from additional visualizations and qualitative examples of generated morphologies. Currently, the paper relies mostly on numerical metrics (such as QD-score and coverage). Providing visual evidence of morphology diversity—clearly demonstrating the variety of morphologies evolved by LOKI on specific tasks—would give readers a more intuitive and convincing demonstration of the method's capabilities.", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank Reviewer  **T5wK**, for the thoughtful feedback and suggestions. We are pleased that you recognize how LOKI creatively integrates different techniques to improve efficiency and diversity in robot morphology optimization. Below we address each of the raised concerns.\n\n> ### 1. Would incorporating a behavioral or performance-aware loss component into the VAE training improve the latent representation for clustering? \nWe agree with the reviewer that morphological similarity does not necessarily imply behavioral similarity, and that our VAE’s latent space, trained solely with a reconstruction loss, primarily captures structural features. Incorporating a behavior-aware loss could indeed enhance the latent representation. However, behavioral information is not available a priori at the clustering stage, as it requires training an optimal policy for each design. \nThat said, in practice, we find that structural features serve as a reasonable proxy for behavioral similarity. As a result, our clustering tends to group morphologies with similar behaviors—for example, spinners and crawlers naturally fall into separate clusters. \n\n> ### 2. Sensitivity to the Number of Clusters and Relation to Population-based Training.\nIn our LOKI framework, the number of clusters is treated as a hyperparameter, similar to the number of cells in MAP-Elites-based methods [22, 23]. Our experiments show that using 40 clusters yields better performance than using 10 or 20. A larger number of clusters allows for finer-grained morphological distinctions but also increases computational cost, as more policies need to be trained. Therefore, this hyperparameter should be selected based on the available computational resources.\nLOKI’s performance is generally robust to the number of clusters, as long as there are enough to avoid overly coarse groupings. For example, using only 10 clusters often results in clusters aligning with broad structural features such as limb count, limiting specialization.\nEach per-cluster evolutionary process resembles population-based training (PBT). Similar to PBT, a population of initial morphologies is sampled from the cluster and periodically evaluated. Underperforming agents are replaced with new morphologies from the cluster. Also similar to PBT, morphologies are evaluated continuously in parallel (simultaneous with policy training), rather than sequentially or only at the end.\n\n> ### 3. Insufficient literature survey and rationale for using QD-based methods as primary baselines rather than Brain-body co-design approaches.\nWe appreciate the reviewer’s suggestion and agree that recent brain-body co-design research deserves clearer discussion in our manuscript. Most of the recent co-design methods [A,B,C,D,E,F]  train a transferable universal controller and they focus on task-specific optimization of morphology and control with no additional objective for diversity and generalization to downstream tasks. LOKI introduces a **cluster-based Quality-Diversity (QD) framework** that co-evolves both morphology and control within behaviorally similar clusters. Unlike methods that aim to train a single universal controller across all morphologies, LOKI leverages structural similarity within each cluster to train per-cluster policies. This is particularly important in high-dimensional and exponentially large design spaces such as UNIMAL, where learning a single controller is especially challenging [G]. Furthermore, this clustering enables localized competition within each group, helping discover morphologies that are optimal within their respective behavioral niches.\nWhile the reviewer suggests that co-design approaches are more appropriate baselines for LOKI, we argue that Quality-Diversity (QD) methods are actually a better fit for comparison. Most co-design methods focus on optimizing morphologies for a specific task and aim to identify one or a few highly performant designs. These approaches typically lack mechanisms to promote morphological diversity. In contrast, QD methods are explicitly designed to discover a diverse set of high-performing solutions across multiple behavioral niches—an objective that aligns closely with the goals of the LOKI framework. LOKI is designed to efficiently discover a diverse population of agents capable of generalizing to a wide range of downstream tasks. While the agents it discovers may not be globally optimal for the locomotion task used during training, they exhibit **niche-optimal behaviors** that are more transferable to other domains, such as manipulation. As such, comparing LOKI to co-design methods that prioritize task-specific optimality alone would not yield a fair or meaningful evaluation.\nAmong co-design methods, DERL [4] is the most relevant baseline, as it also produces a population of 100 morphologies and evaluates their transferability on similar downstream tasks.\n- #### [A]: Luke Strgar et al. Accelerated co-design of robots through morphological pretraining. arXiv preprint arXiv:2502.10862, 2025.\n- #### [B]: Yuxing Wang et al. PreCo: Enhancing Generalization in Co-Design of Modular Soft Robots via Brain-Body Pre-Training. In the ​​proceedings of The 7th Conference on Robot Learning, PMLR 229:478-498, 2023.\n- #### [C]: Muhan Li et al. Generating freeform endoskeletal robots. In International conference on learning representations, 2025.\n- #### [D]: Yuxing Wang et al. Curriculum-based Co-design of Morphology and Control of Voxel-based Soft Robots. In International conference on learning representations, 2023.\n- #### [E]: Haofei Lu et al. Bodygen: Advancing towards efficient embodiment co-design. In The Thirteenth International Conference on Learning Representations, 2025.\n- #### [F]: Ye Yuan et al. Transform2act: Learning a transform-and-control policy for efficient agent design. arXiv preprint arXiv:2110.03659, 2021.\n- #### [G]: Agrim Gupta et al. Metamorph: Learning universal controllers with transformers. arXiv preprint arXiv:2203.11931, 2022.\n\n> ### 4. Visualization of Morphological Diversity.\nThank you for the suggestion. We agree that additional qualitative results would help better illustrate the diversity of the discovered morphologies. To address this, we have provided **videos** showcasing the morphologies and locomotion behaviors of our 100 discovered designs on the supplementary website uploaded in the supplementary material (`index.html`). Due to space limitations in the main paper, we were unable to include more visual examples. We will include more visualizations in the Appendix of the paper for the camera-ready version.\n\n**We hope we have fully addressed the reviewer’s concerns and would greatly appreciate it if the rating could be reconsidered and raised. Thank you.**"}, {"review_id": "xON8msaW1Q", "reviewer": "Reviewer_oRP8", "summary": "This paper proposes an approach to co-evolve morphology and control policies to extend and improve upon previous known methods such as MAP-elites and Deep Evolutionary Reinforcement Learning. Key aspects in the novel approach are the use of shared policies across clusters of similar morphologies to improve generalization and training cost, dynamic local search to improve the efficiency of the evolutionary search and clusters co-evolution of encourage diversity. Experimental evidence is provided in the UNIMAL environments, starting with a flat-terrain locomotion task and extending then to eight different tasks. Baselines include evolutionary-based approaches and show promising results.", "questions": "1. Have you explored dynamic or adaptive clustering during training? If so, can you outline pros and cons of such an approach? \n2. Is it possible to establish more accurately whether generalization comes from the evolutionary processes or simply because of the variety that results from the initial clustering? \n3. How could LOKI be adapted to situations where there are morphological constraints, i.e., more typical real-world robotic applications? Is that just a matter of restricting the random generation? Or would that affect the training of the VAE? \n4. Addressing the weakness nr 6, what non-evolutionary baselines would you consider to assess the algorithm on a broader algorithmic base?", "limitations": "The limitations section touches upon some of the weaknesses that I have pointed out above. However, I feel like more space could be devoted to expand on these.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n1. The approach seems to provide considerable computational gains with respect to other evolutionary approaches \n2. A strength of the algorithm is the ability to generalize to novel tasks\n3. Comparisons considering various metrics (max fitness, QD-score and coverage) are used. \n4. Suitable ablation studies are provided, e.g., with no clusters or variable numbers of clusters and the use of DLS vs mutation based approaches. \n\nWeaknesses:\n\n1. It is not clear how the VAE -based latent space affects performance. \n2. Underexplored relationship between number of clusters and performance. It seems that the performance increases as the number of clusters increases, but that comes with cost and possibly reduced generalization capabilities\n3. The paper does not discuss the fact that while clustering is done initially, morphologies and policies evolve as the evolutionary search is undertaken. This could be a significant limitation in continual learning scenarios where multiple tasks are encountered over a long period of time. \n4. As the algorithm is designed for its ability to generalize, the generalization abilities are not so obvious when considering the fact that the initial clustering is performed on a rich set of morphologies, thereby equipping the population with a varied set of potentially useful morphologies. \n5. Lack of non-evolutionary baselines. The main advantages of the approach are computational advantages with respect to other evolutionary approaches. But how does this algorithm compares with SOTA gradient-descent based approaches that could be used on the same benchmarks? I.e., Metamorph or One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control?", "was_revised": true, "final_justification": "The authors provided meaningful responses to my comments, showing that there is a fair understanding of the implications and weaknesses. I'm not too sure about how these will improve the final version of the paper, but I'm prepared to give the authors the benefit of the doubt, and, as a consequence of the reasonable argumentations made, increase my score.", "rebuttal": "We thank Reviewer  **oRP8**, for your constructive feedback. We are encouraged that you recognized the strengths of our framework such as computational gains and generalization to novel tasks. Below, we address the limitations and recommendations you highlighted:\n>### 1. It is not clear how the VAE -based latent space affects performance.\nWe address this through an ablation study presented in **Appendix A**, where we evaluate the impact of the VAE-based latent space by comparing it to a variant of LOKI that uses clustering over **raw morphology parameters** instead (LOKI w/ raw-param-cluster). Clustering in the raw parameter space fails to capture structural and behavioral similarities between morphologies, resulting in poor generalization of the multi-design policies trained within each cluster. In contrast, the VAE latent space captures meaningful structure by learning a smooth, low-dimensional manifold of the morphology space. As a result, LOKI with raw-parameter clustering performs significantly worse in terms of overall performance, morphological diversity, and downstream task generalization.\n\n>### 2. Underexplored relationship between number of clusters and performance.\n In our LOKI framework, the number of clusters is treated as a hyperparameter, similar to the number of cells in MAP-Elites-based methods [22, 23]. Our experiments show that using 40 clusters yields better performance than using 10 or 20. A larger number of clusters enables finer-grained distinctions between the clusters but also increases the computational budget due to the need for additional policy training. As such, this hyperparameter should be tuned based on the available computational resources. LOKI’s performance is generally robust to the number of clusters, as long as there are enough to avoid overly coarse groupings. For example, using only 10 clusters often results in clusters aligning with broad structural features such as limb count, limiting specialization.\n\n>### 3. Pros and cons of dynamic or adaptive clustering instead of initial clustering.\nThank you for the insightful question. It is true that in the current version of LOKI, clustering is performed only once at the beginning using a VAE-based latent space, and the clusters remain fixed throughout training. LOKI frames co-design as a search problem by constructing a large, predefined pool of morphologies, generated via repeated random mutations to densely and exhaustively cover the design space. Rather than generating new morphologies during optimization, the algorithm samples from these predefined clusters.\nWhile we have not explored dynamic or adaptive clustering during training, we agree that incorporating task-aware or behavior-driven re-clustering is a promising direction. As the policies are being trained, periodically updating clusters based on evolving behavioral or functional similarities could lead to more meaningful groupings and finer specialization. This could be particularly beneficial in continual or lifelong learning scenarios, where task distributions and optimal behaviors may shift over time.\nHowever, this approach presents several challenges. Re-clustering and reassigning morphologies during training may introduce significant computational overhead and could require resetting or transferring policies, potentially disrupting convergence of cluster-specific controllers and destabilizing training. Despite these challenges, we consider dynamic clustering a compelling direction for future work.\n\n>### 4. Explain whether generalization comes from the evolutionary processes or simply because of the variety of a rich set of initial morphologies.\nWhile our initial set of 500k morphologies densely covers the design space, these morphologies are generated through repeated random mutations and are not expected to be inherently optimal for locomotion or manipulation tasks. In fact, most of them are challenging to train for simple locomotion. To illustrate this, we include a **“Random” baseline** in **Table 2 and Figure 6**, which consists of 100 morphologies randomly sampled from the predefined set. The poor performance of the “Random” baseline confirms that simply sampling from this pool is insufficient, indicating that the observed generalization results from the targeted optimization performed by LOKI—not merely from the diversity introduced by the initial clustering.\nThe generalization of our morphologies stems from identifying a diverse set of high-performing designs, each exhibiting optimal behaviors within their respective niche. The downstream tasks require different skills in agility, stability, and manipulation. LOKI discovers such optimal morphologies within each cluster. These top-performing agents display different locomotion behaviors, enabling them to excel in various downstream tasks.\n\n>### 5. How could LOKI be adapted to situations where there are morphological constraints such as real-world applications? Would that affect the training of the VAE?\nLOKI can be adapted to scenarios with morphological constraints by applying those constraints to the initial pool of randomly sampled designs. The VAE learns to represent and reconstruct the morphology space it is trained on, as long as the constrained design space remains sufficiently dense and well-structured. However, if the constraints result in a highly non-convex or sparse design space, the learned latent space may no longer support smooth interpolations that remain within the feasible region. In such cases, it may be beneficial to replace the VAE with a more expressive latent model that can better capture the structure of constrained morphology spaces. This improved latent representation can then be used for clustering.\n\n>### 6. Lack of non-evolutionary baselines. But how does LOKI compare with gradient-descent based approaches that could be used on the same benchmarks? I.e., Metamorph or One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control?\nThe two mentioned approaches, Metamorph and One Policy to Control Them All, propose global policies that generalize across a wide range of agent morphologies. However, they are not co-design methods and thus are not direct baselines for comparison. LOKI, by contrast, focuses on discovering high-performing morphologies rather than solely maximizing policy performance. In fact, we adopt the policy architecture from Metamorph [63] to train multi-agent policies within each cluster.\n\n**We would be grateful if, after reviewing our response and finding your concerns fully addressed, you would consider raising our score.**"}, {"review_id": "JqDN1XtZRM", "reviewer": "Reviewer_Rt4W", "summary": "This paper introduces LOKI, a quality diversity (QD) framework that simultaneously creates high-performing morphologies and policies that control those morphologies. The framework proceeds as follows. First, a large number of morphologies (500k) are sampled from the design space of morphologies and compressed into a latent space with a transformer-based VAE. Within the latent space of the VAE, the 500k morphologies are clustered into a predefined number of clusters (40 clusters in this work). During execution, within each cluster, a \"pool of elites\" is maintained -- the pool contains the top $N_w=20$ highest-performing morphologies. Using PPO, a transformer-based policy is trained to operate these 20 morphologies. Every $f_{diff}=2$ iterations, $N_{sample}$ morphologies are sampled from the cluster and evaluated with the current policy -- essentially, the policy forms an \"evaluator\" that determines which morphologies are best. Ultimately, the product of LOKI is a set of 40 clusters that each have a pool of elites and a corresponding policy that can control those elites. The LOKI framework is evaluated on a number of locomotion benchmarks and compared to baselines in RL and QD, showing that it outperforms existing algorithms in discovering high-performing policies and morphologies, as well as in transferring to downstream tasks.", "questions": "I have left my main comments under the \"Strengths and Weaknesses\" and\n\"Limitations\" sections. If those comments are adequately addressed in the\nrebuttal, I am happy to raise my score. I have reserved this section for minor\ncomments and questions.\n\n- I think it would be helpful to formalize the QD problem statement used in this\n  paper; see, e.g., Chatzilygeroudis 2021 https://arxiv.org/abs/2012.04322 and\n  Fontaine 2021 https://arxiv.org/abs/2106.03894 for examples of formalizing QD.\n- In Figure 1c, it is not clear whether red or blue indicates better\n  performance. Perhaps a color bar would be useful?\n- I am unclear how each morphology is evaluated 8 times on average. With 78,000\n  samples per policy during training (line 225), and 40 policies, and 500,000\n  morphologies total across all the clusters (line 172), shouldn't this come out\n  to $78,000 \\times 40 / 500,000 = 6.24$ evaluations per morphology?\n- In Appendix D Eq. 1 (line 427), are $f_{max}$ and $f_{min}$ the max and min\n  across all clusters of all runs, or only all clusters within a single run? If\n  within a single run, then I think different runs of each algorithm would not\n  be compatible with each other?\n- In Figure 7, it seems the term LOKI (10 clusters) refers to the top 10\n  clusters rather than LOKI with $N_c = 10$. I think it would be more clear to\n  say LOKI (top 10 clusters).", "limitations": "I could not find consideration of broader impacts of the work. I strongly\nencourage including such consideration, at least in the appendix and ideally in\nthe main paper. I am sure there are positive and negative societal impacts of\nlearning to create diverse morphologies. For example, does it affect how we\nthink about biological creatures that adopt their morphology? How does it affect\nthe creation of more capable robots?\n\nNo justifications were provided for the checklist. The instructions for the\nchecklist state \"Please provide a short (1–2 sentence) justification right after\nyour answer (even for NA).\" While I have not factored this into my score, I\nrequest providing this if the paper is accepted.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Significance and Originality:** I am excited to see this paper introduce a\nnew, compelling application for QD algorithms to morphology optimization. The\ncombination of these two areas makes sense because QD is designed to create\ndiverse solutions to problems, and morphology optimization is one area where\ndiverse solutions are an inherent characteristic of the problem, since diverse\nmorphologies are useful in different tasks. At the same time, QD often suffers\nfrom sample inefficiency, as it needs to re-evaluate each new solution\n(morphology) -- a naive application of QD to morphology optimization would\ninvolve training a policy from scratch for each morphology, as is done in the\nMAP-Elites baseline. As such, I find it ingenious that LOKI trains a shared\npolicy for each cluster of morphologies and uses that policy to evaluate new\nmorphologies; this approach effectively overcomes the sample inefficiency\ninherent in QD and allows evaluating many more morphologies.\n\n**Quality:** Overall, the paper is technically sound. The experimental results\nare comprehensive, with evaluations on a wide variety of benchmarks. There are\nstatistical tests and a limitations section.\n\n**Clarity:** The paper is overall clearly written and easy to follow. The\nsections are laid out well and explain the approach, and Section 2 lists a\nnumber of prior works in this area. I appreciate the inclusion of the pseudocode\nin Algorithm 1; it helped me understand LOKI quite easily. I list several parts\nwhere I was confused below.\n\n- It is unclear how random sampling of the morphology parameters is performed.\n  Line 169 seems to suggest that this would be described in the appendix, but I\n  could not find details in the appendix. For example, are limb lengths sampled\n  uniformly at random?\n- The use of Dynamic Local Search (DLS) could be better explained, as not\n  everyone (including myself) is familiar with it. What exactly does DLS refer\n  to in this paper? How does it connect to searching within each cluster? Right\n  now, lines 204-207 explain what DLS does, but not what it _is_. Based on line\n  213, my impression is that DLS refers to sampling randomly morphologies from\n  the cluster. Is that correct?\n- Since the term \"cluster\" $C_k$ is mentioned so many times, I think it would be\n  helpful to formally define it somewhere, e.g., as a set of morphologies or a\n  set of latent embeddings. I think this is important since a cluster is\n  something from which we can \"sample,\" which seems to imply it is a set.\n- I think it can be clarified that the members of each cluster are fixed at the\n  beginning of the search. I initially thought new cluster members were being\n  generated on each iteration (hence my confusion about the definition of a\n  \"cluster\" above). However, it seems that 500k morphologies are sampled at the\n  beginning of the algorithm, and clustered, and from then on, new elites can\n  only come from the morphologies that are already placed in each cluster.\n- I am confused by the use of Coverage as a metric. It seems trivial that LOKI\n  would achieve 100% coverage because it explicitly maintains a policy for each\n  of the 40 clusters. Perhaps this could be clarified in the paper? Similarly,\n  it would help to explain why LOKI with $N_c = 10$ does not achieve 25%\n  coverage (10 / 40), and LOKI with $N_c = 20$ does not achieve 50% coverage. In\n  a similar vein, do the clusters chosen for computing QD Score differ from\n  those used in each run of LOKI?", "was_revised": true, "final_justification": "I truly believe this is an exciting paper for folks in both quality diversity and reinforcement learning. I am happy to say that the authors' responses have addressed my concerns, and I am confident they can make the changes they have promised.", "rebuttal": "We thank Reviewer  **Rt4W** for the constructive comments and feedback. We appreciate your recognition of our paper’s new compelling application for QD algorithms to morphology optimization, the comprehensive experimental results and a well-laid-out approach.  Below, we address each concern you raised and clarify the perceived weaknesses.\n\n>### 1. It is unclear how random sampling of the morphology parameters is performed.\nWe apologize for the confusion and will include a detailed explanation in the Appendix for the camera-ready version. The process of sampling random morphologies in LOKI follows the procedure introduced in prior work, DERL [4] (see Table 1 in the Appendix of the arXiv version). During the population initialization phase, a new morphology is created by first sampling the total number of limbs to grow, followed by a series of mutation operations until the desired number of limbs is reached. These mutations include: growing or deleting limbs, mutating limb parameters, density, degrees of freedom (DoF), gear ratios, and joint angles. For each mutation, the parameters are uniformly sampled from predefined ranges specified in [4]. The key difference is that DERL [4] samples parameter values from discrete sets, while LOKI samples continuously within each range, enabling a denser and more comprehensive coverage of the morphology space. The table below presents the parameter ranges used to create our morphologies. The notation `range(a, b)` denotes a continuous range from minimum `a` to maximum `b`.\n|**Hyperparameter**|**Value**|\n|-|-|\n|Max limbs|11|\n|Limb radius|range(0.02, 0.06)|\n|Limb height|range(0.2, 0.4)|\n|Limb density|range(500., 1000.)|\n|Limb orientation theta|[0, 45, 90, 135, 180, 225, 270, 315]|\n|Limb orientation phi|[90, 135, 180]|\n|Head radius|0.10|\n|Head density|range(500., 1000.)|\n|Joint axis|[x, y, xy]|\n|Motor gear range|range(150., 300.)|\n|Joint limits|[(-30, 0), (0, 30), (-30, 30), (-45, 45), (-45, 0), (0, 45), (-60, 0), (0, 60), (-60, 60), (-90, 0), (0, 90), (-60, 30), (-30, 60)]|\n\n>### 2. What exactly does Dynamic Local Search (DLS) refer to in this paper?\nStochastic Local Search (SLS) algorithms iteratively improve candidate solutions to an optimization problem based on an evaluation function. At each step, the current solution is typically replaced by a neighboring solution with a better evaluation value. Dynamic Local Search (DLS) is a subclass of these algorithms in which the **evaluation function itself is dynamically adapted** whenever a local optimum is encountered, allowing further improvements. In LOKI, we implement a variant of DLS by partially updating the “pool of elites” within each cluster: the two worst-performing morphologies are replaced by new candidates sampled from the same cluster. The evaluation policy is also updated after each step, co-evolving alongside the elite morphologies. This design follows the Dynamic Local Search framework described in Algorithm 54.1, Section 54.3.5 of reference [25] in the paper, from which LOKI draws its search mechanism.\n\n>### 3. It would be helpful to formally define cluster C_k  as a set of morphologies or a set of latent embeddings. It can be clarified that the members of each cluster are fixed at the beginning of the search.\nThank you for the suggestions. We agree that the definition and static nature of the clusters could be stated more clearly. A cluster $C_k$​ is formally defined as a set of latent embeddings of morphologies, where the 500,000 randomly generated morphologies are partitioned into $C_1 \\cup C_2 \\cup \\cdots \\cup C_K$​. These clusters are constructed in advance using the VAE latent embeddings of a diverse, precomputed set of morphologies that densely cover the design space. The clusters remain fixed throughout the evolutionary process, and all candidate morphologies are sampled from within these predefined clusters.\n\n>### 4. Confusion about the coverage metric.\nWe apologize for the confusion and will clarify this in the revised manuscript. LOKI achieves 100% coverage because it explicitly maintains a dedicated policy for each of the 40 clusters. For fair comparison across variants, both the QD-score and coverage are always computed using the same set of 40 reference clusters, regardless of the number of clusters used during training (e.g., $N_c=10,20$). The coverage metric reflects how broadly the discovered solutions span this fixed 40-cluster space. For variants trained with fewer clusters, we assign their 100 final discovered morphologies to the fixed set of 40 reference clusters and compute coverage accordingly.\n\n>### 5. It would be helpful to formalize the QD problem statement used in this paper.\nThank you for the great suggestion. For the camera-ready version, we will include a *Problem Formulation* section and formally define the QD problem as applied in our work.\n\n>### 6. In Figure 1c, it is not clear whether red or blue indicates better performance. Perhaps a color bar would be useful?\nThe small heatmap in the figure indicates that red corresponds to higher reward. We will improve the figure in the camera-ready version.\n\n>### 7. I am unclear how each morphology is evaluated 8 times on average. Shouldn't this come out to 78,000*40/500,000=6.24 evaluations per morphology?\nThank you for identifying a miscalculation in our paper. We will revise the manuscript to correct this error.\n\n>### 8. In Appendix D Eq. , are $f_{max}$ and $f_{min}$ the max and min across all clusters of all runs, or only all clusters within a single run?\nWe appreciate the reviewer’s clarification. $f_{max}$ and $f_{min}$ are computed across all clusters from all runs, not just within a single run. We will revise the manuscript to explicitly clarify this to avoid any potential confusion for readers.\n\n>### 9. In Figure 7, it seems the term LOKI (10 clusters) refers to the top 10 clusters rather than LOKI with N_c=10. I think it would be more clear to say LOKI (top 10 clusters).\nWe will revise the figure label in the camera-ready version to “LOKI (top 10 clusters)” for clarity.\n\n>### 10. Limitations and positive/negative societal impacts.\nWe appreciate the reviewer’s suggestion. We will include a section on societal impact in the appendix and highlight key points in the camera-ready version.\n**Positive impacts**: Our work aims to challenge prevailing assumptions about optimal robot morphology, potentially reshaping how robotic design is approached. By promoting diversity and functionality beyond conventional forms, LOKI encourages exploration of unconventional yet effective morphologies. We believe this contributes toward a more inclusive and biologically inspired understanding of embodiment, potentially serving as a bridge between AI, robotics, and the natural sciences.\n**Negative impacts**: A potential concern is that the ability to autonomously generate a large number of novel and capable morphologies could be misused in contexts that prioritize performance over safety. For instance, this approach could enable rapid prototyping of morphologies for autonomous systems without adequate human oversight, increasing the risk of deploying untested designs in sensitive environments.\n\n**We hope we have fully addressed the reviewer’s concerns and would greatly appreciate it if the rating could be raised. Thank you.**"}]

Author rebuttal:

Meta-review: This paper has received mixed reviews, with one borderline reject, one borderline accept, one accept, and one strong accept. The reviewer giving a borderline reject unfortunately did not react to multiple requests to consider the rebuttal of the authors or other reviews, and a final justification for the rating is missing. Meanwhile, the reply by the authors managed to somewhat satisfy other reviewers, pushing the average score for this work over the line. All reviews and discussions have been carefully considered, and this AC believes that there is sufficient value in this work to be granted publication at NeuRIPS.
Decision: Accept (poster)