LASeR: Towards Diversified and Generalizable Robot Design with Large Language Models
Authors: Junru Song
Yang Yang
Huan Xiao
Wei Peng
Wen Yao
Feifei Wang
Venue: ICLR 2025 Poster (2025)
Keywords: Robot Design Automation
Large Language Model
Voxel-Based Soft Robot
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: This work improves the diversity and inter-task generalizability of robot design processes with the aid of Large Language Models.

Abstract: Recent advances in Large Language Models (LLMs) have stimulated a significant paradigm shift in evolutionary optimization, where hand-crafted search heuristics are gradually replaced with LLMs serving as intelligent search operators. However, these studies still bear some notable limitations, including a challenge to balance exploitation with exploration, often leading to inferior solution diversity, as well as poor generalizability of problem solving across different task settings. These unsolved issues render the prowess of LLMs in robot design automation largely untapped. In this work, we present LASeR -- Large Language Model-Aided Evolutionary Search for Robot Design Automation. Leveraging a novel reflection mechanism termed DiRect, we elicit more knowledgeable exploratory behaviors from LLMs based on past search trajectories, reshaping the exploration-exploitation tradeoff with dual improvements in optimization efficiency and solution diversity. Additionally, with evolution fully grounded in task-related background information, we unprecedentedly uncover the inter-task reasoning capabilities of LLMs, facilitating generalizable design processes that effectively inspire zero-shot robot proposals for new applications. Our simulated experiments on voxel-based soft robots showcase distinct advantages of LASeR over competitive baselines. Code at https://github.com/WoodySJR/LASeR.

Published as a conference paper at ICLR 2025
LASER: TOWARDS DIVERSIFIED AND GENERALIZ- ABLE ROBOT DESIGN WITH LARGE LANGUAGE MOD- ELS
Junru Song', Yang Yang" Huan Xiao*, Wei Peng", Wen Yao Feifei Wang
School of Computer Science, Shanghai Jiao Tong University
2 Intelligent Game and Decision Laboratory
s Schooo of Statistics, Renmin University of China
'Center for Applied Statistics, Renmin University of China songjuunuus jtu. edu. en, bigyangy@gma com (xiachuan001, feifei wang] edu. en. [weipengC098, wendy0782 2126. com
ABSTRACT
Recent advances in Large Language Models (LLMs) have stimulated a sigmificant paradigm shift in evolutionary optimization, where hand-crafted search heuristics are gradually replaced with L.LMs serving as intelligent search operators. How- ever, these studies still bear some notable limitations, including a challenge to balance exploitation with exploration, often leading to inferior solution diversity, well poor generalizability of problem solving across different task settings. These unsolved issues render the prowess of LLMs robot design automation largely untapped. In this work, present LASeR Large Language Model- Aided Evolutionary Search for Robot Design Automation. Leveraging a novel reflection mechanism termed DiRect, elicit more knowledgeable exploratory behaviors from LLMs based on past search trajectories, reshaping the exploration- exploitation tradeoff with dual improvements in optimization efficiency and solu- tion diversity. Additionally, with evolution fully grounded in task-related back- ground information, we unprecedentedly uncover the inter-task reasoning capabil- ities of LLMs, facilitating generalizable design processes that effectively inspire zero-shot robot proposals for new applications. Our simulated experiments on voxel-based soft robots showcase distinct advantages of LASeR over competitive baselines. Code at
INTRODUCTION
In recent years, Large Language Models (LLMs) have demonstrated remarkable reasoning, decision
making, and generalization capabilities (Achiam et al.) 2023) Touvron et 2023) Team et al.) [2023]
Team 2023). sparking a flurry of research interest their application optimization problems.
Earlier efforts embarked on leveraging LLMs to aid traditional search heuristics within evolutionary
algorithms (EAs), such as selecting parent solutions for mutation and crossover (Liu et al. 2022a Ye
et al. 2024) or serving as surrogate models and candidate samplers in Bayesian Optimization (Liu al. 2024b). More and more recent studies have explored the use of LLMs "intelligent search operators". By receiving previously found solutions through prompts, LLMS effectively draw upon their in-context learning and pattern completing abilities to iteratively propose improved candidate solutions (Brahmachary et al. 2024 Huang al., 2024b Yang et al.) 2024 Morris et al.) [024] Romera-Paredes et al./ 2024| Lange et al. (2024). Such LLM-aided evolutionary frameworks have shown great promise in minimizing reliance on handcrafted search heuristics, facilitating convenient problem specification in natural language and rendering evolutionary processes more interpretable.
To date, they have showcased proficiency in classic optimization problems such the Traveling
Salesman Problem and numerical functions, (Liu et al. 2024a Brahmachary et al. 2024 Huang
"Corresponding authors.

---

Published as a conference paper at ICLR 2025
et al.
2024a). as well real-world scenarios spanning code generation (Morris et al. 2024 Romera- Paredes et al. 2024), robotic control (Lange et al.) 2024), protein design (Tran & Hy 2024). etc.
Despite the promising results, contend that existing studies exhibit two major limitations. For one, as many of them have noted that LLMs often strugele to balance exploration and exploita- tion and yield inferior solution diversity (Huang et al. 2024b) TTran & Hy 2024), only expedient
measures have been taken to address this issue, including adjustments to the temperature parameter Yang et al.) 2024 Liu al. 2024a; Pluhacek al., 2024) Ma et al. (022)) or utilizing pre-existing
natural selection techniques such as binary tournament selection and "island models" (Qiu et al., 2024 Paredes et al., 2024). It remains be investigated whether the reasoning capabilities of LLMs could be further harnessed to guide more intelligent exploratory behaviors in the search space. For the other, current LL-M-aided evolutionary approaches generally lack a strong connection to the specific nature of real-world problems, which leads to suboptimal performances and solutions that can not generalize well.
Recently, LLMs have also made their way into the realm of robot design automation. Robot design automation represents a persistent challenge modern robotics that aims to evolve robot morphol- ogy with minimal human intervention (Hu et al., 2022 2023) Song et al. 2024a). However, related work is sparse and only represent rudimentary attempts. To our best knowledge, the only pertinent studies are Zhang 2024), Qiu et al. (2024) and Lehman et al. (2023). While Zhang (2024) uti- lizes LLMS to tune the hyperparameters of traditional EAs, the latter two pioneer the use of L.MMs as search operators for robot design. Nonetheless, they bear the same limitations listed above, which greatly hinder the application of LLMs to robot design automation. In particular, with grow- ing interest in soft robots due their versatility and biomimetic properties, their vast design spaces and intricacy of interaction dynamics among body parts cause existing search algorithms to gener- ally fall short. This highlights the need for more judicious exploration that navigates a variety of design options while ensuring progressive enhancement in functionality (Bhatia et al. 2021 Shah jet al., 202T Song et al. 2024a, SSato & Oka 2024]. Furthermore, as common to have access to repository of pre-designed robots from related tasks when designing for new applications, it is highly relevant to explore the inter-task reasoning capabilities of LLMs to facilitate positive transfer of prior design experience, thus fostering more generalizable design processes.
To address the aforementioned limitations, here we propose LASeR - Large Language Model- Aided Evolutionary Search for Robot Design Automation. LASeR distinguishes itself from previ- ous LLM-dided evolutionary frameworks with a more delicate exploration strategy and generalizable optimization processes. Specifically, we present novel Diversity Reflection Mechanism termed Di- Rect, which strategically instructs an L.IM to reflect upon previously generated designs and suggest viable modifications to enhance diversity while preserving essential functional substructures. This mechanism thus fosters more Knowledgeable exploratory behaviors that closely align with task ob- jectives. Furthermore, by exploiting the abundant descriptive information available in robotic tasks, we not only yield substantially accelerated convergence to figh-performing designs, but also un- precedentedly uncover the potential of LLMs to reason across different tasks and assimilate prior design experience for zero-shot robot proposals in new tasks.
To summarize, our contributions are as follows: (i) By interleaving evolutionary processes with diversity-oriintedd reflective thinking, we reshape the -exooittoonellloiaatiinnn tradeoff of LLdMdaided evolution with simultancous improvements solution diversity and optimization effciency. The for- mer is particularly relevant for enhancing the robustness of robotic systems in volatile environments. (ii) With evolution firmly grounded in the background information of optimization tasks, we unlock the inter-task reasoning capabilities of LLMs in evolutionary computation, hopefully inspiring fu- ture work to further promote the generalizability of LLM-aided evolution across different problem settings. (iii) By unleashing the prowess of LLMs for robot design automation, we also aim in- spire future work that synergizes both robotic design and control with LLMs, achieving closed-loop development of embodied agents.
2 RELATED WORK
Large Language Models Evolutionary Search Operators. Large Language Models (LLMs) represent a class of deep generative neural networks comprising billions or trillions of parameters and pretrained web-scale texual data. In recent years, L.MMs have demonstrated impressive rea-
2

---

Published as a conference paper at ICLR 2025
soning, decision making, and generalization capabilities (Achiam et al. 2023 Touvron et al. 2023 Team et al. 2023 Team (023), which have sparked a flurry of research into exploiting them for op- timization problems (Huang et al. 2024b; Wu et al. 2022). By receiving history search trajectories from the prompt (or context), L.LMs have demonstrated effectiveness as pattern completion engines (Mirchandani et al. 2023), proposing improved solutions and facilitating evolutionary optimization through iterative interactions. Moreover, LLMs are adept at conditioning problem-solving processes on various kinds of prior knowledge expressed in natural language, without needing tedious mathe- matical formulations (Song et al., 2024b). All these favorable attributes position LLMs as promising substitutes for the manually designed search heuristics in traditional evolutionary algorithms (EAs), acting as novel, intelligent search operators. Since Lehman et al.] (2023) introduced this LLM- aided evolutionary paradigm, subsequent studies have extended its methodology and showcased its proficiency in classic optimization tasks like the Traveling Salesman Problem (TSP) and numeri- cal functions (Liu et al., 2024al Brahmachary et al., 2024; Huang et al. 2024a). as well as practical problems spanning prompt optimization (Guo et al.] [223]] Yang et al.] 2024), code generation (Mey- erson et al. 2023, Morris et al., 2024 Cooeraa-Paredee 2024). etc. However, argue that the use of LLMs search operators is still in its early stage, with much of their potential untapped. Notably, existing studies have focused solely on single-task optimization, overlooking the intriguing possibility of LLMs to transfer experience across different tasks. Additionally, although LLMs have been shown to trail behind traditional EAs in balancing ex- ploration and exploitation (Huang et al.) 2024a Tran & Hy 2024). this nuanced aspect has received limited attention from previous research. We am to tackle these limitations in this work.
et al. 2024), robotic control (Lange et al.,
Robot Design Automation. As Artificial Intelligence (AI) continues revolutionize academia and industry. there is an increasing focus on integrating the perceptual and planning capabillitiss of modti- foundation models into various physical embodiments capable of interacting with their environments a research field known as Embodied AI (Roy et al. 2021 Liu et al. 2024c). These advancements highlight the significance of autonomous robot design. Earlier works on robot design automation relied on traditional evolutionary algorithms and primarily targeted rigid robots [Siss] 1994, [hhoronn& & Bidaud [197]] Leger 2012] Wang et al., 2019). In recent years, modular soft robots have garnered broad attention due to their fiexibility. expressiveness, and biomimetic characteristics (Hiller & Lipson 2011 Bhatia et al.) 2021 Medvet et al., 2021). However, these
advantages are accompanied by combinatorially vast design space, necessitating more efficient
search algorithms (Cheney et al.,
2014). Consequently, an emerging line of research resorts to the
cotiimatiiooo-ddssirbuuuue algorithms (EDAs) enhance sample efficiency by explicitly tracking the
distribution of high-performing robot designs. These approaches further leverage deep generative models, such as Generative Adversarial Networks (GANs; Goodfellow et al., (220)) and Variational Autoencoders (VARs; Kingma| 2013), to bolster the representational capacity of EDAs (Hu et al.) 2022 [Song et al. 2024a). Despite their promising results, these models still require problem-spene specific mathematical formulation and neural architecture design, which is highly dependent on domain expertise and poorly generalizable. In this respect, Large Language Models, with their strong in- context learning abilities and extensive prior knowledge, hold the promise to transform the robotic design process (Stella et al. (003)). Nevertheless, the exploration of LLMs in this respect is sparse and warrants further investigation (Lehman et al.
2023 Zhang 2024
Qiu et al. 2024).
3 LASER: LLM-AIDED EVOLUTIONARY SEARCH FOR ROBOT DESIGN AUTOMATION
In this section, we first present an overview of our algorithm, and then delve into the details of our prompt design and the novel Diversity Reflection Mechanism. Subsequently, we describe how LLMs can be instructed to facilitate effective knowledge transfer across different tasks, followed by a brief' introduction to fitness evaluation protocols.
3.1 ALGORITII FRAMEWORK
As illustrated in Figure D a) and detailed by Algorithm 1 in Appendix PP we integrate an LLM into the bi-level optimization framework commonly employed in robot design automation. Specifically, the inner loop optimizes a controller for each robot morphology through reinforcement learning, with the resulting task performance serving as the fitness evaluation. The outer loop evolves
3

---

Published as a conference paper at ICLR 2025
(a) Algorithm Overview of LASeR
Inner Loup
Warm Start OF
Control Optimization & Fitness Evaluation
Termination
Zero-Shot Proposals by LLM
Generated Offspring
Natural Selection
Elite Robot Designs
Outer Loop
Diversity Reflection (DiRect) Mechanism
Prompt
Metadata (task objective, constraints, etc.)
Output
Elite Design-Fitne Fitnnes Pairs
Input
LLM Intelligent Search Operator
Target Fitness
(b) DiRect Mechanism
(c) example Diversity Reflection
Originally Generated Robot Designs
LLM
robot simillar one. Which voxels be modifiie increase diversity, without compromising its fitmess?
pass
fail
Similarity Check
Diversity Reflection
Replace with borizontal actuator. Change (2.1) (2.3) venels
pass
fail
Similarity Check
Please generate the modiflid robot design based your analysis.
append
append
Generated Offspring
discard
Figure 1: (a) algorithm overview of LASeR; (b) the Diversity Reflection (DiRect) Mechanism; (c) an example illustrating how Diversity Reflection works on carrying robot. The illustration takes voxel-based soft robots (VSRs; Bhatia a wwo-dimensional material matrix, where each entry, ranging from 0 to 4. represents the material land denote rigid voxels, soft voxels, horizontal
202I) as an example. Each VSR represented as
type at the corresponding position. actuators, vertical actuators and empty voxels, respectively.
population of robot morphologies by carrying out natural selection and generating new offspring solutions in each generation. Here, instead traditional evolutionary algorithms (EAs) that rely on manually designed search heuristics for generating offspring, an L.M properly prompted to be our search operator. This is achieved by providing the LLM with previously evaluated robots and various kinds of metadata context. However, we still bootstrap the evolutionary process with few generations of conventional EAs before letting the LLM play its part, as this warm start would provide the LLM with an initial momentum (i.e., improving directions) terminates when maximum number of robot evaluations is reached. We term our approach LASeR, short for LLM- Evolutionary Search for Robot Design Automation. The following subsections elaborate the key components of L.ASeR. Note that while the introduction takes voxel-based soft robots (VSRs) proposed in Bhatia et al. (202I) as an example, our approach is readily applicable to other robot types. For a detailed introduction to VSRs, please refer to Bhatia et al. 202
build upon. Evolution
3.2 PROMPT DESIGN
As discussed Section 3.1 the LLM is prompted to propose new offspring designs each gen- eration. As effective prompt design is crucial to elicit desired responses from LLMs, craft well-structured prompt comprising the following essential components:
Task-related metadata: this part includes various auxiliary information to in-context adapt the I.L.M to serve search operator, including task objectives, descriptions of
4

---

Published as a conference paper at ICLR 2025
simulation environment, constraints on robot designs, etc. This component is largely de- rived from the official documents of EvoGym (Bhatia et al. (021). This metadata is largely missing in previous studies. We would showcase in our experiments that by fully ground- ing the evolutionary process on task-related metadata, we not only achieve more efficient design optimization, but also elicit inter-task reasoning capabilities from LLMs that greatly boost the generalization of design experience.
Elite design-fitness pairs: this part includes robot designs that have survived natural se- lection, each accompanied by its fitness score. These designs are sorted in ascending order of fitness, so as to make it easier for the LLM to infer continuation patterns and extrapolate to potentially improved designs.
Target fitness:
we instruct the LLM to extrapolate from the sequence of elite designs and generate new designs that meet pre-specified fitness score (e.g. 120% of current maximal fitness). This is referred to as the "just-ask" query in previous studies (Lim et al. 2024).
Additionally, we include some basic requirements into the system prompt to better align the behavior of the LLM with our intention. For instance, the LLM is strictly demanded to output new robot designs in numpy array format, enclosed between <solution>> and </solution>>, to allow for easier parsing. Even so, LLMs are still not guaranteed to generate valid robot designs in every interaction. To keep runtime and cost under control, we set an upper limit on interactions; once the limit is reached, we revert to conventional EA to generate the remaining offspring. For full prompts please refer to Appendix A
3.3 DIRECT: DIVERSITY REFLECTION MECHANISM
The exploration-exploitatinn tradeoff is longstanding dilemma in evolutionary computation. Prop- erly timed exploratory behavior would contribute to search processes that not only are less prone to getting stuck in local optima. but also produce more diversified solutions. The latter property is particularly relevant for developing robotic systems in dynamic environments, where first-choice robots, once fail, must be immediately replaced with alternatives. However, it has been shown that LLMS often struggle to balance exploration and exploitation, yielding inferior solution diversity to existing EAs (Huang al. 2024a Tran & Hy 2024). While many previous studies addressed this issue by tuning the temperature parameter of LLMS or resorting to diversity-preserving selection techniques, in this work we introduce a novel Diversity Reflection Mechanism (DiRect) that lever- ages the reasoning capabilities of LLMs to guide exploratory behaviors.
The idea of DiRect is straightforward. As depicted in Figure [[b), for each newly proposed robot design, we assess its similarity to previously evaluated designs with probability p. The new design said to fail the similarity check if it shares more than 8 voxels with at least one existing robot. In this case, the DiRect mechanism is triggered. Specifically, the LLM is first prompted to suggest modifications to voxels that could enhance variability without compromising fitness, with existing examples reference. The LLM is then asked to return the modified robot design according to these suggestions. In section 4 we show that this reflection mechanism fosters more beneficial exploratory behavior in the search space, leading to more diversified robot designs while maintaining relevance to the task objective. Figure displays a specific example where DiRect helps modify newly proposed carrying robot. The similarity threshold is important hyper- parameter that controls the performance of LASeR. We include general principles for choosing s, supported by experimental evaluations, in Appendix
3.4
LLM FOR INTER-TASH KNOWLEDGE TRANSFER
The algorithm described thus far treats the robot design of each task independently. starting with a randomly initialized population. However, it is often the case that we already have access to a repository of pre-designed robots from existing tasks when designing for a new one. Under this circumstance, leveraging the prior design experience in one way or another would hopefully afford a boost in sample efficiency. However, to discern functional substructures from highly abstract robot morphologies (e.g. voxel-basee soft robots), as well as to speculate which substructures will benefit task, poses major challenge to humans. Our work represents pioneering effort to exploit the reasoning capabilities of LLMs for this purpose. Specifically, by specifying the characteristics of task A and B, along with collection of high-performing robot designs from task A, we instruct an
5

---

Published as a conference paper at ICLR 2025
LLM to analyze the similarities and differences between the two tasks and infer potentially favorable substructures (such as specific patterns of voxel assembly) for task B. Based on this analysis, the LLM then proposes robot designs for task B. thus enabling zero-shot robot proposals (where "zero- shot" means that no evaluated robot samples from the new task are required). These robot proposals can then serve as informative initialization to initiate further design search for task B.
3.5 FITNESS EVALUATION
In this work, we use the Proximal Policy Optimization (PPO) algorithm to optimize a separate con- troller for each robot design. The PPO algorithm enhances conventional gradient-basee based reinforce- ment learning algorithms by incorporating importance sampling into gradient estimation, allowing for the reuse of sample trajectories across multiple parameter updates. The PPO algorithm alternates between two key phases data collection and policy update- until a predefined number of iterations are completed. With an optimized controller that maps environmental observations appropriate actuation signals, we measure the fitness of a robot by calculating the cumulative reward it receives over a complete episode, which reflects its performance in accomplishing given task. For further details on the PPO algorithm, please refer to Sohhclulmnn et al. (2017).
EXPERIMENTS
We begin this section with an introduction to our experimental setups, and then analyze the results of our comparison and ablation studies in detail. Our experiments are designed to address the following questions:
Q1: Can LASeR outperform state-of-the-art baselines in robot design automation?
Q2: To what extent does DiRect improve the exploration-exploitation tradeoff of LLM-aided evolution?
Q3:
Does task metadata bring additional benefits to single-task robot design automation? More- over, does it aid inter-task experience transfer and enable zero-shot robot design for new tasks?
Previous studies have shown that different temperature parameters and versions of LLMs yield varying evolutionary outcomes. What are the specific impacts of these factors in our context?
Q4:
4.1 EXPERIMENTAL SETUPS
Benchmark Setting. We base our experiments on Evolution Gym (EvoGym: Bhatia et al. 2021 simulation environment designed for voxel-based soft robots (VSRs). In EvoGym, VSRs are repre- sented in grid-like layout and consist of five types of voxels: rigid voxels, soft voxels, horizontal actuators, vertical actuators, and empty voxels. VSRs achieve motion control by altering the vol- umes of actuators either horizontally or vertically according to action signals. For benchmarking of single-task optimization, we select four task instances: one locomotion task, Walker-vO, and three manipulation tasks, Carrier-vO, Pusher-v0 and Catcher-vO. For experiments of inter-task knowl- edge transfer, we use Bridge waldeeeal and UpStepper-vO. A detailed introduction to these tasks is provided in Appendix For more information on EvoGym, please refer to Bhatia al. (2021).
Baselines. We compare our method against the following baselines: (i) Bayesian Optimization (BO) Kushher 1964 Mockus 1974)), classic algorithm for optimizing expensive-to-evalue
functions. It employs probabilistic model as the surrogate function and samples candidate solu- tions based on predicted mean and uncertainty, (ii) Speciated Evolver (SE) (Medvet et al., 2021 ).
a variant of the genetic algorithm (GA: Michalewiczz 2013) that divides the population into species to preserve diversity and prevent premature convergence. (iii) RoboGAN (Hu et al., 2022), an estimation-of -f-istribbtttttttooonn algorithm (EDA) that utilizes the Generative Adversarial Network (GAN) to track the distribution of geigh-performing robot designs and generate new candidate solutions. (iv)
The last baseline, which we term LLM- is adapted from Zhang (2024) that uses LLMs to supervise the hyperparameter tuning of a genetic algorithm. Drawing comparison with LLM- would directly verify the benefits of LLMs serving intelligent search operators. We addition- ally draw comparisons with two latest generative model-basce evolutionary algorithms, MorphVAE (Song et al. 2024a) and OPRO (Yang et al. 2024), with results presented in Appendix F

---

Published as a conference paper at ICLR 2025
Evaluation Metrics. We employ the following metrics to evaluate the performance of various approaches: (i) Maximal Fitness, defined as the fitmess of the best- perffomrf robot design achieved within a specific number of evaluations. This metric is commonly used in robot design automation to assess optimization efficiency. (ii) Diversity: Given the significance of developing diverse robotic ecosystems handle volatile environments, designs" from two perspectives: one is the average edit distance among all pairs of pirf--rorormiing robot designs (Saito & Okal 2024), and the other is the total number of distinct -erfforrroninn robot designs. We further aggregate the two values via weighted averaging, where the latter is multiplied by 0.1 so that they are roughly on the same scale and given equal importance. Please refer to Appendix for for detailed discussion on diversity measurement. We also include an analysis of computational efficiency in Appendix O
measure the diversity of gegh-performing robot
Implementation Details. We use GPin---mini for both LASeR and LLM--Tune, with the tempera- ture parameter set as 0.7. For ablation studies, we additionally try out GPT-3.5-Turbo and tempera-
2024a Saito & Oka 2024: Dong et al. 2023; Bhatia et al.) 202I), we choose the simple yet effective control pro- tocol for fitness evaluation, L.e, Multilayer Perceptron (MLP) the controller for each robot design and PPO algorithm (Schulman et al., (017) for policy training. Following previous studies on VSR design (Song al. 2024a| [Saito & Oka 2024 Dong al., 2023 Bhatia al.) (02T), robot designs are constrained to a 5 bounding box for an expressive yet tractable search space. Nevertheless, as demonstrated in Appendix G our approach is scalable larger design spaces. For fair comparison, each method is permitted 1000 robot evaluations. Experimental results of comparative studies are averaged across five independent runs. Our experiments are conducted on server equipped with Intel Xeon processors running at 2.20 GHz and four NVIDIA Tesla RTX GPUs, with the system operating under Ubuntu 22.04. We relegate additional parameter settings Appendix C For further implementation details, please refer to our code repository
tures of and 1.5. Following the common practice in previous VSR studies (Song et al.
4.2
COMPARISON STUDIES
4.2.1
SNGLEETASK OPTIMIZATION
We begin our analysis by examining single-task optimization performances. As demonstrated in F LASeR nearly consistently outperforms all baselines across the three tasks with significant margins. Specifically, LASeR achieves rapid convergence speeds to optimal robot designs, with only one exception on Walker-vO, where LLM-Tuner demonstrates slightly faster convergence the early stage of evolution but ends up further from optimality. The superior performance of LASeR compared to LLM-Tuner highlights that LLMs have more important roles to play beyond merely tuning hyperparameters for traditional EAs. Results of significance tests are in Appendix D
Maximal Füness
10.7
Maximal Fitmess
Maximal Fitness
12.5
Movimal Fitress
10
0.8
10.6
7.5
0.7
10.5
250
500
750 1000
250 500
750 1000
250
500
750
1000
750 1000 Evaluations
Number Evaluations
Number of Evaluations
Number of Evaluations
(a) Walker-vO
(b) Carrier-v0
(c) Pusher-v0
(d) Catcher-v0
LASeR (ours)
BO
SE
RoboGAN
LLM-Tuner
Figure 2: Comparative results of single-task optimization efficiency.
Table 4.2. further demonstrates the diversity of high-performing robot designs achieved by dif- ferent methods. We observe that LASeR surpasses all baselines in three of the four tasks. We additionally compare the fitness performance of robot designs before and after being modified by DiRect, and find no significant difference (see Appendix E}. This suggests that our Diversity Reflec- tion Mechanism indeed encourages the LLM introduce variability into robot design while keeping
Wee first calculate the 90% quantile of fitnesses obtained by all methods, and consider robot designs with fitness exceeding this threshold as high-performing. ht /i thub. com comm/ooodySUR//AASee

---

Published as a conference paper at ICLR 2025
its functionality largely intact. These results validate the distinct advantage of DiRect to promote beneficial exploratory behaviors directed towards high-performing regions. It is worth noting that the Bayesian Optimization algorithm, known for balance between exploration and exploitation in its acquisition function, actually compromises a great deal of optimization efficiency for exploration and fails to generate high-performing robots in many cases. On the contrary, LASeR reshapes the exploration-explotttionn tradeoff' of LLM-aided evolution to yield dual benefits in optimization ef- ficiency and diversity. For separate results of the edit distance and the number of nigh-performing designs, please refer to Appendix ! However, we note that the diversity of LASeR in Catcher-v0 not as competitive. We suspect this is due to task complexity. Specifically, for more challeng- ing tasks like Catcher-vO, while LLMs are still capable of extrapolating from existing solutions (as shown in Figure 2(d). they struggle to recognize finer-grained functional structures. As a result, they might have difficuity introducing variability without compromising performance. We believe that helping LLMs better understand the roles of different parts within robot morphology as by incorporating images/videos of robots interacting with the environment-coon be promising direction for future research.
Table 1: Comparative results of diversity (reported as mean (std)).
Walker-v0
Carrier-vO
Pusher-vO
Catcher-vO
Average Rank
BO
N/A
11.88 (2.77)
13.68 (2.
19.76 (1.10)
3
SE
5.24 (0.26)
15.26 (1,52)
7.26 (3.91)
13.91 (2.27)
3.25
RoboGAN
N/A
10.94 (N/A)
N/A
18.27 (1.44)
4
LLMM-Tuner
11.60 (4.35)
18.26 (6.00)
14.17 (6.83)
13.89 (7.32)
2.5
LASeR (ours) 23.09 (5.33)
20.87 (4.27)
20.91 (8.85)
8.07 (2.48)
2
Note:
When no more than one high-performing robot design is produced, diversity cannot be calculated. When this is the case across all repeated trials (c.g. BO on Walker-vO), the result is reported as "N/A". When pershperforming robots emerge in only one trial (c.g. RoboGAN in Carrier-vO), the standard deviation is unavailable and reported as "N/A".
4.2.2 INTER-TARE KNOWLEDGE TRANSFER
Now we proceed to examine the ability of LLMs transfer design experience across different tasks. To achieve this purpose, we introduce two more tasks: BridgeWalker-vo and UpStepper- vO. Specifically, both Bridge and UpStepper-v0 bear some resemblance to Walker-vO, but differ in their terrains: BridgeWalker-v0 involves locomotion on a soft rope-bridge, whereas UpStepper-vO requires climbing stairs of varying lengths. The LLM is prompted to generate robot designs for each new task, given clite Walker-vO robot designs. As shown in Figure 3/b the zero- shot proposals by LLM outperform both randomly generated designs and elite Walker-v0 designs evolved by LASeR, in terms of accomplishing the new tasks. This serves as sound evidence that the LLM is not simply replicating examplars in its context, but rather assimilating design experience that is beneficial for new settings. This is largely owing to our incorporation of task-related metadata that provokes inter-task reasoning within the LLM. For illustration, Figure (]a) demonstrates some insights that the LLM drew from Walker-v0 elites transfer to Bridge Walker-vO.
The zero proposals are then leveraged as the initial population for further optimization. Figure Ic-1 c-2) demonstrate that this informative initialization results in faster evolution than start- ing from scratch, and pulls away from baseline algorithms with even greater advantage. Also note that the zero-shot proposals for Bridge Walker-vO turn out to be already near optimal before under- going marginal improvement with evolution. These promising results unprecedentedly uncover the possibility of generalizable evolutionary processes driven by LLMs and hopefully inspire closer in- vestigation in future work. Please note that while here we focus on intuitively similar task instances, we show in Appendix M that this is not strictly necessary for successful experience transfer.
'The result is averaged over rohot designs in each case.
8

---

Published as a conference paper at ICLR 2025
To design high-perrorminn zobota for the "Bridgekalker" task, can leverage insights from the "Walker" designs, The primary differences between the two taske are that "BridgeWalker requizes navigeting on seft zose bridge, which will behave differently under load due to its compliance and deformation characteristics. This suggests that the zobot's structure should be optimized to distribute weight effectivaly and maintain stability while moving.
Here are some key insights dramm from "Nalker" designs: Use Soft Voxels: These surface like zope bridge.
for elastic deformation, wh'ch cructal soft
vertical actuaters should strategleally placed ucovide effecLive locemation while adapting to the bridge's movements,
Actuators
Horizental
center mass low and evenly distributing weight scress the robat's body will help maintein balance.
Neight Distribution: Keeping
Connection Between Voxels: Utilizing connections
enhance flexibility
adaptability the bridge deforms.
5. Combination of Rigid and Soft Voxels: Using rigid voxels where strength ia necessary
while incerporating seft voxels for novement and edeptability.
(a) Example design experience transferred Walker-vO to BridgeWalker-vO.
LASeR (w knowledge transfer)
LASeR (wlo knowledge transfer)
BO
SE
RoboGAN
LLM-Tuner
Average Fimes
6.7
Maxical
Maximal
1.3
6.5
BridgeWalker-v0
UpStegper-v0
Random Sampling II Walker Elkes LLM Preposis (b) Zero-shot performance
Number of Evalustions
Number Evaluations (c-2) UpStepper-v0
(c-1) Bridge WalkereWalee--OO
Figure 3: Results of inter-task knowledge transfer.
4.3
ABLATION STUDIES
4.3.1 EFFECTIVENESS OF DIRECT
As shown Table 4.3.1 the Diversity Reflection Mechanism fosters a robust increase in diversity compared to ablated version. is further demonstrated in Figure the exploratory behaviors led by DiRect also facilitate more efficient navigating of design spaces, leading to reduced sus- ceptibility to local optima and higher optimization efficiency. These results combine to underscore the distinct superiority of DiRect to yield dual benefits in optimization efficiency and diversity by exploiting the reasoning capabilities of LLMs.
Table 2: Ablative results of diversity (reported mean (std))
Walker-v0
Carrier-vo vO
Pusher-vO
Catcher-vO
LASeR
23.09 (5.33)
20.87 (4.27)
20.91 (8.85)
8.07 (2.48)
I LASeR w/o DiRect
16.96 (1,38)
7.28 (1.13)
10.50 (1.04)
5.65 (1.11)
10.7
Maximal Finness
Maxiima Finess
10.0
Maximal Fimess
12.5
10.6
10.0
Moxinal
0.6
10.5
1000
3.0
1000
750 1600
1000
Number Evalustions
Number of Evaluations
Number of Evaluations
Number Evaluations
(a) Walker-vO
(b) Carrier-vO
(c) Pusher-v0
(d) Catcher-ve
LASeR
LAScR (wlo DiRect)
Figure& 4: Ablative results of single-task optimization efficiency. The results of the ablated version are averaged across three repeated runs.

---

Published as a conference paper at ICLR 2025
4.3.2 EFFECTIVENESS OF
METADATA
As explained in Section 3.2 current LLM-aided evolutionary frameworks mostly lack sufficient grounding in task-related background information, which potentially impedes their performances real-world applications. We test this conjecture by removing descriptions of task objectives and simulation environment from our prompts, and see a significant performance drop (Figure [sa)),, hence justifying our prompt design. For finer-grained ablations on individual components of the prompt, please see Appendix K
4.3.3 IMPACT OF LLM VERSION AND TEMPERATURE PARAMETER
Previous work has shown that the temperature parameter of LLMs has an unignorable influence on evolutionary outcomes, with higher temperatures tending yield better results (Pluhacek al. 2024). However, we observe reverse effect where lower temperature turns out slightly more fa- vorable (b). We suspect that this is partly due to the complexity within VSR design, which necessitates precise extrapolation from an ascending sequence of solutions. Any deviation could lead to substantial performance drops, outweighing the benefits of random exploration. Meanwhile, we note that the ablation studies with temperature as 1.5 fail similarity checks only about 70% as often as when temperature equals 0.7. words, higher temperatures would lead to greater but inef- fective variability in candidate solutions so that they could bypass diversity reflection. These results suggest that lower output temperatures are required for our approach to work better. Additionally, observe the same improvement resulted from ure (Jc).. This shows promise of robot design automation directly benefiting from better language models, which puts in a strategic position ride the wave of rapidly progressing LLMs.
up-to-date LLMs in past literature (Fig-
Maximal Fitness
Maximal Fitness
Maximal Fitness
10
10
10
8
8-
LASeR
8
LAScR
LAS:R (lomgerature-1)
LASeR
LASeR (wio took metarlen))
LASeR (lemperature-1..
LASeR (GPT-3.5)
250
500
750
1000
250
500
750
1000
750 Number of Evaluations
250
500
1000
Number of Evaluations
Number of Evaluations
(a) Impact of task metadata
(b) Impact of LLM temperature
(c) Impact of LLM version
Figure 5: Additional ablation studies on Carrier-vO. The results of the ablated versions are averaged
across three repeated runs.
CONCLUDING REMARKS
We present LASeR - Large Language Model-Aided Evolutionary Search for Robot Design Au- tomation. With novel diversity-oriented reflection mechanism termed DiRect, we elicit intelligent exploratory behaviors from LLMs that reshape the pelloaaioo-eeplliiation tradeoff with dual im- provements in optimization efficiency and diversity. We additionally propose to ground robot design on rich task-related metadata and uncover the intriguing inter-task reasoning capabilities of LLMs to foster generalizable design processes across different applications. Our experiments with simulated voxel-based soft robots demonstrate superior performances of our approach compared to competi- tive baselines. Scaling up LASeR for multi-task optimization would hopefully further harness the inter-task reasoning abilities of LLMs to boost sample efficiency. It is also interesting to investigate how LLM-aided control strategies (Wang et al. 2023a) could be integrated into our framework, so that LLMS are not only responsible for action planning, but also enabled to design their own em- bodiments, hence exploiting the synergy between design and control. We leave these for our future work. For more detailed discussion of limitations and open problems, please refer to Appendix N
ACKNOWLEDGEMENTS
This work is funded by National Natural Science Foundation of China (No. 72371221,,, the MOE Project of Key Research Institute of Humanities and Social Sciences (22JJDI 10001), and Zhiqiang Foundation. The authors would like to thank all the anonymous reviewers for their valuable com- ments.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This work uses simulated task environments which have been commonly used in previous research of robot design automation and should not be regarded controversial. Our of Large Language Models is strictly confined to simulated robot design generation without real-worke deployment, and therefore does not involve any safety risks.
REPRODUCIBILITY STATEMENT
Our code is readily available on GitHub. This work uses GPT-3-turbo and GPT-4o-mini, whose APIs are publicly accessible. However, due to the uncontrollable random generator seeds behind closed- source LLMs. experiments involving these models generally suffer from limited reproducibility (Huang ct al. 2024a). Developing reproducible methods for API calls would significantly improve the replicability of research outcomes involving Large Language Models.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
Jagdeep Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, and Wojciech Matusik. Evolution gym: A large-scale benchmark for evolving soft robots. Advances Neural Information Processing Systems, 34:2201-222 2214,444,, 2021.
Shuvayan Brahmachary, Subodh M Joshi, Aniruddha Panda, Kaushik Koneripalli, Arun Ku- mar Sagotra, Harshil Patel, Ankush Sharma, Ameya D Jagtap, and Kaushic Kalyanaraman. Large language model-based evolutionary optimizer: Reasoning with elitism. arXiv preprini arXiv:2403.02005. 2024.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision Aasion-lagguage-acte models transfer web knowledge to robotic control. arXiv preprint arXiv:2307. 15818, 2023.
Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web- scale knowledge for robot manipulation. arXiv preprint orXiv:2410.06758, 2024.
Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution, 7(1): 11-23, 2014.
Olivier Chocron and Philippe Bidaud. Evolutionary algorithms kinematic design of robotic sys- tems. In Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Saal-orrdd Applications. IROS'97, volume 2, pp. 1111-1117. IEEE, 1997.
François Cochevelou, David Bonner, and Martin-Pieree Schmidt. Differentiable soft-robot gener- ation. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 129-137, 2023.
Heng Dong. Junyu Zhang, and Chongjie Zhang. Leveraging hyperbolic embeddings for coarse-to- fine robot design. arXiv preprint arXiv:231 1oki:23311.000 1.0462, 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-14) 2020.
Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532. 2023.

---

Published as a conference paper at ICLR 2025
Jonathan Hiller and Hod Lipson. Automatic design and manufacture of soft robots. IEEE Transac- tions on Robotics, 28(2):457-466 2011.
Jiaheng Hu, Julian Whitman, Matthew Travers, and Howie Choset. Modular robot design optimiza- tion with generative adversarial networks. In 2022 International Conference on Robotics and Automation (ICRA), pp. 4282-42 -288.. IEEE, 2022.
Jiaheng Hu, Julian Whitman, and Howie Choset. Glso: Brammar-guider latent space optimization for sample-efficient robot design automation. In Karen Liu, Dana Kulic, and Jeff Ichnowski (eds.), Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pp. 1321-1331. PMLR, 14-18 Dec 2023. URL https::tp//......... htttps:/pprccee::/
mlr. www.press//ee /20ss/2255//// htc.5/hhh3cccccc..
Beichen Huang, Xingyu Wu, Yu Zhou, Jibin Wu, Liang Feng, Ran Cheng, and Kay Chen Tan. Exploring the true potential: Evaluating the black-box optimization capability of large language models. arXiv preprint arXiv:2404.06296, 2024a.
Sen Huang, Kaixiang Yang. Sheng Qi, and Rui Wang. When large language model meets optimiza- tion. arXiv preprint 1000:22005.0009, 2024b.
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint prXiv:1312.6111 2013.
Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. A scalable pipeline for designing reconfigurable organisms. Proceedings of the National Academy of Sciences, 117(4): 1853-1859, 2020a.
Sam Kriegman, Amir Mohammadi Nasab, Dylan Shah, Hannah Steele, Gabrielle Branin, Michael Levin, Josh Bongard, and Rebecca Kramer-Bottiglio Scalable sim-to-real transfer of soft robot designs. In 2020 3rd IEEE international conference on soft robotics (RoboSoft), pp. 359-366.
IEEE, 2020b.
Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. Kinematic self-replication in reconfigurable organisms. Proceedings of the National Academy of Sciences, 118(49):
e2112672118, 2021.
Harold Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. 1964.
Robert Lange, Yingtao Tian, and Yujin Tang. Large language models as evolution strategies. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 579-582, 2024.
Chris Leger. Darwin2K: An evolutionary approach to automated design for robotics, volume 574. Springer Science Business Media, 2012.
Julie Legrand, Seppe Terryn, Ellen Roels, and Bram Vanderborght. Reconfigurable, multi-material, voxel-based soft robots. IEEE Robotics and Automation Letters, 13): 2023.
Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley. Evolution through large models. In Handbook of Evolutionary Machine Learning, pp. 331-366. Springer, 2023.
Bryan Lim, Manon Flageat, and Antoine Cully. Large language models as in-context ai generators for quality-diversin arXiv preprint 2404522404 2024.
Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, and Yew-Soon Ong. Large language models as evolutionary optimizers. In 2024 IEEE Congress Evolutionary Computation (CEC), pp. 1-8. IEEE, 2024a.
Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela der Schaar. Large language models to enhance bayesian optimization. In The Twelfth International Conference on Learning Repre- sentations, 2024b. hRI.tttss: http::::///ppeerviiiii wwt.wwwwww..eeeeeeeeeeeeeeeeeeee wtt.:e/oorrrmmmmod0ooottttteeeee
12

---

Published as a conference paper at ICLR 2025
Yang Liu, Weixing Chen, Yongjie Bai, Jingzhou Luo, Xinshuai Song, Kaixuan Jiang, Zhida Li, Ganlong Zhao, Junyi Lin, Guanbin Li, et al. Aligning cyber space with physical world: A com- prehensive survey on embodied ai. arXiv preprint arXiv:2407.06886, 2024c.
Pingchuan Ma, Tsun-Hsuan Wang. Minghao Guo, Zhiqing Sun, Joshua Tenenbaum, Daniela Rus, Chuang Gan, and Wojeiech Matusik. Lim and simulation as bilevel optimizers: A new paradigm to advance physical scientific discovery. arXiv preprini arXiv:2405.0978: 2024.
Eric Medvet, Alberto Bartoli, Federico Pigozzi, and Marco Rochelli. Biodiversity in evolved voxel- based soft robots. In Proceedings of the Genetic and Evolutionary Computation Conference, pp- 129-137, 2021.
Elliot Meyerson, Mark Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K. Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. arXiv preprint M.Xiv:2302.12170., 2023.
Zbigniew Michalewicz. Genetic algorithms+ data structures= evolution programs. Springer Sci- ence & Business Media, 2013.
Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Are- nas, Kanishka Rao, Dorsa Sadigh, and Andy Zeng. Large language models as general pattern machines. arXiv preprint arXiv:2307.0477. 2023.
Jonas Mockus. On bayesian methods for seeking the extremum. In Proceedings of the IFIP Techni- cal Conference, pp. 400-404, 1974.
Clint Morris, Michael Jurado, and Jason Zutty. Lim guided evolution-the automation of models advancing models. In Proceedings of the Genetic and Evolutionary Computation Conference, pp- 377-384, 2024.
Michal Pluhacek, Jozef Kovac, Peter Janku, Tomas Kadavy, Roman Senkerik, and Adam Viktorin. A critical examination of large language model capabilities in iteratively refining differential evo- lution algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference Com- panion, pp. 1855-1862, 2024.
Kevin Qiu, Krzysztof Ciebiera, Pawel Fijatkowski, Marek Cygan, and Kukasz Kucinski. Robomorph: Evolving robot morphology using large language models. arXiv preprint arXiv:2407.08626, 2024.
Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog. M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, al. Mathematical discoveries from program search with large language models. Nature, 625(7995):468-477 2024.
Nicholas Roy, Ingmar Posner, Tim Barfoot, Philippe Beaudoin, Yoshua Bengio. Jeannette Bohg, Oliver Brock, Isabelle Depatic, Dieter Fox, Dan Koditschek, et al. From machine learn- ing to robotics: Challenges and opportunities for embodied intelligence. arXiv preprint 10. 2021.
Takumi Saito and Mizuki Oka. Effective design and interpretation in voxel-based soft robotics: A part assembly approach with bayesian optimization. Artificial Life Conference Proceedings 36, volume 2024, pp. 26. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info .... 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: trXiv:/707.06:: 1707.06377, 2017.
Dylan Shah, Joshua Powers, Liana G Tilton, Sam Kriegman, Josh Bongard, and Rebecca Kramer-Bottiglio A soft robot that adapts to environments through shape change. Nature Ma- chine Intelligence, 3(1):51-59, 2021.
Kail Sims. Evolving virtual creatures. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 15-22, 1994.
13

---

Published as a conference paper at ICLR 2025
Junru Song, Yang Yang, Wei Peng, Weien Zhou, Feifei Wang, and Wen Yao. Morphvae: Advancing morphological design of voxel-based soft robots with variational autoencoders. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 10368-10376, 2024a.
Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, and Yutian Chen. Position: Leverage foundational models for black-box optimization. In Forty-first International Conference on Machine Learning. 2024b.
Francesco Stella, Cosimo Della Santina, and Josie Hughes. How can Ilms transform the robotic design process? Nature machine intelligence, 5(6):561-564, 2023.
Luke Strgar, David Matthews, Tyler Hummer, and Sam Kriegman. Evolution and learning in differ- entiable robots. arXiv preprint arXiv:2405. /47112 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud. Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
InternI.M Team. Internlm: A multilingual language model with progressively enhanced capabilities, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307. 009888,, 2023.
Thanh VT Tran and Truong Son Hy. Protein design by directed evolution guided by large language models. IEEE Transactions on Evolutionary Computation, 2024.
Tingwu Wang. Yuhao Zhou, Sanja Fidler, and Jimmy Ba. Neural graph evolution: Towards efficient automatic robot design. arXiv preprint arXiv: 1erXiv::906.... 1e0.. 05370, 2019.
Yen-Jen Wang, Bike Zhang, Jianyu Chen, and Koushil Sreenath. Prompt robot walk with large language models. arXiv preprint arXiv:2309.09969 2023a.
Yuxing Wang, Shuang Wu, Tiantian Zhang. Yongzhe Chang, Haobo Fu, Qiang Fu, and Xueqian Wang. Preco: Enhancing generalization co-design of modular soft robots via brain-body pre- training. In Conference on Robot Learming, pp. 478-498. PMLR, 2023b.
Xingyu Wu, Sheng-hao Wu, Jibin Wu, Liang Feng, and Kay Chen Tan. Evolutionary computation in the era of large language model: Survey and roadmap. arXiv preprint 2rXiv:2401.1003. 2024.
Chengrun Yang. Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe V. Le, Denny Zhou, and Xinyun Chen. Large language models optimizers, 2024.
03409
Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Large language models as heuristics for combinatorial optimization, 2024. /tRp:tts: /ttp/v/rxvvv.//mm www.xii..rq/aaa.. abs 22022 01145
Lechen Zhang. Cuda-accelerated soft robot neural evolution with large language model supervision. aixiv preprint arXiv:2405.00664 2024.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A FULL PROMPTS
A.1 SYSTEM PROMPT FOR GENERATING OFFSPRING ROBOT DESIGNS
Now you will serva as an intel ligent search operator on Fvolutionary Algori each goneration you are given a number of evaluated solutions in the format of numpy array, together with their fitness scores. Pach solution and iis fitnness score are separated by comma. Di ferent solutions seporated by semicolons. The solutione are sorted according to their fitness scores in ascending order. Hiihee fitness scores are better. Your job is to output now soluuion that meets a desired fitness. Flease try your best logically analyze the relationship between the evaluated solutiens scores, and odhere to thig information on while proposing the now solution, A solution is a 5 times 5 natrix, where each entry is integer between zero and four. Please begin the new solution with <solution> and with s/solutions. The now solution should be formatted numpy arroy fashion. the new solution must be distinct from the evaluated solutions. Only generate the new solution. No explanation.
their fitness
A.2 USER PROMPT FOR GENERATING OFFSPRING ROBOT DESIGNS (CARRIER-VO AS AN EXAMPLE)
## Description of VSRa
ke are going to design she structure of a txo-dimansional vexel-based soft sobot [VSR) in a simulation environment. are composed of square-ahaped voxels of different typea, aligned into 5 natrix. Adjacent voxels (that in, in aither the same row or the same column are connected tegethery Voxels located in diagonal poaitions are not connected together, The robot ls subject to gravity, and the botton row touches the ground. There are types voxels available, including soft voxels (for which elastis deformation 1a possible), rigid voxels (which can not detorm), horisontal and vertical actuatora (which can change their sizes horizentally vertically), and empty voxels (which basically mean that the corsesponding pasiticn is emptyl , Empty vexels, rigid vowels, soft voxsin, horizental actuators vertical actuators are repcesented as 0, 1, 2, 3 and 4, respectively.
## Description of simulation environment
Th similtioo represents objects and their environment as 20 mass-spring system in a grid-like layout, overlaappn,,,,, connected voxela. simulation converts
where objects are initialised all objects into a set of paint masses and springs by turning each voxel into a cross-braced square, which may undergo defornation as the simulation progresses. The springs obey Booketa law, Note that adjacent voxels share point nasses springs on their conmor: edge. All point masses in the simulation have the sare mass and the equil lengths of axis-alignes and diagonal springa are constanta for simplicity. However, the spring constents assigned vary based vexel naterial-typs, with ties broken in tavor of the more-rieid apcing, The actuators undergo gradual expanstoon/ootractiio either horizontelly or vertically according to action signals, by changing the lengths of the correspending springs.
# Task description
Your 100 prosose robot designa suitable completing tollowing taak, three-voxel wide box is initialized right abeve the robot, and the robot is required to keep the box on top of its head stably without Letting it slip ott, while 1ocomoting rightwards as quickly as posaible.
## Constraints
There are two constraints VSR designs: 1. all voxels must form entirety and should not fell apart; That 1a, the tour voxels, 1f any, above, be now, to the lett and to the right of non-empty voxel mustr't
be empty at the same time. An exampie that viclates such a constraint is (,,,,,2,,,2) 21, (,,0,1,0,11,, 10, 1,311,,,, 4,,2,,,, in which the voxel '1" in the tourth now and fifth column would fall off because it is not ceneected to any nen-empty vexel; (2) there must be at least one actuator tshat 1s, either 3 or 4), so that the robot could interact with the envi ronment.
15

---

Published as a conference paper at ICLR 2025
Additional requirements
Please carefully analyze the relationship between evaluated solutions and their fitness scozes, and make of this information to propose the solution. Please
empty voxels cleverly that complex functional substructures could be produced to fulfill the purposes of both carrying and locomoting. Note that high-porforming robot design is not necessorily symetric..
tolite desi iniign-fitress paira anitted, sorted in ascending order)
"Just-ask" quary
Now please generate a new robet design that has fitness of (1.2ffuuuze,, max [itness)
A.3 USER PROMPT OF DIRECT
The solution that you just generated is too similar with an existing one, It needs further nedi ricatiinioon to improve diversity. Please decide which voxe la in the solution can be replaced other types of materialss voxels, Please
without harming its fitness score. Change cherocteristics of the evaluated solutiions ond given to you, Meonwhile, make sure that the nodi fication does not violate the constraints VSRs. Now please tell which voxels exactly do think can be alterced, and explain the zeason.
moze
your analysis on
(LIM suggesting modifications)
3ased your analyeis above, please generate the reaulting solution. The number of voxela changed should not exceed three. not provide further texual explanation.
A.4 USER PROMPT INTER-TASK KNOWLEDGE TRANSFER (WALKER-VO- BRIDGEWALKER-VE vO AS AN EXAMPLE)
(Descriptions VaRs, simulation envircament and constraints are same in therefore omitted. )
already have some higa--ernorming rebot designs from task named 'Kaker'' where the robot is required Locomote rightwards as quickly possible on rigid tlat terrain, The ropot designs are as tollows:
(Elite Walker-vO designs omitted.
Now your jeb is to propose ten hirf-pergo robot designs for another task naned 'Bridgeilalke=', where the robot is required to locomote ri rightwards quickly possi ble soft rope-bridge rather rigid terrain. Please analyze potential correlation between knowledge that can be transferred from Walkez to Beldgellalke,, and give ten robot designs that aze suitable for RridgeMal ker, Give your answer in numpy array fashion. Enclose each des' ign between <Solubi and and </Solution>>
tasks, identiff the
B INTRODUCTION TO TASK INSTANCES
In this section, we briefly introduce the tasks adopted for benchmarking in Evolution Gym. This introduction is heavily borrowed from their original paper (Bhatia et al. 2021).
We first define some notations that would be used later.
Denote with p" the position of the center of mass of an object which consists two components Pf and pG. i.e., the positions on x and axis. p" is derived by averaging the positions of all the point-masses that make up object o;
Position:
Velocity: Denote with 8" the velocities of the center of mass of object which consists two components ve and ug. i.e. the velocity on x and y axis. of is computed by averaging the velocities of all point masses that make up object o;
16

---

Published as a conference paper at ICLR 2025
Denote with 8", a vector of length one, the orientation of an object o. Denote the position of point mass of object o as ps. and 8 is computed by averaging over all the angle between the vector p; p" current time and the initial state. This average is weighted by Ilp: - yOll in the initial state.
Orientation:
Other observations: Let of all point masses of object to the center of mass. Let hg(d) characterize the terrain information around a robot below its center mass. More specifically, for some integer I d, the corresponding entry in vector hg(d) will be the highest point of the terrain which is lower than P between a range [z, x 1] voxels from pa in the z-direction.
a vector of length 2n that describes the relative positions
Besides, we would denote the robot as object r. the box that it is trying to manipulate as object b, the number of point masses in r as the observation vector S, and the reward function as R.
B.1 WALKER-VO
Figure 6: Walker-v0
In this task, the robot is required to walk as far as possible on flat terrain. S E RR consists of vt and et with lengths and n. R - Ap" rewards the robot for moving in the positive x-direction. The robot is also given one- time reward of for reaching the end of the terrain.
B.2 CARRIER-VO
Figure 7: Carrier-v0
this task, the robot is required to catch box initialized above it and carries as far as possible. S E RB+6 consists of vh pp p", v" and c" with lengths 2, and respectively. R= Ri + R2, where R2 = 0.5 Apl + 0.5 App rewards the robot and the box for moving in the positive x- direction, and Rg 0 if p > ty and otherwise 10 Ap, penalizes the robot for dropping the box below a threshold height ty. The robot also given a one-time reward of for reaching the end of the terrain.
B.3 PUSHER-VO
Figure 8: Pusher-v0
In this task, the robot is required push a box initialized in front of it. S Rn+6 consists of 2d p" p², and c" with lengths 2. n, 2 and 2 respectively. R R1 R2, where R₁
0.5 Apl. + 0.75 Apt rewards the robot and the box for moving the positive z-direction, and R2 = - App pll penalizes the robot and the box for separating in the z-direction. The robot is also given a one-time reward of for reaching the end of the terrain.
17

---

Published as a conference paper at ICLR 2025
B.4 CATCHER-VO
Figure 9: Catcher-vO
In this task, the robot is required to catch fast-moving and rotating box. The observation space E IR=+7 consists of pd p", v", vb 0 and e,, with lengths 2, 2, and n. respectively. The reward R = R2 + R2. where R1 - A 1r vel rewards the robot for approaching the box in the r-direction, and Rg = 0 if p 2 ty and 10 Ap otherwise penalizes the robot for dropping the box below threshold height ty.
B.5 WALDEEWALKER-O
Figure 10: BridgeWalkers
In this task, the robot required to walk as far as possible on a soft rope-bridge. S E RR+33 consists of o", 0" and c" with lengths 2, 1 and respectively. Rt Ap, rewards the robot for moving in the positive r-direction. The robot also given a one-time reward of for reaching the end of the terrain.
B.6 UPSTEPPER-VO
Figure 11: UpStepper-vO
In this task, the robot is required to mount stairs of varying lengths. S E Ro+14 consists of v", Or, Ap, rewards the robot for moving in the
c" and (5) with lengths 2, 1, n and 11, respectively. positive r-direction. The robot is given one-time reward of 2 for reaching the end of the terrain, and one-tine teetimme penalty of for rotating more than 75 degrees from its original orientation in either direction (after which the environment is reset).
18

---

Published as a conference paper at ICLR 2025
C HYPERPARAMETER SETTINGS
Table 3: Hyperparameter settings
hyperparameter
value
LASeR
robot size (height x width) upper limit of LLM interactions
5 x 5
200
probability of similarity check in DiRect
0.4
similarity threshold in DiRect
no more than 20 identical voxels
percentage of survivors in natural selection
linearly decreasing from 60% to 8%
PPO Policy Training
number of parallel sampling processes
4
number of time steps in each process learning rate in the clip function of PPO number of itcrations number of epochs per iteration number of mini- batihe per epoch X in generalized advantage estimation (GAE)
128
2.5 x 10
0.1
1000
4
0.95
D RESULTS OF SIGNIFICANCE TESTS
We conduct significance tests between LASeR and LLM-Tuner, the most competitive baseline. Since in this work we have chose a sufficiently large number of robot evaluations (i.e., 1000) to give ample opportunity to all algorithms to converge, becomes more relevant to compare the con- vergence speed rather than entire fitness curves. To this end, first average the eventual fitness values obtained by all repeated experiments (denoted as f) within a given task, and then record the number of evaluations that each experiment took to reach this average fitness (denoted as n1). For those that did not reach f. n is simply recorded as 1000. We then conduct a two-tailed f-test to compare the n's of different algorithms. For Carrier-vO, f is 10.69, and is average 719.2 and 979 for LASeR and LLM-Tuner, respectively (p 0.029).. For Pusher-vO, f is 12.95, and n. is on average 528 and 888.6 with p =0.054. For Walker-vO, since none of the experiments of LLM-Tuner reach f=10,65, we instead compare the eventual fitness values achieved by LASeR and LLM-Tuner, which are on average 10.67 and 10.63, with p <0.001. For Catcher-vO, the eventual fitness values of LASSRR and LLM-TTuner are 0.816 and 0.802, with p=0.08.
E COMPARISON OF DIRECT AND RANDOM VOXEL EDITING
To further showcase the effectiveness of our diversity reflection mechanism (DiRect), we re- implemented our experiments with DiRect replaced by random voxel mutations. Specifically, we implemented random editing by substituting the diversity reflection (DiRect) mechanism with ran- dom voxel mutation, which is supported by built-in function of EvoGym. Specifically, we found that the number of voxels edited by DiRect in each design is about 2.61 on average. Thus, for ran- dom mutation, we set the mutation rate to be 0.1, i.e. each voxel will, with a probability of 0.1, be randomly replaced by different material. Given that a robot design consists of 25 voxels, this results in 2.5 voxels being edited on average, which we believe is reasonably close to DiRect editing-
We conducted paired two-tailed Student's s f-test, and found that the fitnesses of randomly mutated robot designs are significantly lower than their pre-editing counterparts (p < 0.001). In contrast, the fitnesses of robot designs before and after DiRect modification show no significant difference (p = 0.19). These results suggest that stochastic exploratory behaviors would disrupt essential functional structures, whereas LLM-aided diversity reflection holds a distinct "informed" nature. Specifically, it builds on successful designs discovered along the evolutionary trajectory, and pro- motes exploration without compromising functionality.
19

---

Published as a conference paper at ICLR 2025
For illustrative purposes, here present two examples of diversity reflection, including both the pre- and post-editing morphologies, along with explanations provided by the LLM for its modifica- tions. It can be seen from Figure 12 that the LLM is indeed able to identify critical substructures within rohot designs, and modifies only the voxel placements that do not affect functionalityy yet promote diversity. These results provide sound evidence that DiRect is reliably functioning intelligent mutation operator.
1. Change the soft vowel at position (0,3) to an empty voxel.
Reasons: This position does not disrupt the structural integrity of the robet it on the edge of the robot design. Removing this soft vowel introduces empty vowel, which can help create diverse structure while still allowing the remaining voxels maintain connectivity.
2. Change the rigid voxel at position (3,0) to a vertical actuator.
Reason: This olteration allows for additional movement capabilities in that region. The vertical actustor facilitate upward mation, enhancing locomotion without violating any constraints,
3. Change the soft vosel at position (2,1) to horizontal actuator.
Reason: By replacing soft voxel with horizontal actustor, we introduce functionality that can aid in latoral movement. This change enhances the overall ficubbiity and capability of the design while keeping structure intact.
(a) An example of DiRect modifications on Walker-v0
1. Change the voxel at (1,2) from vertical actuator to rigid voxel.
this position does significantly contribute pushing mechanism since it is already surrounded by other actuators and soft voxels. Replacing with a rigid voxel will maintain structural integrity while potentia lly enhancing the overall rigidity of the
Reason:
The vertical
2. Change the voxel at (4,4) from soft voxel to a rigid voxel.
Reason: This position already supported by surrounding voxels, and converting it to rigid voxel increases stability at the base the robot. It also ensures that the bottom row maintains with ground, crucial effective pushing.
(b) An example of DiRect modifications on Pusher-vO
Figure 12: Illustrative examples of DiRect modifications. Fach example includes the pre- (left) and post- (right) editing designs, together with the modifications and justifications provided by an LLM to enhance morphological diversity.
F COMPARISON WITH TWO ADDITIONAL BASELINES
We conducted comparative studies between LASeR and two additional baseline algorithms. The first one is OPRO (Yang et al.) 2024), another evolutionary strategy that uses LLMs search operators, which we adapted for voxel-based soft robot (VSR) design. The second one is Morph VAE (Song al. 2024a), state-of-the-an co-design algorithm that does not employ LLMs but is also developed on the EvoGym platform.
As shown in Figure B LASeR consistently outperforms the two baselines in terms of optimization efficiency, reflected by its steeper fitness curves. Here we would like to elarify that we have delib- erately chosen sufficiently large number of robot evaluations hopefully allow all algorithms
20

---

Published as a conference paper at ICLR 2025
to converge for fair comparison. This explains why different algorithms end up with rather similar fitness levels. However, in the context of robot design automation, the convergence rate is an im- portant aspect for evaluating design algorithms, computationally expensive control learning, let alone the manufacturing costs of physical robots when deployed in real-world application. In this regard, LASeR exhibits considerable performance gains.
the evaluation of robot designs usually involves
10.70
Moxinal Finess
10.65
Maximal Finesss
10
10.60
8-
10.55
10.50
200
400
600
800
1000
200
400
600
800
1000
Number Evaluations
Evaluations
(a) Walker-vO
(b) Carrier-vO
0.8
12
Maximal Filness
Maximal Fitess
0.7
10
0.6
0.5
250
500
750
1000
200
400
600
800
1000
Number of Evaluatios
Number of Evaluations
(c) Pusher-vO
(d) Catcher-v0
LASeR (ours)
MorphVAE
OPRO
Figure 13: Comparison between LASeR (red), Morph (blue) and OPRO (green) in terms of optimization efficiency. In the original paper of Morph VAE (Song al. 2024a), two variants proposed that focus on either optimization efficiency or diversity. We conduct three repeated trials for each of them and take an average reflect the overall performance of Morph VAE. This is also the case for diversity calculation.
Furthermore, demonstrated Table LASeR achieves the highest diversity in two of four tasks, while Morph VAE is dominant in the remaining two tasks. Despite the competitive perfor- mance of Morph we note that LASeR holds distinct advantages: (a) with the novel diversity reflection mechanism, LASeR is capable of achieving more favorable trade-off between opti- mization efficiency and diversity, whereas MorphVAF proposes two variants, each of which focuses on one aspect and compromises the other; (b) Morph leverages variational autoencoder to approximate the high-performing robot distribution and generate offspring solutions, which lacks interpretability. On the contrary, LASeR can instruct an LLM to explicitly explain its design choices and thus provide valuable insights for robot design (see Appendix H). LASeR also capable of more intelligent knowledge transfer across different tasks, utilizing the reasoning capabilities of LLMs. Furthermore, the inferior diversity outcomes of OPRO once again reveal the inefficiency of LLMs balance exploitation with exploration on their own without diversity reflection.
G EVALUATION ON 10x10 WALKER-VO
In this work, adhered to the standard setup used in previous VSR studies, specifically 5x5 body size with five different materials, as this configuration is proven already expressive enough for complex and diverse morphological structures to emerge (Song al. 2024a Saito Okal 2024
21

---

Published as a conference paper at ICLR 2025
Table 4: Comparison between LASeR, Morph and OPRO in terms of diversity.
Walker-v0
Carricr-v0
Pusher-vO
Catcher-vO
Morph
16.20 (N/A)
33.16 (16.59)
18.18 (12.48)
11.00 (3.09)
OPRO
18.95 (4.19)
6,01 (2.41)
9,42 (2.08)
6.41 (1,48)
LASeR (ours)
23.09 (5.33)
20.87 (4.27)
20.91 (8.85)
8.07 (2.48)
Dong et al. 2023 Wang et al. 2023a b). Nevertheless, to evaluate the scalability of our approach to larger design spaces, tested both LASeR and LLM--Tuner Walker-vl with 10x 10 body size. Our findings, as presented in Figure 14 demonstrate that LASeR continues to outperform in this larger design space. We attribute this
the baseline in terms of optimization efficiency, success to the unique advantage of LLMs. Specifically, LLMs leverage their reasoning capabilities identify favorable voxel assembly patterns within high-performing designs, instead of relying on random mutations (as seen in genetic algorithms and other heuristics), to generate offspring solutions. This is also demonstrated in Appendix E and H where we show that LLMs are able to provide justifications for their decision making when generating offspring solutions and carrying out diversity reflection. It is worth noting that 10x10 configuration results in design space that is 2.65 10 times larger than the 5x5 case, due to combinatorial explosion. Therefore, the promising results indicate a remarkable potential LLM-basee evolutionary strategies to scale to even larger and more complex robot design problems.
10.40
Maximal Fitness
10.38
10.36
10.34
10.32
10.30
200
400
600
800
1000
Number of Evaluations
LASeR (ours)
LLM-Tunsr
Figure 14: Comparison of LASeR (red) and LLM-Tuner (purple) on 10x10 Walker-vO, The results averaged across four independent runs.
H LLM'S INTERPRETATION OF ROBOT DESIGN
In our preliminary experiments, we included explicit LLM reasoning, where the LLM was prompted explain its design choices (similar to thain-of-thoughhe However, we observed no significant per- formances gains by doing so, and therefore opted to remove the reasoning process to speed up evolution and save API costs. Nevertheless, we would like to clarify that our approach capable of affording higher interpretability. This can readily be achieved by explicitly instructing the LLM to explain its decision-making process, rather than serving as a black box. provide Figure 15 the output generated by GPT-4o-mini after receiving collection of high-performing Catcher-v0 designs and instructed to generate a new one with higher performance. The explanations provided insightful and reasonable, revealing the advantageous structures present in high performing de- signs. This suggests that LLMs are indeed able to align specific voxel assembly patterns with high performance, and leverage these principles generate improved offspring solutions.
22

---

Published as a conference paper at ICLR 2025
The designs provided demonstrate several key principles that contribute to their perfornence in the Catcher-vo task, where the goal is to catch and carry a box Here's summary of observed dasign principles:
1. Structural Stability: The bottom row consistently filled with rigid and soft voxels, which I provide a solid foumdation and enhance stability when carrying loads Rigid voxels are typically positioned towards the bottom
structural points to resist deformation under load.
2. Effective Use of Actuators: The designs incorporate both horizontal and vertical actuators strategically This allows for dynamic adjustments to the rohot's shape to better engage with the carried box The placement actuators near the center of mass or at junctions of flexible structures facilitates effective load distribution and manipulation.
3. Balance of Soft and Rigid Voxels: A mix of soft and rigid vexels provides the flexibility needed for absorbing shocks while also maintsining strucural integrity. Designs often use soft voxels in upper layers to allow for some deformation without compromising overall structure, which can help absorb impacts when catching the box.
4. Optimized Catching Mechanism: Successful designs tend to have a specific configuration that aids maximizing the area available catching the box. This often involves using soft voxels around the central area to create "cushion" effect. The presence of empty voxels strategically placed helps optimiie the weight distribution and maneuverabillly.
5. Symmetry and Connectivity: Many designs uxhibit syumettical features, which can help ensure balanced weight distribution and stability during movement. The connectivity between adjacent voxels I supports resilience against shear forces and promotes coordinated movements, particularly importantt during the catch-and-cary process.
6. Adaptahility: The ability dynamically adjust throngh the actuators enables the rohot adapt to various scenarios- this is critical when dealing with different box sizes or weights.
Here's a proposed new catcher design aimed at enhancing proficiency in the task-
proposed np.ara=n.p 4, 0, 2, [],[1, 4, 3, 3,, 2, 4, 4].[4, 1, 3, 1]. [U, 1, 2]]) Design Features:
1. Enhanced Ntahility: The bottom row consists of two rigid voxels flanking a soft voxel providing
stable base while allowing slight deformation.
2. Improved Actuation: The second row contains two actuators (one vertical and one herizontal).
eenablig more versatile manipulations and adaptations to different box sizes.
3. Strategic Placement Soft Voxels: The arrangement of soft voxels in the middle provides
cushioning during the catch phase, helping to absorb any sudden movements.
4. Central Weight Distribution: The design emphasizes centralized soft voxels, allowing for ceffcctive engagement and support of the box once canght.
5. Flexibility and Adaplation: Additional soft voxels above the actuators ensure adsptability in height and width as needed, improving the ability to secure and carry the box efficiently.
This proposed design aims to leverage the advantages existing high--prrfommin designs while introducing modifications that could enhance its capacity to perform well in the 'Catcher-vo) task. By emphasizing stability, adaptability, and an effective actuation mechanism, this design could potentially outperforu prior iterations.
Figure 15: The LLM-generated output on Catcher-vO. In this example, the LLM is prompted to explain its decision-making process when generating a new robot design.
I VISUALIZATION OF EVOLVED ROBOT DESIGNS
Here we provide visualizations of robot designs evolved by LASeR and baseline algorithms. We take Pusher-vO as an example, and randomly select five algorithm in a single run of experiment. It can be seen from Figure that the robot designs evolved by LASeR seem to exhibit the highest level of variability. However, we would like to make two notes here. First, as we are only able to display a small fraction of robot designs, this visualization could only serve as qualitative and intuitive verification of results presented in Section 4.2. rather than a valid means of evaluation on its own. Second, as we would detail in Appendix JJ the number of evolved performing designs should also be taken into account when measuring morphological
robot designs obtained by each
23

---

Published as a conference paper at ICLR 2025
diversity. To this end, believe the quantitative results reported in Section 4.2.1 still serve the most comprehensive and reliable evaluation of diversity.
Total number of evolved high-perforning robots
LASeR (ours)
203
LLM-Tuner
169
SE
55
BO
17
Figure 16: Visualizations of evolved high perforerfoninng robot designs on Pusher-vO. For each algo- rithm, five designs randomly chosen for display. Annotated on the right are total numbers of high-performing designs obtained. The result of RoboGAN is absent because fails to ohtain robot designs that surpass the fitness threshold (i.e., the 90% quantile of fitnesses achieved by all algo- rithms).
J FURTHER DISCUSSION ON DIVERSITY MEASUREMENT
Diversity is an important aspect for evaluating robotic systems and, in turn, the performance of robot design algorithms, diversified design alternatives are crucial for handling dynamic environments and increasing robustness of robotic systems. To our knowledge, previous studies have predomi- nantly employed two methods for quantifying diversity: (a) averaged measures of distinctiveness within group of robots, such per-voxel entropy (Song et al. 2024a) and pair-wise edit distance 2024); (b) manual categorization of robot designs into distinct classes, followed by the calculation of the Simpson index, which is analogous to an entropy measure of class distribution (Medvet et al. 2021). The latter method becomes impractical when dealing with more abstract mor- phologies without clear subpopulations. The former, the other hand, presents a paradox (Figure 17 including a new robot design into an existing collection can reduce diversity, even the new design is distinct, provided that falls within the distribution of this collection. Here, by "falling within the distribution" we mean that the distance between the new design and existing ones is on average smaller than that within the original collection.
(Saito & Oka
To address the above issue, we incorporate the number of distinct robot designs into our measure- ment correction. Thus, our two measures edit distance (measuring the distinctiveness of evolved designs) and the number of distinct designs complement each other, providing more comprehensive and reasonable characterization of diversity. However, we acknowledge that the weights assigned to these quantities (1.0 and 0.1) are somewhat expedient and primarily intended to bring them onto the same scale. This is based on our preliminary experiments where we found that the number of distinct nigh-performing designs obtained in single run of experiment typi- cally ranged from several dozens to around two hundred, while the edit distance is defined to range between 0 and 25. Given the lack of universally accepted metrics for measuring morphological di- versity, we hope we could inspire future work to devise even more reasonable and comprehensive approaches.
24

---

Published as a conference paper at ICLR 2025
edit distance -
edit distamce
include a new design
edit distunce -3
average edit distance
average edit distance
paradox with diversity measurement. The inclusion of new, distinct robot design decreases, rather than increases, the diversity when solely measured the edit distance. This is counter-intuitive as the addition distinct alternative should benefit diversity.
Figure 17:
We additionally provide the two separate measures of diversity before being weighted averaged (Ta- ble and (6- Our finding suggests that LASSeR has more of advantage in discovering distinct high-pett high-performers than achieving high averaged edit distance. We cannot actually state which ap- proach is more favorable, as both benefit diversity in their own way. However, combining the results of optimization efficiency (i.e., the fitness curves), it is clear that LASeR better balances exploration with exploitation.
Meanwhile, as we pointed out above, we believe that the results in Table somewhat misleading, because edit distance in itself does suffice as valid diversity metric. Even if group robot designs has another group as its subset, the former might still have a lower edit distance (even much lower, due the paradox Figure T7p. The comparative results in Table but we choose to display them to reveal open problem regarding diversity measurement and hopefully inspire future work to investigate further.
be in our favor,
Table 5: Comparative results of the number of distinct high-performing designs.
Walker-vO
Carrier-v0
Pusher-vO
Catcher-vO
BO
N/A
22.00 (6.38)
13.33 (3.86)
3.67 (1.25)
SE
13.25 (3.03)
34.40 (26.62)
6.00 (3,85)
7.00 (2.83)
RoboGAN
N/A
3.00 (N/A)
N/A
3.00 (0.82)
LLM-Tuner
33.75 (41.54)
55.00 (67.02)
46.25 (70.90)
5.67 (1.70)
MorphVAE
2.00 (N/A)
241.17 (179.83)
101.50 (133.73 73))
11.00 (10.18)
OPRO
138.00 (31.73)
17.75 (16.04)
53.25 (20.58)
9.50 (2.29)
LASeR(ours)
165.80 (53.09)
162.00 (43.23)
158.40 (88.68)
28.00 (23.69)
Table 6: Comparative results of average edit distance.
Walker-vO
Carrier-v0
Pusher-vO
Catcher-vO
BO
N/A
9.68 (2.25)
12.35 (2.48)
19.39 (0.98)
SE
(0.46)
11.82 (2.66)
6.66 (4.03)
13.21 (2.01)
RoboGAN
N/A
10.64 (N/A)
N/A
17.97 (1.5)
L.M- Tuner
8.23 (1.79)
12.76 (1.69)
9.54 (3.42)
13.33 (7.21)
MorphVAE
16.00 (N/A)
9.04 (3.36)
8.03 (3.72)
9.90 (4.00)
OPRO
5.15 (1.02)
4.24 (1,46)
4.10 (0.39)
5.46 (1.30)
LASeR(ours)
6.51 (0.09)
4.67 (0.34)
5.07 (0.43)
5.27 (0.31)
25

---

Published as a conference paper at ICLR 2025
K FINER-GRAINED ABLATION ON PROMPT DESIGN
The prompt used in our study consists of three major components: task-related metadata, elite design-fitness pairs, and larget fitness.
The task-related metadata primarily includes descriptions of task objectives and the sim- ulation environment. This component is largely derived from the official documents of EvoGym (Bhatia et al. 2021), with minimal modifications. This metadata, which is often overlooked in previous works on LLM-aided robot design, serves two main purposes: to ground the evolutionary process in the specific context of the problem, and to facilitate the transfer of knowledge between different tasks.
The second component consists of elite design-fitnes pairs previously evaluated, where the designs are sorted according to their fitness in ascending order. This sorting is intended leverage the pattern-com comeeencoomletionn capabilities of L.LMs, a technique shown be effective in prior rescarch (Lange et al. 2024 Yang al. 2024).
The third component, the target fitness (also referred to as the "just-ask query" in Lim et al. (2024)) introduced as means of aligning the LLM's outputs with our desired results.
We have demonstrated the indispensability of task-related metadata in Section 4.3.2 To further jus- tify our prompt design and to complement the intuitive explanations provided above, conducted finer-grained ablation studies and the results are reported in Figure 18] Specifically, we remove the following components one time: (a) the description of the simulation engine: (b) the description of task objectives; (c) the just-ask query (or target fitness); in this case, the LLM is simply prompted generated robot designs with higher fitness; and (d) the ascending ordering of elite design-fitnesss pairs according to fitness. Our findings suggest that removing any of these components leads to performance drops. The just-ask query is proven the most essential, while simulation description and ordering play less important roles.
10.675
10.650
10.625
Muximal Fitness
10.600
10.578
complete prompt
simvlltiion description
wio tskk description
10.525
just-ssk query
ordering
10.500
200
400
1000
Number of Evaluntions
Figure 18: Finer- greinggr ablation studies on prompt design.
We would like to note that the phrasing of these components is intentionally left simple and intu- itive, without applying special techniques of prompt engineering. As such, our experimental results possess certain level of robustness and do not hinge on the specifics of prompt designs. However, would be a promising direction integrate various prompting techniques into our framework for better performances.
L FURTHER DISCUSSION THE SIMILARITY THRESHOLD
In our diversity reflection mechanism (DiRect), a similarity threshold is needed to decide whether a newly generated robot design is overly similar to existing ones and therefore should undergo modifications by diversity reflection. This threshold is indeed crucial hyper-paramette that controls the performance of LAŠeR. The choice of this threshold reflects the extent of diversity that one expects to see in the evolved solutions, and hence should be driven by the user's specific preferences.
26

---

Published as a conference paper at ICLR 2025
For instance, setting it 20 (as did in experiments) means that if newly generated design shares more than 20 identical voxels with any existing solution, it will be modified by DiRect to introduce more variability.
Here, we present some general principles for choosing this parameter. These principles are sup- ported by additional experiments with several different values of threshold (as shown in Figure 19). High similarity thresholds, like 23, are generally not recommended, as they would hinder the beneficial exploration enabled by LLMs. Conversely, excessively low thresholds (such 15) might increase diversity but also risk overly aggressive exploration that compromises functionality and, in turn, harms optimization efficiency. We believe this is due the poor extrapolation performance of LLMs when required to propose robot designs that are much different from given examples. Any moderate values in the middle should lead to desirable performances. that a threshold of 18 leads to further performance gains beyond 20. However, we note that lower thresholds also more frequently trigger DiRect, which means more L.MM API calls. Hence, the threshold choice also involves trade-off between evolutionary performance (including both opti- mization efficiency and diversity) and computational costs, and should be considered case by case. We believe adaptive threshold scheduling, based on problem specifies and evolutionary outcomes, could be a promising direction for future research.
fact, our findings suggest
10.70
Maximal Fitness
10.65
10.60
similarity threshold
10.55
23
18
20
15
10.50
0
250
500
750
1000
Number of Evaluations
Figure 19: Additional experiments with several different similarity thresholds. The diversity results of these experiments are N/A, 28.51, 23.09 and 13.46 in the order of 15, 18, 20 and 23. The diversity not available for 15 because the corresponding experiment failed generate more than one high- performing robot designs.
M GENERALIZATION ACROSS DIFFERENT TASKS
In Section 4.2.2 we focused transferring design experience between task instances that in- tuitively similar (i.e., Walker-vO, BridgeWalker-v0 and UpStepper-vO), These experimental designs largely based on the structural similarities between tasks revealed in Wang al. (2023b). Here we demonstrate that this prior knowledge of task relationships is not strictly necessary for success- ful inter-task generalization. Specifically, we performed an additional zero-shot design experiment (Figure 20) where the LLM was given clite samples of Walker-vO, but instructed to propose designs for significantly different task, Jumper-vO, in which the robot is required to jump high as possi- ble. Despite the discrepancy between task objectives, we find that the LLM is able to dig deeper into the underlying inter-task associations and identify rather general, low-level design principles that are still relevant to the new task. Moreover, the zero-shot designs generated by LLM still outperform randomly sampled ones. See Figure 21 for visualizations of some elite Walker- vO designs, together with Jumper-v0 design generated by LLM. These findings suggest that LLMs possess substantial potential for generalizing experience across seemingly different optimization problems, as long as they not completely imreevant to other.
27

---

Published as a conference paper at ICLR 2025
0.2
The task 'Walker' and Jumppe' are different in their requirements. While the "Walker' task requires horizontal locomotion, the 'Jupper' task demands vertical motion. However, there are some aspects that can be transferred from the 'Walker' task the 'Jumper task:
0.1
1. Use of actuators for shupe chunge: The role of vertical actuators and borizontal actuators is crucial in both tasks. In the 'Wakker task, these actuators help the robot change its shape move forward. Similarly, in the 'Jumper' task, these actuators
Average Fitness
assist the rabot in changing its shape jump high. The springs between voxels would need to compress and then expand rapidly to genernte the upward force required n jump.
Centralized Soft Voxels; Placing soft voxels the middle allows for flexible deformation,
which can be beneficial for absorbing energy during landing and releasing when jumping.
-0.1
Rigid Structure at Base: Having rigid voxels at the base ensures stability while providing
solid foundation for launching into jumps.
Lateral Balanee: Designs should ensure that the mass well-distributed to prevent tipping over
-0.2
during jumps, especially when using actuators.
Random Sampling
Gradual Increase in Height: Building layers of voxels allow for gradual compression and extension, which aids in energ stonge for jumps.
LLM Proposals
(a) Zero-shot performance (b) Design principles that LLM generalizes from Walker-vO Jumper-vO
Figure 20: Additional zero-shot design experiment on Jumper-v0. The zero-shot performance is averaged over 10 robot designs for both random sampling and LLM proposals. Despite the task discrepancy, the LLM still able to identify general principles, such as the importance of actuators for shape change and rigid voxels for stability, to transfer to the new task.
Some the elite Walker-vO designs
An LLLM-proposed Jumper-v0 design
Figure 21: A visualization of some elite Walker-vO designs (left), and the Jumper-vO design gen- erated by LLM (right). The Jumper-v0 design seems different than its Walker-vO counterparts, but they still share some basic design principles, such rigid voxels in the bottom row, soft voxels in the middle rows and well-distributed mass. These features largely mirror the analysis by LLM in Figure 20
N FURTHER DISCUSSION ON LIMITATIONS AND FUTURE WORK
Here expand the discussion of limitations within this work, and outline several open problems that we find promising for future research.
In this work, we developed our approach on voxel-based soft robots (VSRs), and adhered to the 5x5 body size commonly adopted in previous VSR studies. However, would like to note that our approach is conceptually adaptable to larger design spaces (as verified in Appendix G and potentially other types of robots. We look forward to seeing our LLM- based evolutionary framework evaluated in more extensive and complex design problems.
Regarding inter-task generalization, However, we proved that this prior knowledge of task relationships is not necessary for successful transfer of design experience (as demonstrated in Appendix M). It would be promising to investigate how this inter-task reasoning capability of LLMSs could be lever- aged in a broader range of optimization problems.
focused intuitively similar tasks in paper.
Currently, we employed LLMs in the most cost-effective way for robot design automation, that is, through in-context learning. It remains be studied whether fine-tuning LLMs could lead to further performance gains. To our knowledge, fine-tuning Large Language Models typically involves substantially higher costs, both in terms of computational sources and the need of a sufficiently large and carefully curated dataset. However, perfor- mance might not be guaranteed due issues like overfitting and catastrophic forgetting.
28

---

Published as a conference paper at ICLR 2025
Nevertheless, the prospect of a general-purpose LLM fine-tuned for various combinatorial optimization problems is intriguing, and represents an open problem for future research.
In this work, we followed the standard control learning approach used in previous EvoGym- based studies, i.e., PPO algorithm with MLPs serving as control networks. Since our pri- mary focus on the evolutionary capabilities of LLMs, we chose this simple yet effective approach for fitness evaluation. That being said, broad range of alternatives, such as value-based methods and differential simulations (Strgar et al. 2024; Cochevelou et al.) )023), could be tried out, with their impact LLMM-aided evolution examined. We also find it an interesting direction to integrate LLMMaided control strategies (such as those in Wang al., (2023a), Broohnn al. (2023) and Cheang et al. (2024) into our framework, so that LLMs are capable of both designing and controlling their own embodiments.
Morphological diversity is an important aspect for evaluating robot design algorithms, as diversified designs crucial for ensuring the robustness of robotic systems in highly volatile environments. We pointed out the limitations of previous diversity measures in Appendix D Jand proposed to make correction by taking both distinctiveness and the num- ber of distinct designs into account. We hope our work could inspire future work to develop even more reasonable and hopefully universally acceptable metrics for diversity.
Last but not least, while work is based simulation, note that there is ongoing research on the realization of soft robotics in the physical world, using polymers with pneu- self-replicating cells (Kriegman et al. [020a] 2021) and continually narrowing the sim-to-real gap. We believe that with the collective efforts of material scientists, computer scientists, (ech)necha engineers, etc., soft robotics would see rapid advances and finds its way to everyday life in the near future.
matic chambers (Kriegman al. 2020b Legrand al. (023)
O AN ANALYSIS OF COMPUTATIONAL EFFICIENCY
According to the data released on LLM Leaderboard http://attiiiaaaummlssssss/eeeed for GPT-4o-mini, the median rate of output token generation is 99.8 tokens per second, and the latency (i.e,, time to first token) is reported as 0.5 seconds. Given that LASeR makes an average of 130 API calls per generation, with each call involving approximately 180 output tokens (here we assume the worst case where each newly generated robot design triggers DiRect), this results in an overhead of around 5 minutes per generation, or 5 hours in total.
The latency issue could be mitigated with locally deployed LLMs, which are less affected by net- work delays and request queuing. However, believe it is more pertinent compare the overall running time of different methods, with optimization efficiency taken into account. Specifically, by checking the log messages of our programs, find that, for Carrier-vO, in order to reach the same level of fitness, LASeR requires 7 hours, in contrast to the most competitive baseline, LLM- Tuner, which takes 15 hours. For Pusher-vO, the difference is greater: LASeR requires 11 hours, whereas LLM-Tuner takes 46 hours. On Walker-vO, LASeR is even capable of reaching fitness unattainable by baselines. Hence, the rapid convergence enabled by LLMs outweighs the addi- tional computation overhead, rendering the latter perfectly worthwhile.
P PSEUDO CODE OF LASER
29

---

Published as a conference paper at ICLR 2025
Algorithm 1: LASeR: LLM-Aided Evolutionary Search for Robot Design Automation
Input: A task instance T, maximum number of evaluations M, population size N, survival rate r, maximum number of LLM interactions each generation L, probability of similarity check similarity threshold s.
Output: M robot designs together with their fitness scores.
Randomly initialize population P.
//Or initialize with zero-shot proposals from LLM IIS keeps all the evaluated robot designs and their fitness scores Warm start with several generations of traditional EA.
S
while SI < do
attempts t- 0 the top N robot designs among P; survivors while IPI < N and attempls < T. do
//Track the number of LLM interactions //Natural selection
target f'imess max fitness(S)x1.2 //Calculate target fitness 1.2 x current prompt + [metadata of task T, survivors in ascending order, target- //Query the LLM for a new offisring solution
robot + LLM(prompt)
attempts - attempts +
(0, if u < p then
//Enter similarity check with probability
Check the similarity rate s' of robol to existing robot designs. if then
P - P [rrbbol
//Pass the similarity check
else
robot -DiRect(roobl)
//Moiify with Diversity Reflection Mechanism Check the similarity rate s' of robot to existing robot designs.
8' < then P < Probott
//Pass the similarity check after modification
|P| N then
/LLM has not generated enough robots
Generate the remaining offspring with the traditional EA.
for robot P
//Control optimization and fitness evaluation
if robot not in survivors then
Optimize a controller for robot. Evaluate the cumulative reward as its fitness f.
S S (roobot,, f)}
Return: S
30


Reviews:
[{"review_id": "7zhPq4wqk1", "reviewer": "Reviewer_pDhF", "summary": "The paper presents LASeR, an framework leveraging Large Language Models (LLMs) to optimize robot design with evolutionary algorithms. The proposed approach addresses the limitations of existing LLM-driven optimization techniques, such as limited solution diversity and generalizability across tasks. LLM is employed as the intelligent search operator and diversity reflecter, instead of a tool for hyperparameter tuning. By introducing a Diversity Reflection Mechanism (DiRect), LASeR refines the exploration-exploitation tradeoff, enhancing diversity and performance of the robot design automation tasks, comparted to baselines. Through task-grounded prompts, LASeR also enables effective knowledge transfer across different robot design tasks.", "questions": "* What considerations were taken when selecting similarity thresholds? How do you balance the diversity and performance of the generated designs?\n* In Section 3.2.1, you mentioned that the fitness performance of robot designs would not change significantly after being modified by DiRect. Do you have quantative results to support this conclusion? \n* How does the computation time of LASeR compared to the baselines, including LLM-Tuner and the ones without using LLMs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper is clear and well-structured.\n* The idea of applying LLMs in generating offspring for robot design evolutionary algorithms is interesting.\n* Extensive experimental results on EvoGym are provided to validate that the proposed method outforms baselines in both design efficiency/performance and diversity.", "weaknesses": "* The experiments of the paper did not mention the time taken of the LLM-based methods for the evolutionary algorithm, which should be considered into the evaluation of robot design efficiency. \n* The similarity threshold seems to be an important hyperparameter for the proposed method, while it is not discussed in the paper.\n* The experiments in the paper are restricted to relatively simple voxel-based soft robots within predefined settings.\n* The core method does not involve learning or fine-tuning for LLMs besides PPO utilized for the fitness evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Olz7RjINoK", "reviewer": "Reviewer_D75R", "summary": "In this paper, the authors introduce LASER (Large Language Model-Aided Evolutionary Search for Robot Design Automation), a novel approach that leverages Large Language Models (LLMs) to enhance the efficiency and diversity of robot design automation. In  LASER, LLM is integrated into the bi-level optimization framework, and the LLM is prompted to be the mutation operator for generating new robot morphologies.", "questions": "1. The testing tasks such as Walker-v0 and Carrier-v0 used in the paper are too simple, can you test your method on more complex tasks (\"Climber-v0\", \"Catcher-v0\", \"Thrower-v0\" and \"Lifter-v0\"), which I think are more suitable to answer the question \"Does LLM really have an advantage ?\"\n\n2. Can large language models really align a robot's design with its task performance? Is it enough to just use prompt engineering for more complex tasks? Can it be used as a surrogate model to predict the performance of a robot design? Can the authors give some explanations?\n\n3. Can the current framework scale to more complex and larger robot designs (10x10 design space for walking)? If not, what are the potential bottlenecks? In larger design space (10 x 10), does LLM still work well? For some tasks, random combinations of voxels generated by LLM or evolutionary operators don't always work well.\n\n4. To further improve this paper, it is better to show the designed robots by LLM and add analysis of the differences between llm-generated robot designs and  GA-generated robot designs.\n\n5. While the paper demonstrates inter-task knowledge transfer, how well does LASER generalize to tasks that are significantly different from the ones used in the experiments? What are the limitations of this generalization?\n\n6. The authors need to compare their approach with those that also use LLM as a mutation operator， such as openELM (Evolution through Large Models (ELM)) and more recent brain-body co-design methods (does not use LLM) which also use EvoGym platform, to show the effectiveness.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors proposed a reflection mechanism for automated robot design, DiRect, to encourage more knowledgeable exploratory behaviors from LLMs based on past search trajectories. Besides, they effectively leverage the reasoning and decision-making capabilities of LLMs to improve the inner-task transfer propagability.", "weaknesses": "1. The use of LLM as an evolutionary operator (powered by prompt engineering) is interesting, similar ideas such as \"Evolution through Large Models (ELM)\" and [1-2] have been proposed. The paper shows a possible pipeline of integrating LLM into the co-design process of VSR, but does not provide a deeper analysis about \"Why LLM works well?\". The black-box nature of LLMs can make it challenging to understand the reasoning behind the generated designs, Adding more explanations in the LLM's decision-making process would be beneficial.\n\n2. The paper mentions experimenting with different temperatures but does not provide a detailed sensitivity analysis of different prompts. In my opinion, the explanation of the intuition of your designed prompts is more important than the proposed pipeline. \n\n3. This paper is missing a comparison with some important baseline algorithms.\n\n4. The test tasks chosen for this paper are too simple to demonstrate the superiority and validity of large language models.\n\nReferences\n\n[1] Lange, Robert, Yingtao Tian, and Yujin Tang. \"Large language models as evolution strategies.\" Proceedings of the Genetic and Evolutionary Computation Conference Companion. 2024.\n\n[2]Hemberg, Erik, Stephen Moskal, and Una-May O’Reilly. \"Evolving code with a large language model.\" Genetic Programming and Evolvable Machines 25.2 (2024): 21.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6SchGTjVc3", "reviewer": "Reviewer_5ABx", "summary": "The paper investigates the use of large language models in designing and evolving robots. To this end the paper uses an LLM to reflect and propose novel 'soft' robot designs in simulation in an evolutionary design loop. The paper compares its proposed LLM-evolution loop against different baselines and presents in-depth ablations of different effects LLM parameters have on the design loop.", "questions": "I have no questions, overall I think the paper is in a good state and interesting to the ML/robotics community.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- To the best of my knowledge the proposed framework is novel, the usage of LLMs in the problem of robot designs and their evolutions is under-researched\n- The conclusions the paper makes, and its application are relevant to the robot learning community\n- The paper compares its proposed approach versus several baselines\n- The performed ablation studies are very interesting and insightful. I appreciate them.", "weaknesses": "Weaknesses:\n- The environments in which the method is tested are relatively simple. However, I appreciate the hardness of the overall problem; designing and evolving robot hardware is not easy.\n- A critical remark is that while the mean shows (in plots and tables) that the proposed method works, I think it is likely not statistically significant due to the standard deviation and the closeness of the final means.\n- I think the paper could overall more critically discuss its limitations and open problems.\n- The paper should probably cite and discuss this preliminary work discussing the potential of LLMs for the robot design process: Stella, Francesco, Cosimo Della Santina, and Josie Hughes. \"How can LLMs transform the robotic design process?.\" Nature machine intelligence 5, no. 6 (2023): 561-564.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lX265qCEsN", "reviewer": "Reviewer_drYB", "summary": "This work proposes and evaluates an LLM-based evolutionary operator for robot design. The proposed method distinguishes itself from prior related work by incorporating an explicit mechanism for “reflection” and “inter-task knowledge transfer” where the former intends to balance exploitation and exploration and the latter intends to exploit existing robot design datasets whilst leveraging LLMs ability for inter-task reasoning.", "questions": "1. How do you justify the proposed measures of diversity and the weighted averaging used? Can you provide other measures? For example, the proportion of bodies made up of different voxel types? \n\n2. Can you provide concrete examples of morphologies that emerged using your method compared with others? Is the diversity in these collections immediately observable just by looking at the bodies? \n\n3. Can you run these experiments again several times over to provide more meaningful performance measures, confidence intervals and statistical hypothesis testing? \n\n4. How do you explain the early convergence of your method when a primary claim relates to encouraging population level diversity and design exploration? \n\n5. Can you perform additional ablation studies of the diversity reflection aspect? For example, randomly edit voxels and compare results to LLM-based editing? Can you catalog examples of LLM edits that encourage diversity through an evolutionary lineage? \n\n6. Can you run experiments with a larger design space? Perhaps 9x9?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality\n\nThis paper demonstrates originality by identifying the need for and explicitly developing a mechanism aimed at solving lack of diversity in LLM-aided robot design processes. The idea of using existing robot design datasets and inter-task reasoning to transfer or modify it also adds originality to this work. \n\nQuality\n\nThe authors clearly set out their intended contributions and state their scientific hypotheses as well as the experiments they intend to test these hypotheses. The figures throughout the paper are high quality and aesthetically pleasant. The authors do a good job of covering a fairly broad portion of the related literature and providing motivations for their own work. \n\nClarity\n\nThe paper is well-written and easy to follow. The methods and results figures are easy to interpret. The main method figure is done particularly well and makes it easy to understand the relatively involved, multi-step process that is the proposed algorithm. The tables throughout the paper are also well-labeled without superfluous text. \n\nSignificance\n\nJoint design and control of robotic systems is an important problem and provides a canonical example of a combinatorially explosive design space automated methods aspire to solve. The lack of diversity or tendency towards local optima in the morphological design process is a known limitation of evolutionary robotics broadly, so research and development in this area is crucial to advance the field.", "weaknesses": "Small design space\n\nA 5x5, two dimensional design space makes it difficult to discover interesting structures or behaviors, rendering the implications for design (let alone robotics) somewhat unclear. The search space is much smaller than most work over the past three decades, which has been in 2D but at much higher resolution with hundreds of independent motors (https://www.roboticsproceedings.org/rss20/p100.html) or in 3D with hundreds or thousands of voxels (https://www.creativemachineslab.com/soft-robot-evolution.html). The paper would be much more compelling if LLMs were operating over a 3D space where variety in gait patterns more readily appear and the control complexity increases substantially. \n\nWeak notions of diversity and lacking examples\n\nThe paper proposes the measure diversity in terms of the voxel space edit distance of robots and the number of distinct, high-performing robots. The latter is likely a poor measure of diversity as two robots can be highly similar while remaining distinct in terms of the precise voxel layout, and the former is difficult to interpret. Moreover, the paper provides no examples of the morphologies (and diversity) discovered by their algorithm. The overall diversity measure is the weighted average of these two metrics with weights of 1.0 and 0.1. There is no rationale for the selection of these weights outside of an anecdote that weighting the latter by 0.1 makes the two “roughly on the same scale and given equal importance”. \n\nDiversity reflection incomplete ablation \n\nFollowing up on the above point, the paper reports an ablation study to test the effectiveness of their diversity reflection mechanism; however, the ablation does not elucidate whether the LLM is actually providing intelligent mutations that encourage diversity. An additional ablation study should be run wherein random mutations are made to existing designs (in parallel to LLM guided mutations). This would help demonstrate whether the diversity reflection mechanism represents an intelligent operator. \n\nEarly convergence\n\nAlso related to diversity, the proposed algorithm appears to converge very early relative to some other baselines in most cases. This appears to be an indication that the algorithm may be stuck in a local optima, or is it closer to a global optima? If allowed to run for longer would the other methods arrive at a similarly high performing result? If it is indeed discovering something that is close to globally optimal then the fast convergence should indicate that this task (read design space) is too easy. \n\nIt also appears that when using the diversity reflection component the algorithm converges faster, which seems somewhat counter-intuitive as one would expect greater exploration to produce longer convergence times. The fact that it does not leads back to a prior question as to whether the LLM is truly modifying the design in intelligent ways that ultimately produce meaningful diversity in the population. \n\nMarginal gains in performance\n\nWhen all is said and done the proposed method produces marginal gains in performance relative to baselines. In the ablation study with and without diversity reflection performance also does not change substantially whereas the diversity does improve significantly. The diversity metric itself remains questionable (see above). \n\nThe knowledge transfer mechanism also does not appear to itself demonstrate meaningful improvements relative to the LASeR without knowledge transfer. \n\nReproducibility\n\nThe paper states that all experiments are conducted three times and the results are averaged. Since the performance gains are relatively small, additional trials are necessary to demonstrate statistical significance of the results. \n\nMissing related work\n\nThis paper fails to discuss, compare and contrast their work with other similar methods that use LLMs to design robots, for example: https://link.springer.com/chapter/10.1007/978-981-99-3814-8_11\n\nChoices about the control policy and its training also bears at the very least some discussion and comparison to other evolutionary robotics approaches that employ other methods, such as gradient based optimization (https://www.roboticsproceedings.org/rss20/p100.html), for the control problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)