RoboMoRe: LLM-based Robot Co-design via Joint Optimization of Morphology and Reward

## Abstract

Abstract Robot co-design, jointly optimizing morphology and control policy, remains a longstanding challenge in the robotics community, in which many promising robots were generated. However, a key limitation lies in its tendency to converge to sub-optimal designs due to the use of fixed reward functions, which fail to explore the diverse motion modes suitable for different morphologies. Here we propose RoboMoRe, a large language model (LLM)-driven framework that integrates morphology and reward shaping for co-optimization within the robot co-design loop. RoboMoRe performs a dual-stage optimization: in the coarse optimization stage, an LLM-based diversity reflection mechanism is integrated to generate both diverse and high-quality morphology–reward pairs and efficiently explore their distribution. In the fine optimization stage, top candidates are iteratively refined through alternating LLM - guided reward and morphology gradient updates. RoboMoRe could optimize both efficient robot morphology and corresponding suited motion behaviors by reward shaping. The results demonstrate that without any task-specific prompting or predefined reward and morphology templates, RoboMoRe significantly outperform human-engineered design results and competing methods in eight different tasks.

## 1 Introduction

Figure: Figure 1: Diverse motion behaviors and robot morphologies generated by RoboMoRe. Orange: A bio-inspired long-legged robot crawling forward in a climber posture. Blue: A uniform robot advancing by jumping. Purple: A low-center-of-gravity robot rolling forward along the ground.
Refer to caption: extracted/6498497/Fig_teaser_v2.jpg

One of the main goals of artificial intelligence is to develop effective approaches for the creation of embodied intelligent systems [29]. Inspired by natural organisms, where body structure and brain are two key factors for completing any task in a real environment, a successful intelligent robot typically requires concurrently optimizing its structure design and control mechanism [25]. Such a co-design problem has been a long-standing key challenge in the robotics and machine learning communities [32, 20, 34].

Although many existing approaches have achieved promising results, they typically rely on a fixed reward function, which significantly limits the potential of robot co-design [19, 38]. Under such constraints, robots are only capable of learning a narrow range of motion behaviors tied to a single, static objective. For robots with varying morphologies, a single, uniform reward function severely constrains improvements in locomotion performance and the exploration of diverse motion modalities[8, 33, 15].

To address this limitation, we introduce RoboMoRe—a novel LLM-based robot co-design framework that jointly optimizes both Robot Morphology and Reward functions. The core philosophy of RoboMoRe is to identify optimal reward functions tailored to each robot morphology. By tailoring reward functions to match specific morphologies, we can unlock a wider variety of motion behaviors and boost overall performance. For example, a long-legged robot might benefit from a reward that encourages jumping, while a low-profile robot might perform better with a reward that promotes rolling. Please refer to Appendix [B](https://arxiv.org/html/2506.00276v1#A2) for more details.

Our framework follows a Coarse-to-Fine Optimization paradigm. In the Coarse Optimization phase, we introduce a simple and effective Diversity Reflection mechanism in which the LLM observes previously generated morphology-reward pairs and proposes new, diverse candidates for subsequent grid search. This promotes the exploration of the joint distribution space and ensures a wide coverage of viable design alternatives. Then, in the fine optimization phase, we iteratively refine the most promising candidates in an alternating optimization scheme [4]. Specifically, the LLM first refines the reward function based on existing samples, then adjusts the morphology accordingly. This reward-morphology alternating cycle continues until convergence on high-performing design pairs. By fully harnessing the power of LLMs, we aim to inspire a new direction in robot co-design that tightly couples morphology and reward shaping.

In summary, our contributions are three-fold:

- 1.
We propose RoboMoRe, a novel co-design framework that leverages LLM to jointly optimize robot morphology and reward functions.
- 2.
We adopt a Coarse-to-Fine optimization paradigm that combines Diversity Reflection for grid search and alternating refinement for precise optimization.
- 3.
Our quantitative and qualitative evaluations show that RoboMoRe significantly outperforms both human-designed and competing automated methods across eight representative design tasks.

## 2 Related Work

##### Robot Co-design

Early evolutionary robotics demonstrated that co-evolving morphology and control policies outperform sequential design, highlighting the interdependence of “body” and “brain” in agents [6]. Early work framed robot co-design as a graph search problem and addressed using the advanced evolutionary algorithms, yielding progressively more intriguing result [26, 13, 22, 31, 38]. However, as both kinematic complexity and task difficulty grow, approaches such as evolutionary algorithms, Bayesian optimization, and deep reinforcement learning must contend with vast design spaces that demand substantial computational resources and more efficient search strategies, thereby constraining their development. Recently, large language models (LLMs) hold significant promise for transforming the robot design process thanks to their strong in-context learning capabilities and extensive prior knowledge. However, research on LLM-based robot co-design remains limited to only a few early exploratory studies [36, 23, 18, 28]. Among these, LASER [28] has shown promising results in designing soft voxel robots (SVR), particularly with regard to reflecting diversity. However, it still relies on carefully crafted task-specific prompts and a highly narrow method for diversity reflection which can only be applied to SVR, greatly limits its generalizability to other types of robots. In contrast, our work introduces highly general prompts and a more flexible approach to diversity reflection. Most importantly, existing methods share a critical limitation with fixed reward strategies, which significantly hinders their effectiveness in robot co-design.

In general, while these methods demonstrate potential, previous approaches have not incorporated reward shaping. By fixing the reward function and only optimizing robot morphology—i.e., Fixed Reward, Varying Morphology—they have restricted the possibilities of true co-design. In our paper, we argue that reward shaping can encourage different robots to develop distinct and diverse movement strategies, ultimately resulting in superior performance.

##### Reward Shaping

Reward shaping is a pivotal component in the development of robotic control strategies. Early reward engineering methods predominantly relied on manual trial-and-error tuning [17] and inverse reinforcement learning (IRL) [16]. Manual tuning is time-consuming and requires substantial domain expertise, whereas IRL demands costly expert demonstrations and frequently produces opaque, black-box reward functions, thereby limiting its practical applicability. Subsequent researches have investigated automated reward optimization via evolutionary algorithms [9, 7], but these efforts were typically limited to task-specific implementations that only tuned parameters within predefined reward templates. More recently, large language models (LLMs) have been used to generate reward functions for new tasks. For instance, Eureka [21] provides a general framework that, without any task-specific prompting or pre-defined reward templates, can generate reward functions that outperform human-engineered ones. Similarly, Text2Reward [35] shows that LLM-based reward shaping can produce symbolic, interpretable rewards and enable robots to learn novel locomotion behaviors (e.g., Hopper backflip, Ant lying down).

Inspired by these advances, we introduce LLM-driven reward shaping into robot co-design. By generating diverse reward functions with LLMs, RoboMoRe enables robots with varying morphologies to discover novel motion strategies, significantly enhancing their performance.

**Table 1: Comparison of RoboMoRe with related literature.**
| Literature | Method | Mor. Design | Reward Shaping | Diversity | Iterations | Optimum |
| --- | --- | --- | --- | --- | --- | --- |
| EvoGym [5] | BO, EA | ✓ | ✗ | N/A | Slow | Local |
| RoboGrammar [38] | Graph Search | ✓ | ✗ | N/A | Slow | Local |
| RoboMorph [23] | LLM | ✓ | ✗ | Off | Fast | Local |
| LASER [28] | LLM | ✓ | ✗ | On | Fast | Local |
| Eureka [21] | LLM | ✗ | ✓ | Off | Fast | Local |
| RoboMoRe (Ours) | LLM | ✓ | ✓ | On | Fast | Near Global |

## 3 Problem Definition (Robot Co-design)

The Robot Co-design problem involves jointly optimizing a robot’s morphology and control policy to maximize its performance in a given environment. It is defined as a tuple $P=\langle\mathcal{M},\Theta,\mathcal{R},\mathcal{A},F\rangle$, where $\mathcal{M}$ is the environment model, $\Theta$ is the robot morphology design space, and $\mathcal{R}$ is the space of reward functions. $\mathcal{A}(\theta,R):\Theta\times\mathcal{R}\to\Pi$ is the co-design algorithm that takes a robot design $\theta$ and reward function $R$, then learns an optimal policy $\pi$. $F$ is the fitness function which measures real-world performance but can only be accessed through policy execution.

Existing works aim to solve this problem via searching the optimal robot design $\theta^{*}$ that lead to the highest performance given a fixed reward function $R_{0}$:

$$ $\theta^{*}=\arg\max_{\theta\in\Theta}F(\pi_{\theta,R_{0}}),$ (1) $$

However, this can lead to local optimum due to limitation of $R_{0}$. To address this, our objective is to jointly optimize the robot morphology $\theta^{*}$ and reward function $R^{*}$ that lead to the global optimal performance:

$$ $\theta^{*},R^{*}=\arg\max_{\theta\in\Theta,R\in\mathcal{R}}F(\pi_{\theta,R}),$ (2) $$

where the optimized policy is obtained by:

$$ $\pi_{\theta,R}:=\mathcal{A}(\theta,R).$ (3) $$

## 4 Method

In this section, we formally introduce the RoboMoRe pipeline. See Fig. [2](https://arxiv.org/html/2506.00276v1#S4.F2) for an overview. Traditional robot co-design approaches typically optimize morphology under a fixed reward function, which constrains the search space and often results in sub-optimal behaviors misaligned with the robot’s morphology [19].

To address this limitation, RoboMoRe introduces a co-optimization framework that simultaneously explores morphology and reward function space, enabling both components to co-evolve toward globally optimal performance. By dynamically adapting reward functions to suit varying morphologies, RoboMoRe unlocks diverse and morphology-aware motion behaviors, maximizing overall efficiency. For instance, a flexible robot equipped with elongated limbs should be encouraged to optimize its movement through jumping-based locomotion, leveraging its mechanical advantage. In contrast, a heavier robot with short limbs may be better suited for rolling-based movement, as this motion minimizes energy expenditure while maximizing travel distance. This adaptive approach allows RoboMoRe to systematically align each morphology with its most compatible motion strategy, resulting in superior performance across a wide range of robotic configurations.

Figure: Figure 2: The overall pipeline of RoboMoRe. The LLM is provided with the task description, environment code, and masked structure file as the input prompt. The environment code specifies the simulation variables to write reward functions, while the masked structure file provides masked morphology parameters of robots for the LLM to complete. Our algorithm follows a coarse-to-fine strategy. First, the LLM leverages diversity reflection to generate a diverse set of reward functions and morphologies, enabling broad exploration of the reward-morphology design space (the coarse optimization phase). Then, through alternating optimization between morphology and reward, the framework iteratively refines both to arrive at an optimal morphology-reward pair (the fine optimization phase).
Refer to caption: extracted/6498497/Figure-2.jpg

### 4.1 Coarse Optimization: Grid Search with Diversity Reflection

The goal of coarse optimization is to approximate the distribution of morphology-reward pairs near the global optimum. To achieve this, the generated samples must satisfy two key criteria: 1) diversity, meaning the samples should be sufficiently varied to avoid premature convergence to local optima; and 2) high-quality, indicating that the samples should also cluster near the global optimum to improve sampling efficiency.

Traditional approaches typically rely on random sample (e.g., sampling initial candidates from gaussian distribution) [13, 5], which although ensures diversity, often fails to generate samples sufficiently close to the global optimum. As a result, these methods require hundreds of thousands of optimization steps to converge, making them computationally expensive and inefficient. To address this, RoboMoRe leverage LLM and a simple Diversity Reflection mechanism to generate high-quality and diverse samples. We leverage the advanced reasoning, comprehension, and design capabilities of LLMs to generate high-quality samples. For morphology design, we input the task description alongside a masked robot structure file (e.g., an XML file for the MuJoCo simulation environment) into the LLM, prompting it to generate a novel set of robot design parameters. Crucially, we mask all morphology-related parameters (e.g., limb lengths) within the structure file to eliminate bias stemming from human-designed priors, ensuring a fair and unbiased evaluation of the generated designs. For reward functions, following the Eureka, we directly ingest the raw environment source code—excluding any pre-existing reward functions—as contextual information. This enables the LLM to synthesize candidate reward functions that are both tailored to the task and executable in Python, seamlessly integrating into the simulation environment. We impose a strict output format to enforce structural consistency and coherence. Although task-specific prompt engineering could enhance the quality of generated samples [28], RoboMoRe adapts a highly generalizable prompt to ensure scalability in diverse tasks. Please refer to Appendix [A](https://arxiv.org/html/2506.00276v1#A1) for more details in our prompt.

Although LLM can generate high-quality samples, our reward-morphology optimization theory suggests that excessive repetition among samples can lead to sub-optimal local maximum. To address this issue, we propose a simple and effective mechanism called Diversity Reflection to significantly enhance the diversity of LLM-generated designs. The core idea of diversity reflection is intuitive, as illustrated in Fig. [3](https://arxiv.org/html/2506.00276v1#S4.F3). For each newly proposed morphology or reward function, the LLM reflects on previously generated samples and deliberately produces a new design that maximizes diversity relative to past candidates. Owing to its simplicity, Diversity Reflection is task-agnostic and can potentially be seamlessly extended to other domains without any modification. Experimental results in Sec. [6.2](https://arxiv.org/html/2506.00276v1#S6.SS2) demonstrate that diversity reflection consistently improves both sample diversity and optimization performance across diverse tasks.

### 4.2 Fine Optimization: Morphology Reward Alternating Optimization

After obtaining an approximate distribution of morphology and reward function space, we employ an alternating optimization strategy to iteratively refine morphology and reward function, progressively converging toward the global optimum. Specifically, we filter the best-performing candidates from the previous coarse optimization round and feed them into the LLM. The LLM first analyzes the robot morphology samples, identifying morphology optimization momentum $\frac{\partial_{LLM}f}{\partial\theta}$ to iteratively enhance morphology. Subsequently, LLM will examine the reward function samples, refining the reward design to better align with the evolving morphology according to reward optimization momentum$\frac{\partial_{LLM}f}{\partial r}$. Additionally, we retain and explore the most promising coarse optimization results, expanding the search space and uncovering further optimization possibilities, thereby facilitating a more comprehensive search for the global optimum. Please see Appendix [G](https://arxiv.org/html/2506.00276v1#A7) for the algorithm details.

Figure: Figure 3: Overview of Coarse-to-Fine algorithm. (a) In the coarse optimization stage, the LLM first generates diverse reward functions and robot morphologies using a Diversity Reflection mechanism. These reward-morphology pairs are then trained via reinforcement learning to explore and identify the most suitable motion behavior for each robot. (b) In the fine optimization stage, the LLM employs Alternating Optimization to jointly refine both the robot’s reward and morphology, leading to an optimized design and control strategy.
Refer to caption: extracted/6498497/Figure-3.jpg

## 5 Experiment Setup

##### Environments

We implemented our environments using the MuJoCo Gym simulator, covering eight distinct robot design tasks: Ant, Ant-Powered, Ant-Desert, Ant-Jump, Hopper, Half-Cheetah, Swimmer and Walker . These environments encompass diverse physical world types (2D, 3D), environment tasks, search space complexities, ground terrains, and gear powers. For morphology design, we mask certain parameters in the MuJoCo XML files to prevent the LLM from gaining prior knowledge of the robot’s design. For reward function design, the LLM is provided with the official environment code with the reward functions removed. Regarding task descriptions, we use the official descriptions from the environment repository whenever available. Detailed information on robot design morphologies, reward functions, and task descriptions can be found in Appendix [C](https://arxiv.org/html/2506.00276v1#A3).

##### Baselines

We consider 4 strong baselines to highlight performance of RoboMoRe: 1) Bayesian Optimization (BO) is a classic algorithm designed to optimize expensive-to evaluate functions [27, 24]. It employs a probabilistic model (e.g. Gaussian Process) as a surrogate for the objective function and determines where to sample based on predicted mean and uncertainty. 2) Eureka is a general LLM-based reward shaping framework [21]. Similar to an evolutionary algorithm, It uses an LLM to generate a large number of samples, select the best-performing ones, and acts as a heuristic operator for mutation. 3) Eureka (Mor.) is a modified version of the original Eureka framework, specifically focused on robot morphology design. Unlike the standard reward shaping Eureka, Eureka (Mor.) leverages the LLM’s capability to explore and optimize robot morphology parameters. 4) Human represents the template morphology and reward functions from official Mujoco website [30]. As these morphologies and reward functions are written by active reinforcement learning researchers who designed the tasks, these reward functions represent the outcomes of expert-level human reward engineering.

##### Evaluation Metrics

In all experiments, we use a normalized metric efficiency to evaluate performance, which is defined as division of fitness and robot volume.
This choice is crucial for stimulating the LLM’s material-aware design capabilities rather than brute-force scaling. We compute volume using custom scripts for each type of robot. While fitness is also reported (e.g., distance for locomotion, height for jumping tasks), efficiency serves as the core metric in our optimization iterations. Please refer to Appendix [D](https://arxiv.org/html/2506.00276v1#A4) for more implementation details.

## 6 Results

### 6.1 Comparison with Competing Methods

##### RoboMoRe outperforms human design robots and competing methods.

Table [2](https://arxiv.org/html/2506.00276v1#S6.T2) quantitatively compares the results of our method and other strong baselines. The corresponding morphology and reward function can be found in Appendix [H](https://arxiv.org/html/2506.00276v1#A8). It is evident that RoboMoRe consistently generates highly performant designs, significantly outperforming both human-generated and baseline method designs. Notably, RoboMoRe demonstrates a substantial advantage in efficiency, which can be attributed to its ability to generate more structurally efficient robotic designs (Fig. [4](https://arxiv.org/html/2506.00276v1#S6.F4)). We observe a remarkable performance gap between human-designed results and our method, especially in terms of efficiency. We believe this stems from the fact that humans are less likely to approach near-optimal designs for complex tasks, especially for efficiency, leaving substantial room for improvement. In addition, our competitors underperform mainly for two reasons. First, LLM framework Eureka and Eureka (Mor.) fail to consider the importance of diversity during optimization, often producing repetitive and suboptimal offsprings. We further analyze the importance of diversity in Section [6.2](https://arxiv.org/html/2506.00276v1#S6.SS2).

Figure: Figure 4: Optimal designs generated by RoboMoRe. (a) Ant. (b) Ant-Powered, where motor power is doubled. (c) Ant-Desert, with the terrain replaced by desert. (d) Ant-Jump, where the task is changed from locomotion to jumping. (e) Hopper. (f) Half-Cheetah. (g) Swimmer. (h) Walker.
Refer to caption: extracted/6498497/Figure-4.jpg

Second, both methods focus solely on optimizing either the reward function or morphology, which limits their potential and results in suboptimal performance. We further analyze the importance of both reward shaping and morphology design in Section [6.3](https://arxiv.org/html/2506.00276v1#S6.SS3). In addition, we conducted robustness analyses on two cases: Ant-Powered and Ant-Desert. For Ant-Powered, we varied the gear power configurations, while for Ant-Desert, we evaluated performance across different terrains. Detailed results are provided in Appendix [E.2](https://arxiv.org/html/2506.00276v1#A5.SS2) and Appendix [E.3](https://arxiv.org/html/2506.00276v1#A5.SS3).

**Table 2: Performance comparison of human and competing methods across eight tasks. We report both efficiency and fitness metrics. Note that we set efficiency as the optimization goal instead of fitness in implementation.**
| Metric | Method | Ant | Ant-Powered | Ant-Desert | Ant-Jump | Hopper | Half-Cheetah | Swimmer | Walker |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Efficiency $\uparrow$<br>(Fitness/Volume) | Bayesian Optimization | 707.13 | 74.52 | 223.61 | 3.53 | 128.00 | 4242.89 | 57.53 | 111.95 |
| Eureka | 121.54 | 24.91 | 49.82 | 7.60 | 428.11 | 12,157.92 | 23.96 | 139.73 |  |
| Eureka (Mor.) | 5,203.82 | 1,160.94 | 2.51 | 229.55 | 609.04 | 15,531.26 | 16,335.19 | 1,946.60 |  |
| Human | 68.22 | 26.13 | 29.84 | 6.83 | 433.69 | 11,975.34 | 18.86 | 170.29 |  |
| RoboMoRe | 31,038.41 | 32657.10 | 36,995.77 | 902.76 | 3,776.66 | 495,373.71 | 57,627.40 | 6,665.85 |  |
| Fitness$\uparrow$ | Bayesian Optimization | 390.14 | 41.11 | 100.49 | 1.95 | 2.75 | 194.82 | 8.66 | 2.54 |
| Eureka | 8.81 | 4.54 | 9.08 | 1.38 | 6.77 | 257.55 | 2.56 | 3.92 |  |
| Eureka (Mor.) | 143.07 | 41.52 | 0.08 | 5.33 | 1.61 | 145.09 | 89.04 | 2.91 |  |
| Human | 12.43 | 4.76 | 5.44 | 1.24 | 6.86 | 253.69 | 2.01 | 4.78 |  |
| RoboMoRe | 165.18 | 70.30 | 39.60 | 1.48 | 15.10 | 135.71 | 109.35 | 4.17 |  |

### 6.2 Effectiveness of Diversity Reflection

Based on our main assumption, we formulate co-design as a joint optimization of morphology and reward. Therefore, to efficiently reach the global optimum, the initial solution set must satisfy two key properties: (1) high-quality, denoting samples that demonstrate strong task performance and exhibit proximity to the global optimum in the design landscape; and (2) diversity, denoting a sufficient variation among samples to mitigate premature convergence to local optima and facilitate broader exploration of the design space. Experimental results demonstrate that the diversity reflection mechanism empowers RoboMoRe to synthesize solutions that are both high-performing and structurally diverse, thereby enabling more comprehensive and effective exploration of the design landscape.

**Table 3: Performance comparison of RoboMoRe variants in terms of efficiency, morphology diversity, and reward diversity. For clarity, only results from the coarse optimization stage are reported.**
| Metric | Method | Ant | Ant-Powered | Ant-Desert | Ant-Jump | Hopper | Half-Cheetah | Swimmer | Walker |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Efficiency $\uparrow$ | RoboMoRe (Random Sample) | 12.91 | 11.83 | 12.91 | 3.70 | 86.39 | 1,451.44 | 116.44 | 42.17 |
| RoboMoRe (w/o Diversity Reflection) | 136.91 | 45.56 | 52.44 | 32.36 | 111.91 | 1,536.94 | 2,338.88 | 87.54 |  |
| RoboMoRe (w/ Diversity Reflection) | 487.13 | 398.60 | 151.52 | 49.41 | 375.52 | 9,557.07 | 4,607.74 | 149.54 |  |
| Morphology Diversity<br>(Coefficient of Variation) $\uparrow$ | RoboMoRe (Random Sample) | 0.51 | 0.50 | 0.49 | 0.48 | 0.55 | 0.99 | 0.41 | 0.60 |
| RoboMoRe (w/o Diversity Reflection) | 0.44 | 0.46 | 0.57 | 0.44 | 0.53 | 0.82 | 0.29 | 0.43 |  |
| RoboMoRe (w/ Diversity Reflection) | 0.64 | 0.70 | 0.64 | 0.66 | 0.54 | 1.38 | 0.61 | 0.50 |  |
| Reward Diversity<br>(Self-BLEU) $\downarrow$ | RoboMoRe (w/o Diversity Reflection) | 0.70 | 0.67 | 0.70 | 0.57 | 0.70 | 0.75 | 0.74 | 0.72 |
| RoboMoRe (w/ Diversity Reflection) | 0.50 | 0.46 | 0.46 | 0.48 | 0.47 | 0.45 | 0.43 | 0.42 |  |

##### Diversity Reflection enables high quality and diverse morphology parameters and reward functions.

Table [3](https://arxiv.org/html/2506.00276v1#S6.T3) illustrates that the Diversity Reflection mechanism substantially improves both the performance of generated samples and the diversity of rewards and morphologies. We compare our method against two baselines: (1) Random Sample (RS), a commonly used strategy in Evolutionary Algorithms (EA) and Bayesian optimization (BO), where initial designs are sampled from a Gaussian distribution [10, 2], and (2) a setting without Diversity Reflection (w/o DR). Please refer to Appendix [E.1](https://arxiv.org/html/2506.00276v1#A5.SS1) for more details the efficiency comparison. Morphology diversity is quantified using the coefficient of variation [3], while reward diversity is assessed via the Self-BLEU metric [39]. All diversity metrics are averaged over fifty generated samples.

Figure: Figure 5: Qualitative results using three different methods: (a) Random Sample, (b) RoboMoRe (w/o Diversity Reflection), (c) RoboMoRe. It can be observed that RoboMoRe with Diversity Reflection could generate more high-quality and diverse robot morphology examples compared to others (see Table [3](https://arxiv.org/html/2506.00276v1#S6.T3)).
Refer to caption: extracted/6498497/Figure-5.jpg

While Random Sample introduces diversity, it often generates low-quality designs—such as bulky and unbalanced legs for the Ant robot, see Fig. [5](https://arxiv.org/html/2506.00276v1#S6.F5). This low initial quality explains why traditional methods such as EA and BO typically require thousands of iterations to converge. In contrast, LLMs, even without diversity reflection, are capable of generating higher-quality samples thanks to their inherent reasoning and design capabilities. However, without explicitly promoting diversity, such methods fail to explore more performant regions of the design space, resulting in suboptimal performance. RoboMoRe achieves the best overall performance because it effectively leverages Diversity Reflection to generate samples that are both high-quality and diverse, enabling a more efficient and robust design optimization process. Fig. [6](https://arxiv.org/html/2506.00276v1#S6.F6) provides more qualitative results for morphologies and rewards.

Figure: Figure 6: Qualitative results of diverse motion behaviors using Diversity Reflection. (a) The short-legged, low-center-of-gravity robot rolls forward along the ground. (b) The uniform robot propels itself forward with sideways hops. (c) The crab-inspired robot repositions itself and scuttles forward in a crab-walking manner. (d) The agile, lightweight robot advances by executing a side flip. (e) The agile, long-legged robot moves forward with a light hopping gait.
Refer to caption: extracted/6498497/Figure-6.jpg

### 6.3 Effectiveness of Reward Shaping and Morphology Design

##### Reward Shaping can improve performance by inspiring learning morphology-suited motion behaviors.

Table [4](https://arxiv.org/html/2506.00276v1#S6.T4) demonstrates that reward shaping improves performance across most tasks. This is primarily because different robot designs tend to favor different optimal locomotion strategies, and reward shaping enables the discovery of such task-specific optimal behaviors. Notably, our findings indicate that reward shaping yields greater performance gains in tasks characterized by richer morphological parameterization and more complex state spaces. For instance, Half-Cheetah, which has more than 20 morphological parameters and is the most complex design among the evaluated robots benefits the most. In contrast, the simplest robot, Swimmer—characterized by a minimal set of 6 design parameters and a constrained, two-dimensional action space—demonstrates only marginal benefits from reward shaping.

**Table 4: Ablation study of RoboDesign across efficiency metrics under different Reward (Rew.) and Morphology (Mor.) configurations.**
| Method | Metric | Ant | Ant-Powered | Ant-Desert | Ant-Jump | Hopper | Half-Cheetah | Swimmer | Walker |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Efficiency $\uparrow$ | RoboDesign (w/o Mor. & Rew.) | 21.64 | 8.68 | 20.69 | 6.08 | 94.19 | 4,984.63 | 6.82 | 15.36 |
| RoboDesign (w/o Mor.) | 97.74 | 18.04 | 15.79 | 279.42 | 561.61 | 7,945.81 | 15.63 | 22.37 |  |
| RoboDesign (w/o Rew.) | 6,190.39 | 3,026.53 | 3,742.68 | 6.34 | 1,653.59 | 67,992.31 | 22,009.94 | 1401.21 |  |
| RoboDesign (Full) | 10,464.17 | 6,537.45 | 8,001.67 | 396.45 | 1,951.44 | 129,440.98 | 21,931.15 | 1,482.60 |  |

##### Morphology Design still plays a fundamental role.

Although reward shaping can significantly enhance performance by promoting the emergence of novel motion patterns, morphology design remains crucial. When a fixed, template-based morphology is used, performance degrades markedly across all tasks, suggesting that human-designed morphologies are still far from optimal—consistent. Moreover, we observe substantial variation in material usage across different morphology designs, which accounts for the pronounced disparities in efficiency observed in our experiments.

### 6.4 Effectiveness of Coarse-to-Fine Optimization.

Figure: Figure 7: Comparison of the highest efficiency achieved in different cases among RoboMoRe, RoboMoRe (w/o fine stage), and RoboMoRe (w/o coarse stage).
Refer to caption: extracted/6498497/Figure-7.jpg

Both coarse and fine stages contribute significantly to co-design performance. Fig. [7](https://arxiv.org/html/2506.00276v1#S6.F7) highlights the critical role of jointly leveraging both coarse and fine stages in achieving optimal co-design performance. Across all tasks, the fine stage consistently enhances the results of the coarse stage. We attribute this to the greater design space inherent in these more challenging tasks, which, while offering more room for optimization, also make it significantly harder for LLMs to directly generate near-optimal designs. The coarse stage enables broad exploration of the joint morphology–reward space, generating diverse, high-quality candidates. However, in complex or high-dimensional tasks, these alone are insufficient. The fine stage complements by alternately refining morphology and reward using gradient-based feedback. Its effectiveness crucially depends on the search space shaped by the coarse stage. The fine stage alone lags behind the full approach in most cases, but matches or outperforms the coarse stage in simpler environments like Swimmer and Hopper.

## 7 Discussion & Conclusion

RoboMoRe is designed to be task-agnostic and broadly generalizable, but future work should test its robustness on more challenging benchmarks, such as soft robotics platforms [12, 11, 37], where training a single robot can take over a day, making large-scale optimization significantly harder. We also observe that well-performing morphologies under the template reward often remain efficient, while poor ones rarely improve—suggesting early-stage pruning as a viable strategy (Appendix [E.1](https://arxiv.org/html/2506.00276v1#A5.SS1)). Incorporating vision-language models (VLMs) may further enhance adaptability by mimicking human design intuition. Finally, we aim to extend RoboMoRe to real-world tasks, enabling bio-inspired robots to co-optimize morphology and reward for more efficient motion.

In conclusion, we presented RoboMoRe, an LLM-driven framework that integrates reward shaping into the robot co-design process. By adopting a coarse-to-fine optimization strategy—combining Diversity Reflection with alternating LLM-guided refinement—RoboMoRe successfully discovers efficient and high-performing morphology-reward pairs. Across eight robot design tasks, our method consistently outperforms human baselines and prior co-design approaches. We hope RoboMoRe contributes to a deeper understanding of joint optimization in robot design and serves as a springboard for future innovations at the intersection of generative AI and embodied intelligence.

## References

- [1]
Joshua Achiam.
Spinning up in deep reinforcement learning.
2018.
- [2]
Thomas Bäck and Hans-Paul Schwefel.
An overview of evolutionary algorithms for parameter optimization.
Evolutionary computation, 1(1):1–23, 1993.
- [3]
Arthur G Bedeian and Kevin W Mossholder.
On the use of the coefficient of variation as a measure of diversity.
Organizational Research Methods, 3(3):285–297, 2000.
- [4]
James C Bezdek and Richard J Hathaway.
Convergence of alternating optimization.
Neural, Parallel & Scientific Computations, 11(4):351–368, 2003.
- [5]
Jagdeep Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, and Wojciech Matusik.
Evolution gym: A large-scale benchmark for evolving soft robots.
Advances in Neural Information Processing Systems, 34:2201–2214, 2021.
- [6]
Gabriel Bravo-Palacios, Andrea Del Prete, and Patrick M Wensing.
One robot for many tasks: Versatile co-design through stochastic programming.
IEEE Robotics and Automation Letters, 5(2):1680–1687, 2020.
- [7]
Hao-Tien Lewis Chiang, Aleksandra Faust, Marek Fiser, and Anthony Francis.
Learning navigation behaviors end-to-end with autorl.
IEEE Robotics and Automation Letters, 4(2):2007–2014, 2019.
- [8]
Jieming Cui, Tengyu Liu, Ziyu Meng, Jiale Yu, Ran Song, Wei Zhang, Yixin Zhu, and Siyuan Huang.
Grove: A generalized reward for learning open-vocabulary physical skill.
arXiv preprint arXiv:2504.04191, 2025.
- [9]
Aleksandra Faust, Anthony Francis, and Dar Mehta.
Evolving rewards to automate reinforcement learning.
arXiv preprint arXiv:1905.07628, 2019.
- [10]
Peter I Frazier.
A tutorial on bayesian optimization.
arXiv preprint arXiv:1807.02811, 2018.
- [11]
M Gazzola, LH Dudte, AG McCormick, and L Mahadevan.
Forward and inverse problems in the mechanics of soft filaments.
Royal Society open science, 5(6):171628, 2018.
- [12]
Moritz A Graule, Thomas P McCarthy, Clark B Teeple, Justin Werfel, and Robert J Wood.
Somogym: A toolkit for developing and evaluating controllers and reinforcement learning algorithms for soft robots.
IEEE Robotics and Automation Letters, 7(2):4071–4078, 2022.
- [13]
David Ha.
Reinforcement learning for improving agent design.
Artificial life, 25(4):352–365, 2019.
- [14]
Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al.
Soft actor-critic algorithms and applications.
arXiv preprint arXiv:1812.05905, 2018.
- [15]
Xu Han, Qiannan Yang, Xianda Chen, Xiaowen Chu, and Meixin Zhu.
Generating and evolving reward functions for highway driving with large language models.
In 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), pages 831–836. IEEE, 2024.
- [16]
Jonathan Ho and Stefano Ermon.
Generative adversarial imitation learning.
Advances in neural information processing systems, 29, 2016.
- [17]
W Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone.
Reward (mis) design for autonomous driving.
Artificial Intelligence, 316:103829, 2023.
- [18]
Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O Stanley.
Evolution through large models.
In Handbook of evolutionary machine learning, pages 331–366. Springer, 2023.
- [19]
Haofei Lu, Zhe Wu, Junliang Xing, Jianshu Li, Ruoyu Li, Zhe Li, and Yuanchun Shi.
Bodygen: Advancing towards efficient embodiment co-design.
arXiv preprint arXiv:2503.00533, 2025.
- [20]
Pingchuan Ma, Tao Du, John Z Zhang, Kui Wu, Andrew Spielberg, Robert K Katzschmann, and Wojciech Matusik.
Diffaqua: A differentiable computational design pipeline for soft underwater swimmers with shape interpolation.
ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
- [21]
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar.
Eureka: Human-level reward design via coding large language models.
arXiv preprint arXiv:2310.12931, 2023.
- [22]
Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A Efros.
Learning to control self-assembling morphologies: a study of generalization via modularity.
Advances in Neural Information Processing Systems, 32, 2019.
- [23]
Kevin Qiu, Krzysztof Ciebiera, Paweł Fijałkowski, Marek Cygan, and Łukasz Kuciński.
Robomorph: Evolving robot morphology using large language models.
arXiv preprint arXiv:2407.08626, 2024.
- [24]
Carl Edward Rasmussen.
Gaussian processes in machine learning.
In Summer school on machine learning, pages 63–71. Springer, 2003.
- [25]
George Saridis.
Intelligent robotic control.
IEEE Transactions on Automatic Control, 28(5):547–557, 1983.
- [26]
Karl Sims.
Evolving virtual creatures.
In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 699–706. 2023.
- [27]
Jasper Snoek, Hugo Larochelle, and Ryan P. Adams.
Practical bayesian optimization of machine learning algorithms, 2012.
- [28]
Junru Song, Yang Yang, Huan Xiao, Wei Peng, Wen Yao, and Feifei Wang.
Laser: Towards diversified and generalizable robot design with large language models.
In The Thirteenth International Conference on Learning Representations.
- [29]
Luc Steels and Rodney Brooks.
The artificial life route to artificial intelligence: Building embodied, situated agents.
Routledge, 2018.
- [30]
Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al.
Gymnasium: A standard interface for reinforcement learning environments.
arXiv preprint arXiv:2407.17032, 2024.
- [31]
Tingwu Wang, Yuhao Zhou, Sanja Fidler, and Jimmy Ba.
Neural graph evolution: Towards efficient automatic robot design.
arXiv preprint arXiv:1906.05370, 2019.
- [32]
Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua B Tenenbaum, Daniela Rus, and Chuang Gan.
Softzoo: A soft robot co-design benchmark for locomotion in diverse environments.
arXiv preprint arXiv:2303.09555, 2023.
- [33]
Zhaoyue Wang.
Towards socially and morally aware rl agent: Reward design with llm.
arXiv preprint arXiv:2401.12459, 2024.
- [34]
Jie Xu, Andrew Spielberg, Allan Zhao, Daniela Rus, and Wojciech Matusik.
Multi-objective graph heuristic search for terrestrial robot design.
In 2021 IEEE international conference on robotics and automation (ICRA), pages 9863–9869. IEEE, 2021.
- [35]
Yanchao Yang.
Text2reward: Reward shaping with language models for reinforcement learning.
In International Conference on Learning Representations (ICLR), 2024 (07/05/2024-11/05/2024, Vienna, Austria), 2024.
- [36]
Lechen Zhang.
Cuda-accelerated soft robot neural evolution with large language model supervision.
arXiv preprint arXiv:2405.00698, 2024.
- [37]
X Zhang, FK Chan, T Parthasarathy, and M Gazzola.
Modeling and simulation of complex dynamic musculoskeletal architectures.
Nature Communications, 10(1):1–12, 2019.
- [38]
Allan Zhao, Jie Xu, Mina Konaković-Luković, Josephine Hughes, Andrew Spielberg, Daniela Rus, and Wojciech Matusik.
Robogrammar: graph grammar for terrain-optimized robot design.
ACM Transactions on Graphics (TOG), 39(6):1–16, 2020.
- [39]
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu.
Texygen: A benchmarking platform for text generation models.
In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 1097–1100, 2018.

## Appendix

## Appendix A Full Prompts

### A.1 Coarse Optimization Prompts

We take the Ant as an example to illustrate the prompt used in coarse optimization. The prompt for morphology design mainly consists of three parts: the task description, a masked MuJoCo XML file, and the required output format. The MuJoCo XML is masked to prevent LLMs from seeing the human-designed structure in advance. For brevity, we show only part of the MuJoCo XML file and environment code. Please refer to MuJoCo Gym for more details [30].

#### A.1.1 Prompts for Reward Shaping

#### A.1.2 Prompts for Morphology Design

#### A.1.3 Prompts for Diversity Reflection

### A.2 Fine Optimization

#### A.2.1 Prompts for Alternating Refinement

## Appendix B Motivation

To verify our hypothesis, we conducted experiments to quantitatively evaluate the performance of two representative robot morphologies, short leg ant and long leg ant. By designing different reward functions, we encouraged each robot to adopt distinct motion modalities. Specifically, we encouraged the robot to adopt a rolling motion along the ground by augmenting the template reward function with an angular velocity term and a jumping motion on the air by augmenting the template reward function with a bouncing reward term.

As shown in Fig. [8](https://arxiv.org/html/2506.00276v1#A2.F8), the experimental results indicate that the short leg ant, characterized by a bulky body and low center of gravity, exhibits low fitness in the jumping modality but achieves significantly higher fitness when utilizing the rolling modality. Conversely, the long leg ant robot, benefiting from its slender limbs and elevated center of gravity, performs exceptionally well in the jumping modality. However, its motion efficiency drastically decreases under the rolling modality due to frequent overturning and difficulty recovering balance. These results clearly illustrate the strong dependence of robot performance
on appropriate motion modalities, highlighting that optimal efficiency can only be achieved when
morphology and modality are suitably matched.

Figure: Figure 8: (a-d) Motion behaviors of our four cases. (e) Time-series fitness (walk distance) across 100 independent evaluations per design case.
Refer to caption: extracted/6498497/Figure_sup_motivation.jpg

Figure: Table 5: Fitness and Efficiency on four cases.

In addition, Table [5](https://arxiv.org/html/2506.00276v1#A2.T5) presents the fitness and efficiency of the ant robot under different conditions. We observe that the long-leg ant exhibits a clear advantage in terms of efficiency, but only when paired with a well-suited reward function. This finding underscores the importance of co-optimizing both morphology and control strategy in robot design to achieve optimal efficiency.

## Appendix C Environment and Task Details

This section provides additional details on the environments and tasks used in our experiments. We build upon standard environments from MuJoCo Gym, including Ant, Walker, Hopper, Half-Cheetah, and Swimmer, and use their built-in reward functions and morphologies as human-designed baselines for comparison. To further evaluate the robustness of RoboMoRe across diverse settings, we introduce three customized environments: Ant-Desert, Ant-Jump, and Ant-Powered. Task descriptions are used as input to both the Morphology Design Prompt and the Reward Shaping Prompt. The output format constrains the LLM’s response to a set of robot morphology parameters, which are then fed into our custom design scripts to generate a complete MuJoCo XML file, following [13]. We will open-source these design scripts alongside the code.

Figure: Figure 9: Human designed agents in eight different environments for visualization. Purple indicates agents in an augmented gear power.
Refer to caption: extracted/6498497/Fig_environment.jpg

### C.1 Ant

#### C.1.1 Task Description

#### C.1.2 Output Format

### C.2 Ant-Powered

#### C.2.1 Task Description

#### C.2.2 Output Format

### C.3 Ant-Desert

#### C.3.1 Task Description

#### C.3.2 Output Format

### C.4 Ant-Jump

#### C.4.1 Task Description

#### C.4.2 Output Format

### C.5 Hopper

#### C.5.1 Task Description

#### C.5.2 Output Format

### C.6 Half-Cheetah

#### C.6.1 Output Format

### C.7 Swimmer

#### C.7.1 Output Format

### C.8 Walker

#### C.8.1 Task Description

#### C.8.2 Output Format

## Appendix D Implementation Details

**Table 6: Coarse-to-Fine Optimization Hyperparameters**
| Hyperparameter | Value |
| --- | --- |
| $\mathcal{N_{M}}$ | 25 |
| $\mathcal{N_{R}}$ | 5 |
| $k$ | 5% |

In line with standard reinforcement learning practices, we employ distributed trajectory sampling across multiple CPU threads to accelerate training. Each model is trained using four random seeds on a system equipped with one AMD EPYC 7T83 processor and a single NVIDIA RTX 4080 Super GPU. Our framework is built on Python 3.8.20, MuJoCo 2.3.1, Stable-Baselines3 2.4.1, and CUDA 12.4. For all environments considered, training a single policy for 5e5 steps takes approximately 15 minutes using 24 CPU cores and one RTX 4080 Super GPU. GPT-4-turbo serves as the foundation model in most of our experiments. As shown in Table [6](https://arxiv.org/html/2506.00276v1#A4.T6), for coarse optimization, we empirically set the number of morphologies $\mathcal{N_{M}}$ to 25 and the number of reward functions $\mathcal{N_{R}}$ to 5, for trade-off between speed and design quality. For fine optimization, we select $k$ as 5% best candidates from coarse stage. Although this process involves 120 iterations, it remains computationally efficient and practical—especially when compared to conventional methods like Bayesian Optimization or Evolutionary Algorithms, which often require several thousand iterations to converge.

##### Policy Training

We use the Soft Actor-Critic (SAC) algorithm [14] from Stable-Baselines3 as the reinforcement learning backbone for all experiments, ensuring consistency across tasks. Hyperparameters are configured based on the recommendations from Gym SpinningUp [1], which are well-suited for training both human-designed robot morphologies and their corresponding reward functions. To accelerate training, we leverage parallel environments via SubprocVecEnv with 16 instances. Experimental results are averaged over 100 independent runs to mitigate randomness. Each candidate is initially trained for $5\times 10^{5}$ steps for efficiency, and the top-performing candidate is retrained for $10^{6}$ steps to ensure a fair comparison with alternative methods such as Bayesian Optimization (BO) and Eureka—both of which typically converge within this training budget across most tasks [1]. Table [7](https://arxiv.org/html/2506.00276v1#A4.T7) demonstrates detailed parameters for policy training.

**Table 7: SAC Training Hyperparameters**
| Parameter | Value |
| --- | --- |
| Number of environments | 16 |
| Learning rate | 3e-4 |
| Buffer size | 2,000,000 |
| Learning starts | 10,000 |
| Batch size | 1024 |
| $\tau$ | 0.005 |
| $\gamma$ | 0.99 |
| Train frequency | 8 |
| Gradient steps | 4 |
| Policy kwargs | [512, 512] |

##### Comparison Methods

We adopt Bayesian Optimization with a batch size of 1 and 100 iterations, using a Gaussian Process surrogate model with a Matérn 5/2 kernel, automatic relevance determination (ARD), and the Expected Improvement (EI) acquisition function. For Eureka, we follow the recommended configuration with 5 iterations and 16 populations per iteration.

## Appendix E Additional Results

### E.1 Results for Diversity Reflection

To more concretely demonstrate the effectiveness of the Diversity Reflection mechanism, we visualize the 5 × 25 efficiency matrices across multiple tasks for Random Sample, RoboMoRe w/o Diversity Reflection (DR), and RoboMoRe. Due to the large variation in data magnitude, we apply a logarithmic scale to the color map. As shown in Fig. [E.1](https://arxiv.org/html/2506.00276v1#A5.SS1), robots generated by Random Sample generally exhibit low efficiency, whereas those designed by RoboMoRe consistently achieve significantly higher performance across most tasks (see Table [3](https://arxiv.org/html/2506.00276v1#S6.T3)). Notably, while RoboMoRe w/o Dr still achieves reasonably good results, the lack of morphological diversity causes the selected samples to concentrate around local optima, thereby leading to a slightly lower average efficiency compared to the full RoboMoRe configuration.

[Uncaptioned image]: extracted/6498497/Fig_CoarseA_E.jpg

Figure: Figure 10: Coarse optimization results across 8 tasks (a–h) for Random Sample, RoboMoRe w/o Diversity Reflection (DR), and RoboMoRe.
Refer to caption: extracted/6498497/Fig_CoarseF_H.jpg

### E.2 Robustness across Different Gear Powers

To validate the robustness of RoboMoRe-generated designs across varying levels of motor power, we selected the top-performing design from the Ant-Powered task and retrained the model under five additional power settings. These power levels were configured directly through the MuJoCo XML files. The results, summarized in Fig. [11](https://arxiv.org/html/2506.00276v1#A5.F11), show that the design discovered by RoboMoRe demonstrates strong adaptability and maintains consistent performance across a wide range of actuation capabilities.

Figure: Figure 11: Different power settings and corresponding results. (a) Ant-Powered-50. (b) Ant-Powered-100. (c) Ant-Powered-150. (d) Ant-Powered-200. (e) Ant-Powered-250. (f) Ant-Powered-300.
Refer to caption: extracted/6498497/Fig_power_experiment.jpg

### E.3 Robustness across Different Terrains

To further validate the robustness of RoboMoRe-generated designs across diverse terrains, we selected the top-performing design in Ant-Desert task and retrained the model on additional three distinct environments: ground, snow, and hills. These terrains were procedurally generated using Perlin noise and calibrated with realistic friction coefficients to reflect natural surface properties. The results, summarized in Fig. [12](https://arxiv.org/html/2506.00276v1#A5.F12), demonstrate that the design discovered by RoboMoRe exhibits strong adaptability and consistent performance across all tested terrains, highlighting its robustness in diverse physical conditions.

Figure: Figure 12: Different terrain environments and corresponding results. (a) Ant-Desert. (b) Ant-Ground. (c) Ant-Hills. (d) Ant-Snow. (e) Results for Ant-Desert. (f) Results for Ant-Ground. (g) Results for Ant-Hills. (h) Results for Ant-Snow.
Refer to caption: extracted/6498497/Fig_terrains.jpg

## Appendix F Algorithm Details

## Appendix G Algorithm Details

Figure: Algorithm 1 Coarse-to-Fine Optimization

## Appendix H More Visualization Results

### H.1 Comparison of optimal morphology design via different methods

Fig. [13](https://arxiv.org/html/2506.00276v1#A8.F13) presents the optimized morphology design. Evidently, RoboMoRe is capable of producing highly efficient structures, demonstrating its effectiveness in morphology optimization.

Figure: Figure 13: Optimized morphology with comparison methods.
Refer to caption: extracted/6498497/Fig.Compar_Mor.jpg

### H.2 Ant: Best Reward Function

### H.3 Ant-Powered: Best Reward Function

### H.4 Ant-Desert: Best Reward Function

### H.5 Ant-Jump: Best Reward Function

### H.6 Hopper: Best Reward Function

### H.7 Half-Cheetah: Best Reward Function

### H.8 Swimmer: Best Reward Function

### H.9 Walker: Best Reward Function

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: See the weaknesses section.
soundness: 3
strengths: + This work applies emerging LLM capabilities to the classic robot co-design problem.
+  The paper identifies a limitation of LLM-based design -- tendency toward repetitive morphology outputs, and introduces the concept of diversity reflection to address it.
+ The method is evaluated on multiple tasks, though the experiments are performed in simple simulation environments.
summary: This paper introduces RoboMoRe, an LLM-driven framework for robot co-design that jointly optimizes morphology and control policy. Unlike prior methods that rely on fixed reward functions, RoboMoRe uses an LLM to generate and refine morphology-reward pairs through a two-stage process: coarse exploration and fine iterative refinement. Experiments on eight tasks show that the method discovers more efficient designs.
weaknesses: -  The novelty of the work appears limited. The key contributions seem to center on prompt engineering, e.g., prompting the LLM to “reflect” on prior results to increase diversity. The reward shaping and morphology filtering mechanisms also appear straightforward (e.g., discarding repeated designs), and the coarse-to-fine pipeline resembles a standard iterative refinement process, albeit executed via an LLM.

-  The paper does not provide sufficient detail regarding the diversity reflection mechanism. For example, the authors state that “the LLM reflects on previously generated samples and deliberately produces new candidates,” but it is unclear how this reflection is implemented to encourage diversity.

-  Similarly, the description of reward shaping lacks clarity. For example, the paper notes “It is therefore used to evaluate all morphology candidates.” How exactly is this evaluation performed?    

- Line 223 refers to an “LLM-generated reward function,” yet the reward functions for tasks appear manually defined as shown in the paper’s Appendix. This creates confusion. My interpretation is that the manually defined functions compute rewards based on LLM-generated morphologies, rather than the LLM generating the reward function itself. The authors should clarify this distinction.

-  More broadly, the use of the term co-design feels unconvincing. The process appears sequential: morphologies are generated first, and then controllers are optimized. Effective co-design in robotics should explicitly consider the interplay among morphology, motion behavior, and physical constraints during the design process. Robot morphology should be designed with careful consideration of the robot’s motion control and physical constraint requirements. The current framework does not clearly demonstrate such coupling.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1.	With regards to lines 67-68, while different rewards may be more relevant to different designs, modifying the rewards for each design may also inadvertently promote unwanted behaviours for each design (reward hacking). For example, if the ultimate goal is to move forward as fast as possible, while encouraging jumping behaviours could help this objective, it could also lead to unwanted behaviours such as jumping in place if there is a reward introduced for jumping.

2.	Also, if the reward functions of two designs are dependent on the designs themselves, how does one compare the performance of two designs? The performance can no longer be measured in terms of rewards as the reward structure would be different for each agent.  Is there instead some universal measure of performance, independent of the reward function which is considered?

3.	Below eq (1), I am not sure if it is correct to say that it would be a local optimum – because as long as R_0 is the correct reward function for the agent (that is, it accurately captures the quality of behaviours from the agent), $\theta^*$ would indeed be the global optimum. My contention, consistent with 2. above is that as soon as one varies R as in Eq (2), there would be no meaning to ‘high performing’ agents because the performance of two agents would no longer be comparable

4.	If the purpose of stage 1 is to promote diversity, could approaches like diversity if all you need [2] (applied to the design space) be applied?

5.	How is “efficiency” as discussed in the results defined formally?

6.	Why are the resulting designs seemingly all symmetrical? Is symmetry imposed on the designs?

7.	Do the frequency of the stages matter? For example, instead of alternating cycles of course and fine stages, would it possibly be more effective to perform multiple cycles of each stage, before moving to the other stage?

[2] Eysenbach, Benjamin, et al. "Diversity is All You Need: Learning Skills without a Reward Function." International Conference on Learning Representations.
soundness: 2
strengths: The method seems novel and exposes a way to utilise LLMs for morphology design. The results seem promising as well.
summary: The paper presents a 2 stage approach for robot codesign – the first is a coarse stage driven by LLMs to build diversity of designs, and the second is a fine stage where top designs are iteratively refined through alternating LLM guided updates to both reward and morphology. The method is shown to significantly outperform competing methods of agent design.
weaknesses: Some of the parts of this work need to discussed more clearly. For example, it is not clear what efficiency of a design means formally. In addition, there seems to be a lack of comparisons with existing non-LLM methods such as transform2act [1] etc., Even if this is not an equivalent method, it would still be good to include comparisons for better reference. In addition, the idea of refining the rewards is not very clear to me because it inherently changes what a "performing" agent is.

[1] Yuan, Ye, et al. "Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design." International Conference on Learning Representations.

---

confidence: 5
contribution: 3
initial_rating: 8
presentation: 4
questions: see above
soundness: 3
strengths: - Care was taken to evaluate the use of LLMs fairly, eg by masking elements of the XML files to prevent possible training data contamination.
- Using an LLM-generated reward is a relatively novel idea for co-design, albeit learned rewards have been used before in co-design (see below) and Eureka has been used in a behaviour-only RL setting.
- The paper is good to follow, well written and visualisations are used nicely to support the reader's understanding.
summary: The paper introduces the idea of utilising LLMs to judge and augment the fitness value with a generated reward function for the joint co-optimisation of agent policies and embodiment variables.
Overall, the paper presents an interesting approach to reward generation for the problem of co-design. I aprpeciate that it is not quite straightforward to compare against other approaches and find suitable metrics to compare approaches (see comments below). However, I judge this paper as an interesting research into this direction with proividing sufficient details and insight.
weaknesses: - The use of efficiency as fitness/volume is not quite clear to me. Why do you not use torque or fitness/energy instead? It seems to me that the algorithms can easily game this metric by producing as thin geom-elements as possible without increasing actual real world efficiency (asi n, energy spent per forward unit of movement).
- The literature review/discussion of related works is a bit incomplete, as the idea of using learned reward functions has already been explored in previous work (see eg [R1]), albeit to the best of my knowledge not with LLMs.
- In tables 2 and 3, it is not clear to me why the best result is not presented in bold, but only the proposed method. ROboMoRe does not always seem to be the best performing method in the respective metrics.

[R1] Rajani, Chang, et al. "Co-imitation: learning design and behaviour by imitation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 5. 2023.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. How sensitive is the system to prompt phrasing? Did the authors test multiple prompt templates for reward generation, and if so, how consistent were the resulting performance outcomes?
soundness: 4
strengths: 1. Insightful finding on co-design: The study convincingly shows that reward shaping must be robot-dependent, illustrating that task-based optimization alone fails to generalize across morphologies. This is an important and underexplored insight in robot design problem.

2. Timely and important topic: The intersection of LLMs and robotic design is a rapidly growing area with significant potential impact. The authors target an important problem - how generative models can extend the space of feasible robot designs.

3. Sound evaluation protocol: The experiments are thoughtfully designed, combining simulated environments with objective metrics that quantify both control performance and morphological diversity.
summary: The paper presents RoboMoRe, a framework that integrates large language models (LLMs) into the robot co-design process, enabling the joint optimization of morphology and control. By leveraging natural language descriptions and structured reasoning, the system aims to generate robot designs that are both functional and specialized for given tasks. The authors highlight how the LLM facilitates creative design exploration while remaining grounded through simulation-based evaluation. The approach is validated through multiple case studies showing task-specific robot designs and control policies optimized jointly via reinforcement learning and design iteration.
weaknesses: 1. While the integration of LLMs is appealing, much of the technical pipeline (e.g., reinforcement learning for control, parametric morphology search) builds directly on established frameworks, so the question is whether the contribution is appealing enough to a general machine learning audience.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: 1.Since the reward functions are evolved by an LLM, I wonder whether this would implicitly introduce a bias towards robot morphologies with more interpretable motion patterns instead of those with high potential but less intuitive reward functions to evolve? 

2.Could the authors provide a couple of example reward functions for soft voxel robots? Since the dynamics of SVRs are far less intuitive than articulated rigid robots, I wonder whether LLMs can still evolve interpretable reward functions?
soundness: 4
strengths: 1.The paper is well-written and easy to follow. The abundant illustrations and qualitative results greatly help with reader’s understanding. 

2.The paper is well motivated. The design of reward function is left simple and static in previous robot co-design studies. The introduction of reward shaping facilitates more customized and precise fitness evaluation across different morphologies, leading to improved optimization efficiency and overall performances. 

3.I greatly appreciate the authors’ endeavour to examine their method in various settings, such as template-based and free-form robot design, as well as different environmental perturbations. The results validate their adaptability and generality across diverse scenarios. 

4.The authors provide a comprehensive discussion of limitations and future directions, which would greatly inspire further studies.
summary: This paper addresses a fundamental challenge in evolutionary Robotics, i.e., robot design automation, and proposes a novel perspective that evolves reward functions simultaneously in order to account for distinct motion modalities across morphologies. Specifically, the authors introduce a dual-stage strategy: a coarse stage where an LLM is prompted to propose diverse and promising morphology-reward pairs, and a fine stage where the solutions are further optimized. Extensive experiments demonstrate the effectiveness of the proposed approach, RoboMoRe, compared with several baselines.
weaknesses: 1.I believe the motivating example given in Appendix D is worth incorporating into the main text, so that the logic flows more naturally. 

2.The authors could also include a brief outline at the beginning of the appendices so that the contents are more organized.


Decision: Reject