Generating Freeform Endoskeletal Robots
Authors: Muhan Li
Lingji Kong
Sam Kriegman
Venue: ICLR 2025 Spotlight (2025)
Keywords: co-design
agent design
robots
morphology
evolution
locomotion
Primary area: applications to robotics, autonomy, planning
TL;DR: We introduce the multiphysics simulation, de novo design, and universal control of endoskeletal robots with minimal assumptions about the robots’ morphology and behavior.

Abstract: The automatic design of embodied agents (e.g. robots) has existed for 31 years and is experiencing a renaissance of interest in the literature. To date however, the field has remained narrowly focused on two kinds of anatomically simple robots: (1) fully rigid, jointed bodies; and (2) fully soft, jointless bodies. Here we bridge these two extremes with the open ended creation of terrestrial endoskeletal robots: deformable soft bodies that leverage jointed internal skeletons to move efficiently across land. Simultaneous de novo generation of external and internal structures is achieved by (i) modeling 3D endoskeletal body plans as integrated collections of elastic and rigid cells that directly attach to form soft tissues anchored to compound rigid bodies; (ii) encoding these discrete mechanical subsystems into a continuous yet coherent latent embedding; (iii) optimizing the sensorimotor coordination of each decoded design using model-free reinforcement learning; and (iv) navigating this smooth yet highly non-convex latent manifold using evolutionary strategies. This yields an endless stream of novel species of ``higher robots'' that, like all higher animals, harness the mechanical advantages of both elastic tissues and skeletal levers for terrestrial travel. It also provides a plug-and-play experimental platform for benchmarking evolutionary design and representation learning algorithms in complex hierarchical embodied systems.

Published as a conference paper at ICLR 2025
GENERATING FREEFORM ENDOSKELETAL ROBOTS
Muhan Li, Lingji Kong, Sam Kriegman Northwestern University
Figure I: An evolving population of endoskeletal soft robots were encoded by a low dimensional la- tent design genome with minimal morphological assumptions and optimized for locomotion across complex terrains in multiphysics simulation using a shared universal controller that was simultane- ously learned alongside morphological design. Videos and code https://endoskcletall htth:///
ABSTRACT
The automatic design of embodied agents (e.g. robots) has existed for 31 years and is experiencing renaissance of interest in the literature. To date however, the field has remained narrowly focused on two kinds of anatomically simple robots: (1) fully rigid, jointed bodies; and (2) fully soft, jointless bodies. Here we bridge these two extremes with the open ended creation of terrestrial endoskeletal robots: deformable soft bodies that leverage jointed internal skeletons to move efficiently across land. Simultaneous de novo generation of external and internal structures achieved by (i) modeling 3D endoskeletal body plans integrated collections of elastic and rigid cells that directly attach to form soft tissues anchored to com- pound rigid bodies; (ii) encoding these discrete mechanical subsystems into continuous yet coherent latent embedding: (iii) optimizing the sensorimotor co- ordination of each decoded design using model-free reinforcement learning; and (iv) navigating this smooth yet highly non-convex latent manifold using evolution- ary strategies. This yields endless stream of novel species of "higher robots" that, like all higher animals, harness the mechanical advantages of both elastic tissues and skeletal levers for terrestrial travel. It also provides a and--and experimental platform for benchmarking evolutionary design and representation learning algorithms in complex hierarchical embodied systems.
INTRODUCTION
The manual synthesis of a single robot for a single task requires years if not decades of labor- intensive R&D from teams of engineers who rely heavily on expert knowledge, intuition, and hard- earned experience. The automatic synthesis of robots may reveal designs that are different from or beyond what human engineers were previously capable of imagining and would be of great use robots are to be designed and deployed for many disparate tasks in society. However, automating the creation and improvement of a robot's design is notoriously difficult due to the hierarchical and combinatorial nature of the optimization problem: the objective and variables of the robot's s controller optimization depend on the optimizer of the robot's 's discrete mechanical structure (e.g. its distribution of motors and limbs). Considerable effort has been and is being expended to ameliorate (Zhao et al., 2020; Yuan et al., 2022; Strgar et al., 2024) or obviate (Hejna et al., 2021; Gupta et al., 2022; Matthews et al., 2023) these technical challenges. Nevertheless, automatically designed robots

---

Published as a conference paper at ICLR 2025
A
B
M
E
K
Fitness
Latent Dimension 2
Latent Dimension
Figure 2: Designing endoskeletal robots. A population of 64 robot designs (A-C) was sampled from a multivariate normal distribution (D) centered about an initially random point in latent space. The behavior each design in the population was optimized in physics simulation for 20 epochs of reinforcement learning under a shared universal controller. The ten red behavioral traces to the right of A-C project the forward locomotion of these randomly generated designs (from 3D to 2D) into the right hand side of the page, at every other epoch of learning. The fitness of design was based on the best performance it achieved during reinforcement learning. After learning, the mean and covariance matrix of the design distribution were then adapted by evolutionary strategies and a subsequent generation of 64 new designs was sampled from the shifted distribution (E). This process was repeated for dozens of generations (F-J) yielding evolved design distribution (K) that encodes a population designs (L-N) with much higher fitness (blue behavioral traces).
have yet to evolve beyond primitive machines with exceedingly simple body plans, which can be neatly divided into two opposing groups:
1. fully rigid, jointed bodies (Sims, 1994; Lipson Pollack, 2000; Auerbach & Bongard, 2014; Zhao et al., 2020: Hejna et al., 2021; Gupta et al., 2021; Yuan et al., 2022); and
2. fully soft, jointless bodics (Hiller & Lipson, 2012; 2020a; Matthews et al., 2023; Wang al., 2023b; Li et al., 2024; Strgar al., 2024).
Cheney et al., 2018; Kriegman et al.,
Here we introduce the de novo optimization of externally soft yet internally rigid, jointed body plans- endoskeeaallllt robots (Fig. 1)-with with minimal assumptions about the robots' external and in- ternal structures (Fig. 2). Endoskeletal robots contain bones and joints joints-the the lever systems animals to their mechanical advantage they gallop across uneven surfaces, climb up trees, and swing from branch to branch. At the same time, endoskeletal robots contain and are encased by soft de- formable tissues-the springs animals use to conform uneven surfaces, absorb shocks, store and release elastic energy, catapult themselves into the air, and capture complex somatosensory signals about their environment. In order to achieve this confluence of soft and rigid body dynamics within single robot withhut presupposing the topology or the geometry of the robot's bones and tissues novel multi-physics voxel- based simulator (Fig. 4) was created and is introduced here alongside a latent endoskeletal embedding and companion generator that sythensizes elastic and rigid voxel cells into freeform tissues and bones, and connects the bones by joints within a functional whole.
Several existing simulators support two-way coupling between rigid and deformable bodies with joint constraints (Shinar et al., 2008; Kim & Pollard, 2011; Li et al., 2020). However they are not naturally amenable to structural optimization because they do not readily expose building blocks (e.g. voxels) to the optimizer, instead, they require a bespoke volumetric mesh to be supplied for each body plan, and the mesh must be carefully designed to remain numerically stable. Jansson & Vergeest (2003) embedded rigid bodies within a regular grid of passive Hookean springs and masses, which provides suitable building blocks of freeform structure, but does not support joint constraints,
2

---

Published as a conference paper at ICLR 2025
A Endoskeletal Robot
B Rigid Skaleton Sensing
Node Aligment
Actor Output Heed
x x (st)
Edge Node
Graph Transforme
Solt Vorel Sensing
D 3D Space Pooling
Joint Angle Control
G Output weu Nocie
Outsul Edge
Max Pooling
M Pooled Noda
Critic Output Head
V(s,)
Locally Pooled Node
Figure 3: A universal controller for freeform endoskeletal robots. The control of an endoskeletal robot (A) utilizes graph data structure with edges that map bone connections within its skeleton, edge features that track the position, orientation and angle of each joint, and node features that track the position, orientation and movement of each bone (B). The controller also tracks propriocep- tive and mechanosensory input from the position, velocity and strain of the robot's soft tissues (C), which is locally pooled into the center of mass of each bone (D) thereby transforming the soft tis- sue's sensory input from voxel space to a graph that aligns with the skeletal sensory graph (E). The combined rigid+soft sensory graph is then fed as input to graph transformer (F). The graph trans- former distils sensory signatures across the graph into updated node features (G), and updated edge features (H). The Actor (I) takes the updated edge features
input and outputs motor commands (J): the target rotation angle for each joint (K). The updated node features are pooled by globally by channel-wis maximum across the node dimension (L) to retrieve graph-level output (M), which the Critic (N) uses to predict a value function based on the robot's current state information.
actuation, volume preservation under twisting deformations, or the parallelization necessary to scale design to complex bodies composed of large numbers of building blocks. Here, in contrast, we present a massively-paralle -arerlll--aalllll simulator that uses a more accurate, twistable model of elasticity (Hiller & Lipson, 2014) as the basis of building blocks of soft tissues which directly attach to rigid building blocks of bones and articulated skeletons with joint constraints and actuation (Baraff, 2001).
Cheney et al. (2013) evolved robots composed of elastic building blocks they called "muscle", "fat" and "bone". The muscles rhythmically pulsed, expanding and contracting in volume; the fat was passive; and the "bone" was equivalent to fat in density and mass but had an order-of-magniudde higher stiffness (Young's modulus). They found that any further increases to bone stiffness pre- vented the pulsating muscle blocks from sufficiently deforming the robot's jointless body and thus inhibited locomotion. In reality, animal bones are seven orders-of-maad -f-gniitude stiffer than animal fat (Guimarães al., 2020), and pulled by tendons and muscles about joints to rapid generate large yet precise dynamic movements of load bearing limbs. One could simply add joints to this elas- tic model; however, accurate simulations of elastic elements with higher stiffness (and thus higher resonance frequencies) approaching that of animal bones quickly become prohibitively expensive as ever smaller time steps are required to avoid numerical instability. Moreover, the fully "soft" (entirely non-rigid) and jointless robots evolved by Cheney et al. (2013) also lacked sensors, were composed of no more than few hundred elastic blocks, and could only move along perfectly flat surface plane. Here, in contrast, we introduce new approach that combines hundreds of thousands of both elastically deformable and rigid building blocks into multi-physics bodies that use joints and sensors to navigate complex terrestrial environments.
Following Hiller & Lipson (2010), Compositional Pattern Producing Networks (CPPNs; Stanley (2007)) have been used by many others (Cheney et al., 2013; Auerbach & Bongard, 2014; Cheney et al., 2018; Kriegman et al., 2021; Wang et al., 2023a; Cochevelou et al., 2023) to encode robot designs. CPPNs are class of feedforward neural networks with spatial inputs (e.g. cartesian coordi- nates) and hence spatially correlated outputs, which can be used to determine the absence or presence of cell each queried spatial coordinate. CPPNs thus provide simple way "paint" material
3

---

Published as a conference paper at ICLR 2025
A
B
C
D
b
f T
fb: Tb
f Tb
fc T
Figure 4: Simulating endoskeletal robots. Soft tissues (green masses) were modeled by grid of Biter-Blrooulli beams (A) that may twist and stretch and directly attach to bones (blue masses; and C) that follow rigid bodied dynamics with joint constraints (D). More details about the simulator can be found in Sect. 2.1 and Appx. B.
into workspace; however, they struggle to integrate basic structural and functional constraints, such as those which ensure that rigid cells belonging to the same bone form a single contiguous object, that joints are only placed between nearby bones, and that bones remain inside the body. Moreover, one CPPN (its entire architecture) encodes a single design rather than a latent manifold continu- ous variables that encode a generative model of robot designs. Here we learn one such continuous representation for endoskeletal robots and demonstrate the learned representation's 's structural and functional coherence, expressivity, smoothness (Fig. 5) and interpretability (Figs. 13-15).
Prior work encoded robot designs into continuous latent embedding (Hu et al., 2023); however, the design space (Zhao al., 2020) was limited by graph grammars to simple body plans containing a small number of jointed rigid links with predefined shapes. These simplifying assumptions allowed for effective model predictive control (Lowrey et al., 2019), which requires an explicit and accurate model of the robot's dynamics, including its interaction with the environment. Here, in contrast, we use a model-free, end-to-end learning approach to design and control externally body plans with highly non-linear dynamics and multi-physics interactions that are difficult to model explicitly.
More specifically, we here train a single reinforcement learning system-a "universal controller" (Fig. 3) across evolving population of freeform endoskeletal robots. This is achieved by lo- cally pooling (Peng et al., 2020) each robot's s arbitrary arrangement of high-resolution (voxel-level) sensory channels align with its skeletal graph, which may then be fed as input graph trans- former (Yun et al., 2019). Graph transformers were previously employed for universal control of many robots with differing physical layouts (Gupta al., 2022); however, these robots were con- strained to bilaterally symmetrical "stick figures" composed of jointed rigid cylinders (Ventrella, 1994). There was no need learn 3D volumetric representation of sensor integration as the stick figures were already in simple graph form with their sensors directly aligned to the edges of their graph. We here evolve embodied agents beyond the rigid stick figures (Yuan et al., 2022; Gupta et al., 2022) and boneless "blobs" (Wang et al., 2023a; Huang et al., 2024) found in recent work with the multiphysics simulation, freeform design and universal control of endoskeletal robots.
METHODS
In this section we describe how endoskeletal robots were simulated (Sect. 2.1), encoded (Sect. 2.2) and optimized (Sect. 2.3). A symbol table is provided on the very last page of the paper, in Table. 9.
2.1 SIMULATING ENDOSKELETAL ROBOTS
We here introduce novel voxel-based experimental platform that was built from the ground up to support the development of novel representations and algorithms for co-designing the form and function of endoskeletal soft robots. Fig. 4 provides high level overview of the underlying physics simulator, which enables two-way coupling between freeform soft tissues and freeform rigid bones with joint constraints. In this voxel-based simulator, elastic voxels and rigid voxels used as building blocks soft tissues and bones, respectively. Elastic voxels and rigid voxels are subject to their own respective update rules, which are outlined in Appx. B.
4

---

Published as a conference paper at ICLR 2025
A'
B'
D'
G'
K
C'
E
H'
L'
A
F
I
M
B
C
J
N'
D
F
Or
G
H
K
M
N
Figure 5: Interpolating between three points in endoskeletal latent space. Designs sampled from a 2D slice the learned latent embedding (A-O) and their internal jointed skeletons (A'-0',, where A' reveals the skeleton of A and cyan cylinders show the location of hinge joints between bones in each skeleton. The three corner designs (A,K,O) were drawn from arbitrarily selected latent coordi- nates and the 12 designs between them sit with equal spacing along plane of linear interpolation. The visible smoothness, mechanical coherence, and geometric and topological expressiveness the voxel-based latent space facilitated the co-design of morphology and control.
The most challenging part building such a simulator was integrating the dynamics of soft and rigid components, which can be summarized by single equation for all voxels:
M F + F + T(u E F".
where v is single voxel, M E 6 is the generalized mass matrix combining mass and inertia terms, X I is the second derivative of generalized position X with respect time t, F7 g and F E R are the generalized force terms including force and torque terms, Fø is the effect of all elastic beams (Fig. 4A) and F is the effect of all external forces such as gravity and friction. F. is the generalized constraint force term which is only applied on rigid voxels using the indicator function [[( E bone). For more details see Appx.
2.2 ENCODING ENDOSKELETAL MORPHOLOGY
We began by procedurally generating synthetic training plans in 3D voxel space using multi-star graphs (Fig. 10A), which consist of random number of randomly positioned star nodes like this * with variable number of arms radiating from their center
of valid endoskeletal body
5

---

Published as a conference paper at ICLR 2025
points. The radius and length of the bone and the radius of the surrounding soft tissue in each arm were all randomly assigned. Each arm may connect by a joint to an arm of an adjacent star, or remain uncomnected. Synthetic data generation pseudocode and hyperparameters can be found in Appx E. The resulting body was voxelized within a 64 x 64 x 64 cartesian grid. Fach voxel in the grid was assigned an integer label i E [1, K + 2). where i = 1 represents empty space, represents a soft tissue, and i > 2 was used to represent the (at most K) independent rigid bones. A one-hot encoding was used represent the workspace: a tensor with shape (64, 64, 64, k + 2).
A variational autoencoder (VAE; Kingma & Welling (2013)) with four blocks of Voxception-Ress Res modules (Brock al., 2016) was then trained to map voxel space into a highly compressed latent distribution consisting of 512 latent dimensions. Prior work that used voxel-based autoencoders con- sidered fixed dataset, which limited their depth and required data augmentation to avoid overfitting (Brock et al., 2016). Because we can generate new synthetic data points on demand, our training data is unlimited, and the depth of our autoencoder and thus its potential for compress, generalize and capture complex hierarchical features-is not constrained by lack of data.
Each design decoded from the latent space was automatically jointified (as detailed in Fig. 9), and any exposed bone was covered with a thin layer of soft tissue. This tissue patching process was highly targeted; in randomly generated designs, patching was applied to less than 3% of the workspace and covered less than 13% of the robots' bodies. See Appx. F for VAE hyperparameters.
2.3 CO-OPTIMIZING ENDOSKELETAL MORPHOLOGY AND CONTROL
State space. Endoskeletal robots receive high dimensional mechanosensory data 8" {(x", ,", ".".v".... U E soft] with 3D position I, y, 2, strain e and velocity v from a variable number of soft voxels that are distributed at variable locations in 3D space (Fig. 3C), depending on the size, shape and deformation of its soft tissues. In addition to sensing the movement and distor- tion of the soft tissues, proprioception of each rigid bone r and joint j between bones is also taken as input. The state of bones can be represented as s" = {(x", y", 2", .''... q", v", ,w",,... | E bone) where x, y, is the position of the center of mass of the bone, m is the mass, q is quaternion repre- senting the orientation, and v and w are the linear velocity and angular velocity. The state of joints can be represented as s² = {(h,, di. d 0², o llax)...... ... | E joints} where h is the hinge axis direction of joint, d₁ is the vector from the joint position to the center of mass of the first connected bone, d2 is the vector from the joint position to the center of mass of the second connected bone, 0 is the current joint angle, and 0 and Ona are the minimum and maximum joint angle limits, respectively. Soft voxel sensing is locally pooled into nodes at the center of mass of each bone (Fig. 3D) thus aligning soft tissue and skeletal inputs (Fig. 3E) and promoting "reflex partitioning" (Windhorst et al., 1989) whereby motors are affected more by local sensors than distant sensors.
Action space. The action space consists of joint angles, which control the movements and ar- ticulation of the robot's internal skeleton. We use a uniform discrete angle action space AA = {-1.4 rad, -0.7 rad, 0 rad, 0.7 rad, 1.4 rad} for all joints. We selected 1.4 rad as the limit for joint rotation range to prevent extremely large unrealistic deformation of the soft materials surrounding the joints (Bonet & Wood, 2008). We found that a discrete action space greatly simplified and stabilized policy training compared to an otherwise equivalent continuous action space (Fig. 16).
Reward. Reward was based on net displacement of the robot across 100 pairs of observations and actions sampled at 10Hz during a simulation episode of 10 seconds. Rewards were generated per episode in the following way:
ur (x2-1 Fo, Ut-1 yo)
ü, = ut
(3)
||eell
Ve = (o) II-1, Bt - 11-1)
(4)
-Pugee
if ||v.l| <
r = max( Vt, Vr.-Psmali
(5)
otherwise
where 2, and Ye are the and y position of the center of mass of the robot time step r the reward received by the robot, is a small threshold value for detecting stationary, and u, is the history movement vector up the previous frame projected the xy plane. If the norm of u, is

---

Published as a conference paper at ICLR 2025
O, replace ûn with vector (1, 0) pointing in the positive X direction. We apply large penalty if robot is static and clip a negative reward to a smaller penalty if the robot is moving but in an unde- sired direction. Eq. 5 encourages the robot to maintain consistent forward motion while penalizing stagnation or drastic directional changes, which was found to be more robust for identifying designs with high locomotive ability compared using naive reward based on moving distance in fixed direction, such as To = It It-1. This is because the naive reward may discard designs with high locomotive ability in the wrong direction. In some experiments the reward function was augmented to encourage robots to lift themselves above the surface plane using legs.
Policy. We employed Proximal Policy Optimization (PPO; Schulman et al. (2017)) train a single universal controller for an evolving population of 64 endoskeletal robots. A clone of each design in the population was created, yielding batch of 128 designs. This last step intended to broaden the evaluation of each design and so reduce the likelihood of erroneously discarding good design.
At each time step I, the policy's observation S, (s ,',,)) consists of soft voxel observations si joint observations sy, and rigid bone observations sÇ. Given these observations as input, the policy returns actions and value estimates according to the following equations:
s/ = SpatialPool(s")
(6)
n2 = [s[, st]
(7)
nn, ēt) = GraphTransffommr((n s/))
(8)
n; MaxPool((i.)
(9)
V(s,) ResNety (nī)
(10)
Zt.j = ResNet, (er.))
(11)
π (at.,)s = Softmax (Zt,))
(12)
T (ar]ss)) = II π (ac. (8x)
(13)
Soft voxel observations sy are locally pooled (Peng al., 2020) into 64 grid using the robot center of mass as the grid origin. The convoluted feature grid is then queried at the center of masses of each rigid bone, forming locally pooled node features s (Fig. 3D). This operation aligns soft tissue sensory inputs with skeletal sensing. The pooled features are then concatenated with the rigid bone observations S, form the node features n,, shown in Fig. 3F.
A graph transformer (Shi et al., 2021) processes the robot's topology graph, taking the node features n. and edge features s (joint observations) as input, and outputting processed node features n, and processed edge features e; The processed edge features are produced by concatenating processed node features across the two nodes connected by each edge. By max pooling (Fig. 3L) over the processed node features ñ, we obtain a global feature vector nž (Fig. 3M), which summarizes the overall state of the robot. The Critic (Fig. 3N) uses ResNety to take the pooled node feature nž and output the value function (s,)). The Actor (Fig. 3I) processes edge features érj- corresponding to each joint j. and passes them through ResNet, to compute the action logits Zz.j which define the action distribution for joint j. By computing the softmax over the logits Zt,j, probabilities are obtained over the discrete action space A. The overall policy T(or|s:) product of the action distributions over all joints, assuming independence between joints.
thus defined as the
By utilizing graph transformer to process the robot's topology, the policy effectively learns to condition on both the sensory input and the morphology of a robot. This architecture al- lows the actor to generate joint-specific actions while the critic evaluates the discounted score V(st)) = E, EE
The aggregate behavior of a robot across an entire simulation episode was evaluated against fitness function. After 20 epochs of learning, each design in the current population was assigned fitness score equal to the peak fitness it achieved across 40 episodes of simulation.
Fitness.
Evolutionary design. The population of designs fed to the universal controller was optimized by covariance matrix evolutionary strategies (CMA-ES; Hansen & Ostermeier (2001)). Briefly, a mul- tivariate normal distribution of designs is sampled from the latent space and the mean vector is "pulled" toward the latent coordinates of sampled designs with the highest fitness and the covari- ance matrix is adapted to stretch along the most promising dimensions and shrink within others.
7

---

Published as a conference paper at ICLR 2025
Flat Ground
Upright Locomotion
Potholes
Mountain Range
A
B
D
10
E
G
H
0.0
0.0
Fitness
2.6
0.6
0.5
0.2
0.2
0.2
0.0
100
20
100
0.0
10
30
10
Generation of design evolution
Generation of design evolution
Generation of design evolution
Generation of design evolution
Figure 6: Task environments. We considered four task environments: Flat Ground (A), Upright Locomotion (on flat ground; B), Potholes (C) and Mountain Range (D). In each one, two indepen- dent evolutionary trials were conducted, and the peak fitness achieved by each design was averaged across the population before plotting the cumulative max (E-H). The best design is shown for each environment (A-D). Rewarding net displacement over flat ground (A,E) results in the evolution of snakes (A). ran optimization for extra time (100 generations) to see if legs would emerge, but evolution failed to escape the local optima of snakes. Adding second component to the reward function that tracks the proportion of body voxels that fall below a prespecified height threshold during behavior (B,F) resulted in legged locomotion (B), and continued to innovate when provided extra time. Increasing the difficultly of the terrain also promoted morphological diversity (C,D).
Because CMA-ES uses ranked fitness scores instead of raw fitness scores to update the mean design vector, undesirable designs can be assigned large negative rank without interfering with the stabil- ity of the optimizer. Body plans that have less than two joints less than 20% bone, were deemed "invalid designs" and assigned a large negative reward as described below in Sect. 2.3. Less than 15% of the randomly generated robots were invalid under this definition of validity.
RESULTS
In this section, present learned endoskeletal latent embedding and its use in body-brain co- design across four different task environments. We also perform a series of control experiments that isolate the effect of morphological evolution in order to determine its role in achieving high performing endoskeletal robots.
3.1 ENDOSKELETAL LATENT SPACE
The learned latent space is smooth and expressive (Fig. 5) and was found to contain tunable genes that encode specific morphological traits, such as body height, length, width, volume and bone count (Figs. 13-15). The trained VAE is able to encode and decode previously unseen training data points with high accuracy (Fig. 10) while generalizing beyond the training data: Most designs sampled from the latent space look very different from the multi-star graphs (see e.g. Fig. 10A) used train the VAF. For example, skeletal voids (i.e. holes that pass through individual bones) were not present the generated synthetic training data set but they are present in the bones of decoded designs from sampled latent codes (Fig. 5A').
3.2 TASK ENVIRONMENTS
Flat Ground. On perfectly flat ground, selection for forward locomotion resulted in the evolution of snake-like body plans with two joints (Fig. 6A) that are able move high speeds.
8

---

Published as a conference paper at ICLR 2025
Peak fitness
Average fitness
Design freeze
A
Evolution
B
Evolution
Evolved designs: Optimized RL.
Random Search
Random Search
Evolved designs: from
Evolved designs
Evolved designs
Initial designs: Optimized RL
designs
Initial designs
Initial designs: from
Fitness
Mylo A
10 15
30
200 300
400
500
600
100 150
200
Generation of design evolution
Learning epoch
Learning epoch
Figure 7: Evolution learning. population 64 designs was iteratively drawn from an evolving design distribution and used by reinforcement learning to obtain universal controller. The peak fitness achieved by each design across 20 epochs of RL is plotted (with 95% bootstrapped confidence intervals) each generation of design evolution in the Upright Locomotion task environment (A). The dark gray line in A is mean peak fitness across the 64 designs in the population; the shaded region is the CI of the mean. Similarly, the average fitness achieved by each design across its simulation episodes is shown at each learning epoch (in B). An otherwise equivalent experiment with random morphological search instead of evolution strategies was also performed (purple curves in A and B). After 30 generations of evolution (600 total epochs of RL), the 64 designs from the best evolved population (blue dot; gen 23) were compared to the 64 designs from initial randomly- generated population (green dot; gen 0). To do so, the designs within each population were frozen and their behavior was re-optimized (C) starting with the optimized universal controller from gen 23 (Optimized RL). A second comparison was then made in which new universal controller re-learned for each population separately, from scratch (RL from scratch). In both comparisons, the learning conditions were equivalent and the only differences between the populations were morphological. And in both comparisons the evolved designs easily outperformed the initial designs. This indicates that design evolution necessary: it pushed search into regions of the latent space containing better morphologies with higher fitness potential that better support universal controller training.
Upright Locomotion. Given the impressive speed of snakes on flat ground, second term was added to the reward function to promote body plans that exhibit upright legged locomotion. To do the proportion of body voxels that fell within 5 em of the surface plane during behavior was subtracted from the original reward at each step of learning, and the original fitness was divided by the mean proportion of voxels that fell below this threshold across the full simulation episode. This resulted in less serpentine body plans with more joints (Fig. 12). The best evolved design is much more complex, and uses its legs lift its body above the ground during locomotion (Fig. 6B).
Next, we explored a more challenging terrain. In the Potholes environment, there are regularly-spaced depressions ("potholes") along the surface plane. The depth, width, and spacing of the potholes can be easily adjusted in the provided code (indeed any 2D height matrix can be supplied to change the terrain). With relatively small, tightly-spaced potholes, the best evolved designs (Fig. 6C) are longer and more complex than the snakes that dominated flat ground under the same fitness function (Fig. 6A). A common adaptation that evolved for navigating potholes was broad feet, which the robots used to push off and pull against the vertical walls inside the potholes.
Potholes.
Mountain Range. In our fourth and final environment, robots must climb up a slope in order to move away from the origin. In this mountain environment, the best evolved designs (Fig. 6D) were also slightly more complex than the flatland snakes.
3.3 THE NECESSITY OF MORPHOLOGICAL EVOLUTION
Across all four task environments, evolved design populations significantly outperform the randomly-generate initial populations of designs from the first generation of evolution. This could be due to improvements in the universal controller or better body plans, both. In order to isolate the effects of morphological evolution and policy training, we performed the following three control experiments using the Upright Locomotion task environment.

---

Published as a conference paper at ICLR 2025
In the first experiment, extracted the design population that achieved the highest mean reward during evolution (Evolved designs in Fig. 7B), as well as the RL model checkpoint at this point (Optimized RL in Fig. 7C). We also extracted the initial designs that were randomly generated at the very beginning of evolution (gen 0). The initial designs (Fig. 11) were frozen and the controller was re-optimized using this population of frozen designs for 100 epochs starting from optimized RI. checkpoint. In parallel independent experiment, the evolved designs (Fig. 12) were likewise frozen and the controller was re-optimized using the frozen evolved population of designs for 100 epochs starting from optimized RL checkpoint. The same procedure was repeated, this time retraining the control policy from scratch, for both frozen design populations separately. In both cases-statee policy training from scratch or from the RL checkpoint- the frozen evolved designs significantly outperformed the frozen initial designs (Fig. 7C). In both frozen design populations, the universal controller from the checkpoint immediately exhibits the best performance achieved after retraining from scratch. This shows that RL does not suffer catastrophic forgetting: the policy does not forget how to control the initial designs as the population evolves to contain different, better designs.
These results suggest that morphological evolution yields better designs but they do not prove that evolution is necessary. It could be the case that good designs will simply arise by random muta- tions alone, without cumulative selection across discrete generations. As a final control experiment we tested this hypothesis by replacing evolutionary strategies with random search (purple curve in Fig. 7,, Random morphological search performed significantly worse than evolutionary strate- gies. This suggests that morphological evolution (mutation and selection) is indeed necessary.
4 DISCUSSION
In this paper we introduced the computational design of freeform endoskeletal robots. The open- ended nature of the design problem allowed us to procedurally generate a never-ending synthetic data flow of endoskeletal training examples for representation learning, which in turn allowed for effective learning of a much deeper voxel-based autoencoder than those learned in prior work with fixed datasets (Brock et al., 2016). This resulted in a very smooth and highly expressive latent embedding of design space (Fig. 5), which we employed as a genotype space (and the decoder as the genotype-phenotype mapping) for morphological evolution.
We found that a universal controller could be simultancously obtained by reinforcement learning during morphological evolution despite the wide range of endoskeletal geometries and topologies that emerged within the evolving population. We observed across several independent experimen- tal trials that morphological evolution tended to push search into promising regions of latent space consisting of high performing body plans that facilitated policy training. For flat ground, this re- sulted in the evolution of simple two-jointed snakes (Fig. 6A), which in hindsight makes intuitive sense: such bodies are easier to control. But, in the other three task environments we tested, many other, more complex solutions evolved, including legged locomotion (Fig. 6B). This suggests that other researchers may leverage the open- endednessss of this computational design platform to identify which physical environments and fitness pressures lead to the emergence of particular abilities and the embodied structures required to support those abilities
Others may use our simulator as an -oittertitte software library for benchmarking their own voxel-based representations and learning algorithms in the four task environments released here, or by creating their own terrain maps and reward functions based on these examples. To this end, we also provide an example of object manipulation ("dribbling football"; Fig. 8), which we hope inspires others to create their own objects and build more, and more intricate, virtual worlds.
However, there were key limitations of the present work that may be overcome by future work. First, because skeletons were assumed to be inside of each robot, rigid body collisions were not modeled, and thus external rigid horns, claws, shells, etc. were not possible. Also, fluids were not modeled, and thus the simulator was restricted to terrestrial environments and land based behav- iors. But the most important limitation of this paper is that simulated designs were not realized as physical robots. Recent advances in volumetric 3D printing (Darkes -Brkky & Shepherd, 2024) are beginning to enable the fabrication of endoskeletal bodies that flex their muscles but are not yet ca- pable of locomotion. This suggests that artificial endoskeletal systems will soon move through our world, opening the way to evolved endoskeletal robots that begin to approach the sophisticated and perhaps eventually cognitive behavior of evolved endoskeletal organisms.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research was supported by NSF award FRR-2331581, Schmidt Sciences A12050 grant G-22- 64506, Templeton World Charity Foundation award no. 20650, and the Berggruen Institute.
REFERENCES
Joshua E Auerbach and Josh C Bongard. Environmental influence on the evolution of morphological complexity in machines. PLoS Computational Biology, 10(1)=e1003399 2014.
David Baraff. Physically based modeling: Rigid body simulation. SIGGRAPH Course Notes, 2(1): 2-1, 2001.
Joachim Baumgarte. Stabilization of constraints and integrals of motion in dynamical systems. Computer Methods in Applied Mechanics and Engineering, 1(1):1-16, 1972.
Javier Bonet and Richard D. Wood. Nonlinear Continuum Mechanics for Finite Element Analysis. Cambridge University Press, 2nd edition, 2008
Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv: 1608. 1600.04266.446, 2016.
Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding. Proceedings of the Conference Genetic and Evolutionary Computation (GECCO), pp. 167-174. ACM, 2013.
Nick Cheney, Josh Bongard, Vytas SunSpiral, and Hod Lipson. Scalable co-optimization of mor- phology and control in embodied machines. Journal of The Royal Society Interface, 15(143): 20170937, 2018
François Cochevelou, David Bonner, and Martin-Pieen Schmidt. Differentiable soft-robot gener- ation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 139-137. ACM, 2023.
Cameron Parkes-Burker and Robert Shepherd. Volumetric 3D printing endoskeletal soft robots. Advanced Materials, pp. 2402217, 2024.
Kenny Erleben. Numerical methods for linear complementarity problems in physics-based anima- tion. In ACM SIGGRAPH 2013 Courses, pp. 1 1-42, 2013.
Carlos F Guimarães, Luca Gasperini, Alexandra P Marques, and Rui L Reis. The stiffines of living tissues and its implications for tissue engineering. Nature Reviews Materials, 5(5):351-370 2020.
Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature Communications, 12(1):1-12 2021.
Agrim Gupta, Linxi Fan, Surya Ganguli, and Li Fei-Fei. Metamorph: Learning universal controllers with transformers. In Proceedings of the International Conference on Learning Representations (ICLR), 2022.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159-195, 2001.
Donald Hejna, Pieter Abbeel, and Lerrel Pinto. Task-agnostic morphology evolution. In Proceed- ings of the International Conference on Learning Representations (ICLR), 2021.
Jonathan Hiller and Hod Lipson. Evolving amorphous robots. In Proceedings of the Conference on Artificial Life (ALife), pp. 717-724, 2010.
Jonathan Hiller and Hod Lipson. Automatic design and manufacture of soft robots. IEEE Transac- tions on Robotics, 28(2):55-466,, 2012.
Jonathan Hiller and Hod Lipson. Dynamic simulation of soft multimaterial 3D- -D-printed objects. Soft Robotics, 1(1):88-101, 2014.
11

---

Published as a conference paper at ICLR 2025
Jiaheng Hu, Julian Whitman, and Howie Choset. GLSO: Grammar- Srammar-guided latent space optimiza- tion for sample-efficient robot design automation. Proceedings of the Conference on Robot Learning (CoRL). pp. 1321-1331, 2023.
Suning Huang, Boyuan Chen, Huazhe Xu, and Vincent Sitzmann. Dittogym: Learning to control soft shape-shifting robots. In Proceedings of the International Conference on Learning Represen- tations (ICLR), 2024.
Nick Jakobi, Phil Husbands, and Inman Harvey. Noise and the reality gap: The use simulation in evolutionary robotics. In Proceedings of the European Conference on Artificial Life (ECAL), pP- 704-720, 1995.
Johan Jansson and Joris SM Vergeest. Combining deformable-and rigid-body mechanics simulation. The Visual Computer, 19:280-290 2900-290,.00 2003.
Junggon Kim and Naney S Pollard. Fast simulation of skeleton-driven deformable body characters. ACM Transactions on Graphics (TOG), 30(5); 2011.
Diederik P Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. A scalable pipeline for designing reconfigurable organisms. Proceedings of the National Academy of Sciences, 117(4): 1853-1859, 2020a.
Sam Kriegman, Amir Mohammadi Nasab, Dylan Shah, Hannah Steele, Gabrielle Branin, Michael Levin, Josh Bongard, and Rebecca Gramer-Bottiglin Scalable sim-to-real transfer of soft robot designs. In Proceedings of the International Conference on Soft Robotics (RoboSoft), pp. 359- 366, 2020b.
Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. Kinematic self-replication in reconfigurable organisms. Proceedings of the National Academy of Sciences, 118(49):
e21 e2112672115 2021.
Jing Li, Tiantian Liu, and Ladislav Kavan. Soft articulated characters in projective dynamics, JEEE Transactions Visualization and Computer Graphics,
2020.
Muhan Li, David Matthews, and Sam Kriegman. Reinforcement learning for freeform robot design. In Proceedings of the International Conference on Robotics and Automation (ICRA), 2024.
Hod Lipson and Jordan B Pollack. Automatic design and manufacture of robotic lifeforms. Nature, 406(6799):974, 2000.
Kendall Lowrey, Aravind Rajeswaran, Sham Kakade, Emanuel Todorov, and Igor Mordatch. Plan online, learn offline: Efficient learning and exploration via model-based control. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
David Matthews, Andrew Spielberg, Daniela Rus, Sam Kriegman, and Josh Bongard. Efficient auto- matic design of robots. Proceedings of the National Academy of Sciences, 23244))e2005180111 2023.
Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Convo- lutional occupancy networks. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 523-540, 2020.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: arXiv:1707.063: 1707.06377, 2017.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 1548-1554, 2021.
Tamar Shinar, Craig Schroeder, and Ronald Fedkiw. Two-way coupling of rigid and deformable bodies. In Proceedings of the Symposium Computer Animation (SCA), pp. 95-1 2008
12

---

Published as a conference paper at ICLR 2025
Karl Sims. Evolving 3D morphology and behavior by competition. Artificial Life, 1(4): 372, 1994.
Kenneth O Stanley, Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines, 8:131-162, 2007.
Luke Strgar, David Matthews, Tyler Hummer, and Sam Kriegman. Evolution and learning in differ- entiable robots. In Robotics: Science and Systems (RSS), 2024.
Jeffrey Ventrella. Explorations in the emergence of morphology and locomotion behavior in ani- mated characters. In Proceedings of the International Workshop on the Synthesis and Simulation of Living Systems (ALife), pp. 436 441, 1994.
Tsun-Hsuan Wang, Pingchuan Ma, Andrew E. Spielberg, Zhou Xian, Hao Zhang, Joshua B. Tenen- baum, Daniela Rus, and Chuang Gan. Softzoo: A soft robot co-design benchmark for locomotion in diverse environments. In Proceedings of the International Conference on Learning Represen- tations (ICLR), 2023a.
Tsun-Hsuan Wang, Juntian Zheng, Pingchuan Ma, Yilun Du, Byungchul Kim, Andrew E. Spielberg, Joshua B. Tenenbaum, Chuang Gan, and Daniela Rus. Diffusebot: Breeding soft robots with physics-augmented generative diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2023b.
Uwe Windhorst, Thomas M. Hamm, and Douglas G. Stuart. On the function of muscle and reflex partitioning. Behavioral and Brain Sciences, 12(4):629-664 645, 1989.
Ye Yuan. Yuda Song, Zhengyi Luo, Wen Sun, and Kris Kitani. Transform2Act: Learning transform-and-conie and-control policy for efficient agent design. In Proceedings of the International Con- ference on Learning Representations (ICLR), 2022.
Seongjun Yun, Minbyul Jeong, Rachyun Kim, Jaewoo Kang, and Hyunwoo Kim. Graph trans- former networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Allan Zhao, Jie Xu, Mina Konaković-Luković, Josephine Hughes, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. RoboGrammar: Graph grammar for terrain-optimized robot design. ACM Transactions on Graphics (TOG), 39(6):1-16, 2020.
13

---

Published as a conference paper at ICLR 2025
A
SUPPLEMENTAL FIGURES
= 0 sec
sec sec
= 5 sec
= 7.5 sec
Figure 8: Object manipulation. One of the designs that evolved for Upright Locomotion performs new task: it pushes object forward. This design among several others that sampled from the evolving population analyzed in Fig. 7, placed behind a soft sphere (teal), and actuated using the universal controller from the Optimized RL checkpoint in Fig. 7C. Videos of the resulting behaviors and the source code necessary to reproduce our results can be found on our project page. https//endoskelltal. .ithh.............uu...oo..o.
B
D
E
F
Extract contact surface
Build PCA bounding box
Extemal structure
Interaal structure
Bone interface
Add joint
Figure 9: Jointification. The learned latent space encodes generative model of an endoskeletal body plan: external soft tissue geometry (A) and internally segmented skeleton (B). The la- tent space also implicitly encodes the location and orientation of each joint in the skeleton: at the interface of each pair of bones in voxel space (C). In order simulate a joint, voxels along the contact surface of two opposing bones are extracted (D) and Principal Component Analysis used construct an oriented bounded box around them (E). A hinge joint then positioned at the center of mass of the contact surface with a direction along with the longest axis of the bounding box (F).
A
Sample
Sample
Sample
Sample
Reconstruction
Reconstruction
Reconstruction
Reconstruction
Figure 10: Examples of encoding previously unseen sythentic training data (A) and reconstructing each one (B) using the fully optimized VAE. Each column displays a sample/reconstruction pair.
14

---

Published as a conference paper at ICLR 2025
0.72
0.71
0.59
0.54
0.53
0.50
0.50
0.48
0.45
0.44
0.44
0.42
0.40
0.38
0.36
0.35
0.34
0.33
0.33
0.33
0.29
0.28
0.28
0.26
0.26
0.25
0.24
0.24
0.24
0.24
0.24
0.23
0.21
0.21
0.20
0.19
0.19
0.18
0.18
0.17
0.17
0.15
0.13
0.13
0.13
0.10
0.08
0.04
Figure 11: Ancestors. Part of a random initial population, sorted by net displacement (in meters).
2.01
1.88
1.68
1.58
1.53
1.43
1.32
1.30
1.09
1.03
0.99
0.99
0.90
0.89
0.86
0.85
0.83
0.75
0.74
0.73
0.72
0.69
0.68
0.68
0.65
0.58
0.57
0.53
0.51
0.49
0.48
0.47
0.47
0.47
0.45
0.42
0.38
0.37
0.37
0.36
0.36
0.35
0.33
0.31
0.29
0.26
0.25
0.25
Descendants. Part of the evolved population that descended from the initial population shown above in Fig. 11 for Upright Locomotion. Under each design, its displacement in meters.
Figure 12:
15

---

Published as a conference paper at ICLR 2025
A
Correlation
Volume correlation
0.0
-0.5
Feature Index
189
240
361
B
-2.00a
-1.33g
-0.57a
(0o)
0.67a
1.33d
2.00a
Volume
11344
Volume
Corrclation
0.5
Bane count latent correlation
-0.5
Feature index
240
488
507
-2.00a Bone count
-1.33a
-0.67a
# (0o) Bone count
0.67a Bone count
1.33a Bone count 3
2.00c
Bone count
Bone count
Bone count
Correlation
0.5
X size & latent correlation
-0.5
Feature index
189
390
413
-2.00a
-1.33a X Size 12
-0,67a X Size 13
μ (0o) X Size 47
0,67a X Size 62
1.33a X Size 64
2.00c Size 64
X Size 12
Correlation
Y size latent correlation
0.0
Feature index
200
315
361
H
-2.00a
-1.330 Size 12
(0o)
Sizee 12
Size 12
Size
Size
Size
Correlation
Z size latent correlation
-0.5
Feature index
249
335
-2.00a Z ZSize 13
-1.33g Z 2ize 12
-0.67a Z,iize 12
# (0o) Z Size 17
0.67a ZiSize 61
1.33a Z Size 64
2.00c Zize 64
Figure 13: Screening for latent genes. We measured the Pearson correlation coefficient (p) of the 512 latent features and the body volume of one million randomly sampled designs (A). The set of latent features that are highly correlated with volume ((p) 0.1) were extracted as "the gene for volume", the effect of which gene for volume + 20 is shown for the design decoded from the sample mean of the latent space (Іt). The same procedure was repeated identify and modulate genes for bone count (C, D), length (E, F), width (G, H) and height (I, J).
be increased decreased demand (B). The effect of tuning the
16

---

Published as a conference paper at ICLR 2025
A
Value fa
Volume latent gendtype
Feature Index
189
240
390
411
Volume 14830
B
Value a
Volume gendtype
0.25c
Feature index
189
240
Volume 16935
C
Value a
Volume latent genotype
0.
Volume 19590
Feature Index
189
240
361
350
A7
D
Volume latent genotype
Volue la
Volume 23427
Feature index
189
240
580
E
Volue la
Volume genotype
1.00a
Feature index
189
240
Volume = 26403
Volurne latent genotype
Value t
1.250
Feature index
189
240
361
390
Volume 34888
G
Volume latent genotype
Value/ F
1.50a
Feature index
189
240
361
390
671
Volume 45249
Figure 14: Gene editing. The best design that evolved for Upright Locomotion was extracted (A) and the set of latent features found to individually correlated with body volume-the gene for volume-war incrementally increased and the volume of the resulting design measured (B-G). The entire latent genome shown to the right of each decoded design, with features outside the volume gene grayed out. Positively correlated features (240, 390 and 471) were amplified, and neg- atively correlated features (189 and 361) were reversed by 0.25g in each row, yielding progressively thicker body parts within the evolved body shape.
17

---

Published as a conference paper at ICLR 2025
A
1.00a
Volume 14929
Volume 16936
Volume 19590
Volume 23427
Volume 28403
Volume 34887
Volume 45255
B
0o
-0.50a
-1.00a
-1.500
-2.00a
-2.500
-3.000
Bone count
Bone count
Bone count 9
Bone count
Bone count 3
Bone count
Bone count 3
Do
-0.55
-0.50a
-0.75
-1.00a
-1.250
-1.50a
X Size 46
Size 41
X Size 35
Size 32
X Size 30
Size 29
X Size 28
0g
00177
-0.33d
-0.500
-0.670
-0 -0.830
-1.00a
Y Size 58
Y Size 54
Y Size 48
Y Size = 37
Y Size 32
Y Size 27
Y Size 25
E
-0.13r
0,27a
-0.40o
00.53
-0.67a
Z
Size
Figure 15: Programmable morphology. The best design that evolved for Upright Locomotion was extracted (from Fig. 6B.F) and the found genes for volume (A), bone count (B), length (C), width (D) and height (E) were individually modulated to demonstrate the precision with which evolved morphologies can be manually redesigned through "genetic engineering". Some genes afford greater control over specific morphological trait than others; however, this may be attributed to the simplicity of the procedure employed to screen for genes, which only captures pairwise linear associations between a manually-speccine morphological trait and an individual latent feature.
18

---

Published as a conference paper at ICLR 2025
Peak fitness
Average fitness
Discrete
Discrete
Continuous
Continuous
Fitness
15 20
100
200 300
400
500
600
Generation of design evolution
Learning epoch
Figure 16: Discrete and continuous action spaces. Peak (A) and average (B) fitness is plotted for the Upright Locomotion task, as in Fig. 7A and B. Two kinds of action spaces were compared: discrete (gray) and continuous (pink). The discrete action space is {-1.4 rad, -0.7 rad, rad, 0.7 rad, 1.4 rad] and the continuous action space is [-1.4 rad, 1.4 rad]. for all joints. In both panels, each line a mean across the 64 designs in the population and the shaded regions are a 95% bootstrapped CI of the mean. Although both continuous and discrete action spaces viable, discrete set of actions greatly simplified and stabilized universal policy training under the tested conditions.
Best SO far
Average fitness
Universal controller Individual controllers
B
Universal controller
Individual controllers
4
Fitness
2
2
1
10
20
30
40
50
5
10 15 20
25
30
Wall Clock Time [hour]
Learning epoch
Figure 17: Universal and individual controllers. An otherwise equivalent experiment was con- ducted in which the universal controller shared by all 64 designs in the population was replaced with bespoke individual controller for each design. As in Fig. 6, the peak fitness achieved by each design was averaged across the population before plotting the cumulative max (A). This statistic, which we term "best so far", captures the progress of the entire population. Training 64 indepen- dent policies for 30 epochs required 44.8 hours on NVIDIA H100 SXM GPUs. During the same time frame on the same hardware the universal controller can be trained for 451 epochs and thereby achieve much higher fitness. Over the first 30 epochs of learning, the performance of the universal controller not statistically different from that of the individual controllers (B), and since design evolution was only advanced single generation, the designs under universal control were visually similar those with independent controllers, with both populations closely resembling the random initial population (Fig. 11).
19

---

Published as a conference paper at ICLR 2025
B SIMULATION
In this section we describe the underyling soft and rigid body physics behind our endoskeletal robot simulator. We will use the symbol T for both moment and torque to avoid confusion. Subscripts used to indicate the source of an effect and superscripts indicate the target an effect.
B.1 SOFT VOXELS
Following Hiller & Lipson (2014), soft voxels are defined point of mass with rotational inertia in space. Each surface of soft voxel may connect by a Buler-Bernould beam with zero mass to an adjacent soft voxel (Fig. 4A) or rigid voxel (Fig. 4B,C). Each beam exerts moment ты and force fo onto the corresponding connected voxel when deformed from expansion, compression or twisting. Although Euler-Bernoulli beams are more complex to compute than Hookean springs, the additional torsional moment т contributes to preserving volume when the structure is under external torsional stress. Since endoskeletal robots embed joints between rigid bones inside soft tissue and produce large deformations, preserving volume prevents generating invalid force and moment values when soft tissues are radically compressed and thereby stabilizes simulation.
The moment ть and force fo generated by beams can be computed from position (2, v, and rotation 0 of voxels U1 and 12 attached on both ends using a constant stiffness matrix K::
x
TE eel
001
f8
=
[K]
x
(14)
_7b
L
Following Newton's second law, we can express the equation motion for soft voxels as:
m m"x" = Σ f
fox
(15)
0" Σ Tb
ext
where for and Thu represent external force and moment caused by gravity, friction, and contacting force etc. For simplicity, we will define generalized mass, position and force representations M X and pv and rewrite Eq. 15 as:
M Σ
Fg F F with x
x
F =
80 7
M
= m
E
0
(16)
0
1
where E is the identity matrix.
B.2 RIGID VOXELS
Rigid voxels are affected by the beams connected soft voxels (Fig. 4C), generalized external forces Fv and additional generalized constraint forces Fg, as from joints (Fig. 4D). Since rigid voxels belonging to the same rigid bone do not have relative motion, we compute the generalized position X" of the compound rigid bone r and infer X", As each rigid bone will be connected by many beams to its surrounding soft voxels, we will use F7 as the total generalized force exerted by all beams to simplify the representation:
Fr F
(17)
For FT and Fr. torque also generated by forces applied to the center of component rigid voxels. Thus:
F Σ
f and
Σ
f + (x"
(18)
The + (x"
ext
Tex
x")
för
vEr
PEr
where x, represents the center of mass the corresponding rigid bone r.
Contacts modeled between voxel pairs and contact forees thus aligned the center of colliding voxels instead of the center of mass of the rigid bones. To prevent generating a false torque term in Fe the contact force does not contribute to the torque.
20

---

Published as a conference paper at ICLR 2025
Finally, constrained dynamics were used to compute F follows. Constraints were defined as functions between two rigid bones 71 and r2:
c(X", x 2
(19)
For convenience, we now remove superscripts and concatenated X, F, and block-diagonalized M to represent attributes of both bones. By taking the time derivative of constraint function e we obtain the Jacobian matrix which maps global space constraint space:
x(X) JX
(20)
Introducing Lagrange multiplier X, which physically represents applied constraint forces, and can rewrite Eq. 17 as:
MX = F+J with == F Fext
(21)
By substituting Eq. 21 into Eq. 20 we get:
o(X) = Jold + JM + AJ
(22)
where Xold is speed from the last time step; this derivative needs be non-negative if wish the constraint to be satisfied at last time point. We also add a bias factor (Baumgarte, 1972) to correct constraint errors incrementally:
B At
x(X) = J.aaa + (F+ J
Cold (X) 2
(23)
where / is constant that controls the constraint error correction rate in each time step.
Reordering terms in Eq. 23 and we obtain equation system the form of linear complementarity problem (LCP) and then solve À using the projected Gauss-Seidel (PGS) algorithm (Erleben, 2013):
A JM
(24)
B
w
=
JX.s JMM
Cola (X)). At
(25)
AA Z
(26)
X
(27)
A
T
AX +
(28)
Note that Eq. 27 is required since À is in constraint space and it should be in the positive direction that satisfies the constraint. Note also that Eq. 28 required since constraint forces does not add energy to the system and thus should always be perpendicular to the direction of the constraint change.
C FROM SIMULATION TO REALITY: FUTURE WORK
The endoskeletal robots in this paper were confined simulated tasks within a virtual world. Though significant effort was dedicated to ensuring that the simulated mechanies, simulated en- vironments and simulated materials closely resembled physical reality and realizable morphologies (Table 7), it is in general difficult to guarantee that the behavior of system optimized in simulation will transfer with sufficient fidelity to reality (Jakobi et al., 1995). remains to determine the ef- feet that universal controllers have sim2real transfer, since sim2real of universal control has yet to be attempted. But because our approach simultaneously optimizes a population of many robots with differing topological, geometrical, sensory and motor layouts, it realizes the desired behavior in many unique ways and thereby provides many more opportunities for successful sim2real transfer compared other methods that optimize single design (Kriegman al., 2020b). We also pre- dict that the inherent capacity of universal controller to generalize across these differing action and observation spaces will render the policy more transferable across certain differences between simulation and reality, compared to bespoke single-morphology controller.
21

---

Published as a conference paper at ICLR 2025
D
OVERVIEW OF THE ENTIRE CO-DESIGN PIPELINE
Designs
Obs,
Synthetic Examples
VAE
Lalent
Evolution
Sim
Space
Fitness
Action
Figure 18: Data flow. Synthetic examples are used to encode the latent search space in which evolution operates. An evolving population of designs decoded from this space are simulated and used to train a controller. The population is updated based on the fitness of the designs.
E SYNTHETIC DATA GENERATION
Algorithm Procedural Generation of Synthetic Robot Body Plan
1: Input: Node range [mmin Maxx] degrade ratio yb
2: Randomly select number of nodes k in [Kmin Kmax]
3: for each node i = to do
4: Randomly initialize:
5: Max children C₁
6: Limb length T
7: Rigid radius T
8: Soft radius Tsoft.
9: Create StarNode
end for
11: Set node 1 as root (level 0)
12: for each node i =
do
13: Select parent node p with lowest level that can attach more children
14:
Decrease limb length: L, = L; x leevelp
15: Compute offset vector opposite to p's existing children, blend with random unit vector
16:
From p. use Lp and offset to find attachment point; use L, to place center of node i Attach node parent node p
17:
18:
end for
Table Hyperparameters and their ranges used in the procedural generation of synthetic robot body
plans.
Parameter
Range / Values
Description
Global Parameters
Knin
3
Minimum number of nodes (stars) per robot.
Kmax
8
Maximum number of nodes (stars) per robot.
0.9
Degradation ratio multiplier for limb length per hierar- chical level.
Node Parameters (for each node i)
Ci
[2, 4]
Maximum number of children (arms) for node Limb length for node
L
[0.1, 0.3]
Trigid,i
00.02, 0.05]
Radius of the rigid (bone) parts for node i. Radius of the soft (tissue) parts for node i.
Tinft,
0..02, 0.07
22

---

Published as a conference paper at ICLR 2025
F
MORE HYPERPARAMETERS
Table VAE encoder architecture hyperparameters.
Layer
Description
Input
Voxel input, 64 x (k + 2)
VRNO
Voxception-Restee (VRN), kernel size 3 32 channels
Block I
5x VRN, 32 channels, downsample to 32 64 channels
Block 2
5x VRN, 64 channels, downsample to 16 128 channels
Block 3
5x VRN, 128 channels, downsample to 83 256 channels
Block 4
5x VRN, 256 channels, downsample to 43, 512 channels
ResConv
Residual Conv, kernel size 3 512 channels
Max Pool
Max pooling. 512 channels
Linear
in: 512, out: 1024, GeLU
Linear-mu, Linear-logvar
mu: in: 512, out: 512, logvar: in: 512, out: 512
Output
mu: 512, logvar: 512
Table 3: VAB decoder architecture hyperparameters.
Layer
Description
Input
Embedding input, 512
Linear
in: 512, out: 32768, reshaped to 4 x 512
Block
5x VRN, 512 channels, ConvTranspose3D 8 256 channels
Block
5x VRN, 256 channels, ConvTranspose]] to 163, 128 channels
Block
5x VRN, 128 channels, Con/Transpose33 to 323, 64 channels
Block
5x VRN, 64 channels, ConTTranspose3D 64 k 2 channels (voxel prediction)
Output
Voxel logits, 64 x (k 2)
Table 4: RL encoder architecture hyperparameters. The RL encoder is shared by the actor and critic, combining spatial pooling and graph transformer.
Layer
Description
Input
s" (4 channels), s" (14 channels), s (9 channels)
ResNet (voxels wise)
in: 4, out: 64
Feature grid dim: 64 64
Spatial Soecoder-Decodee Net
UNet3D levels:
Peng et al. (2020)
UNet3D groups: 8
UNet3D feature maps: 64
TransformerConv 1 TransformerCom 2
node in: 78 (64 + 14), node out: 256, edge in: 9
node in: 256, node out: 512, edge in: 9
TransformerConv 3
node in: 512, node _ott 512, edge in:
ne: 512
Output
é : 1024 (by concatenate node features n;)
23

---

Published as a conference paper at ICLR 2025
Table 5: Actor architecture hyperparameters.
Layer
Description
Input
Outputs from RI. encoder (see Table 4)
ResNet
in: 1024, out: 256
ResNet 2
in: 256, out: 64
ResNet 3
in: 64, out: ||||]|
Output
ze: |||]
Table 6: Critic architecture hyperparameters.
Layer
Description
Input
ñ, Outputs from RL encoder (see Table 4)
Max Pool
Max pooling, 512 channels
ResNet 1
in: 512, out: 128
ResNet 2
in: 128, out: 32
ResNet
in: 32, out: 1
Output
V(s)):
Table 7: Simulation configuration hyperparameters.
Parameter
Value
Environment
Rigid solver iterations
10
Baumgarte ratio (3)
0.01
Gravity acceleration
9.81 m/s?
Control signal frequency
10 Hz
Max joint motor torque
6 N
Voxel size
0.01 m
Soft Material
Elastic modulus
3 x 10 Pa 800 kg/m2
Density
Poisson's ratio
0.35
Static friction
Dynamic friction
0.8
Rigid Material
Density
1500 kg/m
Static friction
Dynamic friction
0.8
Table 8: Reinforcement learning and evolution hyperparameters.
Parameter
Value
Rl. Hyperparameters
Actor learning rate
6 x 10-5
Critic learning
6 x 10-5
Discount (?)
0.9
Entropy weight
0.01
GAE
Not used 20
Learning epochs
PPO train batch size
64
PPO train steps per epoch
60
Psmall
0.1 [voxels]
Purge
10 [voxels]
Minimum movement threshold (8)
0.001
Population and Rollouts
Population size (Ptarge)
Clones per individual (rollout_num)
2
24

---

Published as a conference paper at ICLR 2025
Table 9: Symbols.
Symbol
Meaning
Simulation
m
Mass
I, V. 2
Position of each axis in space
Average strain of soft voxel Position in space Linear velocity
V
Angular velocity of rigid bone Quaternion representing the orientation
θ
Rotation in space
T
Moment or torque
f
Force
X
Generalized position
Generalized velocity
Generalized acceleration
Generalized velocity from the previous time step
M
Generalized mass matrix Stiffness matrix of beam Inertia
K
F
Generalized force vector Constraint function between two rigid bones Jacobian matrix
c(X)
J
Lagrange multiplier Matrix in the LCP
A
W
Term involving velocities and external forces in the LCP
At
Time step size simulation
B
Constraint error correction rate
Reinforcement Learning
Reward
State
Action
V
Value function
Policy function
Discount factor
Action logits
A
Action space
m
Node features
Edge features
Joint angle
h
Hinge axis direction of joint
d,
Vectors from joint to the center mass of connected bones History movement vector Movement vector between two time steps, or linear velocity
Utilities
Indicator function
E
Identity matrix
Subscript and Superscript Conventions
Beam (subscript)
ext
External (subscript)
Time step in RL (subscript)
Soft voxel (superscript)
Rigid bone (superscript)
Joint (subscrip/ssuperseripp)
1
Local pool (superscript)
25


Reviews:
[{"review_id": "jsKzBAw9t2", "reviewer": "Reviewer_Le5n", "summary": "This work proposes (1) a simulation framework, (2) a generative model, and (3) a co-optimization procedure for endoskeletal robots that consist of both rigid bones and soft tissue, in contrast to rigid-only or soft-only bodies in many previous works. (1) The endoskeletal robots are simulated as a collection of elastic and rigid voxels in a 3D space, with type-specific constraint forces between them. (2) The learning-based generative model is trained by autoencoding a large dataset of synthetic training data (valid voxelized body plans) into a latent space. After training, the decoder generates voxelized body plans from this latent space, which are post-processed into a graph structure defining a simulated robot body. (3) The co-optimization procedure uses two nested training loops, in which the outer \"evolutionary\" loop searches through the latent space with CMA-ES to design better morphology, and the inner \"lifetime\" loop learns a reused actor/critic architecture with PPO to control an arbitrary morphology on control tasks. The agent is rewarded for forward locomotion across a variety of terrains.", "questions": "**Q1: Actor/critic inputs.** What's the rationale for conditioning the actor and critic on different state?\n\n**Q2: Ablation of evolved designs.** On line 498, is that statement a mistake? It does not seem like Optimized RL is doing well with the initial designs. And I'm not sure how catastrophic forgetting is related, i.e. forgetting how to control a bad initial policy?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**S1: Novelty.** Unlike many previous works that focus exclusively on rigid or soft bodies, this work combines them into a single framework. The manuscript provides an extensive related works section for situating this work within the field, which was helpful.\n\n**S2: Clarity.** The manuscript was a pleasure to read. The writing is clear and concise, with appropriate details provided in the extensive appendices. The mathematical notation is simple and consistent. The figures are thoughtfully designed. Overall, great work!\n\n**S3: Quality.** The methods are appropriately applied, and the experimental results are technically sound and informative (especially with the ablations).\n\nOverall, I believe this is a solid work which I will recommend acceptance of, provided that my feedback below is adequately addressed.", "weaknesses": "**W1: Code.** I did not see a link to code, which is critical for reproducibility. Can the authors please provide this during the rebuttal?\n\n**W2: Videos.** It will be useful to see qualitative results (i.e. videos) in addition to the quantitative reward plots. Currently, the reward values aren't contextualized, so it's had to tell if the behavior is actually good, only that it's improving.\n\n**W3: Ablation of discrete action space.** It would be useful to show the comparison of discrete vs continuous action space (line 310), and provide some discussion. Is this a constraint from your simulation, or a learning issue? Wouldn't PPO also be able to control continuous action spaces, as it does in other related sim works?\n\n**W4: Minor figure/text edits.**\n- Figure 2 has nice aesthetics, but could be simplified. It has a lot of extraneous subpanel labels that aren't useful (Figure 5 too). Really the figure has 3 columns (initial, training, final), and the rows are instances of the same population. This message could be highlighted with in-figure annotations, like an arrow indicating the direction of training time.\n- Figure 1 looks pretty, but what is its purpose? It looks like a bunch of molecules floating on a dark ocean, which was confusing as it primed me to think about fluid simulations, (but the sim only works on terrestrial envs, line 533). Also, the entire caption is one really loong sentence.\n- Typos: line 376 sampled, line 398 the the, Fig 7 subpanel A label, Equation 10 star misplaced", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FavP1kFprA", "reviewer": "Reviewer_dYez", "summary": "In this paper, the authors introduced a new brain-body design of freeform endoskeletal robots. The authors introduce a computational framework that integrates the generation of external and internal structures through a combination of 3D modeling, latent space encoding, reinforcement learning, and evolutionary strategies.", "questions": "1. Could the authors provide more insights into the latent space encoding? Specifically, how do specific regions of the latent space correspond to different types of robot designs?\n\n2. Can the proposed method find endoskeletal robot morphologies which can generalize to several tasks?\n\n3. How does the proposed method compare to other state-of-the-art approaches (Such as transform2act, ea+rl bi-level optimization) in terms of design diversity, performance, and computational efficiency?\n\n4. What are the potential real-world applications of the robots designed by your methods? How feasible is the transition from simulated designs to real-world robotic systems? What are the main challenges in this transition?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors integrated 3D modeling, latent space encoding, reinforcement learning, and evolutionary strategies into a cohesive framework for robot design.\n\n2. Optimizing in the latent space significantly reduces the computational burden of the co-design process.\n\n3. The use of a universal controller and simultaneous optimization of morphology and control through reinforcement learning demonstrates the robustness and adaptability of the approach.\n\n4. The research topic of this paper is interesting and significant.", "weaknesses": "1. While the paper focus on the co-design problem of endoskeletal robots, which is innovative, the main contribution of this paper seems to be not clear. Besides, the analysis of scalability and the potential real-world applications are needed.\n\n2. It would be better to have a picture to describe the whole co-design pipeline of the method. \n\n3. The whole co-design pipeline (Learning the latent space description-> Training a universal controller via RL-> Optimizing morphology through EA) is not new. The most important innovation of the article seems to turn out to be the modeling of endoskeletal robots.\n\n4. The authors utilize multiple advanced techniques, which may lead to high computational costs and complexity, might limit practical applications.\n\n5. The latent space encoding, while effective, may lack interpretability, making it difficult to understand the underlying principles that guide the design evolution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I1HwaPeRxc", "reviewer": "Reviewer_uyFn", "summary": "The authors designed and implemented a software library capable of generating three-dimensional freeform endoskeletal robots. Across various tasks, the researchers demonstrate the simulations’ ability to provide a means of benchmarking the evolutionary design and representational learning of complex embodies systems.", "questions": "What insight may explain the evolution of the agents in these specific directions? Can we find similarities of this biologically?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "Provides an easy-to-use computational design of freeform endoskeletal robots and the library software for benchmarking the representations and evolution of these agents over time.", "weaknesses": "The paper's conclusion that \"design evolution finds better designs but they do not imply that evolution (i.e. cumulative selection) is necessary\" appears to contradict the study's methodology, which used reward values that created selection pressure across generations. Please clarify how this conclusion aligns with the reward-based selection used in the experiments, and if possible, provide evidence that random mutations alone could achieve similar results. We recommend rephrasing to better reflect the relationship between mutations and selection pressure while maintaining the valuable insights about design diversity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mYcIHwbVmh", "reviewer": "Reviewer_wJ4t", "summary": "This paper bridges the two opposing types of robots namely fully rigid and fully soft robots in current literature by introducing a novel multi-physics simulator of 3D endoskeletal robots that feature rigid, jointed bodies covered by soft materials. The authors employ a variational autoencoder trained on synthetic body plans to yield smooth and expressive latent representation of morphology. With the aid of a Graph Transformer-based modular control policy trained alongside morphological evolution, the authors showcase the feasibility of their simulation environment and latent representation to support the emergence of complex, lifelike morphologies capable of navigating various terrains. This work lays an important foundation for developing more sophisticated, biomimetic and free-form artificial organisms.", "questions": "1.Some previous studies have demonstrated that Graph Neural Networks tend to have difficulty with long-distance communication, as messages would be diluted in multiple hops [1]. Could you explain why you have chosen a Graph Transformer, instead of a conventional Transformer with full connection between modules (as in [1,2])? \n\n[1] Kurin V, Igl M, Rocktäschel T, et al. My body is a cage: the role of morphology in graph-based incompatible control[C]//9th International Conference on Learning Representations. 2021.\n\n[2] Gupta A, Fan L, Ganguli S, et al. Metamorph: learning universal controllers with transformers[C]//International Conference on Learning Representations. ICLR, 2022.\n\n2.Is your simulation environment currently restricted to locomotion on various terrains? Does it or will it support manipulation tasks, such as carrying or pushing an object? \n\n3.Could you explain why the population is cloned during policy training? \n\nThank you!", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper introduces an open-source simulation environment of 3D endoskeletal robots with four predefined terrains. The voxel-based representation is conceptually simple and meanwhile offers great convenience for future work to benchmark their brain-body co-design algorithms. It also provides an open-endedness to design more complicated terrain maps for specific needs. \n\n2. I appreciate that the authors meticulously designed a suite of control experiments to demonstrate the indispensability of morphological evolution. This is generally absent in existing brain-body co-design studies, which take morphology and control as a whole to evaluate task performance.", "weaknesses": "The introduction seems a bit too verbose. The authors are suggested to only keep the most essential information regarding current state of the art and their contributions, and relegate the remaining to Related Work or Appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)