DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models
Authors: Tsun-Hsuan Wang
Juntian Zheng
Pingchuan Ma
Yilun Du
Byungchul Kim
Andrew Everett Spielberg
Joshua B. Tenenbaum
Chuang Gan
Daniela Rus
Venue: NeurIPS 2023 oral (2023)
Keywords: soft robot
diffusion model
co-design
TL;DR: Use diffusion models augmented by physics-based simulation to breed soft robots

Abstract: Nature evolves creatures with a high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy.  Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. \name bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and (ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation.  We showcase a range of simulated and fabricated robots along with their capabilities. Check our website: https://diffusebot.github.io/

DoffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models
Tsun-Hsuan Wang Juntian Zheng?:3 Pingchuan Ma', Yilun Du', Byungchul Kim Andrew Spielberg "4 Joshua B. Tenenbaum', Chuang Gan 1,3,5.t Daniela Rus!
MITT TTsinghua University, "MIT-IBM Watson AI Lab, "Harvard, UMass Amherst
attps://diffuseooo.itttee github.
Abstract
Nature evolves creatures with high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy. Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present Diffuse- Bot, a
diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with physical dynamical simulation which provides a certificate performance, and ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. We showcase range of simulated and fabricated robots along with their capabilities.
Introduction
Designing dynamical virtual creatures or real-world cyberphysical systems requires reasoning about complex trade-offs in system geometry, components, and behavior. But, what if designing such sys- tems could be made simpler, or even automated wholesale from high-level functional specifications? Freed to focus higher-level tasks, engineers could explore, prototype, and iterate more quickly, focusing more on understanding the problem, and find novel, more performant designs. We present Diffuseloot, first step toward efficient automatic robotic and virtual creature content creation, as an attempt at closing the stubborn gap between the wide diversity and capability of Nature vis-a-vis evolution, and the reiterative quality of modern soft robotics.
Specifically, we leverage co-designing soft robot morphology and control for target tasks. Compared with with previous ap- proaches, DiffuseBot 's learning- bearning-based approach maintains evolutionary algorithms' ability to search over diverse forms while exploiting the efficient nature of gradient-base based optimization. DiffuseBot is made possible by the revolutionary progress of AI-driven content generation, which is now able to synthesize convineing media such images, audio, and animations, conditioned on human input. However, other than raw statistical modeling, these methods are typically task- and physics-oblivious, and tend to provide fundamental reasoning about the performance of generated outputs. We provide the first method for bridging the gap between diffusion processes and the morphological design of cyberphysical systems, guided by physical simulation, enabling the computational creative design of virtual and physical creatures.
crttusion-based algorithms as a means efficiently and generatively
*This work was supported by the NSF EFRI Program (Grant No. 1830901), DSO grant DSOC02107, DARPA Fellowship Grant HR001 121 10007, and MIT-IBM Watson Al Lab. indicates equal advising.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Generative Diffusion Models
Physscc-based Simulation
Physical
Shapes
Augmerted Photical Guanelty Sulliaess
Figure 1: DiffuseBot aims to augment diffusion models with physical utility and designs for high-level functional specifications including rohot geometry, material stiffness, and actuator placement.
While diffusion methods can robustly sample objects with coherent spatial structure from raw noise in step-by-step fashion, several roadblocks preventing existing generative algorithms from being directly applied to physical soft robot co-design. First, while existing diffusion methods can generate 2D or 3D shapes, useful for, say, sampling robot geometry, they do not consider physics, nor are they directly aligned with the robotic task performance. As alternative approach, one might consider learning a diffusion model supervised directly on a dataset of designs mapped to their task performance. This leads us to the second roadblock, that is, that no such dataset exists, and, more crucially, that curating such a dataset would require a prohibitive amount of human effort and would fail transfer novel tasks outside that dataset.
robot
To tackle these challenges, we propose using physical simulation to guide the generative process of pretrained large-scale 3D diffusion models. Diffusion models pretrained for 3D shapes provide an expressive base distribution that can effectively propose reasonable candidate geometries for soft robots. Next, we develop an automatic procedure to convert raw 3D geometry compatible with soft body simulation, i.e. one that parameterizes actuator placement and specifies material stiffness. Finally, in order to sample robots in physics-aware and performance-die driven manner, we apply two methods that leverage physically based simulation. First, we optimize the embeddings that condition the diffusion model, skewing the sampling distribution toward better- performing robots as evaluated by our simulator. Second, we reformulate the sampling process that incorporates co-optimization over structure and control. We showcase the proposed approach of DiffuseBot by demonstrating automatically synthesized, novel robot designs for a wide spectrum of tasks, including balancing, landing, crawling, hurdling, gripping, and moving objects, and demonstrate its superiority to comparable approaches. We further demonstrate DiffuseBot 's amenability to incorporating human semantic input as part of the robot generation process. Finally, we demonstrate the physical realizability of the robots generated by DiffuseBot with a proof-of offf-cc--cccettt-ce 3D printed real-world robot, introducing the possibility of Al-powered end-to-end CAD-CAM pipelines.
representation
In summary, we contribute:
A new framework that augments the ieffusion-bused synthesis with physical dynamical simulation in order to generatively co-design lask-driven soft robots in morphology and control.
Methods for driving robot generation in a task-driven way toward improved physical utility by optimizing input embeddings and incorporating differentiable physics into the diffusion process.
Extensive experiments in simulation to verify the effectiveness of DiffuseBot, extensions to text- conditioned functional robot design, and a proof-of-concept physical robot as a real-world result.
2 Method
In this section, we first formulate the problem (Section 1) and then describe the proposed Diffuse- Bot framework, which consists of diffusion- -baeed 3D shape generation (Section 2.2), a differentiable procedure that converts samples from the diffusion models into soft robots (Section 2.3), technique to optimize embeddings conditioned by the diffusion model for improved physical utility (Section 2.4), and reformulation of diffusion process into co-design optimization (Section 2.4).
2.1 Problem Formulation: Soft Robot Co-design
Soft robot co-design refers to joint optimization of the morphology and control of soft robots. The morphology commonly involves robot geometry, body stiffness, and actuator placement. Control the signal to the actuators prescribed by a given robot morphology. It can be formally defined as,

---

Physics-Augmentee Augmented
[Triiinn]] Embedding
Robotizing
Physics-based Simulation For Soft Robot
3D Diffusion Model
Optimization
Embeddings
Online
Dataset
Noise X
Embeddding optimization
3D Shape
Diffusion Samples
Co design
Diffusion
I Robot Designs Solid Geometry
Sempline Diffusion
sample X
Co-design
Body Stiffness
Actuators
Task
Gradient
Intermediate
3D Xg
Sample
Figure 2: The DiffuseBot framework consists of three modules: (i) robotizing, which converts diffusion samples into physically simulatable soft robot designs (ii) embedding optimization, which iteratively generate new robots to be evaluated for training the conditional embedding (iii) diffusion as co-dexign, which guides the sampling process with co-design gradients from differentiable simulation. Arrow (A): evaluation of robots to guide data distribution, (B): differentiable physics feedback.
min C(V, () min L ((( Φ V) Sh Sn)LL,, where 8h+1 = [(n, us) 2.
2.
where V is robot morphology that includes geometry vi geo- stiffiness Mat and actuator placement vi i u, is actuation with the controller's 's parameters & and dependency on the robot morphology V, Sy is the simulator state, is the environmental dynamics (namely the continuum mechanics of soft robots), and H is robot time horizon (not to confused with the later-on mentioned diffusion time). Co-design poses challenges in optimization including complex interdependencies between body and control variables, ambiguity between competing morphology modifications, trade-offs between flexibility and efficacy in design representations, etc. [61]. In this work, we aim at leveraging the generative power diffusion models in searching for optimal robot designs with (1) the potential to synthesize highly- -divere robots and (2) inherent structural biases in the pre-trained 3D generative models learned from large-scale 3D datasets to achieve efficient optimization.
2.2 3D Shape Generation with Diffusion-based Models
Diffusion-based generative models [24. 52] aim to model a data distribution by augment-
ing it with auxiliary variables {x+] defining Gaussian diffusion process p(xo)
I ((x)) P(- 1 |xx)dxxer with the transition kernel in the forward process g(xx|x+-1 =
N(x; VT -ix-1111111111 3,I)) for some < 8g < 1. For sufficiently large T, we have p(Xr) NN (0,, This formulation enables an analytical marginal at any diffusion time x, = /ā,X0 + VI āge based on clean data Xg. where NO and ā, = I 1 3,.. The goal of the diffusion model (or more precisely the denoiser Eg) is to learn the reverse diffusion process 1 with the loss,
min EE
(c (ccc)
eg <x(xx(xo e, 11
(2)
Intuitively, co learns one-step denoising process that can be used iteratively during sampling to convert random noise p( p(xr) gradually into realistic data p(x.). To achieve controllable generation with conditioning c, the denoising process can be slightly altered via classifier-free guidance [25, 12),
ég frec : co X,, t, Ø) (ca, t, c) cg(x,, t. 2))
where 8 is the guidance scale, Ø is null vector that represents non-conditioning.
2.3 Robotizing Shapes from Diffusion Samples
We adopt Point-E [39] pre-trained diffusion model that is capable of generating diverse and complex 3D shapes, providing a good prior of soft robot geometries. However, the samples from the diffusion model are in the form of surface point cloud and are not readily usable as robots to be evaluated in the physics-based simulation. Here, we describe how to robotize the diffusion samples
X₁ + i and its gradient computation of the objective dc ac 8W
DC 04, + AC 0%
8 gro 8x:
2 8x.
3

---

Algorithm/ Training:
Embedding Optimization
Initialize: D < c 2
while within maximal number of epochs do
Generate data with the diffusion model: Xo po (xolc)).
Evaluate samples with physics- baysics-based simulation: 1(xo) L (V)())).
Aggregate and update datasets: ' + Filter(D (D U {xg. 7
Optimize the embedding e on D using the objective (5).
end while
Solid Geometry. We use a Material Point Method (MPM)-based simulation [61], which takes solid geometries as inputs. This poses two obstacles: (1) conversion from surface point clouds into solid geometries, and (2) the unstructuredness of data in the intermediate samples xx, t o 0. The second issue arises from the fact that the diffusion process at intermediate steps may produce 3D points that do not form a tight surface. First, we leverage the predicted clean sample at each diffusion time 6,
X v āi co (xx,,))
(4)
Vā
This approach is used denoising diffusion implicit model (DDIM) [53] to approximate the unknown clean sample X4 in the reverse process p(x |x, X, X,). Here, we to construct a better-structuree data for simulation. Hence, we break down the robotizing process into X, + Xg + V with gradient Tx. can be trivially derived from (4). To convert the predicted surface points &y into solid geometry, we first reconstruct a surface mesh from X and then evenly sample modify the optimization approach from Shape As Points [40], which provides a differentiable Poisson surface reconstruction that maps control point set V a modified Chamfer Distance loss indicating similarity between V mesh and &n:
09, 0%o. where 0Ka
components as 75g Tx,"
solid point cloud geo within its interior. For mesh reconstruction,
reconstructed surface mesh with vertices V mesh. We calculate
Ciccon
Amesh
Σ
dov, +
Aiagret
Σ
w(v)d(v, V meshh),
mmesh
|&o|
mesh
VERD
in which d(-.-) denotes minimal Euclidean distance between a point and point set, and w(v) denotes a soft interior mask, with w(v) for v outside the mesh, and w(v) 0.1 for inside. The introduced mask term w(v) aims to lower the influence of noisy points inside the mesh which caused by imperfect prediction of Xn from noisy intermediate diffusion samples. The weight parameters are set Amesh = 1 and Auaget 10. We back-propagate &hase to & through the VVunb 3Vant
differentiable Poisson solver, and then apply Adam optimizer on Van to optimize the loss Lrccon- After mesh reconstruction, the solid geometry Vi geo represented by solid interior point cloud is then sampled evenly within the mesh, with sufficient density support the MPM-based simulation. Finally, to integrate the solidification process into diffusion samplers, we still need its gradient 2% We adopt Gaussian kernels on point- wise Euclidean distances as gradients between two point clouds:
&
Ou dv EveR, exp(-ax v'
exp(-allu v|l²)
V gooo V E Xg-
Intuitively, under such Gaussian kernels gradients, each solid point is linearly controlled by predicted surface points near it. In practice, this backward scheme works well for kernel parameter
20. Actuators and Stiffness. solid geometry does not make a robot; in order for the robot to behave, its dynamics must be defined. After sampling solid geometry, we thus need to define material properties and actuator placement. Specifically, we embed actuators in the robot body in the form of muscle fibers that can contract expand to create deformation; further, we define a stiffness parameterization in order to determine the relationship between deformation and restorative elastic force. We adopt constant stiffness for simplicity since it has been shown to trade off with actuation strength [61]; thus we have V (X0)) const and 89. 0. Then, we propose to construct actuators U%e
based on the robot solid geometry V Vanl ( geo (ko)) via clustering; namely, we perform k-means with pre-defined number of clusters on the coordinates of 3D points from the solid geometry Vp L The gradient then becomes 0mmm 00 0%g 2 0&. 0x."
m 0 as empirically we found the clustering is quite stable in terms of label assignment, i.e., with A V yen being small, A AV ast - 0. Overall, we keep only the gradient for the robot geometry empirically it suffices.
where 2. g0

---

Algorithm 2 Sampling: Diffusion As Co-design
Initialize: initial sample Xr ~ I.)
while within maximal number of diffusion steps 2 0 do
Perform regular per-step diffusion update: x; < X(+1- if perform co-design then
while within K steps do
Run update in (6) and (7) to overwrite x, end while
end if end while
2.4 Physics Augmented Diffusion Model
Embedding Optimization. To best leverage the diversity of the generation from large-scale pre-trained diffusion models, we propose to (1) actively generate new data from model and maintain them in buffer, (2) use physics-based simulation the embeddings conditioned by the diffusion model under a skewed data distribution to improve robotic performance in simulation. Curating training dataset on its own alleviates the burden of manual effort to propose performant robot designs. Optimizing the conditional embeddings instead of finetuning the diffusion model eliminates the risk of deteriorating the overall generation and saves the cost of storing model weights for each new task (especially with large models), We follow,
certificate of performance, (3) optimize
min (tt.-//1...ppoxxlleeeeee..oo |||e (x (Xo, e, (), c))1
Note the three major distinctions from (2): (i) the optimization variable is the embedding c not 0 (ii) the denoiser conditioned on the embeddings eg (.....0 c) and (iii) the data distribution is based on the diffusion model pe not the inaccessible real data distribution p and is conditioned on the embeddings c. This adopts an online learning scheme as the sampling distribution is dependent on the changing c. The procedure is briefly summarized in Algorithm 1, where Filter is an operation to drop the oldest data when exceeding the buffer limit. In addition, during this stage, we use fixed prescribed controllers since found empirically that randomly initialized controller may not be sufficiently informative to drive the convergence toward reasonably good solutions; also, enabling the controller to be trainable makes the optimization prohibitively slow and extremely unstable, potentially due to the difficulty of the controller required to be universal to a diverse set of robot designs. After the embedding optimization, we perform conditional generation that synthesizes samples corresponding to robot designs with improved physical utility via classifier-free guidance as in (3).
Diffusion as Co-design. While the optimized embedding already allows us to generate performant robots for some target tasks, we further improve the performance of individual samples by reformulat- ing the diffusion sampling process into a co-design optimization. As described in Section 2.3, we can convert the intermediate sample at any diffusion time x₁ to a robot design V rendering an evolving robot design throughout the diffusion process. However, regular diffusion update [24] much less resembles any gradient- based optimization techniques, which are shown to be effective in soft robot design and control with differentiable simulation [26, 2]. Fortunately, there is synergy between diffusion models and energy-based models [54, 15, 14], which allows a more mendient-descen--iee like update with Markov Chain Monte Carlo (MCMC) sampling [14]. Incorporating the soft robot co-design optimization with differentiable physics [61] into the diffusion sampling process, we have
Design Optim::
x(4) x
(k-1) 0²
(co(x( (x
1) .t) RV (a-1 (( (x1x(((-)))) ofe (6)
o(-1)
Control Optim.: 6(4 di ik (1) + L(V (k-))))) 1
(K) where 6 ~ (O.,I), x = x i- k: is the ratio between two types of design gradients, K is the number of MCMC sampling steps at the current diffusion time, 7 the weight for trading off design and control optimization, and of (0) can be either inherited from the previous diffusion time (k) or
reset to the initialization ( We highlight the high resemblance to gradient-based co-optimization with x, as the design variable and di as the control variable. This procedure is performed once every M diffusion steps (Algorithm 2), where M is a hyperparameter that trade-offs "guidance" strength from physical utility and sampling efficiency. Intuitively, the entire diffusion-as as-co-design process is
5

---

Balancing
Landing
Crawling
Hurdling
Gripping
Moving Box
Figure 3: We consider passive dynamics tasks (halancing, landing), locomotion tasks (crawling, hurdling), and manipulation tasks (gripping, moving box).
Table I: Improved physical utility by augmenting physical simulation with diffusion models.
Embed. Optim.
Diffusion as Co-design
Passive Dynamics Balancing
Locomotion Crawling
Manipulation
Landing
Hurdling
Gripping
Moving a Box
0.081 .164
0.832 217
0.011 012
0.014 020
0.014:008
0,019 020
0.556
0.955 032
0.048 007
0.019 014
0.025 006
0.040
0.653 107
0.964 029
0.081 01s
0.035 030
0.027 004
0.044 021
guided by three types of gradients: (i) cx (k-1) :)) provides a direction for the design toward feasible
3D shapes based on the knowledge pre-training with large-scale datasets (and toward enhanced
physical utility with the optimized embeddings via classifier-fre guidance using ég (x, (#=1)) " c)), (ii) Vx, C( (x)):- provides a direction for the design toward improving co-design objective via differentiable simulation, and (iii) Va, L(,, Φe) provides direction for the controller toward better adaption the current design x, that allows more accurate evaluation of the robot performance.
3 Experiments
3.1 Task Setup
We cover three types of robotics tasks: passive dynamics, locomotion, and manipulation (Figure 3)-
Passive Dynamics tasks include balancing and landing. Balancing initializes the robot on stick- like platform with small contact area with upward velocity that introduces instability; the robot's goal to passively balance itself after dropping on the platform. Landing applies an initial force to the robot toward a target; the robot's goal is to passively land as close to the target as possible.
Locomotion tasks include crawling and hurdling. Crawling
the robot at rest state the ground; the robot must actuate its body to move as far away as possible from the starting position. Hurdling places obstacle in front of the robot; the robot must jump the obstacle.
Manipulation tasks include gripping and moving objects. Gripping places an object underneath the robot; the goal of the robot is to vertically lift the object. Box Moving places a box on the right end of the robot; the robot must move the box to the left.
Please refer the appendix Section D for detailed task descriptions and performance metrics.
3.2 Toward Physical Utility In Diffusion Models
Plysics-augmented diffusion. Table 1, examine the effectiveness of embedding optimization and diffusion as co-design for improving physical utility. For each entry, we draw 100 samples with preset random seeds provide valid sample-level comparison (i.e., setting the step size of co-design optimization to zero in the third row will produce almost identical samples as the second row). We report the average performance with standard deviation the superscript. First, we observe increasing performance across all tasks while incorporating the two proposed techniques, demonstrating the efficacy DiffuseBot. Besides, the sample-level performance does not always monotonically improve, possibly due to the stochasticity within the diffusion process and the low quality gradient from differentiable simulation in some scenarios. For example, in gripping, when the robot fails to pick up the object in the first place, the gradient may be informative and fails to bring proper guidance toward better task performance; similarly in moving a box. addition, we found necessary to include control optimization during the diffusion sampling process, since, at diffusion steps further from zero, the predicted clean sample & (derived from the intermediate sample x,) may differ significantly from the clean sample Xp. leaving the prescribed controller largely unaligned.
Comparison with baselines. In Table 2, we compare with extensive baselines soft robot design representation: particle-based method has each particle possessing its own distinct parameterization of design (geometry, stiffness, actuator); similarly, voxel-based method specifies design in voxel level: implicit function [37] uses shared multi-layer perceptron map coordinates to design;

---

Table Comparison with baselines.
Passive Dynamics
Locomotion
Manipulation
Methads
Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
Particle-hased
0.040
0.863 000
0.019 001
0.006 001
-0.010 001
0.043 027
Voxel-based
0.040:0u0 9uo
0.853 002
0.024-900
0.027-000
-0.009
0.025 022
Implicit Function [37]
0.106-144 147
0,893-024
0,043-024
0.044 003
0.006 012
0.033 080
Diff-CPPN [18]
0.091 086
0.577 425
0.055 023
0.019
0.007 008
0,022
DiffAqua [35]
0.014 023
0.293 459
0.027.015
0.022-011
0.010 001
0.007 008
DiffuseBot
0.706.475
0.965 026
0.092-014
0.031 -011
0.026 002
0.047 U19
Crawling
Gripping
1st Checkpoint
2nd Checkpoint
3rd Checkpoint
4th Checkpoint
Sth Checkpoint
Figure 4: Examples of DiffuseBot evolving robots to solve different tasks.
DiffCPPN [18] uses graphical model composed set of activation function that takes in coor- dinates and outputs design specification; DiffAqua [35] computes the Wasserstein barycenter of of aquatic creatures' meshes and we adapt use more reasonable primitives that include bunny, car, cat, cow, avocado, dog, horse, and sofa. These baselines are commonly used in gradient-basee soft robot co-design [26, 56, 61]. For each baseline method, we run the co-optimization routine for the same number of steps as in the cettuuiioona-co-dess/ii stage in DiffuseBot. To avoid being trapped the local optimum, we run each baseline with 20 different random initializations and choose the best one. Since DiffuseBot is generative method, we draw 20 samples and report the best; this sensible an applications-drive robot, within reasonable sample budget. We observe that method outperforms all baselines. DiffuseBot leverages the knowledge of large-scale pre-trained models that capture the "common sense" of geometry, providing
perspective since we only need to retrieve one performant
well-structured yet flexible prior for soft robot design.
Soft robots bred by DiffuseBot. In Figure 5, we demonstrate the generated soft robots that excel in locomotion and manipulation tasks. We highlight the flexibility of DiffuseBot to generate highly diverse soft robot designs that accommodate various purposes in different robotics tasks. Furthermore, in Figure 4, we show how robots evolve from feasible yet non-necessarily functional design to an improved one that intuitively matches the task objective. By manually inspecting the evolving designs, we found that the role of the embedding optimization is to drive the diverse generations toward a converged, smaller set with elements having higher chance to succeed the task; on the other hand, the role of diffusion co-design brings relatively minor tweaks along with alignment between the control and design. Due to space limit, we refer the reader to our project page for more results.
Crawling
Gripping
100th
Figure 5: Examples of robots bred by DiffuseBot achieve the desired tasks.

---

Performance
0.055
MT
0.016 014
0.080
FT Ours
0.031 -024
Performance
0.065
performance
0.075
0.048-007
0.85
0.00
Table 3: Ablation em- hedding optimization. MT means manually-designee text. FT means finetuning models.
0.855
50 100
150
200
250
200 250
600
Number Of Samples Per Diffusion Step Figure 6: Varying starting point and strength of diffusion as co-design.
Max Diffusion Time
3.3 Ablation Analysis
In this section, we conduct a series of ablation studies to provide deeper understanding of the proposed method. For simplicity, all experiments in this section are done with the crawling task.
Embedding optimization. In Table 3, we compare the optimization of the embedding conditioned by the diffusion models with other alternatives. The pre-trained diffusion model [39] that DiffuseBot is built upon uses CLIP embeddings [43], which allows for textual inputs. Hence, a naive approach is manually design text for the conditional embedding of the diffusion model. The result reported in Table 3 uses "a legged animal or object that can crawl or run fast". We investigated the use of text prompts; in our experience, text was difficult optimize for functional robot design purposes. This is expected since most existing diffusion models perform content generation only in terms of appearance instead of physical utility, which further strengthens the purpose of this work. addition, with exactly the same training objective as in (5), we can instead finetune the diffusion model itself. However, this does not yield better performance, shown in the second entry in Table 3. Empirically, we found there is a higher chance of the generated samples being parts. This suggests that finetuning for physical utility may deteriorate the modeling of sensible 3D shapes and lead to more unstable generations.
with fractured
Diffusion co-design. Recall that the co-design optimization be seamlessly incorporated into any diffusion step. In Figure 6, we examine how the strength of the injected co-design optimization affects the task performance in terms of where to apply throughout the diffusion sampling process and how many times to apply. In the left figure of Figure 6, we sweep through the maximal diffusion time of applying diffusion co-design, i.e., for the data point t 400, we only perform co-design from 400 to t = O. We found that there is sweet spot of when to start applying co-design (at = 200). This is because the intermediate samples larger diffusion time x,, > are extremely under-developed, lacking sufficient connection to the final clean sample Xo, hence failing provide informative guidance by examining its physical utility. Furthermore, we compare against post-diffusion co-design optimization, i.e., run co-design based on the final output of the diffusion (0.064 vs ours 0.081). We allow the same computational budget by running the same number of times of differentiable simulation in DiffuseBot. Our method performs slightly better, potentially due to the flexibility to alter the still-developing diffusion samples. Also note that while our method interleaved into diffusion process, it is still compatible with any post-hoc computation for finetuning.
3.4 Flexibility To Incorporate Human Feedback
Beyond the generative power, diffusion models also provide the flexibility to composite different data distributions. This is especially useful for computational design since empowers to eas- ily incorporate external knowledge, e.g. from human. We follow the compositionality tech- niques introduced in [34, 14], which can be di- rectly integrated into our diffusion as co-design framework. In Figure we demonstrate incorporating human feedback in textual form as "a unicorn" into a crawling robot generated by DiffuseBot. We can see the emergence of the horn-like body part.
DiffuseBot
Feedlback Unicom"
Figure 7: Incorporating human textual feedhack.
3.5 From Virtual Generation To Physical Robot
We further fabricate physical robot for the gripping task as a proof-of-concee to demonstrate the possibility of real-world extension. We use 3D Carbon printer to reconstruct the exact geometry of a generated design and fill the robot body with Voronoi lattice structure to achieve softness.
8

---

For actuators, we employ tendon transmission to realize the contraction force utilized in the soft robot grippe. In our project page, we demonstrate the robots generated by DiffuseBot are capable of picking up an object. Note that physical robot fabrication and real-world transfer have countless non-trivial challenges including stiffness and actuator design, sim-to-real gap, etc. Hence, this experiment is only meant to demonstrate the potential instead of general, robust pipeline toward physical robots, which is left to future work. We refer the reader to the appendix for more details.
4 Related Work
Heuristic Search For Soft Robot Co-Design. Heuristic searches are simple but useful tools for co- designing soft robots. A long line of work has focused on evolutionary algorithms [7, 8, 10], with some including physical demonstrations [22, 31, 32] and recent benchmarks incorporating neural control [5]. These methods are often powered by parameterized by compositional pattern-produce
networks
[58], which parameterize highly expressive search spaces akin to neural networks [50, 511. Similar to [5], [47] combines a heuristic approach with reinforcement learning, and demonstrates resulting designs on physical hardware. Other notable methods include partile-filter-basee approaches [1 and simulated annealing [60]. Heuristic search methods tend to be less efficient than gradient-base or learning-baac -basen algorithms, but can reasoning about large search spaces; our approach employs the highly expressive diffusion processes, while leveraging the differentiable nature of neural networks and physical simulation for more efficient and gradient-directen search.
Gradient-Based Soft Robot Co-Optimization. A differentiable simulator is one in which useful analytical derivatives of any system variable with respect any other system variable is efficiently
queryable; the recent advent of soft differentiable simulation environments [26, 56, 13, 33, 41, 42, 61] has accelerated the exploration of gradient-based co-optimiation methods. [26, 56] demonstrated how differentiable simulators can be used to co-optimize very high-dimensional spatially varying material and open- ppen-loop/neural controller parameters. [38] presented gradient- -ased search of shape parameters for soft manipulators. Meanwhile, [61] showed how actuation and geometry could be co-optimized, while analyzing the trade-offs of design space complexity and exploration in the search procedure. DiffuseBot borrows ideas from gradient-based optimization in guiding the design search physics-aware way, especially in the context of control.
Learning-Based Soft Robot Co-Design Methods. Though relatively nascent, learning- based ap- proaches (including DiffuseBot) can re-use design samples build knowledge about problem. Further, dataset-based minibatch optimization algorithms are more robust to local minima than single-iterate pure optimization approaches. [57] demonstrated how gradient-based search could be combined with learned-models; a soft robot proprioceptive model was continually updated by simula- tion data from interleaved control/material co-optimization. Other work employed learning-baa based methods in the context of leveraging available datasets. [35] learned parameterized representation of geometry and actuators from basis shape geometries tractable interpolation over high-dimensional search spaces. [55] leveraged motion data and sparsifying neurons to simultaneously learn sensor placement and neural soft robotic tasks such as proprioception and grasp classification.
Diffusion Models for Content Generation. Diffusson models [24, 52] have emerged as the de-facto standard for generating content in continuous domains such as images [44, 46], 3D content [66, 65], controls [28, 9, IJ. videos [23, 49, 16], and materials [63, 62, 48). In this paper, we explore how diffusion models in combination with differentiable physics may be used to design new robots. Most similar to our work. [64] uses differentiable physics to help guide human motion synthesis. However, while [64] uses differentiable physics to refine motions in the last few timesteps of diffusion sampling. we tightly integrate diffcrentiable physics wih sampling throughout the diffusion sampling procedure through MCMC We further uses differentiable simulation to define a reward objective through which we may optimize generative embeddings that represent our desirable robot structure.
5 Conclusion
We presented DiffuseBot, framework that augments physics- based simulation with a diffusion process capable of generating performant soft robots for diverse set of tasks including passive dynamics, locomotion, and manipulation. We demonstrated the efficacy of diffusion- generation with extensive experiments, presented a method for incorporating human feedback, and prototyped a physical robot counterpart. DiffuseBot is first step toward generative invention of soft machines, with the potential to accelerate design cycles, discover novel devices, and provide building blocks for downstream applications in automated computational creativity and computer-assister design.

---

References
[1] Anurag Ajay, Yilun Du Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022.
[2] Moritz Bächer, Espen Knoop, and Christian Schumacher. Design and control of soft robots using differentiable simulation. Current Robotics Reports, 2(2):2112211, 2021.
[3] Nicholas W Bartlett, Michael T Tolley, Johannes TB Overvelde, James C Weaver, Bobak Mosadegh, Katia Bertoldi, George M Whitesides, and Robert Wood. A 3d-printed, functionally graded soft robot powered by combustion. Science, 349(6244):161-166) 2015.
[4] James M Bern, Kai-Hung Chang, and Stelian Coros. Interactive design of animated plushies. ACM Transactions on Graphics (TOG), 36(4):1-11, 2017.
[5] Jagdeep Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, and Wojciech Matusik. Evolution gym: large-scale benchmark for evolving soft robots. Advances Neural Information Processing Systems, 34:2201-2224-11 2021.
[6] Antnooy' Chen, Stelian Coros. Jennifer Mankoff, and Scott Hudson. Encore: 3d printed augmentation of everyday objects with printed-over, affixed and interlocked attachments. In Proceedings of the 28th Annual ACM Symposium on User Interface Software Technology, pages 73-82, 2015.
[7] Nicholas Cheney, Jeff Clune, and Hod Lipson. Evolved clettrophysiological soft robots. In Artificial life conference proceedings, pages 222-229. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-inte .... 2014.
[8] Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. Unshackling evolution: evolving soft robots with multiple materials and powerful generative encoding. ACM SIGEVOlution, 7(1):11-22) 2014.
[9] Cheng Chi, Siyuan Feng. Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. arXiv preprint 23Xi:23333.0417,, 2023.
[10] Francesco Corucci, Nick Cheney, Francesco Giorgio- Serchi,, Josh Bongard, and Cecilia Laschi. Evolving soft locomotion in aquatic and terrestrial environments: effeets of material properties and environmental transitions. Soft robotics, 5(4):475 495, 2018
[11] Raphael Deimel, Patrick Irmisch, Vincent Wall, and Oliver Brock. Automated co-design of soft hand morphology and control strategy for grasping. In 2017 IEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pages 1213-1218. IEEE, 2017.
[12] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs image synthesis. Advances in Neural Information Processing Systems, 2021.
[13] Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. Diffpd: Differentiable projective dynamics. ACM Transactions on Gruphics (TOG), 41(2):1-21, 2021.
[14] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fer- gus. Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. arXiv preprint 1rXiv:2302.11552 2023.
[15] Yilun Du and Igor Mordatch. Implicit generation and generalization in energy- energy-based models. arXiv preprint arXiv: 08889, 2019.
[16] Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. arXiv e-prints, pages arXiv-2302, 2023.
10

---

[17] Bin Fang, Fuchun Sun, Linyuan Wu, Fukang Liu, Xiangxiang Wang, Haiming Huang. Wenbing Huang, Huaping Liu, and Li Wen. Multimode grasping soft gripper achieved by layer jamming structure and tendon-driven mechanism. Soft Robotics, (0):233-249, 2022.
[18] Chrisantha Fernando, Dylan Banarse, Malcolm Reynolds, Frederic Besse, David Pfau, Max Jaderberg, Mare Lanctot, and Daan Wierstra. Convolution by evolution: Differentiable pattern producing networks. In Proceedings of the Genetic and Evolutionary Computation Conference
2016, pages 109-116, 2016.
[19] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208 220.0.2.000...18 2022.
[20] Debkalpa Goswami, Shuai Liu, Aniket Pal, Lucas G Silva, and Ramses V Martinez. 3d- architected soft machines with topologically encoded motion. Advanced Functional Materials, 29(24): 1808718 13, 2019.
[21] Sehoon Ha, Peng Xu, Zhenyu Tan, Sergey Levine, and Jie Tan. Learning to walk in the real world with minimal human effort. arXiv preprint arXiv:2002.0885 08550, 2020.
[22] Jonathan Hiller and Hod Lipson. Automatic design and manufacture of soft robots. IEEE Transactions on Robotics, 28(2):457-466, 2011.
[23] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao. Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint 1rXiv:2200002300 2022.
[24] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6855 2020.
[25] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint wrXi::2207.12598 2022.
[26] Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu. Daniela Rus, and Wojciech Matusik. Chainqueen: A real-time differentiable physical simulator for soft robotics. In 2019 International conference on robotics and automation (ICRA), pages 6265-6271. IEEE, 2019.
[27] Hyunki In. Useok Jeong, Haemin Lee, and Kyu-Jin Cho. A novel slack-enabling tendon drive that improves efficiency, size, and safety in soft wearable robots. IEEE/ASME Transactions on Mechatromics, 22(1):59-70, 2016
[28] Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning. 2022.
[29] Adrien Kaiser, Jose Alonso Ybanez Zepeda, and Tamy Boubekeur. survey of simple geometric primitives detection methods for captured 3d data. In Computer Graphics Forum, volume 38, pages 167-196. Wiley Online Library, 2019.
[30] Byungchul Kim, Useok Jeong, Brian Byunghyun Kang, and Kyu-Jin Cho. Slider-tendon linear actuator with under-actuation and fast-connection for soft wearable robots. IEEE/ASME Transactions on Mechatronics, 26(6):2932-2943 2021.
[31] Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. A scalable pipeline for designing reconfigurable organisms. Proceedings of the National Academy of Sciences, 117(4):1833-185) 2020.
[32] Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. Kinematic self- replication in reconfigurable organisms. Proceedings of the National Academy of Sciences, 118(49):e21 12672118, 2021.
[33] Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojeiech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Transactions on Graphics (TOG), 42(1):1-20, 2022.
11

---

[34] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. arXiv preprint arXiv:2206.01714, 2022.
[35] Pingchuan Ma, Tao Du, John Z Zhang, Kui Wu, Andrew Spielberg, Robert K Katzschmann, and Wojeiech Matusik. Diffaqua: A differentiable computational design pipeline for soft underwater swimmers with shape interpolation. ACM Transactions on Graphics (TOG), 40(4):1-14, 2021.
[36] Jonás Martínez, Jérémie Dumas, and Sylvain Lefebvre. Procedural voronoi foams for additive manufacturing, ACM Transactions on Graphics (TOG), 35(4):1-12, 2016.
[37] Lars Mescheder, Michael Oechsle, Michael Niemeyer. Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1460-4470, 2019.
[38] Thomas Morzadec, Damien Marcha, and Christian Duriez. Toward shape optimization of soft robots. In 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), pages 521-526. IEEE, 2019.
[39] Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv preprint 1rXiv:2212.08751, 2022.
[40] Songyou Peng, Chiyu "Max" Jiang, Yiyi Liao, Michael Niemeyer, Mare Pollefeys, and Andreas Geiger. Shape as points: differentiable poisson solver. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[41] Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.02 168, 2020.
[42] Yiling Qiao, Junbang Liang. Vladlen Koltun, and Ming Lin. Differentiable simulation of soft multi-body systems. Advances in Neural Information Processing Systems, 34:111224 34:17123-171355
2021.
[43] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748 8763. PMLR, 2021.
[44] Aditya Ramesh, Mikhail Pavlov, Scott Gray Gabriel Goh, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. arXiv preprint arXiv22102.12092 2021.
[45] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics, 2011.
[46] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans. Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprinl arXiv:2205. /487, 2022.
[47] Charles Schaff, Audrey Sedal, and Matthew R Walter. Soft robots learn to crawl: Jointly optimizing design and control with sim-to-real transfer. arXiv preprint arXiv:2202.04575, 2022.
[48] Ame Schneuing, Yuanqi Du Arian Jamasb Charles Harris, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Michael Bronstein Max Welling, and Bruno Correia. Structure-based drug design with equivariant diffusion models. arXiv preprint arXi::2210002303. 2022.
[49] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209. 14792, 2022.
12

---

[50] Lawrence Smith, Travis Hainsworth, Jacob Haimes, and Robert MacCurdy. Automated synthesis of bending pneumatic soft actuators. In 2022 IEEE 5th International Conference on Soft Roboties (RoboSoft), pages 358-363. IEEE, 2022.
[51] Lawrence Smith and Robert MacCurdy, Soroforge: End-to-end soft actuator design. IEEE Transactions on Automation Science and Engineering, 2023.
[52] Jascha Sohk-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
[53] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:20100002502, 2020.
[54] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[55] Andrew Spielberg, Alexander Amini, Lillian Chin, Wojeiech Matusik, and Daniela Rus. Co- learning of task and sensor placement for soft robotics. IEEE Robotics and Automation Letters, 6(2):1208-12)) 12108 2021.
[56] Andrew Spielberg, Tao Du, Yuanming Hu, Daniela Rus, and Wojeiech Matusik. Advanced soft robot modeling in chainqueen. Robolica, 41(1):74-10, 2023.
[57] Andrew Spielberg, Allan Zhao, Yuanming Hu, Tao Du Wojeiech Matusik, and Daniela Rus. Learning-in-the- -nophe-oohoo optimization: End-to-end control and co-design of soft robots through learned deep latent representations. Advances in Neural Information Processing Systems, 32, 2019.
[58] Kenneth O Stanley. Compositional pattern producing networks: A novel abstraction of develop- ment. Genetic programming and evolvable machines, 8:131-162 2007.
[59] Michael T Tolley, Robert F Shepherd, Michael Karpelson, Nicholas W Bartlett, Kevin C Galloway, Michael Wehner, Rui Nunes, George M Whitesides, and Robert J Wood. An untethered jumping soft robot. In 2014 JEEE/RSJ International Conference Intelligent Robots and Systems, pages 561-566. IEEE, 2014.
[60] Merel Van Diepen and Kristina Shea. A spatial grammar method for the computational design synthesis of virtual soft locomotion robots. Journal of Mechanical Design, 141(10), 2019.
[61] Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua Tenenbaum, Daniela Rus, and Chuang Gan. Softzoo: A soft robot co-design benchmark for locomotion diverse environments. In The Eleventh International Conference on Learning Representations, 2023.
[62] Tian Xie, Xiang Fu, Octavian Sagavian-Eugen Ganea, Regina Barzilay, and Tommi S Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Representations, 2021.
[63] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, , and Jian Tang. GeoDiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations, 2021.
[64] Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guiee guided human motion diffusion model. arXiv preprint arXiv:2212.025000 2022.
[65] Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. arXiv preprint arXiv:2210.06998, 2022.
[66] Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through poxnt-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5826-5835. 2021.
13

---

A Background On Point-E
Point-E [39] is a diffusion-based generative model that produces 3D point clouds from text or images. The Point-E pipeline consists of three stages: first, it generates single synthetic view using a text-to-image diffusion model; second, it produces a coarse, low-resolution 3D point cloud (1024 points) using second diffusion model which is conditioned on the generated image; third, it upsamples/ diffusion model. The two diffusion models operating on point clouds use permutation invariant transformer architecture with different model sizes. The entire model is trained on Point-E's curated dataset of several million 3D models and associated metadata which captures generic distribution of common 3D shapes, providing a suitable and sufficiently diverse prior for robot geometry. The diffused data is set of points, each point possessing 6 feature dimensions: 3 for spatial coordinates and 3 for colors. We ignore the color channels in this work. The conditioning for the synthesized image in the first stage relies on embeddings computed from a pre-trained ViT-L/14 CLIP model; in the embedding optimization of DiffuseBot, the variables to be optimized is exactly the same embedding. Diffusion as co-design is only performed in the second stage (coarse point cloud generation) since the third stage is merely an upsampling which produces only minor modifications to robot designs. We refer the reader the original paper [39] for more details.
the coarse point cloud to a high-resolution one (4096 points) with a third
B Theoretical Motivation
Online learning in embedding optimization. In Section 2.4, we discuss how to online collect dataset to optimize the embedding toward improved physical utility. Given simplified version of (5)
min Ha (xolec)) (xx,, c)]
(8)
where, for notation simplicity, we drop t [1, Nee; 0,I) in the sampling distribution, and summarize [||e (x, e, t,, c)l as g(x, c). We can rewrite the expectation term as,
f Pa (x(x))
pe(c)xo) Po(c)
g(xo, c)dr
(9)
which allows to sample from po (xp) (i.e., generating samples from the diffusion model) and reweight the loss with (e(e)x)); the latter scaling term is essentially proportional to a normalized task perfor- pale) mance. Empirically, we can maintain a buffer for the online dataset and train the embedding with the sampling distribution biased toward higher task performance; we use list store samples with top-k performance in our implementation (we also tried reshaping the sampling distribution like prioritized experience replay in reinforcement learning but we found less stability and more hyperparameters required in training compared to our simpler top-k approach).
Connection to MCMC. In diffusion sampling, the simplest way perform reverse denoising process as in Section 2.2 follows [24].
pa (xxx-11x)) = N (11( (x, Va
3e
k: 31 1 1 āc
VI-oi (x t));
(10)
Here, the denoising term can either be unconditional ea(xe: () or conditional via classifier-free guidance é fe,ccassifeertteee (x, C) as in (3). We use the latter to incorporate the optimized embedding. aim to introduce physical utility during diffusion
To further leverage physics-basee simulation, sampling process. One possibility is to utilize classifier- faased guidance [12],
Eo classifier-based == (x,,
/I - a, Vx, log p(c(x:)
(11)
where p(c|x;) can be conceptually viewed as improving physical utility and Vx, log p(c x,) can be obtained using differentiable physics and the unconditional score. Note that we slightly abuse the notation here by overloading with conditioning from differentiable simulation during sampling other than the classifier-free guidance using the optimized embedding. However, combining (10) and (11) much less resembles any cradient-based optimization techniques, which are shown to be effective in soft robot co-design with differentiable simulation [26, 2]. Fortunately, drawing a connection to energy- based models [54, 15, 14], yet another alternative to incorporate conditioning in diffusion
14

---

Table 4: Configuration of embedding optimization.
Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
Buffer Size
600
600
60
600
60
Min. Buffer Size
60
60
60
60
60
60
Num. Samples / Epoch
60
60
60
60
60
60
Train Iter. Epoch
Buffer Top-K
12
Batch Size
6
6
6
6
6
Table 5: Configuration of diffusion co-design.
Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
mmax
400
150
400
400
400
400
biin
0
0
0
At
50
25
50
50
50
50
K
5
5
10 10-4_
10 10-4 |
10 3
10 10-4,
,
10 -8
10 10-4., 8
K
10a
104
10
10
104
10
7
0.01
0.001
0.001
0.001
Renorm Scale
10
models is Markov Chain Monte Carlo (MCMC) sampling [14], where we use Unadjusted Langevin Dynamies,
(k-1) X; =x (K) where (A) ~ N (R) ;x + o² Vx logp(c|x (k-1) 11 ),
(12)
2
where K is the number of samples in the current MCMC with k as indexing, o² is pre-defined variance, and x; (0) Xt-1- In the context of diffusion models, this procedure is commonly performed within single diffusion step to drive the sample toward higher-density regime under the intermediate distribution pxx,) = qxx, (x))p(x)) (2)))(((dd))) dx, at diffusion time t. Inspired by its resemblance to gradient ascent with stochasticity from the added Gaussian noise of variance o², we establish connection to design optimization, reformulating diffusion process co-design optimization in (6) and (7).
Specifically, we can apply Bayes rule decompose the score of p(c| (*--1)).
Vx logp(c/x;) Vx logp(x;/c) V logp(x;)
(13)
where Vx (xx) is simply the denoiser output Eø (x,,, t) and V log p(x/(())))) is the gradient of task performance with respect to the intermediate sample of the diffusion model from differentiable physical simulation and robotizing process. Overall, this leads (6).
C Implementation Details In Algorithm
In this section, we provide more implementation details and experimental configurations of Diffuse- Bot and other baselines, In Table 4, we list the configurations of the embedding optimization. With respect Algorithm 1, "buffer" refers to the online dataset D.
Buffer Size is the capacity of the dataset.
Min. Buffer Size is the minimum of data filled in the buffer before training starts.
Num. Samples/ Epoch is number of new samples collected in each epoch, where epoch here refers to a new round data collection online learning.
Train Iter. Epoch is number of training iterations per epoch.
Buffer Top-K K is the number of datapoints with top-k performance being retained in the Filter step atop the most up-to-date data.
Batch Size is the batch size in the embedding optimization.
15

---

Direct SIDF
DiffuseBot
04:0
0430
9422
Number
Wamber Beralion
Figure 8: Comparison between Direct SDF DiffuseBot on conversion to solid geometry robotizing.
In Table 5, we list the configurations of diffusion co-design. We follow (6)(7) for:
K is number of MCMC sampling steps at the current diffusion time.
d is the standard deviation related to the MCMC step size.
k is the ratio between two types of design gradients.
y is the weight for trading off design and control optimization.
Imax and min are the maximal and minimal diffusion time to perform diffusion as co-design,
respectively.
At is the diffusion time interval to perform diffusion as co-design.
For baselines, use learning rates for control optimization following in Table 5; for particle-based and based approaches, we use learning rate 0.01 for design optimization; for implicit function and diff- 1fff- use learning rate 0.001 for design optimization. For the inputs of implicit function and
we use x, y, Z coordinates in the local workspace, the distance to the workspace center on the xy, yz planes, and the radius from the center. For the network architecture of implicit function, we use a 2-layer multilayer perceptron with hidden size 32 and Tanh activation. For the network architecture of diff-CPPN, we Sin, Sigmoid, Tanh, Gaussian, SELU, Softplus, Clamped activations with 5 hidden layers and 28 graph nodes in each layer.
Hyperparameters are chosen mostly based intuition and balancing numerical scale with very little tuning. In the following, we briefly discuss the design choices all hyperparameters listed in Table and Table 4. For min buffer size, samples per epoch, training iteration per epoch, and batch size, roughly make sufficiently diverse the data used in the optimization and use the same setting for all tasks. For buffer size, start with 60 and if we observe instability in optimization, to 10 times, 600 (similar to online on- on-policy reinforcement learning); note that buffer size refers to the maximal size and increasing this won't affect runtime. For buffer Top-K, we start with 6 and we observe limited diversity of generation throughout the optimization (or lack of exploration), double it. For imaz: min, and At, roughly inspect how structured the generation in terms of achieving the desired robotic task to determine tmaz and modify At accordingly to match the similar number of performing MCMC sampling (e.g., Emex/ At: 400/50
increase
150/25). For the number of MCMC steps K, we simply set for passive tasks and 5 for active tasks by intuition. For a, we simply follow one of the settings in [14]. For the guidance scale λs and renorm scale, we check the numerical values between and gradient from differentiable simulation and try to make them roughly in the similar magnitude, and set the same scale for all tasks for simplicity. For y, we set 0.001 for trajectory optimization and 0,01 for parameterized controllers based on our experience of working with differentiable physics. Overall, from our empirical findings, the only hyperparameters that may be sensitive include buffer size and buffer Top-K K for optimization stability and generation diversity, and guidance scales, which need be tuned match the numerical magnitude of other terms so as to take proper effect.
D Details In Task Setup
In this section, we provide more details of all tasks setup including environment configuration and prescribed actuator and controller if any (as mentioned in Section 2.4; it is fixed during embedding optimization and will be co-optimize during diffusion as co-design). We select tasks that
can cover wide spectrum of existing roboties tasks: we briefly categorize tasks into passive dy- namics, locomotion, and manipulation. Note that passive dynamics tasks are explicitly considered
16

---

here since there is no active control of robot bodies, making optimization on robot design direct factor toward physical utility.
2. only involve lower-level control/motion without the complications of long-term or higher-level task planning: we select tasks that mostly involve few motor skills, e.g., in manipulation, instead of pick and place, we simply aim at picking up/gripping an object.
3. are commonly considered in other soft robot co-design literature: all proposed active tasks are widely used in the soft robot community, including crawling [8, 10, 47, 61], hurdling/jumping
[26, 59, 3], and manipulating objects [5, 11, 38].
4. may induce more visible difference in robot designs between the performing and the non- performing ones to facilitate evaluation and algorithmic development: we select tasks more based on heuristics and intuition, e.g., in crawling, we expect leg-like structures may outperform other random designs.
We build our environments on top of SoftZoo [61] and employ the Material Point Method for simulation. Each environment is composed of boundary conditions that include impenetrable ground, and, in the case of fixed objects or body parts, glued particles. All tasks units without direct real-world physical correspondence and last for 100 steps with each step consisting of 17 simulation substeps. In the following, when describing the size of 3D shape without explicitly mentioning the axis, we follow the order: length (x, in the direction when we talk about left and right), height (y, parallel with the gravity direction), and width (z). All tasks are demonstrated in Figure in the main paper.
Balancing. The robot is initialized atop stick-like platform of shape 0.02-umit 0.05-unit X 0.02-umit. The robot is givenan initial upward velocity of a 0.5-unitssccond and allowed to free fall under gravity. The goal is for the robot to passively balance itself after dropping again on the platform; the performance is measured as the intersection over union (IoU) between the space occupied by the robot during the first simulation step and the space occupied by the robot during the last simulation step. The robot geometry is confined to a 0.08-unit x 0.08-unit x 0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).
Landing. The robot initialized to be 0.08-unit to the right and 0.045-unit above the landing target with size of 0.02-unit x 0.05-unit x 0.02-unit. The robot is given an initial velocity of 0.5- unit/second to the right. The goal of the robot is to land at the target; the performance is measured as the exponential to the power of the negative distance between the target and the robot in the last frame e -1p seaee
-Pn "I, where Pyz is the position of the robot or object at the last frame with horizon H. The robot geometry is confined to 0.08-unit x 0.08-unit x 0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).
Crawling. The robot is initialized at rest on the ground. The goal of the robot is to actuate its body to move as far away as possible from the starting position; the performance is measured as the distance traveled Ilg Po roboo ||. where roobt is the position of the robot in the x axis a certain frame with horizon H. The robot geometry is confined to a 0.08-unit x 0.08-unit x 0.08-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the xz (nn-verrtcal) plane into 4 groups. Each group contains an actuator with its direction parallel to gravity. The prescribed controller is a composition of four sine waves with frequency 30hz, amplitude 0.3, and phases 0.5xT, 1.5TT, 0, and for the four actuators.
Hurdling. An obstacle of shape 0.01-unit X 0.03-unit 0.12-unit placed in 0.07-umit front of the robot. The goal of the robot is to jump as far as possible, with high distances achieved only by bounding over the obstacle. The performance is measured as the distance traveled. The robot geometry is confined to a 0.08-umit x 0.08-umit x 0.08-umit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups. Each group contains an actuator aligned parallel to gravity. The prescribed controller takes the form of open-loop, per-step actuation sequences, set to linearly-increasing values from (0.0, 0.0) to (1.0, 0.3) between the first and the thirtieth frames and zeros afterward for the two actuators (the first value of the aforementioned actuation corresponds to the actuator closer to the obstacle) respectively.
Gripping. An object of shape 0.03-unit x 0.03-unit x 0.03-unit is placed 0.08-unit underneath the robot. The goal of the robot is to vertically lift the object; the performance is measured as the
vertical distance of the object being lifted [|p.
object
pb
g.mnjectj, || where p. r.jject is the position of the
17

---

object in the axis at a certain frame with horizon H. Within a 0.06-umit x 0.08-umit x 0. 0mii- workspace. we decompose the robot into a base and two attached submodules, and we set the output of DiffuseBot or other robot design algorithms one of the submodules and make constrain the other submodule to be a mirror copy; conceptually we design "the finger of a parallel gripper." The base is clamped/glued to the upper boundary in z and given large polyhedral volume to serve as the attachment point of the gripper finger submodules. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups; each submodule has one pair of actuators. Each actuator is aligned parallel to gravity. Overall, one pair of actuators comprises the "inner" part of the gripper and the other comprises the "outer" part. Suppose the actuation of the two pairs of actuators is denoted in the format of (actuation of the outer part, actuation of the inner part), the prescribed controller is per-frame actuation, set to (i) finearly-increasie values from (0.0, 0.0) to (0.0, 1.0) between the first and the fiftieth frames, and (ii) then dinearly- values from (1.0, 0.0) to (0.0, 0.0) between the fiftieth and the last frames.
Moving a box. A 0.03-unit x 0.03-unit x 0.03-unit cube is placed on the right end of the robot (half body length of the robot to the right from the robot center). The goal of the robot is to move the box to the left; the performance is measured as the distance that the box is moved to the right with x. 1. where p. 2...ejjcct is the position of the object in the x axis certain frame with horizon H. The robot geometry is confined a 0.1 6-unit 0.06-unit x 0.06-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the height direction into groups. Each group contains an actuator aligned parallel to the ground. The prescribed controller is per-frame actuation, initialized to linearly-increasing values from (0.0, 0.0) (1.0, 0.0) between the first and last frame for the lower and the upper actuator respectively.
respect to its initial position [|p s.ojeet Po
E Analysis On Robotizing
As mentioned in Section 2.3, Material Point Method simulation requires solid geometry for simulation; thus, we need to convert the surface point cloud from Point-E [39] to a volume. The most direct means of converting the point cloud to a solid geometry is to compute the signed distance function (SDF), and populate the interior of the SDF using rejection sampling. We refer to this baseline as Direct SDF Here, we use a pretrained wansformer-based model provided by Point-E E as the SDF. In Figure 8, we compare Direct SDF with our approach described in Section 2.3 Solid Geometry. We perform robotizing on intermediate samples at t-300. We observe that Direct SDF fails to produce well-structured solid geometry since it trained with watertight on geometry, and thus the conversion cannot gracefully handle generated point clouds that do not exhibit such watergith structure. This is specifically common in the intermediate diffusion sample X as x, is essentially a Gaussian- noise- corrupted version of the clean surface point cloud. In contrast, the robotizing of DiffuseBot produces a much well-structured solid geometry since it explicitly handles the noisy interior 3D points by introducing a tailored loss in Shape As Points optimization [40] (see Section 2.3). In addition, better-structured robot geometry is critical to obtaín not only a more accurate evaluation of a robot design at the forward pass of the simulation but also the gradients from differentiable physics at the backward pass. In Figure& 8, we further perform co-optimization on the two robots obtained by Direct SDF and DiffuseBot; we observe more stably increasing trend of task performance in our approach, demonstrating that the gradient is usually more informative with a better-structured robot. Empirically, while Direct SDF sometimes still provides improving trend with unstructured robots in co-design optimization, the performance gain is not as stable as DiffuseBot.
F
Additional Visualizations Of Experiments
In this section, we show more visualization for a diverse set of the experimental analysis. Please visit our project page (attps://diffusebot wtt.futt.....tuub...o..... io/) for animated videos.
Generated robots performing various tasks. In Figure 9, we display a montage of a generated robot's motion for each task; this is the unabridged demonstration of Figure 5 in the main paper.
Physics-guided robot generation In Figure 10, we show how DiffuseBot evolves robots throughout the embedding optimization process; this is the full demonstration of Figure 4 in the main paper. Note
18

---

Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
1st Frame
30th Frame
60th Frame
100th Frame
Figure Examples of robots bred by DiffuseBot to achieve the all presented tasks.
that DiffuseBot is a generative algorithm and thus results presented are for demonstration purposes and reflect only a single fixed random seed.
Diffusion process of robot generation. In Figure 11, show how robots evolve throughout the diffusion process at different diffusion time via x, with from T' to O: not to be confused by Figure 10 where all generations are obtained via full diffusion sampling Xo and different conditioning embeddings c. This figure demonstrates the robotized samples across diffusion times, gradually converging the final functioning robots approaches
Comparison with baselines. In Figure 12, we present robots generated from different baselines. Note that Point-E is essentially DiffuseBot without embedding optimization and diffusion as co-design.
Incorporating human feedback. Figure 13, we showcase more examples of incorporating human textual feedback to the generation process of DiffuseBot. As briefly mentioned in Section 3.4 in the main paper, we leverage the compositionality of diffusion-based generative models [34, 14]. Specifically, this is done by combining two types of classifier-free guidance in (3); source
19

---

Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
1st Checkpoint
2nd Checkpoint
3rd Checkpoint
4th Checkpoint
Figure Examples DiffuseBot evolving robots to solve all presented tasks.
of guidance derived from the task-driven embedding optimization while the other is derived from embeddings from human-provided textual descriptions. These two conditional scores, namely (X t, Cphysios) and e (xe, Chuman). can then be integrated into the diffusion as co-design framework as in (6). We refer the reader to the original papers for more details and theoretical justification. We demonstrate examples of: the balancing robot with additional text prompt ball". the crawling robot with additional text prompt "a star", the hurdling robot with additional text prompt "a pair of wings". and the moving-a- a-bonng--boxxx robot with additional text prompt "thick". Interestingly, human feedback is introduced as an augmented "trait" to the original robot. For example, while the hurdling robot keeps tall, lengthy geometry that beneficial for storing energy for jumping, wing-like structure appears at the top of the robot body instead of overwriting the entire geometry. This allows users to design specific parts for composition while also embedding fabrication and aesthetic priorities and constraints through text. We leave future explorations of these ideas for future work.
20

---

Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
= 600
400
200
t=
Figure 11: Demonstrations of generations changing from noises to rohots with physical utility throughout difftusion sampling process. We present snapshots at diffusion times = 600, 400, 200, for all presented tasks.
G Hardware Design and Fabrication
To create real-world counterpart of the DiffuseBot, fabricated proof-of-concept physical robot gripper, as illustrated Figure 14(a). The gripper was designed to match the shape of the digital gripper and successfully demonstrated its ability to grasp objects, as shown in Figure 15. video demonstrating the grasping process can be found on our project page (https:/ //iffusebot... github. io/).
During the translation from the simulated design to physical robot, we aimed to minimize differences between the simulated and physical designs. However, due to hardware and fabrication limitations, certain modifications were necessary.
One such challenge involved replicating the arbitrary contraction force at the particles from the simulation in the physical world. To address this, we employed tendon transmission as the actuation method for the real-world gripper, which was found to be suitable for emulating interaction forces
21

---

Implicit Function
Diff-CPPN
Point-E
DiffuseBot
Balancing
Landing
Crawling
Hurdling
Gripping
Moving a Box
Figure 12: Comparison of robots generated by different baselines.
from the digital world without introducing significant discrepancies. To enhance the stability of the tendon-driven actuation, we incorporated rigid base ("Base" Figure 14(a)) above the gripper itself ("Gripper" in Figure 14(a)). This base was designed to withstand the reaction force generated by the tendon-driven actuation. Furthermore, we added four tendon routing points (represented by small dotted circles in Figure 14(b)) on each finger to securely fix the tendon path. By utilizing four Teflon tubes (shown in Figure 14(b),, we were able to position complex components such as motors, batteries, and controllers away from the gripper, reducing its complexity.
The actuation strategy the simulated gripper applies equal contraction force to both fingers simulta- neously. To replicate this strategy, we employed underactuated tendon routing in the development of the gripper. This approach eliminates the need for four separate actuators, thereby reducing the complexity of the robot. We used tendon-driven actuators specifically designed for underactuated tendon-driven robots as they solve practical issues such as size and friction issues that commonly arise in such systems [30].
22

---

Balancing Robot from DifluseBal
Belancing Bobot Tex: A Ball"
Box moving Robet From Diffuseãc:
Box moving Robot lext "Thck"'
ddle
Crawler From Diflusefot
Frawlor Test 'A Round Dist"
Hurdler From Difuusefoot
Hurdler Text 'A Pai Of Wines"
Two wingss
Croular shape
Figure 13: More examples of incorporating human textual feedback into robot generation by DiffuseBot.
(a)
(b)
(c)
Actustor 2
Base
Actuator
Teflon
tube
Gripper
Figure (a) shows fabricated mrof-of-concep physical robot gripper; (b) describes detailed robot configura- tion; (c) represents interior structure design used to soften the robot body.
The soft gripper was 3D-printed using digital light projection (DLP) type 3D printer (Carbon M1 printer, Carbon Inc.) and commercially available elastomeric polyurethane (EPU 40, Carbon Inc.). To enhance the softness of the robot body beyond the inherent softness of the material itself, we infilled the finger body with Voronoi lattice structure, metamaterial useful for creating soft structure with tunable effective isotropic stiffnnes [20, 36]. We generated Voronoi lattice foam with point spacing of 2.5mm, and beam thickness of 0.4 mm as shown in the red boundary of Fig. 14(c). Finally, we tested the soft gripper, designed and fabricated as described above, to verify its ability to grasp an object, as shown Figure 15.
Although, at present, the compilation of the virtual robot to a physical, digitally fabricated counterpart involves manual post-processing of algorithm's output, most, not all of these steps could be automated. Our method outputs a point cloud (defining geometry). actuator placements, and an open-loop controller, along with a prescribed stiffness. Since we can casily convert the point cloud into a 3D triangle mesh, the geometry can be created by almost any 3D printing method. In order realize an effective stiffness and material behavior, stochastic lattices, specifically Voronoi foams, have been used [36, 20] in the past and employed here in order to match target material properties. Given the actuator placement, tendons [27, 30] can be aligned with the prescribed (contiguous) regions. Since lattice is used, threading tendons through the robot body is simple, and we note that even more complex routings have been studied in detail the literature [4]. Creating attachment points for the tendons is relatively simple geometry processing problem [6]. Thus, converting virtual robot specification that incorporates geometry, material, and actuation can be automated in straightforward way.
More discussion on fabrication.
23

---

(a)
(b)
(c)
Grasping motion of the real world soft gripper from DiffuseBot. Time order from (a) to (c). The demo video can be seen at our project page.
Figure 15:
We note that when controlled, the physical robot may not always match the virtual robot's motion. This is the sim-to-real gap, and is significantly harder to overcome in our case than translating the virtual robot to physical hardware. Significant literature has been invested in specifically tackling the sim-to-real gap, and in our case would require its own dedicated project; however, we note that often hardware can be adapted to work by modifying only the control policies using feedback from real-world experiments, often even with little human intervention [21].
Quantitative analysis. In Figure 16, we provide a detailed analysis the gap between simulation and physical robot. In order to explore the quantitative gap between the behavior of the physical robot and the simulated robot, we conducted similar setups are commonly adopted in soft robot literature [17]. The objective was to measure the
experiment with the following conditions, where
change in distance between two tips when we pull/release two tendons one for closing the gripper (flexion) and the other for opening it (extension). The tendons were pulled or released in increments and decrements of 2mm.
When contracting the tendon to flex or extend the fingers, both simulation and real robot results show -g--saped graphs. The pattern in the physical robot plot is a commonly observed phenomenon called hysteresis. However, the main difference between the simulation and real-world cases can be seen when releasing the tendon from fully contracted state. In the real robot experiment, the tip distance changes rapidly, while in the simulation, the opposite effect is observed.
One plausible explanation for this disparity could be attributed to the friction direction and elongation of the tendons. During the transition from tendon contraction to tendon release, the tension of the tendon at the end-effector may change suddenly due to the change of the friction direction. Also, since we only control the motor position (not the tendon position) to pull/release the tendon with 2mm step, the exact tendon length may not be exactly the same when we consider the tendon elongation.
Additional real robots. In Figure 17, we show two other fabricated robots for the moving box task and the crawling task. For moving a box, we can achieve similar motion as in the simulation; the only slight difference occurs when the stiffness the physical robot is not as soft as that in the simulation, falling bit short to perform the extremely curly motion in the simulation. For crawling, we found more significant sim-to-real gap where the physical robot cannot achieve at all the galloping like llie motion in the simulation; instead, the robot crawls more alike inchworm movement, pushing the body in a quasi-static way. The major reason is the damping of the material (elastomeric polyurethane, EPU 40, Carbon Inc.) prohibit from releasing sufficient energy for dynamical motion.
Potential solutions. Given that the gap between simulation and real robot performance seems to originate from the trauttioo/ransmissioon method, our future work will focus on developing a tendon- driven actuation simulation framework. This framework aims address the differences and improve the accuracy of our simulations. We are exploring other possible explanations for the sim-to-real gap and will investigate any additional factors that may contribute to the observed discrepancies. Overall, as for a high-level general solution, we believe (1) adjusting parameters based on observed sim to real gap and repeat the design process or (2) building a more accurate physics-based simulation (which can be straightforwardyy plug-and-played in DiffuseBot) can largely bridge the sim-to-real gap of fabricating physical robots; or more interestingly, connecting generative models to commercial-level design and simulation softwares.
24

---

Simulation
Real
Tip Distance (unit)
0.02
Tip Distance (mm)
Molton
Extendion
6:0
Hestur
Actuation Type
0.90
Contracting
-0.92
Extension
Heston
3.0
0.1
0.2
0.3
0.4
0.5
10
Actuation (unit)
Tendon Pulled Length (mm)
Figure 16: Quantitative analysis of behavior between simulation and physical robots.
Moving Box
Crawling
Figure 17: Other proof-of-concee real robot fabrication for moving box and crawling,
H
More Discussions
H.1 Limitation
The major limitation of DiffuseBot is that make simplification in the parameterization of actuators and stiffness; we make dependencies of the two design specifications on robot geometry (check more technical details in Section 2.3 paragraph Actuators and Stiffness. This works well with properly-crafted mapping from geometry to the others yet limits the potential by human prior with little use of the generative power. While this may be reasonable as properly-set actuators and stiffness based on geometry (hand-tuned empirically in this work) roughly reflects task performance, more flexible parameterization can definitely lead improved performance. Potential remedy can be using part-based 3D generative models for actuators and direct optimization for stiffness. Another limitation is the gap between simulated results and real robots. While the hardware experiment has shown as a proof-of. coooo--f--oncepp that minimally demonstrates the potential, physical robot fabrication and real-world transfer have countless non-trivial challenges including stiffness and actuator design, sim-to-real gap, etc. This may require studies on more high-fidelity physics-basee simulation, which can be straightforwardly plugged into DiffuseBot.
H.2 Conditioning Beyond Text
The use of textual inputs additional to the embeddings optimized toward physical utility is achieved by both being able be consumed by the diffusion model produce guidance for the diffusion process. More concretely speaking, in DiffuseBot, we use the CLIP feature extractor as in Point-E and it allows to extract embedding for both text and image modalities, which then be used condition in the diffusion model. Thus, we can also incorporate images as inputs and perform the exact same procedure that of the textual inputs. Theoretically, the textual inputs are incorporated via following the intuition in end of the paragraph Diffusion as Co-design in Section 2.4, where the textual inputs additionally provide gradients toward following the textual specification. Similarly, the image inputs can also be processed to provide gradients since CLIP embeddings live in joint space of images and languages. More interestingly. if build DiffuseBot on models other than Point-E, which can consume embeddings for other modalities like audio as conditioning, we can then straightforwardly perform robot design generation guided by the other corresponding input formats (and meanwhile, toward physical utility). Note that this critical feature of compositionality across different sources of guidance throughout the reverse diffusion process is one of the biggest advantages of using diffusion-baan based models as opposed to other types of generative models.
25

---

H.3 Connection To Text Inversion
There some synergy between text inversion in [19] and embedding optimization in DiffuseBot. Both of them aim at tuning the embedding toward reflecting certain properties of the output generation, i.e. describing the output generated images in [19] and toward improved physical utility DiffuseBot. The major difference lies in the nuance of the data/samples used to carry out the optimization. Text inversion performs a direct optimization using latent diffusion model loss (Fq. (2) in [19]), which computes losses on noisy samples/latents corrupted from the real dataset. On the other hand, it is tricky to think about real dataset in robot design (as discussed in the second paragraph of Section and the paragraph Embedding Optimization in Section 2.4), embedding optimization in DiffuseBot computes losses on noisy samples corrupted from self-generated data filtered by robot performance (as in Algorithm and Section 2.4). Conceptually, it is more like mixture of diffusion model training and online imitation learning like DAGGER [45].
H.4 Connection To Diffusion Models With Physical Plausibility
A potential and interesting way to adapt DiffuseBot to other applications like motion planning or control [28, 1] to view a generated robot as one snapshot/frame of motion/state sequence and the physics prior can be the dynamics constraint across timesteps (e.g.. robot dynamics or contact dynamics that enforce non-penetration). The physics prior can be injected similarly to diffusion co-design that propagates the enforcement of physical plausibility of generated states from differentiable physics-based simulation to diffusion samples. For example, considering states in two consecutive timesteps, violation of physical constraints regarding robot dynamics or interaction with the environment. Then, we can compute gradients with respect either control or design variables; for gradients in control, this will essentially augment works like [28, 1] with classifier-based guidance to achieve physical plausibility; for gradients in design, this will much resemble optimizing toward the motion sequence of a chape-shifting robot.
compute loss in the differentiable simulation measure the
H.S Parameterization Of Actuator And Stiffines
The goal of DiffuseBot is to demonstrate the potential of using diffusion models to generate soft robot design and to leverage the knowledge of the pre-trained generative models learned from a large-scale 3D dataset. Under this setting, the generated output of the diffusion model can only provide the geometry information of robot designs, leading to our design choice of having full dependency of actuator and stiffness the geometry. This may be reasonable simplification as prior works [48] have shown geometry along with properly-set actuator and stiffness (we take manual efforts to design proper mapping from geometry to actuator and stiffness in this work) roughly reflect the performance of a soft robot design. For better generality, one potential remedy is to optimize actuator and stiffness independently from the geometry generated by the diffusion model, i.e., apply DiffuseBot and do direct optimization on actuator and stiffness afterward or at the same time. Another interesting direction may be, for actuators, to leverage part-based models [29] to decompose a holistic geometry into parts (or different actuator regions in soft robots).
26


Reviews:
[{"review_id": "utQhJeAnGr", "reviewer": "Reviewer_5Fqn", "summary": "This paper proposes a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks called DiffuseBot. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. In the experiment, they showed a range of simulated and fabricated robots along with their capabilities.", "questions": "1. In Section 2 before 2.1, there was no reference to 2.2. Is it fine?\n2. I did not find the explicit definition of bold c in Eq. (5).  \n3. L159: what does the slash mean? \n4. In the experiments, I want to know why such tasks were selected (i..e, motivation for the tasks). \n5. I cannot find the explanation about the performance metric such as Tables 1 and 2 (sometimes having minus values). This information is important and should be mentioned in the main text.\n6. The (short) introduction of the baseline models and the reasons can be mentioned. \n7. The figure at the bottom of page 7 was Table 4, but may be incorrect. And Figure 4 in L233 may be incorrect. \n8. Section 3.4: The paper discusses the use of textual inputs and is very interesting. Can the authors discuss the potential for using other input formats?\n", "limitations": "In the conclusion section, there seems to be less limited information about this work from the experimental results. In other parts, general limitations were mentioned.\n", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper introduces a new framework that augments diffusion-based synthesis with physical dynamical simulation in order to co-design task-driven soft robots in morphology and control.\n2. The method leverages optimizing input embeddings and incorporating differentiable physics into the diffusion process for driving robot generation in a task-driven way toward improved physical utility. \n3. They performed experiments in simulation to verify the effectiveness of DiffuseBot, extensions to text-conditioned functional robot design, and a proof-of-concept physical robot as a real-world result. \n", "weaknesses": "The presentations in this paper were sometimes unclear, as asked in the following placeholder. The paper admitted that there are countless non-trivial challenges in the physical robot fabrication and real-world transfer, including stiffness and actuator design, and the sim-to-real gap. However, in my opinion, other contributions of this paper may outperform the weaknesses. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer 5Fqn for positive comments on the soundness and the contribution of our work. We address the remaining questions as below.\n\n**Challenges of physical robot fabrication.**\n\nThanks for recognizing the contribution of our work in spite of these non-trivial challenges of real-world transfer. We further conduct a simple quantitative analysis on sim-to-real transfer. Please check out more details in the response to reviewer 7Vvn along with the experimental results shown in the one-page pdf in the global response as well as the in-depth discussion on manufacturing physical robots in the response to reviewer syw2.\nNote that this experiment only serves a preliminary study and more comprehensive and in-depth analysis along with further contribution should be done in the future research. We will also dedicate a paragraph to discuss these challenges and potential remedy in the limitation section in the revision.\n\n**No reference to 2.2.**\n\nWe will add the following in the first paragraph of section 2 in the revision,\n\n“… then describe the proposed DiffuseBot framework, which consists of diffusion-based 3D shape generation (Section 2.2), a differentiable procedure …”\n\n**Explicit definition of \\bold c.**\n\nThe **c** in Eq. (5) is the embedding to be optimized. We will add a clearer definition at line 133 right after Eq. (5) in the revision.\n\n**L159: what does the slash mean?**\n\nIt is a typo and it should be a period. We will update this in the revision.\n\n**Motivation for the tasks.**\n\nAt a high level, we select tasks that \n1. can cover a wide spectrum of existing robotics tasks: we briefly categorize tasks into passive dynamics, locomotion, and manipulation. Note that passive dynamics tasks are explicitly considered here since there is no active control of robot bodies, making optimization on robot design a direct factor toward physical utility.\n2. only involve lower-level control/motion without the complications of long-term or higher-level task planning: we select tasks that mostly involve few motor skills, e.g., in manipulation, instead of pick and place, we simply aim at picking up/gripping an object.\n3. are commonly considered in other soft robot co-design literature: all proposed active tasks are widely used in the soft robot community, including crawling [5,7,35,48], hurdling/jumping [19,A1,A2], and manipulating objects [3,8,27]. \n4. may induce more visible difference in robot designs between the performing and the non-performing ones to facilitate evaluation and algorithmic development: we select tasks more based on heuristics and intuition, e.g., in crawling, we expect leg-like structures may outperform other random designs.\n\nWe will include the above discussion in the appendix section D in the revision.\n\n**Explanation about the performance metric.**\n\nThe performance metrics are described in the appendix section D. We will add brief description as below and a pointer in section 3.1,\n\n“We refer the reader to the appendix Section D for more detailed task descriptions and performance metrics.”\n\n**Brief introduction to the baselines.**\n\nIn the revision, we will add the following paragraph in section 3.2 with more details in the appendix: \n\n“In Table 2, we compare with extensive baselines of soft robot design representation: particle-based method has each particle possessing its own distinct parameterization of design (geometry, stiffness, actuator); similarly, voxel-based method specifies design in voxel level; implicit function uses use a shared multi-layer perceptron to map coordinates to design; DiffCPPN uses a graphical model composed of a set of activation function that takes in coordinates and outputs design specification. These baselines are commonly used in gradient-based soft robot co-design [19,44,48]. ”\n\n**Incorrect labeling of Table 4.**\n\nThanks for catching these typos. Table 4 at the bottom of page 7 should be labeled as Figure X and the reference of Figure 4 in L233 should be Figure X. We will fix the labeling and referencing in the revision. (After fixing this issue in the manuscript, X is 6 and the original Figure 6 becomes Figure 7)\n\n**Other input formats than texts.**\n\nThe use of textual inputs additional to the embeddings optimized toward physical utility is achieved by both being able to be consumed by the diffusion model to produce guidance for the diffusion process $\\epsilon$. More concretely speaking, in DiffuseBot, we use the CLIP feature extractor as in Point-E and it allows to extract embedding for both text and image modalities, which can then be used as a condition $\\mathbf{c}$ in the diffusion model. Thus, we can also incorporate images as inputs and perform the exact same procedure as that of the textual inputs. Theoretically, the textual inputs are incorporated via following the intuition in lines 162-165, where the textual inputs additionally provide gradients toward following the textual specification. Similarly, the image inputs can also be processed to provide gradients since CLIP embeddings live in a joint space of images and languages. More interestingly, if we build DiffuseBot on models other than Point-E, which can consume embeddings for other modalities like audio as conditioning, we can then straightforwardly perform robot design generation guided by the other corresponding input formats (and meanwhile, toward physical utility). Note that this critical feature of compositionality across different sources of guidance throughout the reverse diffusion process is one of the biggest advantages of using diffusion-based models as opposed to other types of generative models.\n\nWe will include this discussion in the appendix in the revision.\n\n**Limitation in the conclusion.**\n\nPlease check the paragraph about limitations in the global response.\n\n**Reference**\n\n[A1] Tolley, M.T., et al., An untethered jumping soft robot. IROS 2014.\n\n[A2] Bartlett, N.W., et al., A 3D-printed, functionally graded soft robot powered by combustion. Science 2015.\n"}, {"review_id": "2SmfiVU1Pe", "reviewer": "Reviewer_7Vvn", "summary": "The paper introduces DiffuseBot, a system that aims to simplify and automate the design of soft robots in simulation and real-world systems. DiffuseBot uses diffusion-based algorithms to co-design soft robot morphology and control for specific tasks, combining the diversity of evolutionary algorithms with the efficiency of gradient-based optimization. The system is made possible by advancements in AI-driven content generation.\n\nHowever, existing generative algorithms face challenges when applied to physical soft robot co-design, such as the lack of consideration for physics and task performance. To overcome these, DiffuseBot uses physical simulation to guide the generative process of pretrained large-scale 3D diffusion models. It also develops an automatic procedure to convert raw 3D geometry into a format compatible with soft body simulation.\n\nThe system optimizes the embeddings that condition the diffusion model, skewing the sampling distribution toward better-performing robots as evaluated by a simulator. It also reformulates the sampling process to incorporate co-optimization over structure and control.\n\nDiffuseBot has been tested on a wide range of tasks, demonstrating its superiority to comparable approaches. It also allows for human input in the robot generation process and has been used to create a proof-of-concept 3D-printed real-world robot. The paper contributes a new framework that augments the diffusion-based synthesis with differentiable physics simulation, methods for driving robot generation in a task-driven way toward improved physical utility, and extensive experiments in simulation to verify the effectiveness of DiffuseBot.\n", "questions": "In your paper and supplementary materials, you've provided a qualitative discussion on the challenges of translating simulated results into the fabrication of real robots based on the design developed in simulation. Could you delve deeper into this issue by providing more detailed, quantitative results that highlight the discrepancies between the behavior of the simulated robot and its real-world counterpart? Additionally, could you propose potential solutions aimed at minimizing this gap between simulation and reality?", "limitations": "The paper effectively addresses all identified limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper is robust and comprehensive in its approach. It introduces an innovative method that applies diffusion models to the co-design of robots, representing a significant contribution to the field. The authors have ensured thorough experimental coverage by testing their system, DiffuseBot, on a diverse range of tasks. This extensive testing underscores the versatility and applicability of the proposed method. Furthermore, the paper is not limited to theoretical constructs but extends to practical, real-world applications. The authors demonstrate this by providing a proof-of-concept 3D-printed real-world robot, thereby solidifying the relevance and potential of their research in real-world scenarios. ", "weaknesses": "The paper does not exhibit any significant shortcomings or areas of concern.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate reviewer 7Vvn for recognizing our paper as a robust, comprehensive and innovative work supported by extensive experiments and a proof-of-concept physical robot to demonstrate the potential of future research. We address the remaining suggestions as the below.\n\n**Simulation and physical robot: quantitative analysis and potential future solution.**\n\nThanks for bringing this extremely interesting question. To start off, we would like to highlight that the main focus of our work is to showcase the fascinating possibility of applying diffusion-based generative models to soft robot co-design by augmenting physics, and the hardware experiment is more of a proof-of-concept that minimally demonstrates the potential. The sim-to-real issue in soft robots involves materials, actuation, fabrication, and many other factors, and is an open and extremely challenging research question.\n\nIn order to explore the quantitative gap between the behavior of the physical robot and the simulated robot, we conducted an experiment with the following conditions, where similar setups are commonly adopted in soft robot literature [A1]. The objective was to measure the change in distance between two tips when we pull/release two tendons - one for closing the gripper (flexion) and the other for opening it (extension). The tendons were pulled or released in increments and decrements of 2mm, and the results are depicted in Figure A1 in the one-page pdf in the global response.\n\nWhen contracting the tendon to flex or extend the fingers, both simulation and real robot results show log-shaped graphs. The pattern in the physical robot plot is a commonly observed phenomenon called hysteresis. However, the main difference between the simulation and real-world cases can be seen when releasing the tendon from a fully contracted state. In the real robot experiment, the tip distance changes rapidly, while in the simulation, the opposite effect is observed.\n\nOne plausible explanation for this disparity could be attributed to the friction direction and elongation of the tendons. During the transition from tendon contraction to tendon release, the tension of the tendon at the end-effector may change suddenly due to the change of the friction direction. Also, since we only control the motor position (not the tendon position) to pull/release the tendon with 2mm step, the exact tendon length may not be exactly the same when we consider the tendon elongation.\n\nGiven that the gap between simulation and real robot performance seems to originate from the actuation/transmission method, our future work will focus on developing a tendon-driven actuation simulation framework. This framework aims to address the differences and improve the accuracy of our simulations. We are exploring other possible explanations for the sim-to-real gap and will investigate any additional factors that may contribute to the observed discrepancies. Overall, as for a high-level general solution, we believe (1) adjusting parameters based on observed sim to real gap and repeat the design process or (2) building a more accurate physics-based simulation (which can be straightforwardly plug-and-played in DiffuseBot) can largely bridge the sim-to-real gap of fabricating physical robots; or more interestingly, connecting generative models to commercial-level design and simulation softwares.\n\n**References**\n\n[A1] Fang, B., Sun, F., Wu, L., Liu, F., Wang, X., Huang, H., Huang, W., Liu, H. and Wen, L., 2022. Multimode grasping soft gripper achieved by layer jamming structure and tendon-driven mechanism. Soft Robotics, 9(2), pp.233-249.\n"}, {"review_id": "u78CXxuHHt", "reviewer": "Reviewer_zr6A", "summary": "This paper introduces DiffuseBot, a physics-augmented diffusion model designed for generating and optimizing the morphologies and control mechanisms of soft robots. DiffuseBot aims to bridge the gap between virtually generated content and physical utility in the domain of soft robotics. Firstly, it combines the diffusion process with a physical simulation that serves as a performance certificate, thereby ensuring the feasibility and effectiveness of the generated designs. Secondly, it details a co-design procedure that simultaneously optimizes the physical design and control of the soft robots, leveraging insights from differentiable simulation. The paper validates the efficacy of this approach by presenting a variety of both simulated and physically fabricated robots, along with their diverse capabilities.", "questions": "\n1. I do not fully understand how the k-means clustering is performed for actuator and stiffness generation. Specifically, what kind of feature is used for clustering?\n\n2. In line 86, which structural biases are you referring to ?\n", "limitations": "see weakness.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. In general, the paper is well written, with only minor flaws. Even those unfamiliar with soft robot design will find the paper easy to comprehend.\n\n2. Although diffusion models are expressive and powerful, their performance for tasks dealing with physical tasks often falls short. Thus, injecting a physics prior or 'physics-augmented diffusion model' is crucial. I think the method proposed in this paper is interesting and promising. \n\n3. The evaluation is comprehensive and thoughtful. The physical robot is impressive.\n", "weaknesses": "Overall, I did not identify any major weaknesses in the paper, but here are a few points that could strengthen it:\n\n1. While the writing is generally clear, certain sections could benefit from clearer exposition, such as:\n*  The section on diffusion as co-design is not very intuitive, especially for audiences not familiar with soft robot design. Specifically, it should be clearer how gradient-based optimization benefits robot design and what exactly line 152's \"synergy\" means.\n* It would be helpful if the authors clarify that the \"condition\" in this work actually refers to text.\n2. The robot's actuator and stiffness seem oversimplified, having only constant stiffness. Given that the gradient of $\\Psi_{act}$ is almost zero, it appears that the actuator and stiffness are solely determined by the geometry.\n3. A similar idea of tuning in the embedding space is proposed in[1]. A discussion and connection to this existing work could be interesting.  \n4. In general, the method the paper uses to inject a physics prior into the generation process could be applicable to more general scenarios. Works like Diffuser[2] or Decision Diffuser[3] generate state sequences with diffusion models, but the generated states can sometimes be physically implausible. A deeper discussion about the potential of the method could make the paper stronger.\n\n[1] Gal, Rinon, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik and Daniel Cohen-Or. “An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion.”, ICLR, 2023.\n\n[2] Janner, Michael, Yilun Du, Joshua B. Tenenbaum and Sergey Levine. “Planning with Diffusion for Flexible Behavior Synthesis.”, ICML, 2022. \n\n[3] Ajay, Anurag, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, T. Jaakkola and Pulkit Agrawal. “Is Conditional Generative Modeling all you need for Decision-Making?” ICLR, 2023. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer zr6A for acknowledging that our approach is interesting and promising. We address remaining questions as the below.\n\n**Clearer exposition in diffusion as co-design.**\n\nGradient-based optimization is shown to achieve more efficient and effective design search in soft robot co-design [19,25,44,48], especially with soft robots having a continuum of bodies and non-rigid contact. Thus, we aim to connect gradient-based optimization to diffusion-based generative processes in our work.\n\nThe “synergy” in line 152 is between diffusion models and energy-based models, not robot co-design; it draws a connection to MCMC sampling in energy-based models and makes the update of diffusion process more “gradient-descent-like”. This allows us to elegantly formulate gradient-based optimization, which is commonly used in soft robot co-design, in the diffusion-based generative process. Please refer to [11,12,42] for more details about the theory and the appendix section D paragraph “Connection to MCMC” for more complete theoretical motivation.\n\nMore precisely, the condition refers to the embedding either from text inputs, or from image inputs, or directly optimized in the Embedding Optimization stage to achieve physical utility. \n\nWe will improve the exposition in section 2.4 paragraph “Diffusion as Co-design” in the revision.\n\n**Actuator and stiffness.**\n\nThe goal of DiffuseBot is to demonstrate the potential of using diffusion models to generate soft robot design and to leverage the knowledge of the pre-trained generative models learned from a large-scale 3D dataset. Under this setting, the generated output of the diffusion model can only provide the geometry information of robot designs, leading to our design choice of having full dependency of actuator and stiffness on the geometry. This may be a reasonable simplification as prior works [48] have shown geometry along with properly-set actuator and stiffness (we take manual efforts to design proper mapping from geometry to actuator and stiffness in this work) roughly reflect the performance of a soft robot design. For better generality, one potential remedy is to optimize actuator and stiffness independently from the geometry generated by the diffusion model, i.e., apply DiffuseBot and do direct optimization on actuator and stiffness afterward or at the same time. Another interesting direction may be, for actuators, to leverage part-based models [A5] to decompose a holistic geometry into parts (or different actuator regions in soft robots).\n\n**The connection to [A1].**\n\nThere is some synergy between text inversion in [A1] and embedding optimization in DiffuseBot. Both of them aim at tuning the embedding toward reflecting certain properties of the output generation, i.e., describing the output generated images in [A1] and toward improved physical utility in DiffuseBot. The major difference lies in the nuance of the data/samples used to carry out the optimization. Text inversion performs a direct optimization using latent diffusion model loss (Eq. (2) in [A1]), which computes losses on noisy samples/latents corrupted from the real dataset. On the other hand, it is tricky to think about real dataset in robot design (as discussed in line 40-44 and line 125-130), embedding optimization in DiffuseBot computes losses on noisy samples corrupted from self-generated data filtered by robot performance (as in Algorithm 1 and section 2.4). Conceptually, it is more like a mixture of diffusion model training and online imitation learning like DAGGER [A4].\n\nWe will include this discussion in the revision.\n\n**Discussion on more general applications [A2,A3].**\n\nA potential and interesting way to adapt DiffuseBot to other applications like motion planning or control [A2,A3] is to view a generated robot as one snapshot/frame of a motion/state sequence and the physics prior can be the dynamics constraint across timesteps (e.g., robot dynamics or contact dynamics that enforce non-penetration). The physics prior can be injected similarly to diffusion as co-design that propagates the enforcement of physical plausibility of generated states from differentiable physics-based simulation to diffusion samples. For example, considering states in two consecutive timesteps, we can compute loss in the differentiable simulation to measure the violation of physical constraints regarding robot dynamics or interaction with the environment. Then, we can compute gradients with respect to either control or design variables; for gradients in control, this will essentially augment works like [A2,A3] with classifier-based guidance to achieve physical plausibility; for gradients in design, this will much resemble optimizing toward the motion sequence of a shape-shifting robot.\n\nWe will include this discussion in the revision.\n\n**Features used in k-means clustering.**\n\nWe use the 3D coordinates offset by the center of the geometry as the feature for k-means clustering. We will make it clearer in line 120 in the revision.\n\n**Structural biases in line 86.**\n\nThe structural biases refer to the knowledge in the pre-trained 3D generative models learned from large-scale 3D datasets. The idea is, instead of searching for good robot designs from scratch, explore in the space of what a 3D generative model has learned, which provides biases toward diverse and sensible 3D structures. We will provide a clearer description in the revision.\n\n**References**\n\n[A1] Gal et al. “An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion.”, ICLR, 2023.\n\n[A2] Janner et al. “Planning with Diffusion for Flexible Behavior Synthesis.”, ICML, 2022.\n\n[A3] Ajay et al. “Is Conditional Generative Modeling all you need for Decision-Making?” ICLR, 2023.\n\n[A4] Ross et al. A reduction of imitation learning and structured prediction to no-regret online learning. AISTATS 2011.\n\n[A5] Kaiser et al. A survey of simple geometric primitives detection methods for captured 3D data. CG 2019.\n"}, {"review_id": "LV473OI8Vl", "reviewer": "Reviewer_syw2", "summary": "This paper presents DiffuseBot, a framework that uses physics-augmented diffusion models to generate soft robot designs and control strategies for various tasks. The authors propose to optimize the embeddings conditioned by the diffusion model to improve the physical utility of the generated robots, and to reformulate the diffusion sampling process as a co-design optimization that leverages differentiable simulation. The authors demonstrate the effectiveness of their method on several tasks, such as balancing, landing, crawling, hurdling, gripping, and moving objects. They also show how to incorporate human feedback and fabricate a physical robot prototype.", "questions": "- Can limitations of the current pipeline be discussed and included in the paper?\n- Can the paper includes discussions about manufacturing?\n- How design choices are set? How sensitive is the current pipeline to hyper-parameters?\n- How to interpret the metric reported in Table 1, 2? \"We report the average performance with standard deviation in the superscript.\" -- Is the performance success rate?", "limitations": "Please include a section about limitations. ", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written and clear. The proposed method is novel and interesting, combining diffusion models for shape generation, physics-based simulation and co-design optimization. The paper provides extensive experimental comparisons with baselines and ablation studies on both latent optimization and co-design to validate the proposed method. The paper also shows some fun qualitative results of diverse robot designs that function under passive dynamics, locomotion tasks and manipulation tasks. ", "weaknesses": "- The paper does not discuss the limitations or failure scenarios of the proposed method. In addition, discussions on design choices can be help: how hyper-parameters are chosen, such as the guidance scale, or the number of MCMC steps? \n- Figure 1 shows that the motivation of this work is to deploy the optimized soft robot into the real world. However, though I may miss it, I did not find discussions in the paper about the feasibility of manufacturing the resulting soft robots, such as the soft gripper. Given that the actuators are currently assumed to be muscle fibers, it can be hard for manufacturing. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer syw2 recognizing our work as well-written, novel, and with extensive results. We address the remaining questions as the below.\n\n**Limitations or failure scenarios.**\n\nPlease check the paragraph about limitations in the global response.\n\n**Hyper-parameter choices.**\n\nHyperparameters are chosen mostly based on intuition and balancing numerical scale with very little tuning. In the following, we briefly discuss the design choices of all hyperparameters listed in Table 5 and 6 in the appendix. For min buffer size, samples per epoch, training iteration per epoch, and batch size, we roughly make sufficiently diverse the data used in the optimization and use the same setting for all tasks. For buffer size, we start with 60 and if we observe instability in optimization, we increase to 10 times, 600 (similar to online on-policy reinforcement learning); note that buffer size refers to the maximal size and increasing this won’t affect runtime. For buffer Top-K, we start with 6 and if we observe limited diversity of generation throughout the optimization (or lack of exploration), we double it. For $t_{max}$, $t_{min}$, and $\\Delta t$ (we made some typos in Table 6, all 60’s should be 50’s), we roughly inspect how structured the generation in terms of achieving the desired robotic task to determine $t_{max}$ and modify $\\Delta t$ accordingly to match the similar number of performing MCMC sampling (e.g., $t_{max}$/$\\Delta t$: 400 / 50 $\\approx$ 150 / 25). For the number of MCMC steps K, we simply set 3 for passive tasks and 5 for active tasks by intuition. For $\\sigma$, we simply follow one of the settings in [11]. For the guidance scale $\\kappa$ and renorm scale, we check the numerical values between $\\epsilon$ and gradient from differentiable simulation and try to make them roughly in the similar magnitude, and set the same scale for all tasks for simplicity. For $\\gamma$, we set 0.001 for trajectory optimization and 0.01 for parameterized controllers based on our experience of working with differentiable physics. Overall, from our empirical findings, the only hyperparameters that may be sensitive include buffer size and buffer Top-K for optimization stability and generation diversity, and guidance scales, which need to be tuned to match the numerical magnitude of other terms so as to take proper effect.\n\nWe will include the above descriptions in the appendix section C.\n\n**Details on robot manufacturing.**\n\nThe details of the physical robot experiment and the manufacturing is in the appendix section G. We describe how to build muscle fiber with tendon-driven actuators, how to achieve soft bodies with lattice structure, and how to fabricate the physical robot with a carbon 3D printer. Please find more details in the section G along with videos of using the soft gripper to pick up various types of objects in the project site (link shown in line 685). In addition, we further conduct a simple quantitative analysis on the behavior of simulation and the physical robot. Please check out more details in the response to reviewer 7Vvn.\n\nAlthough, at present, the compilation of the virtual robot to a physical, digitally fabricated counterpart involves manual post-processing of algorithm's output, most, if not all of these steps could be automated. Our method outputs a point cloud (defining geometry), actuator placements, and an open-loop controller, along with a prescribed stiffness. Since we can easily convert the point cloud into a 3D triangle mesh, the geometry can be created by almost any 3D printing method. In order to realize an effective stiffness and material behavior, stochastic lattices, specifically Voronoi foams, have been used [A1,A2] in the past and employed here in order to match target material properties.  Given the actuator placement, tendons [A3,A4] can be aligned with the prescribed (contiguous) regions. Since a lattice is used, threading tendons through the robot body is simple, and we note that even more complex routings have been studied in detail in the literature [A5]. Creating attachment points for the tendons is a relatively simple geometry processing problem [A6]. Thus, converting a virtual robot to a specification that incorporates geometry, material, and actuation can be automated in a straightforward way.\n\nWe note that when controlled, the physical robot may not always match the virtual robot's motion.  This is the sim-to-real gap, and is significantly harder to overcome in our case than translating the virtual robot to physical hardware.  Significant literature has been invested in specifically tackling the sim-to-real gap, and in our case would require its own dedicated project; however, we note that often hardware can be adapted to work by modifying only the control policies using feedback from real-world experiments, often even with little human intervention [A7].\n\n**Metrics.**\n\nThey are more of a soft version of success rate. The definition of the metric in Table 1 and 2 is in the appendix section D. We will add a pointer at line 182 in section 3.1 as,\n\n“We refer the reader to the appendix Section D for more detailed task descriptions and performance metrics.”\n\n**References**\n\n[A1] Martínez et al. \"Procedural voronoi foams for additive manufacturing.\" TOG 2016.\n\n[A2] Goswami et al. 3D‐architected soft machines with topologically encoded motion. Advanced functional materials 2019.\n\n[A3] In et al. A novel slack-enabling tendon drive that improves efficiency, size, and safety in soft wearable robots. ToM 2016.\n\n[A4] Kim et al. Slider-tendon linear actuator with under-actuation and fast-connection for soft wearable robots. ToM 2021.\n\n[A5] Bern et al. \"Interactive design of animated plushies.\" TOG 2017.\n\n[A6] Chen et al. Encore: 3D printed augmentation of everyday objects with printed-over, affixed and interlocked attachments. UIST 2015.\n\n[A7] Ha et al. Learning to Walk in the Real World with Minimal Human Effort. CoRL 2021."}, {"review_id": "q8AFj3ziVc", "reviewer": "Reviewer_vV19", "summary": "This work proposed a new framework that augments the diffusion-based synthesis with physical dynamical simulation in order to generatively co-design task-driven soft robots in morphology and control. The extensive experiments in simulation to verify the effectiveness of DiffuseBot.", "questions": "Please give more detailed clarification and experiments results to address my concerns. ", "limitations": "1. The novelty is limited and the motivation of using diffusion model is not well illustrated.\n2. The baseline is weak and old, the evaluation way is not agreed. The ablation studies are not enough.\n3. The role and the improvement bring by physic-augmented component is not well studied.\n4. The limitation part is absent in the submitted version. I hope the authors can discuss the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow.\n2. The visualization is good and help me understand how the method works.\n3. The application of diffusion is always be encouraged and diffusion model is an interesting and promising backbone.\n\n", "weaknesses": "1. The novelty is limited and more like an incremental work that applied diffusion model into 3D soft body generation task. The relationship and difference with  Softzoo mentioned in the paper should be more clearly illustrated.\n2. The baseline for comparison is relatively weak, consisting of some outdated works from a few years ago. The paper should includes more strong and recent baselines.\n3. The evaluation way is not agreed. 'To avoid being trapped in the local optimum, we run each baseline with 20 different random initializations and choose the best one. Since DiffuseBot is a generative method, we draw 20 samples and report the best' . If the baseline is sensitive to the initialization seed, more detailed results should be reported rather not choose the best one. And the 20 runs should report the mean and var rather not the best. The explanation about the generative model is not convincing enough for me.\n4. The novelty is limited and more like an incremental work applying diffusion model in 3D generation domain. \n5. The physics augmented component is not included in ablation studies and this will harm the convincing conclusion in the paper.\n6. Why diffusion not other generative model, such as VAE or transformer? The ablation studies should be listed or the paper will be likely the incremental work.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer vV19 for bringing up several concerns. We provide additional experimental results (see the one-page pdf in the global response) and discussion as the below.\n\n**Limited novelty.**\n\nWe aim to design robots with physical function and simple pattern generation will not get us there. We hence propose an entirely new framework by leveraging the computational power of a physics simulator to achieve meaningful results. Generative models are far from sufficient as:\n- The generated contents of most large-scale pre-trained generative models don’t reason about physics and fail to achieve any physical utility or functionality.\n- Unlike most training of generative modeling, computational robot design normally doesn’t have access to real data (which in our case, a dataset of well-performing robots). Instead, we need to leverage the physics-based simulation that evaluates the performance of a robot.\n\nPlease refer to 1-3 paragraphs in the introduction for more detailed justification. Thus, we make the following technical contributions,\n- introduce a new framework that augments the diffusion-based synthesis with physical dynamical simulation in order to generatively co-design task-driven soft robots in morphology and control (the last paragraph in the introduction).\n- propose a method to robotize 3D shapes from diffusion samples for meaningful evaluation in physics-based simulation (section 2.3).\n- present methods for driving robot generation toward improved physical utility by optimizing input embeddings and incorporating differentiable physics into the diffusion process (section 2.4).\n\n**Stronger baselines.**\n\nAs opposed to pure 3D generative modeling, soft robot co-design remains relatively unexplored. We believe most prior methods are covered as baselines for benchmarking, e.g., we adapt one of the most well-known yet a bit old method CPPN [49] to Diff-CPPN that can be used with the more recent and powerful techniques via differentiable physics. Nevertheless, we further compare with a very recent paper, DiffAqua [27]. Originally, we didn’t include this baseline since this method is designed for swimming tasks and lacks generality across a wide range of tasks. Briefly, DiffAqua proposes to compute Wasserstein barycenter among a set of primitives of underwater creatures. We report mean and standard deviation for all tasks in Table A1.\n\nWe can observe that DiffuseBot outperforms DiffAqua across all tasks. There is a natural tradeoff in the method between choosing a larger set of primitives for potentially better performance across diverse tasks and obtaining good solutions in Wasserstein barycenter optimization. To this end, we believe leveraging the power of large-scale pre-trained 3D generative models remains a more scalable and general method toward soft robot co-design.\n\n**Report the mean and var.**\n\nWe report the best results since normally the soft robot co-design problem expects to only produce one final robot design that can achieve high performance for a certain task (somewhat similar to other applications like drug discovery). To make the analysis more thorough, we report the mean and standard deviation in Table A2. Most results lead to conclusions that are consistent with Table 2 (using the best), except for hurdling, where the implicit function (IF) gives slightly better performance. However, IF is much more unstable, indicated as the much larger 0.63 standard deviation.\n\n**Ablation on physics components.**\n\nThe ablation for the physics-augmenting components is shown in Table 1, Table 3, and Table 4 (this is the figure with caption “Varying starting...”; it is incorrectly labeled as table).\n- In Table 1, we show the results of improved physical utility by augmenting physical simulation with diffusion models; specifically, we demonstrate how the 3D diffusion model (Point-E) works poorly (1st row), and how the proposed components in DiffuseBot greatly enhance the task performance (2nd, 3rd rows). Please refer to section 3.2 paragraph “Physics-augmented diffusion” for more detailed discussion.\n- In Table 3 and Table 4, we conduct more fine-grained ablation studies on embedding optimization and diffusion as co-design. Please refer to section 3.3 for more detailed discussion.\n\n**Baselines such as VAE or transformer.**\n\nWhile there exist many 3D generative models other than diffusion-based models (including VAE [A1,A2], normalizing flow [A3], GAN [A4], etc.), it is non-trivial to incorporate physics priors into the generative process. Compared to other generative models, diffusion models allow us to elegantly build theoretical constructs to incorporate external knowledge like physic-based simulation as guidance throughout the iterative generative process, which is also the major technical contribution in DiffuseBot. Besides, diffusion models have emerged as the de-facto of content generation, inspiring our work to harness such power to soft robot design application.\n\nTo further strengthen the paper, in Table A3, we compare with a recent VAE-based 3D generative model [A1], which outperforms other widely adopted baselines [A2-A6]. Given it is an open question to augment physics in VAE-based models (and, in fact, any other generative models), we perform direct optimization of co-design on the generated samples of [A1] to leverage physics-based simulation. With more advanced ways to inject physics prior into generative process as proposed in DiffuseBot, much superior performance can be achieved. Lastly, DiffuseBot uses a transformer-based architecture to generate 3D point clouds as in Point-E.\n\n**Limitations.**\n\nPlease check the paragraph about limitations in the global response.\n\n**References**\n\n[A1] Cheng. Autoregressive 3d... ECCV 2022.\n\n[A2] Kim. Setvae: Learning hierarchical... CVPR 2021.\n\n[A3] Yang. Pointflow: 3d point... ICCV 2019.\n\n[A4] Wu. Multimodal shape... ECCV 2020.\n\n[A5] Luo. Diffusion...point cloud generation. CVPR 2021.\n\n[A6] Zhou. 3d..point-voxel diffusion. ICCV 2021."}]

Author rebuttal:
We thank all reviewers for their thoughtful and constructive feedback. We are encouraged to hear the reviewers acknowledge,
- that the proposed approach is robust, innovative, and extends beyond theoretical construct to practical, real-world applications (reviewer 7Vvn), interesting and promising as a crucial solution to inject a physics prior to diffusion models (reviewer zr6A), novel and interesting (reviewer syw2), and introduces a new framework that augments diffusion-based synthesis with physical dynamical simulation (reviewer 5Fqn);
- that the paper is well-written and easy to follow with overall clarity (reviewer zr6A, syw2, vV19);
- that the results verify the effectiveness of DiffuseBot (reviewer 5Fqn), are with thorough experimental coverage and extensive testing (reviewer 7Vvn), are comprehensive and thoughtful (reviewer zr6A), and provides extensive comparisons with baselines and ablation studies (reviewer syw2);
- that the proof-of-concept physical robot solidifies the relevance and potential of the research in real-world scenarios (reviewer 7Vvn), is impressive (reviewer zr6A). 

In response to feedback, we provide individual responses below to address the remaining concerns from each reviewer to improve clarity of missing details and to provide additional discussion that strengthen our paper. Briefly, we summarize the added experiments and revision to the paper,
- Add quantitative analysis on the behavior of simulation and physical robots along with further discussion on physical robot fabrication.
- Add a paragraph for the discussion on limitations.
- Comparison to an additional baseline of a more recent soft robot co-design method.
- Comparison to an additional baseline of a VAE-based generative model.
- Report additional statistics including mean and standard deviation for baseline comparison.
- Add more clarification to the paper and discussion on relevant works.

For more details, please check individual responses. We thank all reviewers’ for their time and efforts! We hope our responses have persuasively addressed all remaining concerns. Please don’t hesitate to let us know of any additional comments or feedback on improvement.

Note that we include all additional experimental results in the one-page pdf submitted along with this global rebuttal response.

**A paragraph dedicated to limitations.** Re reviewer 5Fqn, syw2, vV19. We will add this to the revision.

“The major limitation of DiffuseBot is that we make a simplification in the parameterization of actuators and stiffness; we make dependencies of the two design specifications on robot geometry (check more technical details in section 2.3 paragraph Actuators and Stiffness. This works well with properly-crafted mapping from geometry to the others yet limits the potential by human prior with little use of the generative power. While this may be reasonable as properly-set actuators and stiffness based on geometry (hand-tuned empirically in this work) roughly reflects task performance, a more flexible parameterization can definitely lead to improved performance. Potential remedy can be using part-based 3D generative models for actuators and direct optimization for stiffness. Another limitation is the gap between simulated results and real robots. While the hardware experiment has shown as a proof-of-concept that minimally demonstrates the potential, physical robot fabrication and real-world transfer have countless non-trivial challenges including stiffness and actuator design, sim-to-real gap, etc. This may require studies on more high-fidelity physics-based simulation, which can be straightforwardly plugged into DiffuseBot.”


Meta-review: This paper presents DiffuseBot, a physics augmented diffusion model that can be used to design soft robots and the control in simulation and in the real-world. The initial reviews were mostly positive. The concerns raised in the reviews were sufficiently addressed in the rebuttal. All reviewers unanimously voted for acceptance. Please incorporate the contents in the rebuttal and reviewer's suggestions into the final version of this paper.
Decision: Accept (oral)