Title: DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models
Venue: NeurIPS 2023
Accepted: yes
Keywords: soft robot

diffusion model

co-design
Abstract: Nature evolves creatures with a high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy.  Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. \name bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and (ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation.  We showcase a range of simulated and fabricated robots along with their capabilities. Check our website: https://diffusebot.github.io/

DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models

Abstract

Nature evolves creatures with a high complexity of morphological and behavioral intelligence, meanwhile computational methods lag in approaching that diversity and efficacy. Co-optimization of artificial creatures' morphology and control in silico shows promise for applications in physical soft robotics and virtual character creation; such approaches, however, require developing new learning algorithms that can reason about function atop pure structure. In this paper, we present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. We showcase a range of simulated and fabricated robots along with their capabilities.

1 Introduction

Designing dynamical virtual creatures or real-world cyberphysical systems requires reasoning about complex trade-offs in system geometry, components, and behavior. But, what if designing such systems could be made simpler, or even automated wholesale from high-level functional specifications? Freed to focus on higher-level tasks, engineers could explore, prototype, and iterate more quickly, focusing more on understanding the problem, and find novel, more performant designs. We present DiffuseBot , a first step toward efficient automatic robotic and virtual creature content creation, as an attempt at closing the stubborn gap between the wide diversity and capability of Nature vis-a-vis evolution, and the reiterative quality of modern soft robotics.

Specifically, we leverage diffusion-based algorithms as a means of efficiently and generatively co-designing soft robot morphology and control for target tasks. Compared with with previous approaches, DiffuseBot 's learning-based approach maintains evolutionary algorithms' ability to search over diverse forms while exploiting the efficient nature of gradient-based optimization. DiffuseBot is made possible by the revolutionary progress of AI-driven content generation, which is now able to synthesize convincing media such as images, audio, and animations, conditioned on human input. However, other than raw statistical modeling, these methods are typically task- and physics-oblivious, and tend to provide no fundamental reasoning about the performance of generated outputs. We provide the first method for bridging the gap between diffusion processes and the morphological design of cyberphysical systems, guided by physical simulation, enabling the computational creative design of virtual and physical creatures.

,

Physical Robot In The Real World

Physics-based Simulation

Generative AI: Diffusion Models

Augmented Physical Utility:

Geometry + Stiffness + Actuator

While diffusion methods can robustly sample objects with coherent spatial structure from raw noise in a step-by-step fashion, several roadblocks preventing existing generative algorithms from being directly applied to physical soft robot co-design. First, while existing diffusion methods can generate 2D or 3D shapes, useful for, say, sampling robot geometry, they do not consider physics, nor are they directly aligned with the robotic task performance. As an alternative approach, one might consider learning a diffusion model supervised directly on a dataset of highly-performant robot designs mapped to their task performance. This leads us to the second roadblock, that is, that no such dataset exists, and, more crucially, that curating such a dataset would require a prohibitive amount of human effort and would fail to transfer to novel tasks outside that dataset.

To tackle these challenges, we propose using physical simulation to guide the generative process of pretrained large-scale 3D diffusion models. Diffusion models pretrained for 3D shapes provide an expressive base distribution that can effectively propose reasonable candidate geometries for soft robots. Next, we develop an automatic procedure to convert raw 3D geometry to a representation compatible with soft body simulation, i.e. one that parameterizes actuator placement and specifies material stiffness. Finally, in order to sample robots in a physics-aware and performance-driven manner, we apply two methods that leverage physically based simulation. First, we optimize the embeddings that condition the diffusion model, skewing the sampling distribution toward betterperforming robots as evaluated by our simulator. Second, we reformulate the sampling process that incorporates co-optimization over structure and control. We showcase the proposed approach of DiffuseBot by demonstrating automatically synthesized, novel robot designs for a wide spectrum of tasks, including balancing, landing, crawling, hurdling, gripping, and moving objects, and demonstrate its superiority to comparable approaches. We further demonstrate DiffuseBot 's amenability to incorporating human semantic input as part of the robot generation process. Finally, we demonstrate the physical realizability of the robots generated by DiffuseBot with a proof-of-concept 3D printed real-world robot, introducing the possibility of AI-powered end-to-end CAD-CAM pipelines.

In summary, we contribute:

A new framework that augments the diffusion-based synthesis with physical dynamical simulation in order to generatively co-design task-driven soft robots in morphology and control.

Methods for driving robot generation in a task-driven way toward improved physical utility by optimizing input embeddings and incorporating differentiable physics into the diffusion process.

Extensive experiments in simulation to verify the effectiveness of DiffuseBot, extensions to textconditioned functional robot design, and a proof-of-concept physical robot as a real-world result.

2 Method

In this section, we first formulate the problem (Section 2.1) and then describe the proposed DiffuseBot framework, which consists of diffusion-based 3D shape generation (Section 2.2), a differentiable procedure that converts samples from the diffusion models into soft robots (Section 2.3), a technique to optimize embeddings conditioned by the diffusion model for improved physical utility (Section 2.4), and a reformulation of diffusion process into co-design optimization (Section 2.4).

2.1 Problem Formulation: Soft Robot Co-design

Soft robot co-design refers to a joint optimization of the morphology and control of soft robots. The morphology commonly involves robot geometry, body stiffness, and actuator placement. Control is the signal to the actuators prescribed by a given robot morphology. It can be formally defined as,

t

Physics-Augmented

3D Diffusion Model

Noise

!

!

Intermediate

Sample

!

"

3D Shape

!

#

Embeddings

Diffusion as

Co-design

Embedding Optimization

Robotizing

Diffusion Samples

↕

+ Solid Geometry

+ Body Stiffness

+ Actuators

Physics-based

Simulation For

[Training] Embedding

Optimization

[Sampling] Diffusion

as Co-design

!

"$%

!

"

!

"&%

3D Shape

Online

Dataset

Intermediate

Sample

Task

Gradient

A

B

A

B

where Ψ is robot morphology that includes geometry Ψ geo , stiffness Ψ st, and actuator placement Ψ act ; u h is actuation with the controller's parameters φ and dependency on the robot morphology Ψ , s h is the simulator state, f is the environmental dynamics (namely the continuum mechanics of soft robots), and H is robot time horizon (not to be confused with the later-on mentioned diffusion time). Co-design poses challenges in optimization including complex interdependencies between body and control variables, ambiguity between competing morphology modifications, trade-offs between flexibility and efficacy in design representations, etc. [61]. In this work, we aim at leveraging the generative power of diffusion models in searching for optimal robot designs with (1) the potential to synthesize highly-diverse robots and (2) inherent structural biases in the pre-trained 3D generative models learned from large-scale 3D datasets to achieve efficient optimization.

2.2 3D Shape Generation with Diffusion-based Models

Diffusion-based generative models [24, 52] aim to model a data distribution by augmenting it with auxiliary variables { x t } T t =1 defining a Gaussian diffusion process p ( x 0 ) = ∫ p ( x T ) ∏ T t =1 p ( x t -1 | x t ) d x 1: T with the transition kernel in the forward process q ( x t | x t -1 ) = N ( x t ; √ 1 -β t x t -1 , β t I ) for some 0 < β t < 1 . For sufficiently large T , we have p ( x T ) ≈ N ( 0 , I ) . This formulation enables an analytical marginal at any diffusion time x t = √ ¯ α t x 0 + √ 1 -¯ α t /epsilon1 based on clean data x 0 , where /epsilon1 ∼ N ( 0 , I ) and ¯ α t = ∏ t i =1 1 -β i . The goal of the diffusion model (or more precisely the denoiser /epsilon1 θ ) is to learn the reverse diffusion process p ( x t -1 | x t ) with the loss,

Intuitively, /epsilon1 θ learns a one-step denoising process that can be used iteratively during sampling to convert random noise p ( x T ) gradually into realistic data p ( x 0 ) . To achieve controllable generation with conditioning c , the denoising process can be slightly altered via classifier-free guidance [25, 12],

where s is the guidance scale, ∅ is a null vector that represents non-conditioning.

2.3 Robotizing 3D Shapes from Diffusion Samples

We adopt Point-E [39] as a pre-trained diffusion model that is capable of generating diverse and complex 3D shapes, providing a good prior of soft robot geometries. However, the samples from the diffusion model x t are in the form of surface point cloud and are not readily usable as robots to be evaluated in the physics-based simulation. Here, we describe how to robotize the diffusion samples

x

Ψ

and its gradient computation of the objective

d

d

x

L

t

=

∂

∂

Ψ

L

∂

Ψ

∂

geo

x

t

+

∂

∂

Ψ

L

∂

Ψ

∂

x

st

t

+

∂

∂

Ψ

L

∂

Ψ

∂

act

x

t

.

act

/negationslash

Solid Geometry. We use a Material Point Method (MPM)-based simulation [61], which takes solid geometries as inputs. This poses two obstacles: (1) conversion from surface point clouds into solid geometries, and (2) the unstructuredness of data in the intermediate samples x t , t = 0 . The second issue arises from the fact that the diffusion process at intermediate steps may produce 3D points that do not form a tight surface. First, we leverage the predicted clean sample at each diffusion time t ,

This approach is used in denoising diffusion implicit model (DDIM) [53] to approximate the unknown clean sample x 0 in the reverse process p ( x t -1 | x t , ˆ x 0 ) . Here, we use it to construct a better-structured data for simulation. Hence, we break down the robotizing process into x t ↦→ ˆ x 0 ↦→ Ψ with gradient components as ∂ Ψ · ∂ ˆ x 0 ∂ ˆ x 0 ∂ x t , where ∂ ˆ x 0 ∂ x t can be trivially derived from (4). To convert the predicted surface points ˆ x 0 into solid geometry, we first reconstruct a surface mesh from ˆ x 0 , and then evenly sample a solid point cloud Ψ geo within its interior. For mesh reconstruction, we modify the optimization approach from Shape As Points [40], which provides a differentiable Poisson surface reconstruction that maps a control point set V ctrl to a reconstructed surface mesh with vertices V mesh. We calculate a modified Chamfer Distance loss indicating similarity between V mesh and ˆ x 0 :

in which d ( · , · ) denotes minimal Euclidean distance between a point and a point set, and w ( v ) denotes a soft interior mask, with w ( v ) = 1 for v outside the mesh, and w ( v ) = 0 . 1 for v inside. The introduced mask term w ( v ) aims to lower the influence of noisy points inside the mesh, which is caused by imperfect prediction of ˆ x 0 from noisy intermediate diffusion samples. The weight parameters are set to λ mesh = 1 and λ target = 10 . We back-propagate ∂ L recon ∂ V mesh to ∂ L recon ∂ V ctrl through the differentiable Poisson solver, and then apply an Adam optimizer on V ctrl to optimize the loss L recon . After mesh reconstruction, the solid geometry Ψ geo represented by a solid interior point cloud is then sampled evenly within the mesh, with sufficient density to support the MPM-based simulation. Finally, to integrate the solidification process into diffusion samplers, we still need its gradient ∂ Ψ geo ˆ x 0 . We adopt Gaussian kernels on point-wise Euclidean distances as gradients between two point clouds:

Intuitively, under such Gaussian kernels gradients, each solid point is linearly controlled by predicted surface points near it. In practice, this backward scheme works well for kernel parameter α = 20 .

Actuators and Stiffness. A solid geometry does not make a robot; in order for the robot to behave, its dynamics must be defined. After sampling a solid geometry, we thus need to define material properties and actuator placement. Specifically, we embed actuators in the robot body in the form of muscle fibers that can contract or expand to create deformation; further, we define a stiffness parameterization in order to determine the relationship between deformation and restorative elastic force. We adopt constant stiffness for simplicity since it has been shown to trade off with actuation strength [61]; thus we have Ψ st (ˆ x 0 ) = const and ∂ Ψ st ∂ ˆ x 0 = 0 . Then, we propose to construct actuators based on the robot solid geometry Ψ act ( Ψ geo (ˆ x 0 )) via clustering; namely, we perform k-means with pre-defined number of clusters on the coordinates of 3D points from the solid geometry Ψ geo. The gradient then becomes ∂ Ψ act ∂ Ψ geo ∂ Ψ geo ∂ ˆ x 0 ∂ ˆ x 0 ∂ x t , where ∂ Ψ act ∂ Ψ geo ≈ 0 as empirically we found the clustering is quite stable in terms of label assignment, i.e., with ∆Ψ geo being small, ∆Ψ act → 0 . Overall, we keep only the gradient for the robot geometry as empirically it suffices.

2.4 Physics Augmented Diffusion Model

Embedding Optimization. To best leverage the diversity of the generation from a large-scale pre-trained diffusion models, we propose to (1) actively generate new data from model and maintain them in a buffer, (2) use physics-based simulation as a certificate of performance, (3) optimize the embeddings conditioned by the diffusion model under a skewed data distribution to improve robotic performance in simulation. Curating a training dataset on its own alleviates the burden of manual effort to propose performant robot designs. Optimizing the conditional embeddings instead of finetuning the diffusion model eliminates the risk of deteriorating the overall generation and saves the cost of storing model weights for each new task (especially with large models). We follow,

Note the three major distinctions from (2): (i) the optimization variable is the embedding c not θ (ii) the denoiser is conditioned on the embeddings /epsilon1 θ ( . . . , c ) and (iii) the data distribution is based on the diffusion model p θ not the inaccessible real data distribution p and is conditioned on the embeddings c . This adopts an online learning scheme as the sampling distribution is dependent on the changing c . The procedure is briefly summarized in Algorithm 1, where Filter is an operation to drop the oldest data when exceeding the buffer limit. In addition, during this stage, we use fixed prescribed controllers since we found empirically that a randomly initialized controller may not be sufficiently informative to drive the convergence toward reasonably good solutions; also, enabling the controller to be trainable makes the optimization prohibitively slow and extremely unstable, potentially due to the difficulty of the controller required to be universal to a diverse set of robot designs. After the embedding optimization, we perform conditional generation that synthesizes samples corresponding to robot designs with improved physical utility via classifier-free guidance as in (3).

Diffusion as Co-design. While the optimized embedding already allows us to generate performant robots for some target tasks, we further improve the performance of individual samples by reformulating the diffusion sampling process into a co-design optimization. As described in Section 2.3, we can convert the intermediate sample at any diffusion time x t to a robot design Ψ , rendering an evolving robot design throughout the diffusion process . However, regular diffusion update [24] much less resembles any gradient-based optimization techniques, which are shown to be effective in soft robot design and control with differentiable simulation [26, 2]. Fortunately, there is a synergy between diffusion models and energy-based models [54, 15, 14], which allows a more gradient-descent-like update with Markov Chain Monte Carlo (MCMC) sampling [14]. Incorporating the soft robot co-design optimization with differentiable physics [61] into the diffusion sampling process, we have

where /epsilon1 ∼ N ( 0 , I ) , x (0) t = x ( K ) t -1 , κ is the ratio between two types of design gradients, K is the number of MCMC sampling steps at the current diffusion time, γ is the weight for trading off design and control optimization, and φ (0) t can be either inherited from the previous diffusion time φ ( K ) t -1 or reset to the initialization φ (0) T . We highlight the high resemblance to gradient-based co-optimization with x t as the design variable and φ t as the control variable. This procedure is performed once every M diffusion steps (Algorithm 2), where M is a hyperparameter that trade-offs "guidance" strength from physical utility and sampling efficiency. Intuitively, the entire diffusion-as-co-design process is

Balancing

Landing

Crawling

Hurdling

Moving A Box

guided by three types of gradients: (i) /epsilon1 θ ( x ( k -1) t , · ) provides a direction for the design toward feasible 3D shapes based on the knowledge of pre-training with large-scale datasets (and toward enhanced physical utility with the optimized embeddings via classifier-free guidance using ˆ /epsilon1 θ ( x ( k -1) t , · , c ) ), (ii) ∇ x t L ( Ψ ( x t ) , · ) provides a direction for the design toward improving co-design objective L via differentiable simulation, and (iii) ∇ φ t L ( · , φ t ) provides a direction for the controller toward a better adaption to the current design x t that allows more accurate evaluation of the robot performance.

3 Experiments

3.1 Task Setup

We cover three types of robotics tasks: passive dynamics, locomotion, and manipulation (Figure 3).

Passive Dynamics tasks include balancing and landing. Balancing initializes the robot on a sticklike platform with small contact area with an upward velocity that introduces instability; the robot's goal is to passively balance itself after dropping on the platform. Landing applies an initial force to the robot toward a target; the robot's goal is to passively land as close to the target as possible.

Locomotion tasks include crawling and hurdling. Crawling sets the robot at a rest state on the ground; the robot must actuate its body to move as far away as possible from the starting position. Hurdling places an obstacle in front of the robot; the robot must jump over the obstacle.

Manipulation tasks include gripping and moving objects. Gripping places an object underneath the robot; the goal of the robot is to vertically lift the object. Box Moving places a box on the right end of the robot; the robot must move the box to the left.

Please refer to the appendix Section D for more detailed task descriptions and performance metrics.

3.2 Toward Physical Utility In Diffusion Models

Physics-augmented diffusion. In Table 1, we examine the effectiveness of embedding optimization and diffusion as co-design for improving physical utility. For each entry, we draw 100 samples with preset random seeds to provide valid sample-level comparison (i.e., setting the step size of co-design optimization to zero in the third row will produce almost identical samples as the second row). We report the average performance with standard deviation in the superscript. First, we observe increasing performance across all tasks while incorporating the two proposed techniques, demonstrating the efficacy of DiffuseBot. Besides, the sample-level performance does not always monotonically improve, possibly due to the stochasticity within the diffusion process and the low quality of gradient from differentiable simulation in some scenarios. For example, in gripping, when the robot fails to pick up the object in the first place, the gradient may be informative and fails to bring proper guidance toward better task performance; similarly in moving a box. In addition, we found it necessary to include control optimization during the diffusion sampling process, since, at diffusion steps further from zero, the predicted clean sample ˆ x 0 (derived from the intermediate sample x t ) may differ significantly from the clean sample x 0 , leaving the prescribed controller largely unaligned.

Comparison with baselines. In Table 2, we compare with extensive baselines of soft robot design representation: particle-based method has each particle possessing its own distinct parameterization of design (geometry, stiffness, actuator); similarly, voxel-based method specifies design in voxel level; implicit function [37] uses use a shared multi-layer perceptron to map coordinates to design;

Methods

Passive Dynamics

Locomotion

Manipulation

Balancing

Landing

Crawling

Hurdling

Moving a Box

Particle-based

0.040

.

000

0.863

.

005

0.019

.

001

0.006

.

001

-0.010

.

001

0.043

.

027

Voxel-based

0.040

.

000

0.853

.

002

0.024

.

000

0.027

.

000

-0.009

.

000

0.025

.

022

Implicit Function [37]

0.106

.

147

0.893

.

033

0.043

.

024

0.044

.

063

0.006

.

012

0.033

.

030

Diff-CPPN [18]

0.091

.

088

0.577

.

425

0.055

.

023

0.019

.

029

0.007

.

008

0.022

.

017

DiffAqua [35]

0.014

.

023

0.293

.

459

0.027

.

015

0.022

.

011

0.010

.

001

0.007

.

008

DiffuseBot

0.706

.

078

0.965

.

026

0.092

.

016

0.031

.

011

0.026

.

002

0.047

.

019

Crawling

1st Checkpoint

2nd Checkpoint

3rd Checkpoint

5th Checkpoint

心

DiffCPPN [18] uses a graphical model composed of a set of activation function that takes in coordinates and outputs design specification; DiffAqua [35] computes the Wasserstein barycenter of a set of aquatic creatures' meshes and we adapt to use more reasonable primitives that include bunny, car, cat, cow, avocado, dog, horse, and sofa. These baselines are commonly used in gradient-based soft robot co-design [26, 56, 61]. For each baseline method, we run the co-optimization routine for the same number of steps as in the diffusion-as-co-design stage in DiffuseBot. To avoid being trapped in the local optimum, we run each baseline with 20 different random initializations and choose the best one. Since DiffuseBot is a generative method, we draw 20 samples and report the best; this is sensible an applications-driven perspective since we only need to retrieve one performant robot, within a reasonable sample budget. We observe that our method outperforms all baselines. DiffuseBot leverages the knowledge of large-scale pre-trained models that capture the 'common sense' of geometry, providing a more well-structured yet flexible prior for soft robot design.

Soft robots bred by DiffuseBot. In Figure 5, we demonstrate the generated soft robots that excel in locomotion and manipulation tasks. We highlight the flexibility of DiffuseBot to generate highly diverse soft robot designs that accommodate various purposes in different robotics tasks. Furthermore, in Figure 4, we show how robots evolve from a feasible yet non-necessarily functional design to an improved one that intuitively matches the task objective. By manually inspecting the evolving designs, we found that the role of the embedding optimization is to drive the diverse generations toward a converged, smaller set with elements having higher chance to succeed the task; on the other hand, the role of diffusion as co-design brings relatively minor tweaks along with alignment between the control and design. Due to space limit, we refer the reader to our project page for more results.

Crawling

1st Frame

30th Frame

60th Frame

80th Frame

100th Frame

3.3 Ablation Analysis

In this section, we conduct a series of ablation studies to provide a deeper understanding of the proposed method. For simplicity, all experiments in this section are done with the crawling task.

Embedding optimization. In Table 3, we compare the optimization of the embedding conditioned by the diffusion models with other alternatives. The pre-trained diffusion model [39] that DiffuseBot is built upon uses CLIP embeddings [43], which allows for textual inputs. Hence, a naive approach is to manually design text for the conditional embedding of the diffusion model. The result reported in Table 3 uses 'a legged animal or object that can crawl or run fast' . We investigated the use of text prompts; in our experience, text was difficult to optimize for functional robot design purposes. This is expected since most existing diffusion models perform content generation only in terms of appearance instead of physical utility, which further strengthens the purpose of this work. In addition, with exactly the same training objective as in (5), we can instead finetune the diffusion model itself. However, this does not yield better performance, as shown in the second entry in Table 3. Empirically, we found there is a higher chance of the generated samples being non-well-structured with fractured parts. This suggests that finetuning for physical utility may deteriorate the modeling of sensible 3D shapes and lead to more unstable generations.

Diffusion as co-design. Recall that the co-design optimization can be seamlessly incorporated into any diffusion step. In Figure 6, we examine how the strength of the injected co-design optimization affects the task performance in terms of where to apply throughout the diffusion sampling process and how many times to apply. In the left figure of Figure 6, we sweep through the maximal diffusion time of applying diffusion as co-design, i.e., for the data point at t = 400 , we only perform co-design from t = 400 to t = 0 . We found that there is a sweet spot of when to start applying co-design (at t ≈ 200 ). This is because the intermediate samples at larger diffusion time x t , t /greatermuch 0 are extremely under-developed, lacking sufficient connection to the final clean sample x 0 , hence failing to provide informative guidance by examining its physical utility. Furthermore, we compare against post-diffusion co-design optimization, i.e., run co-design based on the final output of the diffusion ( 0 . 064 vs ours 0 . 081 ). We allow the same computational budget by running the same number of times of differentiable simulation as in DiffuseBot. Our method performs slightly better, potentially due to the flexibility to alter the still-developing diffusion samples. Also note that while our method is interleaved into diffusion process, it is still compatible with any post-hoc computation for finetuning.

3.4 Flexibility To Incorporate Human Feedback

Beyond the generative power, diffusion models also provide the flexibility to composite different data distributions. This is especially useful for computational design since it empowers to easily incorporate external knowledge, e.g. from human. We follow the compositionality techniques introduced in [34, 14], which can be directly integrated into our diffusion as co-design

Crawler + Human Feedback

'A Unicorn'

Crawler From DiffuseBot

framework. In Figure 7, we demonstrate incorporating human feedback in textual form as 'a unicorn' into a crawling robot generated by DiffuseBot. We can see the emergence of the horn-like body part.

3.5 From Virtual Generation To Physical Robot

We further fabricate a physical robot for the gripping task as a proof-of-concept to demonstrate the possibility of real-world extension. We use a 3D Carbon printer to reconstruct the exact geometry of a generated design and fill the robot body with a Voronoi lattice structure to achieve softness.

For actuators, we employ tendon transmission to realize the contraction force utilized in the soft robot gripper. In our project page, we demonstrate the robots generated by DiffuseBot are capable of picking up an object. Note that physical robot fabrication and real-world transfer have countless non-trivial challenges including stiffness and actuator design, sim-to-real gap, etc. Hence, this experiment is only meant to demonstrate the potential instead of a general, robust pipeline toward physical robots, which is left to future work. We refer the reader to the appendix for more details.

4 Related Work

Heuristic Search For Soft Robot Co-Design. Heuristic searches are simple but useful tools for codesigning soft robots. A long line of work has focused on evolutionary algorithms [7, 8, 10], with some including physical demonstrations [22, 31, 32] and recent benchmarks incorporating neural control [5]. These methods are often powered by parameterized by compositional pattern-producing networks [58], which parameterize highly expressive search spaces akin to neural networks [50, 51]. Similar to [5], [47] combines a heuristic approach with reinforcement learning, and demonstrates resulting designs on physical hardware. Other notable methods include particle-filter-based approaches [11] and simulated annealing [60]. Heuristic search methods tend to be less efficient than gradient-based or learning-based algorithms, but can reasoning about large search spaces; our approach employs the highly expressive diffusion processes, while leveraging the differentiable nature of neural networks and physical simulation for more efficient and gradient-directed search.

Gradient-Based Soft Robot Co-Optimization. A differentiable simulator is one in which useful analytical derivatives of any system variable with respect to any other system variable is efficiently queryable; the recent advent of soft differentiable simulation environments [26, 56, 13, 33, 41, 42, 61] has accelerated the exploration of gradient-based co-optimiation methods. [26, 56] demonstrated how differentiable simulators can be used to co-optimize very high-dimensional spatially varying material and open-loop/neural controller parameters. [38] presented gradient-based search of shape parameters for soft manipulators. Meanwhile, [61] showed how actuation and geometry could be co-optimized, while analyzing the trade-offs of design space complexity and exploration in the search procedure. DiffuseBot borrows ideas from gradient-based optimization in guiding the design search in a physics-aware way, especially in the context of control.

Learning-Based Soft Robot Co-Design Methods. Though relatively nascent, learning-based approaches (including DiffuseBot ) can re-use design samples to build knowledge about a problem. Further, dataset-based minibatch optimization algorithms are more robust to local minima than single-iterate pure optimization approaches. [57] demonstrated how gradient-based search could be combined with learned-models; a soft robot proprioceptive model was continually updated by simulation data from interleaved control/material co-optimization. Other work employed learning-based methods in the context of leveraging available datasets. [35] learned a parameterized representation of geometry and actuators from basis shape geometries tractable interpolation over high-dimensional search spaces. [55] leveraged motion data and sparsifying neurons to simultaneously learn sensor placement and neural soft robotic tasks such as proprioception and grasp classification.

Diffusion Models for Content Generation. Diffusion models [24, 52] have emerged as the de-facto standard for generating content in continuous domains such as images [44, 46], 3D content [66, 65], controls [28, 9, 1], videos [23, 49, 16], and materials [63, 62, 48]. In this paper, we explore how diffusion models in combination with differentiable physics may be used to design new robots. Most similar to our work, [64] uses differentiable physics to help guide human motion synthesis. However, while [64] uses differentiable physics to refine motions in the last few timesteps of diffusion sampling, we tightly integrate differentiable physics wih sampling throughout the diffusion sampling procedure through MCMC. We further uses differentiable simulation to define a reward objective through which we may optimize generative embeddings that represent our desirable robot structure.

5 Conclusion

We presented DiffuseBot, a framework that augments physics-based simulation with a diffusion process capable of generating performant soft robots for a diverse set of tasks including passive dynamics, locomotion, and manipulation. We demonstrated the efficacy of diffusion-based generation with extensive experiments, presented a method for incorporating human feedback, and prototyped a physical robot counterpart. DiffuseBot is a first step toward generative invention of soft machines, with the potential to accelerate design cycles, discover novel devices, and provide building blocks for downstream applications in automated computational creativity and computer-assisted design.

References

Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657 , 2022.

Moritz Bächer, Espen Knoop, and Christian Schumacher. Design and control of soft robots using differentiable simulation. Current Robotics Reports , 2(2):211-221, 2021.

Nicholas W Bartlett, Michael T Tolley, Johannes TB Overvelde, James C Weaver, Bobak Mosadegh, Katia Bertoldi, George M Whitesides, and Robert J Wood. A 3d-printed, functionally graded soft robot powered by combustion. Science , 349(6244):161-165, 2015.

James M Bern, Kai-Hung Chang, and Stelian Coros. Interactive design of animated plushies. ACM Transactions on Graphics (TOG) , 36(4):1-11, 2017.

Jagdeep Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, and Wojciech Matusik. Evolution gym: A large-scale benchmark for evolving soft robots. Advances in Neural Information Processing Systems , 34:2201-2214, 2021.

Xiang'Anthony' Chen, Stelian Coros, Jennifer Mankoff, and Scott E Hudson. Encore: 3d printed augmentation of everyday objects with printed-over, affixed and interlocked attachments. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology , pages 73-82, 2015.

Nicholas Cheney, Jeff Clune, and Hod Lipson. Evolved electrophysiological soft robots. In Artificial life conference proceedings , pages 222-229. MIT Press One Rogers Street, Cambridge, MA02142-1209, USA journals-info ..., 2014.

Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution , 7(1):11-23, 2014.

Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. arXiv preprint arXiv:2303.04137 , 2023.

Francesco Corucci, Nick Cheney, Francesco Giorgio-Serchi, Josh Bongard, and Cecilia Laschi. Evolving soft locomotion in aquatic and terrestrial environments: effects of material properties and environmental transitions. Soft robotics , 5(4):475-495, 2018.

Raphael Deimel, Patrick Irmisch, Vincent Wall, and Oliver Brock. Automated co-design of soft hand morphology and control strategy for grasping. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1213-1218. IEEE, 2017.

Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems , 2021.

Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. Diffpd: Differentiable projective dynamics. ACM Transactions on Graphics (TOG) , 41(2):1-21, 2021.

Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc. arXiv preprint arXiv:2302.11552 , 2023.

Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903.08689 , 2019.

Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. arXiv e-prints , pages arXiv-2302, 2023.

Bin Fang, Fuchun Sun, Linyuan Wu, Fukang Liu, Xiangxiang Wang, Haiming Huang, Wenbing Huang, Huaping Liu, and Li Wen. Multimode grasping soft gripper achieved by layer jamming structure and tendon-driven mechanism. Soft Robotics , 9(2):233-249, 2022.

Chrisantha Fernando, Dylan Banarse, Malcolm Reynolds, Frederic Besse, David Pfau, Max Jaderberg, Marc Lanctot, and Daan Wierstra. Convolution by evolution: Differentiable pattern producing networks. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 , pages 109-116, 2016.

Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618 , 2022.

Debkalpa Goswami, Shuai Liu, Aniket Pal, Lucas G Silva, and Ramses V Martinez. 3darchitected soft machines with topologically encoded motion. Advanced Functional Materials , 29(24):1808713, 2019.

Sehoon Ha, Peng Xu, Zhenyu Tan, Sergey Levine, and Jie Tan. Learning to walk in the real world with minimal human effort. arXiv preprint arXiv:2002.08550 , 2020.

Jonathan Hiller and Hod Lipson. Automatic design and manufacture of soft robots. IEEE Transactions on Robotics , 28(2):457-466, 2011.

Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 , 2022.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840-6851, 2020.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 , 2022.

Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu, Daniela Rus, and Wojciech Matusik. Chainqueen: A real-time differentiable physical simulator for soft robotics. In 2019 International conference on robotics and automation (ICRA) , pages 6265-6271. IEEE, 2019.

Hyunki In, Useok Jeong, Haemin Lee, and Kyu-Jin Cho. A novel slack-enabling tendon drive that improves efficiency, size, and safety in soft wearable robots. IEEE/ASME Transactions on Mechatronics , 22(1):59-70, 2016.

Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning , 2022.

Adrien Kaiser, Jose Alonso Ybanez Zepeda, and Tamy Boubekeur. A survey of simple geometric primitives detection methods for captured 3d data. In Computer Graphics Forum , volume 38, pages 167-196. Wiley Online Library, 2019.

Byungchul Kim, Useok Jeong, Brian Byunghyun Kang, and Kyu-Jin Cho. Slider-tendon linear actuator with under-actuation and fast-connection for soft wearable robots. IEEE/ASME Transactions on Mechatronics , 26(6):2932-2943, 2021.

Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. A scalable pipeline for designing reconfigurable organisms. Proceedings of the National Academy of Sciences , 117(4):1853-1859, 2020.

Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh Bongard. Kinematic selfreplication in reconfigurable organisms. Proceedings of the National Academy of Sciences , 118(49):e2112672118, 2021.

Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojciech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Transactions on Graphics (TOG) , 42(1):1-20, 2022.

Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. arXiv preprint arXiv:2206.01714 , 2022.

Pingchuan Ma, Tao Du, John Z Zhang, Kui Wu, Andrew Spielberg, Robert K Katzschmann, and Wojciech Matusik. Diffaqua: A differentiable computational design pipeline for soft underwater swimmers with shape interpolation. ACM Transactions on Graphics (TOG) , 40(4):1-14, 2021.

Jonàs Martínez, Jérémie Dumas, and Sylvain Lefebvre. Procedural voronoi foams for additive manufacturing. ACM Transactions on Graphics (TOG) , 35(4):1-12, 2016.

Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4460-4470, 2019.

Thomas Morzadec, Damien Marcha, and Christian Duriez. Toward shape optimization of soft robots. In 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft) , pages 521-526. IEEE, 2019.

Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751 , 2022.

Songyou Peng, Chiyu "Max" Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger. Shape as points: A differentiable poisson solver. In Advances in Neural Information Processing Systems (NeurIPS) , 2021.

Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C Lin. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.02168 , 2020.

Yiling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Differentiable simulation of soft multi-body systems. Advances in Neural Information Processing Systems , 34:17123-17135, 2021.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Aditya Ramesh, Mikhail Pavlov, Scott Gray Gabriel Goh, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. arXiv preprint arXiv:2102.12092 , 2021.

Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics , 2011.

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487 , 2022.

Charles Schaff, Audrey Sedal, and Matthew R Walter. Soft robots learn to crawl: Jointly optimizing design and control with sim-to-real transfer. arXiv preprint arXiv:2202.04575 , 2022.

Arne Schneuing, Yuanqi Du, Arian Jamasb Charles Harris, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Michael Bronstein Max Welling, and Bruno Correia. Structure-based drug design with equivariant diffusion models. arXiv preprint arXiv:2210.02303 , 2022.

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 , 2022.

Lawrence Smith, Travis Hainsworth, Jacob Haimes, and Robert MacCurdy. Automated synthesis of bending pneumatic soft actuators. In 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft) , pages 358-363. IEEE, 2022.

Lawrence Smith and Robert MacCurdy. Soroforge: End-to-end soft actuator design. IEEE Transactions on Automation Science and Engineering , 2023.

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning , pages 2256-2265. PMLR, 2015.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 , 2020.

Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems , 32, 2019.

Andrew Spielberg, Alexander Amini, Lillian Chin, Wojciech Matusik, and Daniela Rus. Colearning of task and sensor placement for soft robotics. IEEE Robotics and Automation Letters , 6(2):1208-1215, 2021.

Andrew Spielberg, Tao Du, Yuanming Hu, Daniela Rus, and Wojciech Matusik. Advanced soft robot modeling in chainqueen. Robotica , 41(1):74-104, 2023.

Andrew Spielberg, Allan Zhao, Yuanming Hu, Tao Du, Wojciech Matusik, and Daniela Rus. Learning-in-the-loop optimization: End-to-end control and co-design of soft robots through learned deep latent representations. Advances in Neural Information Processing Systems , 32, 2019.

Kenneth O Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic programming and evolvable machines , 8:131-162, 2007.

Michael T Tolley, Robert F Shepherd, Michael Karpelson, Nicholas W Bartlett, Kevin C Galloway, Michael Wehner, Rui Nunes, George M Whitesides, and Robert J Wood. An untethered jumping soft robot. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 561-566. IEEE, 2014.

Merel Van Diepen and Kristina Shea. A spatial grammar method for the computational design synthesis of virtual soft locomotion robots. Journal of Mechanical Design , 141(10), 2019.

Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua B. Tenenbaum, Daniela Rus, and Chuang Gan. Softzoo: A soft robot co-design benchmark for locomotion in diverse environments. In The Eleventh International Conference on Learning Representations , 2023.

Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi S Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Representations , 2021.

Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, , and Jian Tang. GeoDiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations , 2021.

Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. arXiv preprint arXiv:2212.02500 , 2022.

Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. arXiv preprint arXiv:2210.06978 , 2022.

Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5826-5835, 2021.

A Background On Point-E

Point-E [39] is a diffusion-based generative model that produces 3D point clouds from text or images. The Point-E pipeline consists of three stages: first, it generates a single synthetic view using a text-to-image diffusion model; second, it produces a coarse, low-resolution 3D point cloud (1024 points) using a second diffusion model which is conditioned on the generated image; third, it upsamples/'densifies' the coarse point cloud to a high-resolution one (4096 points) with a third diffusion model. The two diffusion models operating on point clouds use a permutation invariant transformer architecture with different model sizes. The entire model is trained on Point-E's curated dataset of several million 3D models and associated metadata which captures a generic distribution of common 3D shapes, providing a suitable and sufficiently diverse prior for robot geometry. The diffused data is a set of points, each point possessing 6 feature dimensions: 3 for spatial coordinates and 3 for colors. We ignore the color channels in this work. The conditioning for the synthesized image in the first stage relies on embeddings computed from a pre-trained ViT-L/14 CLIP model; in the embedding optimization of DiffuseBot, the variables to be optimized is exactly the same embedding. Diffusion as co-design is only performed in the second stage (coarse point cloud generation) since the third stage is merely an upsampling which produces only minor modifications to robot designs. We refer the reader to the original paper [39] for more details.

B Theoretical Motivation

Online learning in embedding optimization. In Section 2.4, we discuss how to online collect a dataset to optimize the embedding toward improved physical utility. Given a simplified version of (5)

where, for notation simplicity, we drop t ∼ [1 , T ] , N ( /epsilon1 ; 0 , I ) in the sampling distribution, and summarize [ || /epsilon1 -/epsilon1 θ ( x t ( x 0 , /epsilon1 , t ) , t, c ) || 2 ] as g ( x , c ) . We can rewrite the expectation term as,

which allows to sample from p θ ( x 0 ) (i.e., generating samples from the diffusion model) and reweight the loss with p θ ( c | x 0 ) p θ ( c ) ; the latter scaling term is essentially proportional to a normalized task performance. Empirically, we can maintain a buffer for the online dataset and train the embedding with the sampling distribution biased toward higher task performance; we use a list to store samples with top-k performance in our implementation (we also tried reshaping the sampling distribution like prioritized experience replay in reinforcement learning but we found less stability and more hyperparameters required in training compared to our simpler top-k approach).

Connection to MCMC. In diffusion sampling, the simplest way to perform reverse denoising process as in Section 2.2 follows [24],

Here, the denoising term can either be unconditional /epsilon1 θ ( x t , t ) or conditional via classifier-free guidance ˆ /epsilon1 θ , classifier-free ( x t , t, c ) as in (3). We use the latter to incorporate the optimized embedding. To further leverage physics-based simulation, we aim to introduce physical utility during diffusion sampling process. One possibility is to utilize classifier-based guidance [12],

where p ( c | x t ) can be conceptually viewed as improving physical utility and ∇ x t log p ( c | x t ) can be obtained using differentiable physics and the unconditional score. Note that we slightly abuse the notation here by overloading c with conditioning from differentiable simulation during sampling other than the classifier-free guidance using the optimized embedding. However, combining (10) and (11) much less resembles any gradient-based optimization techniques, which are shown to be effective in soft robot co-design with differentiable simulation [26, 2]. Fortunately, drawing a connection to energy-based models [54, 15, 14], yet another alternative to incorporate conditioning in diffusion

models is Markov Chain Monte Carlo (MCMC) sampling [14], where we use Unadjusted Langevin Dynamics,

where K is the number of samples in the current MCMC with k as indexing, σ 2 is a pre-defined variance, and x (0) t = x t -1 . In the context of diffusion models, this procedure is commonly performed within a single diffusion step to drive the sample toward higher-density regime under the intermediate distribution p ( x t ) = ∫ q ( x t | x 0 ) p ( x 0 ) d x 0 at diffusion time t . Inspired by its resemblance to gradient ascent with stochasticity from the added Gaussian noise of variance σ 2 , we establish a connection to design optimization, reformulating diffusion process as co-design optimization as in (6) and (7). Specifically, we can apply Bayes rule to decompose the score of p ( c | x ( k -1) t ) ,

where ∇ x log p ( x t ) is simply the denoiser output /epsilon1 θ ( x t , t ) and ∇ x log p ( x t | c ) is the gradient of task performance with respect to the intermediate sample of the diffusion model from differentiable physical simulation and robotizing process. Overall, this leads to (6).

C Implementation Details In Algorithm

In this section, we provide more implementation details and experimental configurations of DiffuseBot and other baselines. In Table 4, we list the configurations of the embedding optimization. With respect to Algorithm 1, 'buffer' refers to the online dataset D .

Buffer Size is the capacity of the dataset.

Min. Buffer Size is the minimum of data filled in the buffer before training starts.

Num. Samples / Epoch is number of new samples collected in each epoch, where epoch here refers to a new round of data collection in online learning.

Train Iter. / Epoch is number of training iterations per epoch.

Buffer Top-K is the number of datapoints with top-k performance being retained in the Filter step atop the most up-to-date data.

Batch Size is the batch size in the embedding optimization.

Direct SDF

DiffuseBot

0.030

0.080

0.029

0.078

Performance

Performance

0.076

0.027

0.074

0.026

0.072

0.025

0.070

0.024

0.068

2

NumberofIteration

4

6

8

10

Q

2

Number of Iteration

4

6

8

10

In Table 5, we list the configurations of diffusion as co-design. We follow (6)(7) for:

K is number of MCMC sampling steps at the current diffusion time.

σ is the standard deviation related to the MCMC step size.

κ is the ratio between two types of design gradients.

γ is the weight for trading off design and control optimization.

t max and t min are the maximal and minimal diffusion time to perform diffusion as co-design, respectively.

∆ t is the diffusion time interval to perform diffusion as co-design.

For baselines, we use learning rates for control optimization following γ in Table 5; for particle-based and voxel-based approaches, we use learning rate 0 . 01 for design optimization; for implicit function and diff-CPPN, we use learning rate 0 . 001 for design optimization. For the inputs of implicit function and diff-CPPN, we use x, y, z coordinates in the local workspace, the distance to the workspace center on the xy, xz, yz planes, and the radius from the center. For the network architecture of implicit function, we use a 2-layer multilayer perceptron with hidden size 32 and Tanh activation. For the network architecture of diff-CPPN, we use Sin, Sigmoid, Tanh, Gaussian, SELU, Softplus, Clamped activations with 5 hidden layers and 28 graph nodes in each layer.

Hyperparameters are chosen mostly based on intuition and balancing numerical scale with very little tuning. In the following, we briefly discuss the design choices of all hyperparameters listed in Table 5 and Table 4. For min buffer size, samples per epoch, training iteration per epoch, and batch size, we roughly make sufficiently diverse the data used in the optimization and use the same setting for all tasks. For buffer size, we start with 60 and if we observe instability in optimization, we increase to 10 times, 600 (similar to online on-policy reinforcement learning); note that buffer size refers to the maximal size and increasing this won't affect runtime. For buffer Top-K, we start with 6 and if we observe limited diversity of generation throughout the optimization (or lack of exploration), we double it. For t max , t min , and ∆ t , we roughly inspect how structured the generation in terms of achieving the desired robotic task to determine t max and modify ∆ t accordingly to match the similar number of performing MCMC sampling (e.g., t max / ∆ t : 400 / 50 ≈ 150 / 25 ). For the number of MCMC steps K , we simply set 3 for passive tasks and 5 for active tasks by intuition. For σ , we simply follow one of the settings in [14]. For the guidance scale κ and renorm scale, we check the numerical values between /epsilon1 and gradient from differentiable simulation and try to make them roughly in the similar magnitude, and set the same scale for all tasks for simplicity. For γ , we set 0.001 for trajectory optimization and 0.01 for parameterized controllers based on our experience of working with differentiable physics. Overall, from our empirical findings, the only hyperparameters that may be sensitive include buffer size and buffer Top-K for optimization stability and generation diversity, and guidance scales, which need to be tuned to match the numerical magnitude of other terms so as to take proper effect.

D Details In Task Setup

In this section, we provide more details of all tasks setup including environment configuration and prescribed actuator and controller if any (as mentioned in Section 2.4; it is fixed during embedding optimization and will be co-optimize during diffusion as co-design). We select tasks that

1. can cover a wide spectrum of existing robotics tasks: we briefly categorize tasks into passive dynamics, locomotion, and manipulation. Note that passive dynamics tasks are explicitly considered

here since there is no active control of robot bodies, making optimization on robot design a direct factor toward physical utility.

only involve lower-level control/motion without the complications of long-term or higher-level task planning: we select tasks that mostly involve few motor skills, e.g., in manipulation, instead of pick and place, we simply aim at picking up/gripping an object.

are commonly considered in other soft robot co-design literature: all proposed active tasks are widely used in the soft robot community, including crawling [8, 10, 47, 61], hurdling/jumping [26, 59, 3], and manipulating objects [5, 11, 38].

may induce more visible difference in robot designs between the performing and the nonperforming ones to facilitate evaluation and algorithmic development: we select tasks more based on heuristics and intuition, e.g., in crawling, we expect leg-like structures may outperform other random designs.

We build our environments on top of SoftZoo [61] and employ the Material Point Method for simulation. Each environment is composed of boundary conditions that include impenetrable ground, and, in the case of fixed objects or body parts, glued particles. All tasks use units without direct real-world physical correspondence and last for 100 steps with each step consisting of 17 simulation substeps. In the following, when describing the size of a 3D shape without explicitly mentioning the axis, we follow the order: length (x, in the direction when we talk about left and right), height (y, parallel with the gravity direction), and width (z). All tasks are demonstrated in Figure 3 in the main paper.

Balancing. The robot is initialized atop a stick-like platform of shape 0.02-unit × 0.05-unit × 0.02-unit. The robot is givenan initial upward velocity of a 0.5-unit/second and allowed to free fall under gravity. The goal is for the robot to passively balance itself after dropping again on the platform; the performance is measured as the intersection over union (IoU) between the space occupied by the robot during the first simulation step and the space occupied by the robot during the last simulation step. The robot geometry is confined to a 0.08-unit × 0.08-unit × 0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).

Landing. The robot is initialized to be 0.08-unit to the right and 0.045-unit above the landing target with size of 0.02-unit × 0.05-unit × 0.02-unit. The robot is given an initial velocity of 0.5unit/second to the right. The goal of the robot is to land at the target; the performance is measured as the exponential to the power of the negative distance between the target and the robot in the last frame e -|| p object H -p robot H || , where p · H is the position of the robot or object at the last frame with horizon H . The robot geometry is confined to a 0.08-unit × 0.08-unit × 0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).

Crawling. The robot is initialized at rest on the ground. The goal of the robot is to actuate its body to move as far away as possible from the starting position; the performance is measured as the distance traveled || p x, robot H -p x, robot 0 || , where p x, robot · is the position of the robot in the x axis at a certain frame with horizon H . The robot geometry is confined to a 0.08-unit × 0.08-unit × 0.08-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the xz (non-vertical) plane into 4 groups. Each group contains an actuator with its direction parallel to gravity. The prescribed controller is a composition of four sine waves with frequency as 30hz, amplitude as 0.3, and phases as 0.5 π , 1.5 π , 0, and π for the four actuators.

Hurdling. An obstacle of shape 0.01-unit × 0.03-unit × 0.12-unit is placed in 0.07-unit front of the robot. The goal of the robot is to jump as far as possible, with high distances achieved only by bounding over the obstacle. The performance is measured as the distance traveled. The robot geometry is confined to a 0.08-unit × 0.08-unit × 0.08-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups. Each group contains an actuator aligned parallel to gravity. The prescribed controller takes the form of open-loop, per-step actuation sequences, set to linearly-increasing values from (0.0, 0.0) to (1.0, 0.3) between the first and the thirtieth frames and zeros afterward for the two actuators (the first value of the aforementioned actuation corresponds to the actuator closer to the obstacle) respectively.

Gripping. An object of shape 0.03-unit × 0.03-unit × 0.03-unit is placed 0.08-unit underneath the robot. The goal of the robot is to vertically lift the object; the performance is measured as the vertical distance of the object being lifted || p y, object H -p y, object 0 || , where p y, object · is the position of the

object in the y axis at a certain frame with horizon H . Within a 0.06-unit × 0.08-unit × 0.12-unit workspace, we decompose the robot into a base and two attached submodules, and we set the output of DiffuseBot or other robot design algorithms as one of the submodules and make constrain the other submodule to be a mirror copy; conceptually we design 'the finger of a parallel gripper.' The base is clamped/glued to the upper boundary in z and given a large polyhedral volume to serve as the attachment point of the gripper finger submodules. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups; each submodule has one pair of actuators. Each actuator is aligned parallel to gravity. Overall, one pair of actuators comprises the "inner" part of the gripper and the other comprises the "outer" part. Suppose the actuation of the two pairs of actuators is denoted in the format of (actuation of the outer part, actuation of the inner part), the prescribed controller is per-frame actuation, set to (i) linearly-increasing values from (0.0, 0.0) to (0.0, 1.0) between the first and the fiftieth frames, and (ii) then linearly-decreasing values from (1.0, 0.0) to (0.0, 0.0) between the fiftieth and the last frames.

Moving a box. A 0.03-unit × 0.03-unit × 0.03-unit cube is placed on the right end of the robot (half a body length of the robot to the right from the robot center). The goal of the robot is to move the box to the left; the performance is measured as the distance that the box is moved to the right with respect to its initial position || p x, object H -p x, object 0 || , where p x, object · is the position of the object in the x axis at a certain frame with horizon H . The robot geometry is confined to a 0.16-unit × 0.06-unit × 0.06-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the height direction into 2 groups. Each group contains an actuator aligned parallel to the ground. The prescribed controller is per-frame actuation, initialized to linearly-increasing values from (0.0, 0.0) to (1.0, 0.0) between the first and last frame for the lower and the upper actuator respectively.

E Analysis On Robotizing

As mentioned in Section 2.3, Material Point Method simulation requires solid geometry for simulation; thus, we need to convert the surface point cloud from Point-E [39] to a volume. The most direct means of converting the point cloud to a solid geometry is to compute the signed distance function (SDF), and populate the interior of the SDF using rejection sampling. We refer to this baseline as Direct SDF . Here, we use a pretrained transformer-based model provided by Point-E as the SDF. In Figure 8, we compare Direct SDF with our approach described in Section 2.3 Solid Geometry. We perform robotizing on intermediate samples at t = 300. We observe that Direct SDF fails to produce well-structured solid geometry since it is trained with watertight on geometry, and thus the conversion cannot gracefully handle generated point clouds that do not exhibit such watergith structure. This is specifically common in the intermediate diffusion sample x t as x t is essentially a Gaussian-noisecorrupted version of the clean surface point cloud. In contrast, the robotizing of DiffuseBot produces a much well-structured solid geometry since it explicitly handles the noisy interior 3D points by introducing a tailored loss as in Shape As Points optimization [40] (see Section 2.3). In addition, a better-structured robot geometry is critical to obtain not only a more accurate evaluation of a robot design at the forward pass of the simulation but also the gradients from differentiable physics at the backward pass. In Figure 8, we further perform co-optimization on the two robots obtained by Direct SDF and DiffuseBot; we observe a more stably increasing trend of task performance in our approach, demonstrating that the gradient is usually more informative with a better-structured robot. Empirically, while Direct SDF sometimes still provides improving trend with unstructured robots in co-design optimization, the performance gain is not as stable as DiffuseBot.

F Additional Visualizations Of Experiments

In this section, we show more visualization for a diverse set of the experimental analysis. Please visit our project page ( https://diffusebot.github.io/ ) for animated videos.

Generated robots performing various tasks. In Figure 9, we display a montage of a generated robot's motion for each task; this is the unabridged demonstration of Figure 5 in the main paper.

Physics-guided robot generation In Figure 10, we show how DiffuseBot evolves robots throughout the embedding optimization process; this is the full demonstration of Figure 4 in the main paper. Note

Crawling

1st Frame

30th Frame

60th Frame

100th Frame

Balancing

Landing

Hurdling

Moving a Box

that DiffuseBot is a generative algorithm and thus results presented are for demonstration purposes and reflect only a single fixed random seed.

Diffusion process of robot generation. In Figure 11, we show how robots evolve throughout the diffusion process at different diffusion time via x t with t from T to 0 ; not to be confused by Figure 10 where all generations are obtained via a full diffusion sampling x 0 and different conditioning embeddings c . This figure demonstrates the robotized samples across diffusion times, gradually converging to the final functioning robots as t approaches 0 .

Comparison with baselines. In Figure 12, we present robots generated from different baselines. Note that Point-E is essentially DiffuseBot without embedding optimization and diffusion as co-design.

Incorporating human feedback. In Figure 13, we showcase more examples of incorporating human textual feedback to the generation process of DiffuseBot. As briefly mentioned in Section 3.4 in the main paper, we leverage the compositionality of diffusion-based generative models [34, 14]. Specifically, this is done by combining two types of classifier-free guidance as in (3); one source

Crawling

1st Checkpoint

2nd Checkpoint

3rd Checkpoint

Balancing

Landing

Hurdling

Moving a Box

of guidance is derived from the task-driven embedding optimization while the other is derived from embeddings from human-provided textual descriptions. These two conditional scores, namely /epsilon1 θ ( x t , t, c physics ) and /epsilon1 θ ( x t , t, c human ) , can then be integrated into the diffusion as co-design framework as in (6). We refer the reader to the original papers for more details and theoretical justification. We demonstrate examples of: the balancing robot with additional text prompt "a ball" , the crawling robot with additional text prompt "a star" , the hurdling robot with additional text prompt "a pair of wings" , and the moving-a-box robot with additional text prompt "thick" . Interestingly, human feedback is introduced as an augmented "trait" to the original robot. For example, while the hurdling robot keeps the tall, lengthy geometry that is beneficial for storing energy for jumping, a wing-like structure appears at the top of the robot body instead of overwriting the entire geometry. This allows users to design specific parts for composition while also embedding fabrication and aesthetic priorities and constraints through text. We leave future explorations of these ideas for future work.

Crawling

t = 600

t = 400

t = 200

t = 0

Balancing

Landing

Hurdling

Moving a Box

G Hardware Design and Fabrication

To create a real-world counterpart of the DiffuseBot, we fabricated a proof-of-concept physical robot gripper, as illustrated in Figure 14(a). The gripper was designed to match the shape of the digital gripper and successfully demonstrated its ability to grasp objects, as shown in Figure 15. A video demonstrating the grasping process can be found on our project page ( https://diffusebot. github.io/ ).

During the translation from the simulated design to a physical robot, we aimed to minimize differences between the simulated and physical designs. However, due to hardware and fabrication limitations, certain modifications were necessary.

One such challenge involved replicating the arbitrary contraction force at the particles from the simulation in the physical world. To address this, we employed tendon transmission as the actuation method for the real-world gripper, which was found to be suitable for emulating interaction forces

Crawling

Balancing

Landing

Hurdling

Moving a Box

Implicit Function

Diff-CPPN

Point-E

DiffuseBot

from the digital world without introducing significant discrepancies. To enhance the stability of the tendon-driven actuation, we incorporated a rigid base ("Base" in Figure 14(a)) above the gripper itself ("Gripper" in Figure 14(a)). This base was designed to withstand the reaction force generated by the tendon-driven actuation. Furthermore, we added four tendon routing points (represented by small dotted circles in Figure 14(b)) on each finger to securely fix the tendon path. By utilizing four Teflon tubes (shown in Figure 14(b)), we were able to position complex components such as motors, batteries, and controllers away from the gripper, reducing its complexity.

The actuation strategy of the simulated gripper applies equal contraction force to both fingers simultaneously. To replicate this strategy, we employed underactuated tendon routing in the development of the gripper. This approach eliminates the need for four separate actuators, thereby reducing the complexity of the robot. We used tendon-driven actuators specifically designed for underactuated tendon-driven robots as they solve practical issues such as size and friction issues that commonly arise in such systems [30].

Crawler + Text 'A Round Disk'

Crawler From DiffuseBot

Hurdler From DiffuseBot

Hurdler + Text 'A Pair Of Wings'

Balancing Robot From DiffuseBot

Box-moving Robot From DiffuseBot

Box-moving Robot + Text 'Thick'

A sphere in

the middle

Circular shape

Two wings

Actuator

1

Actuator 2

(a)

(b)

(c)

Base

Gripper

Teflon

tube

The soft gripper was 3D-printed using a digital light projection (DLP) type 3D printer (Carbon M1 printer, Carbon Inc.) and commercially available elastomeric polyurethane (EPU 40, Carbon Inc.). To enhance the softness of the robot body beyond the inherent softness of the material itself, we infilled the finger body with a Voronoi lattice structure, a metamaterial useful for creating a soft structure with tunable effective isotropic stiffness [20, 36]. We generated a Voronoi lattice foam with point spacing of 2.5mm, and a beam thickness of 0.4 mm as shown in the red boundary of Fig. 14(c). Finally, we tested the soft gripper, designed and fabricated as described above, to verify its ability to grasp an object, as shown in Figure 15.

More discussion on fabrication. Although, at present, the compilation of the virtual robot to a physical, digitally fabricated counterpart involves manual post-processing of algorithm's output, most, if not all of these steps could be automated. Our method outputs a point cloud (defining geometry), actuator placements, and an open-loop controller, along with a prescribed stiffness. Since we can easily convert the point cloud into a 3D triangle mesh, the geometry can be created by almost any 3D printing method. In order to realize an effective stiffness and material behavior, stochastic lattices, specifically Voronoi foams, have been used [36, 20] in the past and employed here in order to match target material properties. Given the actuator placement, tendons [27, 30] can be aligned with the prescribed (contiguous) regions. Since a lattice is used, threading tendons through the robot body is simple, and we note that even more complex routings have been studied in detail in the literature [4]. Creating attachment points for the tendons is a relatively simple geometry processing problem [6]. Thus, converting a virtual robot to a specification that incorporates geometry, material, and actuation can be automated in a straightforward way.

(a)

(b)

(c)

We note that when controlled, the physical robot may not always match the virtual robot's motion. This is the sim-to-real gap, and is significantly harder to overcome in our case than translating the virtual robot to physical hardware. Significant literature has been invested in specifically tackling the sim-to-real gap, and in our case would require its own dedicated project; however, we note that often hardware can be adapted to work by modifying only the control policies using feedback from real-world experiments, often even with little human intervention [21].

Quantitative analysis . In Figure 16, we provide a detailed analysis on the gap between simulation and physical robot. In order to explore the quantitative gap between the behavior of the physical robot and the simulated robot, we conducted an experiment with the following conditions, where similar setups are commonly adopted in soft robot literature [17]. The objective was to measure the change in distance between two tips when we pull/release two tendons - one for closing the gripper (flexion) and the other for opening it (extension). The tendons were pulled or released in increments and decrements of 2mm.

When contracting the tendon to flex or extend the fingers, both simulation and real robot results show log-shaped graphs. The pattern in the physical robot plot is a commonly observed phenomenon called hysteresis. However, the main difference between the simulation and real-world cases can be seen when releasing the tendon from a fully contracted state. In the real robot experiment, the tip distance changes rapidly, while in the simulation, the opposite effect is observed.

One plausible explanation for this disparity could be attributed to the friction direction and elongation of the tendons. During the transition from tendon contraction to tendon release, the tension of the tendon at the end-effector may change suddenly due to the change of the friction direction. Also, since we only control the motor position (not the tendon position) to pull/release the tendon with 2mm step, the exact tendon length may not be exactly the same when we consider the tendon elongation.

Additional real robots. In Figure 17, we show two other fabricated robots for the moving a box task and the crawling task. For moving a box, we can achieve similar motion as in the simulation; the only slight difference occurs when the stiffness in the physical robot is not as soft as that in the simulation, falling a bit short to perform the extremely curly motion in the simulation. For crawling, we found a more significant sim-to-real gap where the physical robot cannot achieve at all the galloping-like motion in the simulation; instead, the robot crawls more alike a inchworm movement, pushing the body in a quasi-static way. The major reason is the damping of the material (elastomeric polyurethane, EPU 40, Carbon Inc.) prohibit from releasing sufficient energy for dynamical motion.

Potential solutions . Given that the gap between simulation and real robot performance seems to originate from the actuation/transmission method, our future work will focus on developing a tendondriven actuation simulation framework. This framework aims to address the differences and improve the accuracy of our simulations. We are exploring other possible explanations for the sim-to-real gap and will investigate any additional factors that may contribute to the observed discrepancies. Overall, as for a high-level general solution, we believe (1) adjusting parameters based on observed sim to real gap and repeat the design process or (2) building a more accurate physics-based simulation (which can be straightforwardly plug-and-played in DiffuseBot) can largely bridge the sim-to-real gap of fabricating physical robots; or more interestingly, connecting generative models to commercial-level design and simulation softwares.

!"#$%&'(%

)*$"'(%

(unit

istance

Di

C

Moving a Box

Crawling

H More Discussions

H.1 Limitation

The major limitation of DiffuseBot is that we make a simplification in the parameterization of actuators and stiffness; we make dependencies of the two design specifications on robot geometry (check more technical details in Section 2.3 paragraph Actuators and Stiffness . This works well with properly-crafted mapping from geometry to the others yet limits the potential by human prior with little use of the generative power. While this may be reasonable as properly-set actuators and stiffness based on geometry (hand-tuned empirically in this work) roughly reflects task performance, a more flexible parameterization can definitely lead to improved performance. Potential remedy can be using part-based 3D generative models for actuators and direct optimization for stiffness. Another limitation is the gap between simulated results and real robots. While the hardware experiment has shown as a proof-of-concept that minimally demonstrates the potential, physical robot fabrication and real-world transfer have countless non-trivial challenges including stiffness and actuator design, sim-to-real gap, etc. This may require studies on more high-fidelity physics-based simulation, which can be straightforwardly plugged into DiffuseBot.

H.2 Conditioning Beyond Text

The use of textual inputs additional to the embeddings optimized toward physical utility is achieved by both being able to be consumed by the diffusion model to produce guidance for the diffusion process . More concretely speaking, in DiffuseBot, we use the CLIP feature extractor as in Point-E and it allows to extract embedding for both text and image modalities, which can then be used as a condition in the diffusion model. Thus, we can also incorporate images as inputs and perform the exact same procedure as that of the textual inputs. Theoretically, the textual inputs are incorporated via following the intuition in end of the paragraph Diffusion as Co-design in Section 2.4, where the textual inputs additionally provide gradients toward following the textual specification. Similarly, the image inputs can also be processed to provide gradients since CLIP embeddings live in a joint space of images and languages. More interestingly, if we build DiffuseBot on models other than Point-E, which can consume embeddings for other modalities like audio as conditioning, we can then straightforwardly perform robot design generation guided by the other corresponding input formats (and meanwhile, toward physical utility). Note that this critical feature of compositionality across different sources of guidance throughout the reverse diffusion process is one of the biggest advantages of using diffusion-based models as opposed to other types of generative models.

H.3 Connection To Text Inversion

There is some synergy between text inversion in [19] and embedding optimization in DiffuseBot. Both of them aim at tuning the embedding toward reflecting certain properties of the output generation, i.e., describing the output generated images in [19] and toward improved physical utility in DiffuseBot. The major difference lies in the nuance of the data/samples used to carry out the optimization. Text inversion performs a direct optimization using latent diffusion model loss (Eq. (2) in [19]), which computes losses on noisy samples/latents corrupted from the real dataset. On the other hand, it is tricky to think about real dataset in robot design (as discussed in the second paragraph of Section 1 and the paragraph Embedding Optimization in Section 2.4), embedding optimization in DiffuseBot computes losses on noisy samples corrupted from self-generated data filtered by robot performance (as in Algorithm 1 and Section 2.4). Conceptually, it is more like a mixture of diffusion model training and online imitation learning like DAGGER [45].

H.4 Connection To Diffusion Models With Physical Plausibility

A potential and interesting way to adapt DiffuseBot to other applications like motion planning or control [28, 1] is to view a generated robot as one snapshot/frame of a motion/state sequence and the physics prior can be the dynamics constraint across timesteps (e.g., robot dynamics or contact dynamics that enforce non-penetration). The physics prior can be injected similarly to diffusion as co-design that propagates the enforcement of physical plausibility of generated states from differentiable physics-based simulation to diffusion samples. For example, considering states in two consecutive timesteps, we can compute loss in the differentiable simulation to measure the violation of physical constraints regarding robot dynamics or interaction with the environment. Then, we can compute gradients with respect to either control or design variables; for gradients in control, this will essentially augment works like [28, 1] with classifier-based guidance to achieve physical plausibility; for gradients in design, this will much resemble optimizing toward the motion sequence of a shape-shifting robot.

H.5 Parameterization Of Actuator And Stiffness

The goal of DiffuseBot is to demonstrate the potential of using diffusion models to generate soft robot design and to leverage the knowledge of the pre-trained generative models learned from a large-scale 3D dataset. Under this setting, the generated output of the diffusion model can only provide the geometry information of robot designs, leading to our design choice of having full dependency of actuator and stiffness on the geometry. This may be a reasonable simplification as prior works [48] have shown geometry along with properly-set actuator and stiffness (we take manual efforts to design proper mapping from geometry to actuator and stiffness in this work) roughly reflect the performance of a soft robot design. For better generality, one potential remedy is to optimize actuator and stiffness independently from the geometry generated by the diffusion model, i.e., apply DiffuseBot and do direct optimization on actuator and stiffness afterward or at the same time. Another interesting direction may be, for actuators, to leverage part-based models [29] to decompose a holistic geometry into parts (or different actuator regions in soft robots).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposed a new framework that augments the diffusion-based synthesis with physical dynamical simulation in order to generatively co-design task-driven soft robots in morphology and control. The extensive experiments in simulation to verify the effectiveness of DiffuseBot.\n\nSTRENGTHS:\n1. The paper is well-written and easy to follow.\n2. The visualization is good and help me understand how the method works.\n3. The application of diffusion is always be encouraged and diffusion model is an interesting and promising backbone.\n\nWEAKNESSES:\n1. The novelty is limited and more like an incremental work that applied diffusion model into 3D soft body generation task. The relationship and difference with  Softzoo mentioned in the paper should be more clearly illustrated.\n2. The baseline for comparison is relatively weak, consisting of some outdated works from a few years ago. The paper should includes more strong and recent baselines.\n3. The evaluation way is not agreed. 'To avoid being trapped in the local optimum, we run each baseline with 20 different random initializations and choose the best one. Since DiffuseBot is a generative method, we draw 20 samples and report the best' . If the baseline is sensitive to the initialization seed, more detailed results should be reported rather not choose the best one. And the 20 runs should report the mean and var rather not the best. The explanation about the generative model is not convincing enough for me.\n4. The novelty is limited and more like an incremental work applying diffusion model in 3D generation domain. \n5. The physics augmented component is not included in ablation studies and this will harm the convincing conclusion in the paper.\n6. Why diffusion not other generative model, such as VAE or transformer? The ablation studies should be listed or the paper will be likely the incremental work.\n\nQUESTIONS:\nPlease give more detailed clarification and experiments results to address my concerns.\n\nLIMITATIONS:\n1. The novelty is limited and the motivation of using diffusion model is not well illustrated.\n2. The baseline is weak and old, the evaluation way is not agreed. The ablation studies are not enough.\n3. The role and the improvement bring by physic-augmented component is not well studied.\n4. The limitation part is absent in the submitted version. I hope the authors can discuss the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents DiffuseBot, a framework that uses physics-augmented diffusion models to generate soft robot designs and control strategies for various tasks. The authors propose to optimize the embeddings conditioned by the diffusion model to improve the physical utility of the generated robots, and to reformulate the diffusion sampling process as a co-design optimization that leverages differentiable simulation. The authors demonstrate the effectiveness of their method on several tasks, such as balancing, landing, crawling, hurdling, gripping, and moving objects. They also show how to incorporate human feedback and fabricate a physical robot prototype.\n\nSTRENGTHS:\nThe paper is well-written and clear. The proposed method is novel and interesting, combining diffusion models for shape generation, physics-based simulation and co-design optimization. The paper provides extensive experimental comparisons with baselines and ablation studies on both latent optimization and co-design to validate the proposed method. The paper also shows some fun qualitative results of diverse robot designs that function under passive dynamics, locomotion tasks and manipulation tasks.\n\nWEAKNESSES:\n- The paper does not discuss the limitations or failure scenarios of the proposed method. In addition, discussions on design choices can be help: how hyper-parameters are chosen, such as the guidance scale, or the number of MCMC steps? \n- Figure 1 shows that the motivation of this work is to deploy the optimized soft robot into the real world. However, though I may miss it, I did not find discussions in the paper about the feasibility of manufacturing the resulting soft robots, such as the soft gripper. Given that the actuators are currently assumed to be muscle fibers, it can be hard for manufacturing.\n\nQUESTIONS:\n- Can limitations of the current pipeline be discussed and included in the paper?\n- Can the paper includes discussions about manufacturing?\n- How design choices are set? How sensitive is the current pipeline to hyper-parameters?\n- How to interpret the metric reported in Table 1, 2? \"We report the average performance with standard deviation in the superscript.\" -- Is the performance success rate?\n\nLIMITATIONS:\nPlease include a section about limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DiffuseBot, a physics-augmented diffusion model designed for generating and optimizing the morphologies and control mechanisms of soft robots. DiffuseBot aims to bridge the gap between virtually generated content and physical utility in the domain of soft robotics. Firstly, it combines the diffusion process with a physical simulation that serves as a performance certificate, thereby ensuring the feasibility and effectiveness of the generated designs. Secondly, it details a co-design procedure that simultaneously optimizes the physical design and control of the soft robots, leveraging insights from differentiable simulation. The paper validates the efficacy of this approach by presenting a variety of both simulated and physically fabricated robots, along with their diverse capabilities.\n\nSTRENGTHS:\n1. In general, the paper is well written, with only minor flaws. Even those unfamiliar with soft robot design will find the paper easy to comprehend.\n\n2. Although diffusion models are expressive and powerful, their performance for tasks dealing with physical tasks often falls short. Thus, injecting a physics prior or 'physics-augmented diffusion model' is crucial. I think the method proposed in this paper is interesting and promising. \n\n3. The evaluation is comprehensive and thoughtful. The physical robot is impressive.\n\nWEAKNESSES:\nOverall, I did not identify any major weaknesses in the paper, but here are a few points that could strengthen it:\n\n1. While the writing is generally clear, certain sections could benefit from clearer exposition, such as:\n*  The section on diffusion as co-design is not very intuitive, especially for audiences not familiar with soft robot design. Specifically, it should be clearer how gradient-based optimization benefits robot design and what exactly line 152's \"synergy\" means.\n* It would be helpful if the authors clarify that the \"condition\" in this work actually refers to text.\n2. The robot's actuator and stiffness seem oversimplified, having only constant stiffness. Given that the gradient of $\\Psi_{act}$ is almost zero, it appears that the actuator and stiffness are solely determined by the geometry.\n3. A similar idea of tuning in the embedding space is proposed in[1]. A discussion and connection to this existing work could be interesting.  \n4. In general, the method the paper uses to inject a physics prior into the generation process could be applicable to more general scenarios. Works like Diffuser[2] or Decision Diffuser[3] generate state sequences with diffusion models, but the generated states can sometimes be physically implausible. A deeper discussion about the potential of the method could make the paper stronger.\n\n[1] Gal, Rinon, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik and Daniel Cohen-Or. “An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion.”, ICLR, 2023.\n\n[2] Janner, Michael, Yilun Du, Joshua B. Tenenbaum and Sergey Levine. “Planning with Diffusion for Flexible Behavior Synthesis.”, ICML, 2022. \n\n[3] Ajay, Anurag, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, T. Jaakkola and Pulkit Agrawal. “Is Conditional Generative Modeling all you need for Decision-Making?” ICLR, 2023.\n\nQUESTIONS:\n1. I do not fully understand how the k-means clustering is performed for actuator and stiffness generation. Specifically, what kind of feature is used for clustering?\n\n2. In line 86, which structural biases are you referring to ?\n\nLIMITATIONS:\nsee weakness.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces DiffuseBot, a system that aims to simplify and automate the design of soft robots in simulation and real-world systems. DiffuseBot uses diffusion-based algorithms to co-design soft robot morphology and control for specific tasks, combining the diversity of evolutionary algorithms with the efficiency of gradient-based optimization. The system is made possible by advancements in AI-driven content generation.\n\nHowever, existing generative algorithms face challenges when applied to physical soft robot co-design, such as the lack of consideration for physics and task performance. To overcome these, DiffuseBot uses physical simulation to guide the generative process of pretrained large-scale 3D diffusion models. It also develops an automatic procedure to convert raw 3D geometry into a format compatible with soft body simulation.\n\nThe system optimizes the embeddings that condition the diffusion model, skewing the sampling distribution toward better-performing robots as evaluated by a simulator. It also reformulates the sampling process to incorporate co-optimization over structure and control.\n\nDiffuseBot has been tested on a wide range of tasks, demonstrating its superiority to comparable approaches. It also allows for human input in the robot generation process and has been used to create a proof-of-concept 3D-printed real-world robot. The paper contributes a new framework that augments the diffusion-based synthesis with differentiable physics simulation, methods for driving robot generation in a task-driven way toward improved physical utility, and extensive experiments in simulation to verify the effectiveness of DiffuseBot.\n\nSTRENGTHS:\nThis paper is robust and comprehensive in its approach. It introduces an innovative method that applies diffusion models to the co-design of robots, representing a significant contribution to the field. The authors have ensured thorough experimental coverage by testing their system, DiffuseBot, on a diverse range of tasks. This extensive testing underscores the versatility and applicability of the proposed method. Furthermore, the paper is not limited to theoretical constructs but extends to practical, real-world applications. The authors demonstrate this by providing a proof-of-concept 3D-printed real-world robot, thereby solidifying the relevance and potential of their research in real-world scenarios.\n\nWEAKNESSES:\nThe paper does not exhibit any significant shortcomings or areas of concern.\n\nQUESTIONS:\nIn your paper and supplementary materials, you've provided a qualitative discussion on the challenges of translating simulated results into the fabrication of real robots based on the design developed in simulation. Could you delve deeper into this issue by providing more detailed, quantitative results that highlight the discrepancies between the behavior of the simulated robot and its real-world counterpart? Additionally, could you propose potential solutions aimed at minimizing this gap between simulation and reality?\n\nLIMITATIONS:\nThe paper effectively addresses all identified limitations.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks called DiffuseBot. DiffuseBot bridges the gap between virtually generated content and physical utility by (i) augmenting the diffusion process with a physical dynamical simulation which provides a certificate of performance, and ii) introducing a co-design procedure that jointly optimizes physical design and control by leveraging information about physical sensitivities from differentiable simulation. In the experiment, they showed a range of simulated and fabricated robots along with their capabilities.\n\nSTRENGTHS:\n1. The paper introduces a new framework that augments diffusion-based synthesis with physical dynamical simulation in order to co-design task-driven soft robots in morphology and control.\n2. The method leverages optimizing input embeddings and incorporating differentiable physics into the diffusion process for driving robot generation in a task-driven way toward improved physical utility. \n3. They performed experiments in simulation to verify the effectiveness of DiffuseBot, extensions to text-conditioned functional robot design, and a proof-of-concept physical robot as a real-world result.\n\nWEAKNESSES:\nThe presentations in this paper were sometimes unclear, as asked in the following placeholder. The paper admitted that there are countless non-trivial challenges in the physical robot fabrication and real-world transfer, including stiffness and actuator design, and the sim-to-real gap. However, in my opinion, other contributions of this paper may outperform the weaknesses.\n\nQUESTIONS:\n1. In Section 2 before 2.1, there was no reference to 2.2. Is it fine?\n2. I did not find the explicit definition of bold c in Eq. (5).  \n3. L159: what does the slash mean? \n4. In the experiments, I want to know why such tasks were selected (i..e, motivation for the tasks). \n5. I cannot find the explanation about the performance metric such as Tables 1 and 2 (sometimes having minus values). This information is important and should be mentioned in the main text.\n6. The (short) introduction of the baseline models and the reasons can be mentioned. \n7. The figure at the bottom of page 7 was Table 4, but may be incorrect. And Figure 4 in L233 may be incorrect. \n8. Section 3.4: The paper discusses the use of textual inputs and is very interesting. Can the authors discuss the potential for using other input formats?\n\nLIMITATIONS:\nIn the conclusion section, there seems to be less limited information about this work from the experimental results. In other parts, general limitations were mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}