SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Buffer Size is the capacity of the dataset.
- Min. Buffer Size is the minimum of data filled in the buffer before training starts.
- Num. Samples / Epoch is number of new samples collected in each epoch, where epoch here refers to a new round of data collection in online learning.
- Train Iter. / Epoch is number of training iterations per epoch.
- Buffer Top-K is the number of datapoints with top-k performance being retained in the Filter step atop the most up-to-date data.
- Batch Size is the batch size in the embedding optimization.
Figure 8: Comparison between Direct SDF and DiffuseBot on conversion to solid geometry for robotizing.
In Table 5, we list the configurations of diffusion as co-design. We follow (6)(7) for:
-  $K$  is number of MCMC sampling steps at the current diffusion time.
-  $\sigma$  is the standard deviation related to the MCMC step size.
-  $\kappa$  is the ratio between two types of design gradients.
-  $\gamma$  is the weight for trading off design and control optimization.
-  $t_{\mathrm{max}}$  and  $t_{\mathrm{min}}$  are the maximal and minimal diffusion time to perform diffusion as co-design, respectively.
-  $\Delta t$  is the diffusion time interval to perform diffusion as co-design.
For baselines, we use learning rates for control optimization following  $\gamma$  in Table 5; for particle-based and voxel-based approaches, we use learning rate 0.01 for design optimization; for implicit function and diff-CPPN, we use learning rate 0.001 for design optimization. For the inputs of implicit function and diff-CPPN, we use x, y, z coordinates in the local workspace, the distance to the workspace center on the xy, xz, yz planes, and the radius from the center. For the network architecture of implicit function, we use a 2-layer multilayer perceptron with hidden size 32 and Tanh activation. For the network architecture of diff-CPPN, we use Sin, Sigmoid, Tanh, Gaussian, SELU, Softplus, Clamped activations with 5 hidden layers and 28 graph nodes in each layer.
Hyperparameters are chosen mostly based on intuition and balancing numerical scale with very little tuning. In the following, we briefly discuss the design choices of all hyperparameters listed in Table 5 and Table 4. For min buffer size, samples per epoch, training iteration per epoch, and batch size, we roughly make sufficiently diverse the data used in the optimization and use the same setting for all tasks. For buffer size, we start with 60 and if we observe instability in optimization, we increase to 10 times, 600 (similar to online on-policy reinforcement learning); note that buffer size refers to the maximal size and increasing this won't affect runtime. For buffer Top-K, we start with 6 and if we observe limited diversity of generation throughout the optimization (or lack of exploration), we double it. For  $t_{max}$ ,  $t_{min}$ , and  $\Delta t$ , we roughly inspect how structured the generation in terms of achieving the desired robotic task to determine  $t_{max}$  and modify  $\Delta t$  accordingly to match the similar number of performing MCMC sampling (e.g.,  $t_{max} / \Delta t$ :  $400 / 50 \approx 150 / 25$ ). For the number of MCMC steps  $K$ , we simply set 3 for passive tasks and 5 for active tasks by intuition. For  $\sigma$ , we simply follow one of the settings in [14]. For the guidance scale  $\kappa$  and renorm scale, we check the numerical values between  $\epsilon$  and gradient from differentiable simulation and try to make them roughly in the similar magnitude, and set the same scale for all tasks for simplicity. For  $\gamma$ , we set 0.001 for trajectory optimization and 0.01 for parameterized controllers based on our experience of working with differentiable physics. Overall, from our empirical findings, the only hyperparameters that may be sensitive include buffer size and buffer Top-K for optimization stability and generation diversity, and guidance scales, which need to be tuned to match the numerical magnitude of other terms so as to take proper effect.
D Details In Task Setup
In this section, we provide more details of all tasks setup including environment configuration and prescribed actuator and controller if any (as mentioned in Section 2.4; it is fixed during embedding optimization and will be co-optimize during diffusion as co-design). We select tasks that
1. can cover a wide spectrum of existing robotics tasks: we briefly categorize tasks into passive dynamics, locomotion, and manipulation. Note that passive dynamics tasks are explicitly considered
here since there is no active control of robot bodies, making optimization on robot design a direct factor toward physical utility.
2. only involve lower-level control/motion without the complications of long-term or higher-level task planning: we select tasks that mostly involve few motor skills, e.g., in manipulation, instead of pick and place, we simply aim at picking up/gripping an object.
3. are commonly considered in other soft robot co-design literature: all proposed active tasks are widely used in the soft robot community, including crawling [8, 10, 47, 61], hurdle/jumping [26, 59, 3], and manipulating objects [5, 11, 38].
4. may induce more visible difference in robot designs between the performing and the non-performing ones to facilitate evaluation and algorithmic development: we select tasks more based on heuristics and intuition, e.g., in crawling, we expect leg-like structures may outperform other random designs.
We build our environments on top of SoftZoo [61] and employ the Material Point Method for simulation. Each environment is composed of boundary conditions that include impenetrable ground, and, in the case of fixed objects or body parts, glued particles. All tasks use units without direct real-world physical correspondence and last for 100 steps with each step consisting of 17 simulation substeps. In the following, when describing the size of a 3D shape without explicitly mentioning the axis, we follow the order: length (x, in the direction when we talk about left and right), height (y, parallel with the gravity direction), and width (z). All tasks are demonstrated in Figure 3 in the main paper.
Balancing. The robot is initialized atop a stick-like platform of shape  $0.02$ -unit  $\times 0.05$ -unit  $\times$  0.02-unit. The robot is given an initial upward velocity of a 0.5-unit/second and allowed to free fall under gravity. The goal is for the robot to passively balance itself after dropping again on the platform; the performance is measured as the intersection over union (IoU) between the space occupied by the robot during the first simulation step and the space occupied by the robot during the last simulation step. The robot geometry is confined to a 0.08-unit  $\times$  0.08-unit  $\times$  0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).
Landing. The robot is initialized to be 0.08-unit to the right and 0.045-unit above the landing target with size of 0.02-unit  $\times$  0.05-unit  $\times$  0.02-unit. The robot is given an initial velocity of 0.5-unit/second to the right. The goal of the robot is to land at the target; the performance is measured as the exponential to the power of the negative distance between the target and the robot in the last frame  $e^{-||p_H^{\mathrm{object}} - p_H^{\mathrm{robot}}||}$ , where  $p_H^{\prime}$  is the position of the robot or object at the last frame with horizon  $H$ . The robot geometry is confined to a 0.08-unit  $\times$  0.08-unit  $\times$  0.08-unit workspace. There is no prescribed actuator placement or controller (passive dynamics).
Crawling. The robot is initialized at rest on the ground. The goal of the robot is to actuate its body to move as far away as possible from the starting position; the performance is measured as the distance traveled  $||p_H^{x,\mathrm{robot}} - p_0^{x,\mathrm{robot}}||$ , where  $p_{\cdot}^{x,\mathrm{robot}}$  is the position of the robot in the x axis at a certain frame with horizon  $H$ . The robot geometry is confined to a 0.08-unit  $\times 0.08$ -unit  $\times 0.08$ -unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the xz (non-vertical) plane into 4 groups. Each group contains an actuator with its direction parallel to gravity. The prescribed controller is a composition of four sine waves with frequency as 30hz, amplitude as 0.3, and phases as  $0.5\pi$ ,  $1.5\pi$ , 0, and  $\pi$  for the four actuators.
Hurdling. An obstacle of shape  $0.01$ -unit  $\times 0.03$ -unit  $\times 0.12$ -unit is placed in 0.07-unit front of the robot. The goal of the robot is to jump as far as possible, with high distances achieved only by bounding over the obstacle. The performance is measured as the distance traveled. The robot geometry is confined to a  $0.08$ -unit  $\times 0.08$ -unit  $\times 0.08$ -unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups. Each group contains an actuator aligned parallel to gravity. The prescribed controller takes the form of open-loop, per-step actuation sequences, set to linearly-increasing values from (0.0, 0.0) to (1.0, 0.3) between the first and the thirtieth frames and zeros afterward for the two actuators (the first value of the aforementioned actuation corresponds to the actuator closer to the obstacle) respectively.
Gripping. An object of shape  $0.03$ -unit  $\times 0.03$ -unit  $\times 0.03$ -unit is placed 0.08-unit underneath the robot. The goal of the robot is to vertically lift the object; the performance is measured as the vertical distance of the object being lifted  $\left\| p_H^{y,\text{object}} - p_0^{y,\text{object}} \right\|$ , where  $p_{-}^{y,\text{object}}$  is the position of the
object in the y axis at a certain frame with horizon  $H$ . Within a 0.06-unit × 0.08-unit × 0.12-unit workspace, we decompose the robot into a base and two attached submodules, and we set the output of DiffuseBot or other robot design algorithms as one of the submodules and make constrain the other submodule to be a mirror copy; conceptually we design "the finger of a parallel gripper." The base is clamped/glued to the upper boundary in z and given a large polyhedral volume to serve as the attachment point of the gripper finger submodules. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the length direction into 2 groups; each submodule has one pair of actuators. Each actuator is aligned parallel to gravity. Overall, one pair of actuators comprises the "inner" part of the gripper and the other comprises the "outer" part. Suppose the actuation of the two pairs of actuators is denoted in the format of (actuation of the outer part, actuation of the inner part), the prescribed controller is per-frame actuation, set to (i) linearly-increasing values from (0.0, 0.0) to (0.0, 1.0) between the first and the fiftieth frames, and (ii) then linearly-decreasing values from (1.0, 0.0) to (0.0, 0.0) between the fiftieth and the last frames.
Moving a box. A 0.03-unit  $\times$  0.03-unit  $\times$  0.03-unit cube is placed on the right end of the robot (half a body length of the robot to the right from the robot center). The goal of the robot is to move the box to the left; the performance is measured as the distance that the box is moved to the right with respect to its initial position  $||p_H^{x,\mathrm{object}} - p_0^{x,\mathrm{object}}||$ , where  $p^{\mathrm{x,object}}$  is the position of the object in the x axis at a certain frame with horizon  $H$ . The robot geometry is confined to a 0.16-unit  $\times$  0.06-unit  $\times$  0.06-unit workspace. The actuator placement is computed by clustering the local coordinates of the robot centered at its average position in the height direction into 2 groups. Each group contains an actuator aligned parallel to the ground. The prescribed controller is per-frame actuation, initialized to linearly-increasing values from (0.0, 0.0) to (1.0, 0.0) between the first and last frame for the lower and the upper actuator respectively.
E Analysis On Robotizing
As mentioned in Section 2.3, Material Point Method simulation requires solid geometry for simulation; thus, we need to convert the surface point cloud from Point-E [39] to a volume. The most direct means of converting the point cloud to a solid geometry is to compute the signed distance function (SDF), and populate the interior of the SDF using rejection sampling. We refer to this baseline as Direct SDF. Here, we use a pretrained transformer-based model provided by Point-E as the SDF. In Figure 8, we compare Direct SDF with our approach described in Section 2.3 Solid Geometry. We perform robotizing on intermediate samples at  $t = 300$ . We observe that Direct SDF fails to produce well-structured solid geometry since it is trained with watertight on geometry, and thus the conversion cannot gracefully handle generated point clouds that do not exhibit such watergith structure. This is specifically common in the intermediate diffusion sample  $\mathbf{x}_t$  as  $\mathbf{x}_t$  is essentially a Gaussian-noise-corrupted version of the clean surface point cloud. In contrast, the robotizing of DiffuseBot produces a much well-structured solid geometry since it explicitly handles the noisy interior 3D points by introducing a tailored loss as in Shape As Points optimization [40] (see Section 2.3). In addition, a better-structured robot geometry is critical to obtain not only a more accurate evaluation of a robot design at the forward pass of the simulation but also the gradients from differentiable physics at the backward pass. In Figure 8, we further perform co-optimization on the two robots obtained by Direct SDF and DiffuseBot; we observe a more stably increasing trend of task performance in our approach, demonstrating that the gradient is usually more informative with a better-structured robot. Empirically, while Direct SDF sometimes still provides improving trend with unstructured robots in co-design optimization, the performance gain is not as stable as DiffuseBot.
F Additional Visualizations Of Experiments
In this section, we show more visualization for a diverse set of the experimental analysis. Please visit our project page (https://diffusebot.github.io/) for animated videos.
Generated robots performing various tasks. In Figure 9, we display a montage of a generated robot's motion for each task; this is the unabridged demonstration of Figure 5 in the main paper.
Physics-guided robot generation In Figure 10, we show how DiffuseBot evolves robots throughout the embedding optimization process; this is the full demonstration of Figure 4 in the main paper. Note
Figure 9: Examples of robots bred by DiffuseBot to achieve the all presented tasks.
that DiffuseBot is a generative algorithm and thus results presented are for demonstration purposes and reflect only a single fixed random seed.
Diffusion process of robot generation. In Figure 11, we show how robots evolve throughout the diffusion process at different diffusion time via  $\mathbf{x}_t$  with  $t$  from  $T$  to 0; not to be confused by Figure 10 where all generations are obtained via a full diffusion sampling  $\mathbf{x}_0$  and different conditioning embeddings  $\mathbf{c}$ . This figure demonstrates the robotized samples across diffusion times, gradually converging to the final functioning robots as  $t$  approaches 0.
Comparison with baselines. In Figure 12, we present robots generated from different baselines. Note that Point-E is essentially DiffuseBot without embedding optimization and diffusion as co-design.
Incorporating human feedback. In Figure 13, we showcase more examples of incorporating human textual feedback to the generation process of DiffuseBot. As briefly mentioned in Section 3.4 in the main paper, we leverage the compositionality of diffusion-based generative models [34, 14]. Specifically, this is done by combining two types of classifier-free guidance as in (3); one source
Figure 10: Examples of DiffuseBot evolving robots to solve all presented tasks.
of guidance is derived from the task-driven embedding optimization while the other is derived from embeddings from human-provided textual descriptions. These two conditional scores, namely  $\epsilon_{\theta}(\mathbf{x}_t,t,\mathbf{c}_{\mathrm{physics}})$  and  $\epsilon_{\theta}(\mathbf{x}_t,t,\mathbf{c}_{\mathrm{human}})$ , can then be integrated into the diffusion as co-design framework as in (6). We refer the reader to the original papers for more details and theoretical justification. We demonstrate examples of: the balancing robot with additional text prompt "a ball", the crawling robot with additional text prompt "a star", the hurdlebot with additional text prompt "a pair of wings", and the moving-a-box robot with additional text prompt "thick". Interestingly, human feedback is introduced as an augmented "trait" to the original robot. For example, while the hurdlebot keeps the tall, lengthy geometry that is beneficial for storing energy for jumping, a wing-like structure appears at the top of the robot body instead of overwriting the entire geometry. This allows users to design specific parts for composition while also embedding fabrication and aesthetic priorities and constraints through text. We leave future explorations of these ideas for future work.
Figure 11: Demonstrations of generations changing from noises to robots with physical utility throughout the diffusion sampling process. We present snapshots at diffusion times  $t = 600, 400, 200, 0$  for all presented tasks.
G Hardware Design and Fabrication
To create a real-world counterpart of the DiffuseBot, we fabricated a proof-of-concept physical robot gripper, as illustrated in Figure 14(a). The gripper was designed to match the shape of the digital gripper and successfully demonstrated its ability to grasp objects, as shown in Figure 15. A video demonstrating the grasping process can be found on our project page (https://diffusebot.github.io/).
During the translation from the simulated design to a physical robot, we aimed to minimize differences between the simulated and physical designs. However, due to hardware and fabrication limitations, certain modifications were necessary.
One such challenge involved replicating the arbitrary contraction force at the particles from the simulation in the physical world. To address this, we employed tendon transmission as the actuation method for the real-world gripper, which was found to be suitable for emulating interaction forces
Figure 12: Comparison of robots generated by different baselines.
from the digital world without introducing significant discrepancies. To enhance the stability of the tendon-driven actuation, we incorporated a rigid base ("Base" in Figure 14(a)) above the gripper itself ("Gripper" in Figure 14(a)). This base was designed to withstand the reaction force generated by the tendon-driven actuation. Furthermore, we added four tendon routing points (represented by small dotted circles in Figure 14(b)) on each finger to securely fix the tendon path. By utilizing four Teflon tubes (shown in Figure 14(b)), we were able to position complex components such as motors, batteries, and controllers away from the gripper, reducing its complexity.
The actuation strategy of the simulated gripper applies equal contraction force to both fingers simultaneously. To replicate this strategy, we employed underactuated tendon routing in the development of the gripper. This approach eliminates the need for four separate actuators, thereby reducing the complexity of the robot. We used tendon-driven actuators specifically designed for underactuated tendon-driven robots as they solve practical issues such as size and friction issues that commonly arise in such systems [30].
Figure 13: More examples of incorporating human textual feedback into robot generation by DiffuseBot.
(b)
(c)
Figure 14: (a) shows a fabricated proof-of-concept physical robot gripper; (b) describes detailed robot configuration; (c) represents interior structure design used to soften the robot body.
The soft gripper was 3D-printed using a digital light projection (DLP) type 3D printer (Carbon M1 printer, Carbon Inc.) and commercially available elastomeric polyurethane (EPU 40, Carbon Inc.). To enhance the softness of the robot body beyond the inherent softness of the material itself, we inflated the finger body with a Voronoi lattice structure, a metamaterial useful for creating a soft structure with tunable effective isotropic stiffness [20, 36]. We generated a Voronoi lattice foam with point spacing of  $2.5\mathrm{mm}$ , and a beam thickness of  $0.4\mathrm{mm}$  as shown in the red boundary of Fig. 14(c). Finally, we tested the soft gripper, designed and fabricated as described above, to verify its ability to grasp an object, as shown in Figure 15.
More discussion on fabrication. Although, at present, the compilation of the virtual robot to a physical, digitally fabricated counterpart involves manual post-processing of algorithm's output, most, if not all of these steps could be automated. Our method outputs a point cloud (defining geometry), actuator placements, and an open-loop controller, along with a prescribed stiffness. Since we can easily convert the point cloud into a 3D triangle mesh, the geometry can be created by almost any 3D printing method. In order to realize an effective stiffness and material behavior, stochastic lattices, specifically Voronoi foams, have been used [36, 20] in the past and employed here in order to match target material properties. Given the actuator placement, tendons [27, 30] can be aligned with the prescribed (contiguous) regions. Since a lattice is used, threading tendons through the robot body is simple, and we note that even more complex routings have been studied in detail in the literature [4]. Creating attachment points for the tendons is a relatively simple geometry processing problem [6]. Thus, converting a virtual robot to a specification that incorporates geometry, material, and actuation can be automated in a straightforward way.
(b)
Figure 15: Grasping motion of the real-world soft gripper from DiffuseBot. Time order from (a) to (c). The demo video can be seen at our project page.
We note that when controlled, the physical robot may not always match the virtual robot's motion. This is the sim-to-real gap, and is significantly harder to overcome in our case than translating the virtual robot to physical hardware. Significant literature has been invested in specifically tackling the sim-to-real gap, and in our case would require its own dedicated project; however, we note that often hardware can be adapted to work by modifying only the control policies using feedback from real-world experiments, often even with little human intervention [21].
Quantitative analysis. In Figure 16, we provide a detailed analysis on the gap between simulation and physical robot. In order to explore the quantitative gap between the behavior of the physical robot and the simulated robot, we conducted an experiment with the following conditions, where similar setups are commonly adopted in soft robot literature [17]. The objective was to measure the change in distance between two tips when we pull/release two tendons - one for closing the gripper (flexion) and the other for opening it (extension). The tendons were pulled or released in increments and decrements of  $2\mathrm{mm}$ .
When contracting the tendon to flex or extend the fingers, both simulation and real robot results show log-shaped graphs. The pattern in the physical robot plot is a commonly observed phenomenon called hysteresis. However, the main difference between the simulation and real-world cases can be seen when releasing the tendon from a fully contracted state. In the real robot experiment, the tip distance changes rapidly, while in the simulation, the opposite effect is observed.