2020 3rd IEEE International Conference on Soft Robotics (RoboSoft)
Yale University, USA

Scalable sim-to-real transfer of soft robot designs
Sam Kriegman1 ,Amir Mohammadi Nasab2 , Dylan Shah2 , Hannah Steele2 , Gabrielle Branin2 ,Michael Levin3 , Josh Bongard1 , Rebecca Kramer-Bottiglio2
1

Fig. 1.

University of Vermont, 2 Yale University, 3 Tufts University

The top 100 simulated 2-by-2-by-2 configurations of passive (cyan) and volumetrically-actuating (red) voxels (a) were manufactured in reality (b).

Abstract—The manual design of soft robots and their controllers is notoriously challenging, but it could be augmented—or,in some cases, entirely replaced—by automated design tools.
Machine learning algorithms can automatically propose, test,
and refine designs in simulation, and the most promising ones
can then be manufactured in reality (sim2real). However, it iscurrently not known how to guarantee that behavior generated
in simulation can be preserved when deployed in reality. Although many previous studies have devised training protocols
that facilitate sim2real transfer of control polices, little to nowork has investigated the simulation-reality gap as a functionof morphology. This is due in part to an overall lack of tools
capable of systematically designing and rapidly manufacturing robots. Here we introduce a low cost, open source, andmodular soft robot design and construction kit, and use it tosimulate, fabricate, and measure the simulation-reality gap ofminimally complex yet soft, locomoting machines. We prove
the scalability of this approach by transferring an order ofmagnitude more robot designs from simulation to reality thanany other method. The kit and its instructions can be found here:
github.com/skriegman/sim2real4designs

I. I NTRODUCTION
The simulation-reality gap1 for rigid-bodied robots is
steadily closing. Computational models of rigid body dynamics can now be regularized and tuned so that control
policies optimized in simulation are just as successful whentested on the physical system [2, 11]. The reality gap forsoft robots, on the other hand, remains uncharted. It could
be wider than the gap for rigid bodies, or not. Soft bodies are
more challenging to accurately simulate, design, and precisely
1 Henceforth, “the reality gap”—as coined by Jakobi et al. [12].

978-1-7281-6570-7/20/$31.00 ©2020 IEEE

control. But, they are also, by definition, more permissive tosimulation inaccuracies, design flaws, and control precision: A
soft gripper or foot will passively conform to complex objects
and terrain, reducing the burden on the simulator to perfectly
capture any single, “true” surface contact geometry.
Quantifying which soft robot designs, policies and behaviors
can be faithfully simulated is critical not only for robotics[14], but also synthetic approaches to understand functional
plasticity of biological systems during development and regeneration [16]. For both domains, testing candidate hypotheses
in reality is expensive, time consuming, and, in some cases,
dangerous. With the recent development of several high-space,many-body, GPU-accelerated soft body simulators [10, 23],
sim2real for soft robotics and synthetic biology has become
more feasible. However, because these simulators have yet tobe employed to design physical systems, their transferability
is currently unknown.
Previous work has demonstrated methods that promote
successful sim2real transferal of soft object manipulation but
not soft robot behavior. For example, a rigid-bodied robot arm
was successfully trained in simulation to fold towels and drape
pieces of cloth over a hanger [25]. However, the reality gap
was not quantified beyond a binary success rate for each task.
Additionally, the robot’s geometry was fixed and controllers
were then optimized for it, whereas in the work reported here,
the robot’s geometry is part of the solution space.
Hiller and Lipson [8] evolved the overall geometry anddistribution of hard and soft materials in simulation, andtransferred the structures and passive dynamics of various

359

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

cantilever beams. In a separate experiment that included actuating materials, Hiller and Lipson evolved the morphology
and behavior of soft robots in simulation, and then built
one of the evolved designs physically. However, in order totransfer the simulated behavior of this one design, the physicalrobot needed to be placed in a pressure and vacuum chamber,whereas the hundreds of soft robot designs built here can beinternally pressurized and actuated.
More recently, Kriegman et al. [15] subjected a simulated
soft robot (composed of elastic voxels) to a series of damage
scenarios that removed increasingly more of the robot’s structure. In each scenario, the robot was challenged to recoverfunction (locomotion) by deforming its remnant structure,
without changing its predamage control policy. A pair ofrecovery strategies, automatically discovered by an evolutionary algorithm in simulation, were transferred to reality (using
silicone “voxels”), but function was not: The physical system
could deform its resting structure as dictated by the recovery
strategy, but it could not locomote, before or after damage.
The physical robot was heavy, had high friction feet, and wassymmetrically actuated in phase, so it just oscillated in place.
To determine the particular challenges and opportunities ofsoft robot transferal, it would be useful to greatly scale up
the number of design/policy pairs transferred. To this end,
we present a soft robot design and construction kit based on
the silicone voxel modules used in [15], but miniaturized toincrease stability, simplified to improve reproducibility, andarbitrarily actuated to permit the transferal of locomotion.
Other modular yet rigid-bodied robot design and construction kits exist, such as Molecubes [37]. However, our kit iseasier, faster, cheaper, and safer to use. In short, silicone ismolded into hollow voxels, and tubing is attached to supply
low pressure actuation from a hand pump, causing volumetricchanges in one or more of the voxels (Figs. 2 and 3). For
simple behaviors robust to actuation noise, there is no need
to use a highly-pressurized air supply or program microcontrollers for open-loop control. There are also no expensive
motors or power supplies.
Here, we employ the kit as an instrument to measure thereality gap as a function of morphology (Table I). To doso, we fabricated 108 morphologies (transferal of structure)and compared the behavior of nine simulated designs to theirsilicone equivalents (transferal of behavior). We hope that the
kit’s affordability, safety, speed, and simplicity will generateincreasingly more, and more reproducible, data about the
automated design of increasingly competent soft machines.

TABLE IS UMMARY OF PUBLISHED SIM 2 REAL TRANSFERENCE .
Author/citation

Year

Controllers

Morphologies

Miglino et al. [26]Jakobi et al. [12]Harvey et al. [7]Lipson and Pollack [21]Bongard et al. [2]Hiller and Lipson [8]Koos et al. [13]Moeckel et al. [27]Caluwaerts et al. [3]Cully et al. [5]Cellucci et al. [4]Tobin et al. [36]Rusu et al. [34]Peng et al. [29]Pinto et al. [32]Tan et al. [35]Golemo et al. [6]Matas et al. [25]Kwiatkowski and Lipson [17]Hwangbo et al. [11]Kriegman et al. [15]Nachum et al. [28]Akkaya et al. [1]Rosser et al. [33]Kriegman et al. [16]The results presented here

1994
1995
1997
2000
2006
2011
2012
2013
2014
2015
2017
2017
2017
2018
2018
2018
2018
2018
2019
2019
2019
2019
2019
2019
2020
2020

1
2
4
3
34
1
2
1
2
10
1
1
1
1
3
2
1
3
2
3
1
3
1
1
1
1

1
1
1
3
2
5
2
1
1
10
3
1
1
1
1
1
1
1
2
1
5
1
1
16
5
108

B. The simulation.
We used the soft-body physics engine Voxelyze [9] tosimulate robots composed of actuating and/or passive, elastic
voxels. The simulator models the distance between adjacentvoxels as Euler-Bernoulli beams (critically damped; ζ = 1).
Additionally, a collision detection system monitors the distance between the voxels on the surface of the robot at eachtimestep. If a pair of surface voxels are detected to collide(intersect), a temporary beam (underdamped; ζ = 0.8) isconstructed between the two until the collision is resolved.
Designs were simulated with a gravitational acceleration of-9.81 m/s2 , and initialized on top of an infinite surface planeat z = 0. Coulomb friction is applied to voxels in contact withthe surface plane. Voxels were simulated to have 1 cm3 resting
volume (resting beam lengths), with Young’s modulus 107
Pa, Poisson’s ratio 0.35, and coefficients of static and kinetic
friction of 1 and 0.5, respectively. These hyperparameters wereadopted from [15]. For more details about how the physics are
actually modeled, see [9].
Volumetric actuation was implemented by varying the rest
length between voxels, in all three dimensions, when computing the elastic force between them. Volumetric expansion

II. M ETHODS
A. The design space.
Following [8] and [15], our kit uses elastic voxels as building blocks of structure. Here, we considered a 2-by-2-by-2
cartesian lattice workspace, within which voxels were connected together to form a robot. At each x,y,z coordinate,voxels could either be passive, volumetrically actuated, orabsent, yielding a total of 38 = 6561 different configurations.
We evaluated each configuration in simulation.

Fig. 2. A random morphology in the design space shown at atmospheric(resting; a), positive (expanding; b), and negative (compressing; c) pressure.

360

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

where xt , yt are the final coordinates of the design at the endof the evaluation period.

Fig. 3. Manufacturing modular soft robots. Hollow, silicone voxels werecreated by partially filling an open-face mold with silicone (a), using a spatulato spread it along the interior walls (b), and then securing the mold to a 1-axis
rotational molding machine (c). This process allowed excess silicone to dripout of the mold, while spreading the remaining silicone into a thin uniform
layer. The cured, bottomless voxels were then appropriately arranged andconnected for each x,y slice of the design, and bonded with a shared bottom
layer (d). Finally, tubing was attached (e), and the slices were stacked andbonded to form the design (f). Video: youtu.be/jbQ2T7jIYRU.

in simulation and reality are both roughly spherical (Fig. 2b),but compression in reality is more complex and difficult tosimulate: the voxels buckle (Fig. 2c). So volumetric actuation
was here limited to expansion only (+90% rest volume). The
active voxels expand in phase with each other as dictated bya central pattern generator: a sine wave with frequency 4 Hz
and amplitude 1.9 cm3 . When the sine wave is at or below
zero, the active voxels remain at their resting volume (1 cm3 ).
This produced quasistatic dynamics.
Each design was simulated for 8831 timesteps, with astepsize of 0.000453 seconds, resulting in a total simulation
time of 4 seconds. During the first 552 times steps (0.25 sec),
the design was allowed to settle under gravity before actuation
begins, ensuring that movement (if any) is a result of actuation,rather than passively falling forward. Just before actuation, thedesign’s initial center of mass is recorded as (x0 , y0 , z0 ). The
active voxels are then actuated for 3.75 sec at 4 Hz, or 15
actuation cycles.
An exhaustive search of all 6561 designs (in batches of50) took 58 CPU hours (1.8 wall-clock hours) on a single
AMD Ryzen threadripper 1950X 16-core/32-thread processor.
Fitness was taken to be the net displacement (away from theorigin in any direction) of the design’s center of mass, in termsof euclidean distance in the plane, where the origin is definedby the x, y components of the design’s initial center of mass
(x0 , y0 ). Fitness is thus defined as:p
F = (xt x0 )2 + (yt y0 )2 ,(1)

C. Reality.
Following Kriegman et al. [15], simulated voxels wererealized physically as pneumatically-actuated, hollow silicone
voxels. The physical robot in [15] was constructed to transfer
symmetrical shape change, so its actuated voxels were distributed symmetrically and hooked into a single pressure inlet.
Thus, pressure oscillations occurred symmetrically in phase,and the robot could only pulse in place. Moreover, due tothin voxel walls relative to overall voxel size, and the tubing
and glue used to bond them together, the robot in [15] could
not fully support its own weight. The robot was lifted off theground by placing it on top of a small petri dish, positioned
underneath a segment of entirely passive voxels in the centerof the robot’s ventral surface. This permitted ventral (and more
extreme global) changes in surface curvature, yielding successful sim2real transfer of shape change, but not locomotion.
The construction kit presented here rectifies the weight issue
by miniaturizing the voxels—voxel length was halved (from
3cm to 1.5cm) and the wall thickness remained the same(1mm), reducing voxel mass from 4.3g to 1.2g (including
tubing but not pneumatic connectors). Further, the inter-voxel
tubing and glue was replaced with holes punched through thewalls of adjacent active voxels in the same x,y slice, before
attaching them with a shared bottom layer (Fig. 3d). Finally,
locomotion is now possible because separate contiguous sections of voxels in each slice can be arbitrarily actuated in orout of phase with other sections across the body.
D. The build protocol.
The voxels were manufactured using a single-axis rotational
molding machine.2 First, an open-face mold was fabricated byinterlacing 26 acrylic strips into a flat base, to form a latticeof cubic concavities, resembling an ice-cube tray (Fig. 3a).
Mold components were laser-cut (VLS2.30, Universal Laser
System) from a flat acrylic sheet with a thickness of 0.025
inch. Next, silicone (Dragon Skin 10 Fast; Smooth-On, Inc.)
was poured into the acrylic mold (Fig. 3a), and a spatula wasused to spread the silicone along the interior walls of eachcavity (Fig. 3b). Colored pigment was added to each batch ofsilicone to indicate whether the voxel was active or passive,
simplifying the assembly process. Here we used pink for active
voxels and blue or yellow for passive voxels.
The mold was then flipped upside down and secured to a
1-axis rotational molding machine. The machine was clamped
to a table with binder clips, angled 45◦ relative to horizontal,
and set to rotate 90◦ every 45 seconds (Fig. 3c). This allowed
the silicone to flow and evenly coat the walls of the mold, as
excess silicone dripped out. After the voxels partially cured for
25 minutes at room temperature, the mold was moved to an
incubator, with a temperature of 60◦ C for another 20 minutes.
(Without an incubator, the silicone will take 75 minutes tofully cure at room temperature.)
2 The required materials are listed at the end of the manuscript in Table II.

361

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

quinary control space was mapped to 2D, by nesting pairs ofdimensions within each other.
Here, the 8D ternary morphology space was reduced to
2D by plotting pairs of dimensions nested within each other(Fig. 5). The pixel in the exact center of Fig. 5, for instance,
represents the configuration consisting entirely of passive
voxels, and thus cannot locomote (F = 0). Likewise, thepixel in the top right-hand corner of the heatmap represents
the configuration of all active voxels (Fig. 6d), which actuatedsymmetrically in phase, and thus (given its flat ventral surface)
could not locomote across the flat ground plane (F = 0).
Finally, the pixel in the bottom left-hand corner contains novoxels at all, and thus F = 0.
For locomotion, a good design obviously needs to havea body, rather than none at all. With open-loop, in-phaseactuation, designs also need to have asymmetrical mass and/or
actuator distributions, or they will not generate any forward
movement. However it is not clear, even for this minimal
design space, exactly which asymmetrical designs will yield
the highest fitness. Yet we can see small clusters and linesof similarly colored pixels in Fig. 5, representing morphologically similar designs with similar fitness. This suggeststhat these configurations and substructures would be relatively
stable under random mutations or errors in fabrication.
Because fitness was measured by displacement in any
direction away from the origin (Eq. 1), there are four configurations—rotations, in the x,y plane, of a single geometry
and distribution of passive and active voxels—with different

active

The above steps were then repeated to add an additional
layer of silicone. Once the second layer cured, the bottomlessvoxels were removed from the mold using an X-Acto knife,and excess silicone around their edges was trimmed.
In the next step, each x,y slice (or dorsal plane) of thedesign was assembled by using Sil-Poxy (Smooth-On, Inc.) tobond adjacent voxels and prevent the slice from shifting. Holes
were then punched between adjacent active voxels so that
contiguous collections of voxels could be actuated together inphase. Each actuator group needed to contain at least one voxelon the surface of the design so that it could be controlled by an
external pressure inlet. To create the bottom layer, two 1mmthick rulers were attached to an acrylic substrate using doublesided tape and silicone was poured in the space between them.
Then, the slice of bottomless voxels was flipped, open-side
down, onto this uncured silicone layer (Fig. 3d).
After the bottom layer cured, a thin layer of silicone wasapplied with a popsicle stick along the outermost portions of
the interstices of the voxels, bonding adjacent voxels (without
gluing over inter-voxel holes). Then, the slice was cut fromthe silicone sheet and a hole was poked into the side of one
exterior voxel from each group of active voxels. Next, a 1/32”
ID silicone tube was inserted into the hole, and glued in placewith Sil-Poxy, applied with a Q-tip (Fig. 3e). The end of thistube was then connected to a straight pneumatic connector,
which was connected to 1/16” ID silicone tubing.
Occasional imperfections in alignment, silicone thickness,or inter-voxel hole sizes would result in leaky structures. Leaks
were detected by filling a beaker with water, submerging thevoxels, and inflating them. Bubbles would emanate from leaks,
which were repaired with Sil-Poxy. After repairing any leaks,
the slices were stacked on top of each other and bonded
together using a thin layer of silicone (Fig. 3e). Finally, theselayers were connected pneumatically with assorted pneumaticconnectors, attached to 1/16” ID silicone tubes.

none

To test the effects of morphology on fitness and sim2realtransfer success, it is useful to first visualize the design space.
However, because there areeight cartesian voxel coordinates in the chosen workspace,the design space here is eight
dimensional, which is difficult
to draw (let alone conceptualize) without dimensionality
reduction. By nesting the dimensions of a search space
onto a single plot (Fig. 4), theentire space can be visualized
Fig. 4. The 2D tessellation of 8D
as a 2D heatmap. This strategy ternary vector space used in Fig. 5.
was used by Cully et al. [5] toneatly visualize the predicted fitness of a very large libraryof control policies, as a function of the time a robot’s six
limbs were in contact with the simulated ground plane: 6D

vox 2
passive

III. R ESULTS

none

passive
vox 1

activeFig. 5. Simulating modular soft robots. The design space is plotted asa heatmap, containing one cell for each of the 6561 possible configurations.
Lighter colored cells are fitter designs (Eq. 1). Each design is defined by avector of eight ternary values, indicating what kind of voxel (none, passive,or active) the design contains at the eight lattice points in the 2 × 2 × 2
workspace. The 8D ternary vector is reduced to a 2D heatmap by nestingpairs of dimensions within each other: four, nested 3 × 3 grids result in a
34 × 34 = 81 × 81 overall heatmap.

362

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

Fig. 6. Measuring transferal from simulation to reality. Nine designs (a-i) were evaluated three times each in reality (green-to-blue gradient colored curves
in a'-i'). The behavioral trajectories start at the origin (green) and end at the robot’s final XY destination (blue) (in centimeters). The simulated movement
tracks (yellow-to-pink curves) are superimposed on top of the real ones. The relative volume (normalized by rest volume) was also recorded for each design
at four points during actuation under water (a''-i''). The simulated and real behavior of designs e and f can be observed here: youtu.be/UqjvmkYa9u4.

behaviors (they moved in different directions) but very similar(if not identical) fitness. There were also some configurations
that, when rotated upward (in the x,z or y,z plane) fell into
the same basic orientation and behavior but with a slightly
different heading. Thus, configurations with similar fitness(similarly colored pixels) are reflected across multiple, nested
planes of symmetry in Fig. 5. These symmetries can also beseen in the manufactured robots (Fig. 1b). The uniqueness ofdesigns (i.e., the size of the search space of morphologies) istherefore a function of how behavior is measured.
Fig. 6 shows the behavior of nine different designs in simulation and reality. The real robot was actuated 90 times at 6 kPapressure on a surface covered with cornstarch (Argo®, ACH
Food Companies, Inc.) to reduce friction, and is compared to
23 simulated actuation cycles. Seven of the nine designs filled
the cubic workspace with passive and active voxels, while theother two share a more complex geometry: a single-voxel limb
attached to the face of a 2-by-2 plane of voxels (Fig. 6e,f).
In one, the limb is active (Fig. 6e), in the other it is passive(Fig. 6f). These two designs achieved the two highest fitness
scores (Eq. 1), in both simulation and reality.
By this measure, the reality gap appears small. However,these simulated designs move very differently from their manufactured equivalents. The simulated morphology in Fig. 6epushes off its active limb, whereas in reality the design uses its
limb to pull itself forward, in the opposite direction. Likewise,
the simulated morphology in Fig. 6f pushes off its active 2by-2 torso, whereas in reality the design uses its torso to pullitself forward, in the opposite direction.
Majidi et al. [24] showed that the interfacial shear strengthand coefficient of friction of the surface on which their soft
robot undulated determined the direction of locomotion. They
decomposed friction into load- and area controlled terms forpoint and surface contacts, respectively. On slippery surfaces
with low interfacial shear resistance, the robot anchored about
the point contact (expanded section) for locomotion and pulled
its surface contact (passive segment). However, on surfaces
with high interfacial shear resistance, the robot anchored about
the surface contact and pulled the point contact toward it.
We hypothesize that such differences in tribological properties
could have caused our designs to move in opposite directions
in simulation and reality.
In an attempt to test this hypothesis and reduce thesimulation-reality discrepancies that cause the virtual configurations in Fig. 6 to move differently than their physicalrealizations, we performed a grid search of various simulation
hyperparameters, including the coefficients of static and kinetic friction. However, we could not identify a pair of friction
coefficients that resulted in correct movement heading for allnine of the behaving designs (Fig. 6a'-i'). This could be due toeither low precision or low accuracy of the model. To isolate
and test the former possibility, we increased the resolutionof the simulated surface contact geometry by modeling eachsilicone voxel as a 3-by-3-by-3 group of simulated “subvoxels”(Fig. 7), and then re-ran the parameter sweep. Still, we could
not find friction settings in which the simulated movement
direction matched the ground truth across all designs simultaneously. This suggests that the accuracy of Coulomb friction
model may be insufficient to model this type of movement.
The Coulomb approximation assumes that friction is simply

363

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

Fig. 7. A higher resolution model in which each silicone voxel is approximated by a 3-by-3-by-3 group of simulated subvoxels: a high-res voxel.
The design in a and b are high-res instantiations of those in Figs. 6e and6f, respectively. Spherical volumetric expansion in a high-res voxel (c) wasapproximated by increasing the rest length between the centermost subvoxel
and the subvoxels at center of each face (green subvoxels in d).

proportional to the vertical component N of the reaction force,and independent of the contact area. However, friction is also
a function of the surface area and interfacial shear strengthτ , a fixed constant which is mostly governed by adhesion
or mechanical interlocking between the contacting surfaces.
A better model would thus consider friction as a functionof both the normal force and the interfacial shear strength.
However, before fundamentally changing the simulator, weplan to evaluate designs in noisy environments with imperfectcontrol over actuation characteristics to avoid ascribing highfitness to designs that exploited unrealistic properties of thesimulation [12]. Additionally, data from reality could be used
to automatically tune the geometry and resolution of the simulated finite elements [2], or to predict the kinds of behaviors
that are more likely to successfully transfer [13], and whichshould be tested next [2]. Concurrently, we are investigating
additional physical surfaces with varied tribological properties
in an attempt to match reality to simulation.
IV. D ISCUSSION
In this paper, we introduced a low cost, open source
platform for designing and rapidly building soft robots, andused it to transfer 108 different morphologies (voxels on acartesian grid) from simulation to reality. We then measured
the reality gap as function of the robot’s design (geometry anddistribution of passive and actuating voxels) by tracking the
behavior of nine transferred morphologies. Under one measure(net displacement) the reality gap appeared rather small, but
under another (velocity) the gap was much wider, likely due
to oversimplified tribological contacts between the simulated
ground plane and the robot’s ventral surface [24].
Although most of the transferred designs (99 out of the108) were not actuated in reality, they nevertheless served an
important function: they were sketches. Sketches let us think
more clearly about the behavior or properties (e.g., stability)of a design without investing the additional time and resources
required to fully build and examine the design itself. Sketches,in other words, greatly increase the breadth of exploration indesign space. All sim2real methods embrace this utility ofsimplifying sketches. Simulation, after all, is also a sketch.
However, there is a tacit assumption in robotics about DepthFirst. A typical sim2real experiment begins by sending acomplicated robot design across the reality gap, and thenendeavors to learn transferable policies that control the morphology in all its complexity. But this is not how most design
proceeds. An architect first roughly sketches a structure, say,a bridge, on the back of a napkin. A diversity of designs
are then generated, tweaked, discarded or provisionally accepted—at this shallow level of napkin realism—before moredetailed blueprints are drawn under more stringent constraints.
Blueprints, too, undergo this breadthwise evolution, before themost promising are realized physically, first as scale models(built from matchsticks and glue instead of concrete and steel),then, finally, at full scale and cost. This incrementally weeds
out nontransferable features and adds mechanical complexity
only when and where it is necessary to do so, rather thanglobally from the outset.
The assumption that the reality gap can be bridged by policy
search alone, with a single robot design, is groundless. The
desired behavior of a robot is typically much more complicatedthan that of architecture. This suggests the necessity of more,
not less, sketches. Soft robots are more complicated still. This
makes their automated design that much more appealing, but
implies the need for even greater breadth of sketches, at moreintermediate levels of realism. Though not every experimentwill need to start from a blank slate. Instead, designers(whether human or AI) could leverage prior knowledge toreject truly awful designs before sketching them in the firstplace. The designs transferred here add to a growing database(prior probabilities) about which and how well different soft
robot designs and behaviors can be realized physically. Ourconstruction kit has the potential to further increase this databy lowering not only cost and build times but also the barrierof entry to soft robotics for non-experts.
The generality of such data beyond robotics is currently
not known, but it could also have important implications fordevelopmental biology and regenerative medicine. The bioelectric and genetic control policies that orchestrate adaptive
remodeling of growth and form in organisms are not yet
understood, but could, in future, be reverse-engineered bymachine learning, and then controlled externally to induce
regeneration in otherwise non-regenerative organisms (suchas adult humans), or to reprogram otherwise unbounded cancerous growth toward functional organogenesis [18, 19, 20,
22, 30, 31]. However, such advances in regenerative medicine
and synthetic morphology will only be possible if hypotheses
generated in simulation are transferable and testable in reality.
ACKNOWLEDGEMENTS
Sponsored by NSF award EFRI-1830870, DARPA contract
HR0011-18-2-0022, and NASA STRF-80NSSC17K0164.

364

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

TABLE II
T HE CONSTRUCTION KIT FOR MAKING FIFTY 2- BY-2- BY-2 DESIGNS .
No.

Part

Cost

Notes

1
2
3
4
5
6

Dragon Skin 10 FAST, 2 lbs kit (Smooth-On, Inc.)
Sil-Poxy, 3 ounce tube (Smooth-On, Inc.)
1/32” ID silicone tubing, 52 ft
1/16” pneumatic plastic connectors, 200 pieces
1/16” ID silicone tubing, 65.6 ft
60 mL Plastic Syringe (McMaster–Carr, Inc.)$32.21$30.72$34.32$100$62.98$3.13

Created voxel bodies and connected x,y slices.
Secured tubing and repaired air leaks in six-sided voxels.
50A shore hardness; connected voxels to actuation system.
Straight and T shaped.
50A shore hardness; part of actuation system.
Used to hand-actuate voxels while checking for leaks.

71000 mL beaker (PYREX™ VISTA™ Griffin, Fisher
Scientific International, Inc.)$14

Voxels were inflated in this beaker filled with water to detect leaks.

8
9
10
11
12
13
14
15
16
17
18
19
20

Hole punch 1/4 rectangle (Fiskars)
X-Acto Knife (McMaster–Carr, Inc.)
100 mL mixing cups (VWR international), pack of 100
Spatula (McMaster–Carr, Inc.)
2 Scrap acrylic sheets, 12”×12”×1/8”
2-1mm thick 30 cm metal rulers
Double-sided tape
Simple 30 cm metal ruler
Popsicle stick (11.3 cm × 1 cm), box of 1000
Cotton-tipped applicators, 6 inch, box of 1000
Disposable Gloves (Halyard Inc.), box of 100
2 Acrylic sheets, 12”×12”×1/8” (McMaster–Carr, Inc.)
41 mm binder clips, pack of 12$10.35
$4.11
$66
$7.50
$10.58
$6.99
$4.99
$3.99
$13.49$8.99$8.95$18.30$7.99

Created holes between voxels within a slice so they actuated as a unit.
Cut bottomless voxels out of acrylic mold, and trim the edges.
Where silicone was mixed.
Spread the Dragon Skin on the edges of the acrylic mold.
Collected scrap Dragon Skin; the surface where voxel bottom were created.
Used to set a thickness for sixth voxel side (bottom layer).
Adhered the rulers to a scrap acrylic sheet.
Spread a thin sheet of Dragon Skin onto scrap acrylic sheet for sixth side.
Applied thin layer of silicone to bond adjacent voxels and x,y slices.
(McKesson Corp.) Used to spread Sil-Poxy.
Wore when handling uncured silicone.
Used to manufacture the open-face acrylic mold with laser-cut.
Held the acrylic mold onto the rotational molding machine.

Acrylic sheet, 12”×12”×1/4” (McMaster–Carr, Inc.)
Acrylic sheet, 12”×12”×1/4” (McMaster–Carr, Inc.)
Pololu 4756 DC rotational motor
Pololu 1999 mounting hum
Arduino Uno Microcontroller
Arduino Motor Shield
12 V Power Supply
8020 T-Slotted Solid 1” beams (McMaster-Carr, Inc.)
8020 screws and T-nuts“M3×10” screws
Irwin QuickGrip 12”×2.75” Clamp$17.34$17.34$39.95$7.95$22$19.95$10.42$12.31$7.92$1.24$15.99Supports for rotational machine; 2× triangular plates, 3× motor mount.
Mounting plate; holes were cut to minimize weight.
Used for rotational molding machine.
Used to mount rotational molding machine.
Controlled rotation timing and degree.
Controlled rotation motor.
Powers the Arduino; 12mm×2.1×5.5mm barrel jack.
2×10 cm, 2×40 cm; supports for rotational molding machine.
Connected 8020 beams.
Used to mount rotational molding machine.
Held rotational molding machine to table.

Grand total:$622

The rotational molding machine:
21
22
23
24
25
25
26
27
28
29
30

R EFERENCES[1] Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz
Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias
Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik’s cube with a robot hand. arXiv preprint, 2019. URL
https://arxiv.org/abs/1910.07113.
[2] Josh Bongard, Victor Zykov, and Hod Lipson. Resilientmachines through continuous self-modeling. Science, 314(5802):1118–1121, 2006. URL https://doi.org/10.1126/science.
1133687.
[3] Ken Caluwaerts, Jérémie Despraz, Atıl Işçen, Andrew P Sabelhaus, Jonathan Bruce, Benjamin Schrauwen, and Vytas SunSpiral. Design and control of compliant tensegrity robots
through simulation and hardware validation. Journal of theroyal society interface, 11(98):20140520, 2014. URL https://doi.org/10.1098/rsif.2014.0520.
[4] Daniel Cellucci, Robert MacCurdy, Hod Lipson, and SebastianRisi. 1d printing of recyclable robots. IEEE Robotics andAutomation Letters, 2(4):1964–1971, 2017. URL https://doi.
org/10.1109/LRA.2017.2716418.
[5] Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste
Mouret. Robots that can adapt like animals. Nature, 521:503–
507, 2015. URL https://doi.org/10.1038/nature14422.
[6] Florian Golemo, Adrien Ali Taiga, Aaron Courville, and PierreYves Oudeyer. Sim-to-real transfer with neural-augmented

robot simulation. In Proceedings of The 2nd Conferenceon Robot Learning, volume 87 of Proceedings of Machine
Learning Research, pages 817–828. PMLR, 2018. URL http://proceedings.mlr.press/v87/golemo18a.html.
[7] I. Harvey, P. Husbands, D. Cliff, A. Thompson, and N. Jakobi.
Evolutionary robotics: the sussex approach. Robotics andAutonomous Systems, 20(2):205 – 224, 1997. ISSN 0921-8890.
URL https://doi.org/10.1016/S0921-8890(96)00067-X.
[8] Jonathan Hiller and Hod Lipson. Automatic design and manufacture of soft robots. IEEE Transactions on Robotics, 28(2):457–466, 2011. URL https://doi.org/10.1109/TRO.2011.
2172702.
[9] Jonathan Hiller and Hod Lipson. Dynamic simulation of soft
multimaterial 3D-printed objects. Soft Robotics, 1(1):88–101,
2014. URL https://doi.org/10.1089/soro.2013.0010.
[10] Daniel Holden, Bang Chi Duong, Sayantan Datta, and Derek
Nowrouzezahrai. Subspace neural physics: fast data-driven
interactive simulation. In Proceedings of the 18th annual ACM
SIGGRAPH/Eurographics Symposium on Computer Animation,
page 6. ACM, 2019. URL https://doi.org/10.1145/3309486.
3340245.
[11] Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter.
Learning agile and dynamic motor skills for legged robots.
Science Robotics, 4(26), 2019. URL https://doi.org/10.1126/

365

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.

scirobotics.aau5872.
[12] Nick Jakobi, Phil Husbands, and Inman Harvey. Noise andthe reality gap: The use of simulation in evolutionary robotics.
In European Conference on Artificial Life, pages 704–720.
Springer, 1995. URL https://doi.org/10.1007/3-540-59496-5
337.
[13] Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Doncieux.
The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1):122–145, 2012. URL https://doi.org/10.1109/TEVC.
2012.2185849.
[14] Sam Kriegman. Why virtual creatures matter. Nature Machine Intelligence, 1(10), 2019. URL https://doi.org/10.1038/
s42256-019-0102-8.
[15] Sam Kriegman, Stephanie Walker, Dylan Shah, Michael Levin,
Rebecca Kramer-Bottiglio, and Josh Bongard. Automatedshapeshifting for function recovery in damaged robots. In
Proceedings of Robotics: Science and Systems, 2019. URL
https://doi.org/10.15607/RSS.2019.XV.028.
[16] Sam Kriegman, Douglas Blackiston, Michael Levin, and Josh
Bongard. A scalable pipeline for designing reconfigurable
organisms. Proceedings of the National Academy of Sciences,
117(4):1853–1859, 2020. URL https://doi.org/10.1073/pnas.
1910837117.
[17] Robert Kwiatkowski and Hod Lipson. Task-agnostic selfmodeling machines. Science Robotics, 4(26), 2019. URL
http://doi.org/10.1126/scirobotics.aau9354.
[18] Michael Levin. Morphogenetic fields in embryogenesis, regeneration, and cancer: non-local control of complex patterning.
Biosystems, 109(3):243–261, 2012. URL https://doi.org/10.
1016/j.biosystems.2012.04.005.
[19] Michael Levin. Reprogramming cells and tissue patterning via
bioelectrical pathways: molecular mechanisms and biomedical
opportunities. Wiley Interdisciplinary Reviews: Systems Biology
and Medicine, 5(6):657–676, 2013. URL https://doi.org/10.
1002/wsbm.1236.
[20] Michael Levin. The computational boundary of a “self”:
developmental bioelectricity drives multicellularity and scalefree cognition. Frontiers in Psychology, 10, 2019. URL
https://doi.org/10.3389/fpsyg.2019.02688.
[21] Hod Lipson and Jordan B Pollack. Automatic design andmanufacture of robotic lifeforms. Nature, 406(6799):974, 2000.
URL https://doi.org/10.1038/35023115.
[22] Daniel Lobo, Mauricio Solano, George A Bubenik, and Michael
Levin. A linear-encoding model explains the variability of thetarget morphology in regeneration. Journal of The Royal SocietyInterface, 11(92):20130918, 2014. URL https://doi.org/10.1098/
rsif.2013.0918.
[23] Miles Macklin, Kenny Erleben, Matthias Müller, Nuttapong
Chentanez, Stefan Jeschke, and Viktor Makoviychuk. Nonsmooth newton methods for deformable multi-body dynamics.
ACM Transactions on Graphics (TOG), 38(5):140, 2019. URL
https://doi.org/10.1145/3338695.
[24] Carmel Majidi, Robert F Shepherd, Rebecca K Kramer,
George M Whitesides, and Robert J Wood. Influence of
surface traction on soft robot undulation. The International
Journal of Robotics Research, 32(13):1577–1584, 2013. URL
https://doi.org/10.1177/0278364913498432.
[25] Jan Matas, Stephen James, and Andrew J Davison. Sim-toreal reinforcement learning for deformable object manipulation.
In Proceedings of The 2nd Conference on Robot Learning,volume 87 of Proceedings of Machine Learning Research, pages
734–743. PMLR, 2018. URL http://proceedings.mlr.press/v87/
matas18a.html.
[26] Orazio Miglino, Kourosh Nafasi, and Charles E. Taylor. Selection for wandering behavior in a small robot. Artificial Life, 2(1):101–116, 1994. URL https://doi.org/10.1162/artl.1994.2.1.

101.
[27] Rico Moeckel, Yura N Perov, Anh The Nguyen, Massimo Vespignani, Stéphane Bonardi, Soha Pouya, Alexander
Sproewitz, Jesse van den Kieboom, Frédéric Wilhelm, andAuke Jan Ijspeert. Gait optimization for roombots modular
robots—matching simulation and reality. In The IEEE/RSJ
International Conference on Intelligent Robots and Systems(IROS), pages 3265–3272, 2013. URL https://doi.org/10.1109/
IROS.2013.6696820.
[28] Ofir Nachum, Michael Ahn, Hugo Ponte, Shixiang Gu, andVikash Kumar. Multi-agent manipulation via locomotion using
hierarchical sim2real. In Proceedings of The 3rd Conference on
Robot Learning, 2019. URL https://arxiv.org/abs/1908.05224.
[29] Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, andPieter Abbeel. Sim-to-real transfer of robotic control withdynamics randomization. In The IEEE International Conference on Robotics and Automation (ICRA), 2018. URL
https://doi.org/10.1109/ICRA.2018.8460528.
[30] Giovanni Pezzulo and Michael Levin. Re-membering the
body: applications of computational neuroscience to the topdown control of regeneration of limbs and other complex
organs. Integrative Biology, 7(12):1487–1517, 2015. URL
https://doi.org/10.1039/C5IB00221D.
[31] Giovanni Pezzulo and Michael Levin. Top-down models inbiology: explanation and control of complex living systems
above the molecular level. Journal of The Royal SocietyInterface, 13(124):20160555, 2016. URL https://doi.org/10.
1098/rsif.2016.0555.
[32] Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech
Zaremba, and Pieter Abbeel. Asymmetric actor critic for imagebased robot learning. In Proceedings of Robotics: Science andSystems, 2018. URL https://doi.org/10.15607/RSS.2018.XIV.
008.
[33] Kent Rosser, Jia Kok, Javaan Chahl, and Josh Bongard.
Sim2real transfer degrades non-monotonically with morphological complexity for flapping wing design. arXiv preprint, 2019.
URL https://arxiv.org/abs/1910.13790.
[34] Andrei A. Rusu, Matej Večerı́k, Thomas Rothörl, Nicolas Heess,Razvan Pascanu, and Raia Hadsell. Sim-to-real robot learningfrom pixels with progressive nets. In Proceedings of the 1st Annual Conference on Robot Learning, volume 78 of Proceedingsof Machine Learning Research, pages 262–270. PMLR, 2017.
URL http://proceedings.mlr.press/v78/rusu17a.html.
[35] Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei
Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke.
Sim-to-real: Learning agile locomotion for quadruped robots.
In Proceedings of Robotics: Science and Systems, 2018. URL
https://doi.org/10.15607/RSS.2018.XIV.010.
[36] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech
Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world.
In The IEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS), 2017. URL https://doi.org/10.1109/IROS.
2017.8202133.
[37] Victor Zykov, Andrew Chan, and Hod Lipson. Molecubes: An
open-source modular robotics kit. In IROS Self-Reconfigurable
Robotics Workshop, 2007. URL http://molecubes.org.

366

Authorized licensed use limited to: Yale University. Downloaded on July 07,2020 at 19:33:53 UTC from IEEE Xplore. Restrictions apply.