2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
October 1-5, 2023. Detroit, USA

Real2Sim2Real Transfer for Control of Cable-driven Robots
via a Differentiable Physics Engine

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) | 978-1-6654-9190-7/23/$31.00 ©2023 IEEE | DOI: 10.1109/IROS55552.2023.10341811

Kun Wang1 , William R. Johnson III2 , Shiyang Lu1 , Xiaonan Huang2 ,Joran Booth2 , Rebecca Kramer-Bottiglio2 , Mridul Aanjaneya1 , and Kostas Bekris1
Abstract— Tensegrity robots, composed of rigid rods andflexible cables, exhibit high strength-to-weight ratios and significant deformations, which enable them to navigate unstructured
terrains and survive harsh impacts. They are hard to control,
however, due to high dimensionality, complex dynamics, anda coupled architecture. Physics-based simulation is a promising avenue for developing locomotion policies that can betransferred to real robots. Nevertheless, modeling tensegrity
robots is a complex task due to a substantial sim2real gap.
To address this issue, this paper describes a Real2Sim2Real(R2S2R) strategy for tensegrity robots. This strategy is based ona differentiable physics engine that can be trained given limited
data from a real robot. These data include offline measurementsof physical properties, such as mass and geometry for various
robot components, and the observation of a trajectory usinga random control policy. With the data from the real robot,
the engine can be iteratively refined and used to discover
locomotion policies that are directly transferable to the realrobot. Beyond the R2S2R pipeline, key contributions of thiswork include computing non-zero gradients at contact points,a loss function for matching tensegrity locomotion gaits, anda trajectory segmentation technique that avoids conflicts ingradient evaluation during training. Multiple iterations of theR2S2R process are demonstrated and evaluated on a real 3-bar
tensegrity robot.

I. I NTRODUCTION
Tensegrity robots are actuated systems composed of rigid
struts (rods) and flexible elements (cables) connected to form
lightweight, deformable structures. Their natural compliancemakes them adaptable and safe robots that are well-suited formany applications, such as manipulation [1], locomotion [2],morphing airfoils [3], and spacecraft landing [4].
At the same time, tensegrity robots are difficult to accurately model and control due to their many degrees of
freedom (DoF) and complex dynamics [5]. The difficultyin modeling has led some researchers to propose modelfree solutions for learning control [6], but these strategies still encounter challenges since they require a large
amount of training data, and collecting trajectories fromtensegrities is time-consuming, cumbersome, and expensive.
The authors have previously introduced a differentiable, but
modular and explainable, physics engine [7]–[9] as a dataefficient tool to identify physical parameters and developThis work has been supported by NSF Robust Intelligence award
1956027. The opinions expressed here are those of the authors and do notreflect the positions of the sponsor. 1 Computer Science, Rutgers University,
NJ 08901, USA. Email: {kun.wang2012, shiyang.lu,
mridul.aanjaneya, kostas.bekris}@rutgers.edu.
2 Mech. Eng. & Material Sc., Yale University, New Haven, CT, USA. Email:{will.johnson, xiaonan.huang, joran.booth,rebecca.kramer}@yale.edu.
978-1-6654-9190-7/23/$31.00 ©2023 IEEE

Fig. 1: The target platform: a 3-bar prismatic tensegrity robot with
9 stretch sensors. The 6 short cables are contracted and extended
by the motors in the 3D-printed housings on each rod.

locomotion policies for tensegrity robots. Yet, the transfer ofsimulated policies to real hardware is typically impeded by
the so-called simulation-to-reality (sim2real) gap [10], [11].
Sim2real transfer can be improved by tuning the simulation
to minimize differences between predicted and real robot
trajectories for the same controls.
To overcome the sim2real gap, many methods are applied [11]–[15], however, the robot is assumed to be
rigid [16] as a prior. Systems that include compliance, suchas tensegrity robots, are less frequently targeted, as compliance makes closing the sim2real gap more challenging.
Only recently have Real2Sim2Real (R2S2R) frameworks
been developed that focus on deformable elements (e.g., asystem that manipulates deformable cables [17]).
This work applies the principle of R2S2R to the control of
deformable robots composed of both rigid and soft elements.
More specifically, this work introduces an R2S2R pipeline
for tensegrity robots, through which a policy learned on adifferentiable engine is transferred to a 3-bar tensegrity robot(Fig. 1) by first training the engine with data from the realrobot. This paper contributes:• A complete pipeline for identifying the parameters of adifferentiable simulator from real tensegrity robot trajectories, generating locomotion policies in the simulator, andtransferring the policies back to the real robot.
• A method to compute non-zero gradients at contact points
to enable efficient learning of contact parameters in the
optimization step of the engine’s identification process.
• A loss function and trajectory segmentation strategy to
avoid conflicts in gradient direction during training undernoisy observations. Non-convex trajectories lead to gradients with opposite directions at different time steps. Thus,
the trajectory is segmented into convex segments to ensure
the gradient directions computed are aligned.

2534

Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

II. R ELATED W ORK
Sim2real transfer has been applied in autonomous underwater vehicles [18], drones [19], [20], muscles [21],quadruped robots [22], soft robots [23]–[25], and graspingmanipulators [26]. To the best of the authors’ knowledge,
this is the first work that mitigates the sim2real gap for atensegrity robot.
Differentiable physics has been actively applied to system identification. Compared to artificial evolution approaches (e.g., genetic algorithms, particle swarm optimizations, and covariance matrix adaptation evolution strategies)
and domain-randomization methods, gradient-based methods
like differential physics are data-efficient and can lead tofaster convergence for complex robot systems [27], [28]. We
previously developed a differentiable engine for tensegrity
robots [7]–[9], although prior work has been limited to only
sim2sim transfer and never demonstrated on a real tensegrity
robot.
Prior work on tensegrity locomotion [29]–[31] hasachieved complex behaviors, sometimes on uneven terrain,using the NASA Tensegrity Robotics Toolkit (NTRT) simulator [32], which was manually tuned to match a real platform [33], [34]. Many prior approaches use reinforcement
learning (RL) to learn policies given sparse inputs, which
can be provided by onboard sensors [30] and aim to address
the large data requirements of RL [31], including by training
in simulation. Simulated locomotion, however, is hard to
replicate on a real platform, even after hand-tuning, whichemphasizes the importance of training a simulator that can
produce policies that can be transferred to a real system. A
website accompanying this paper with videos and additional
evaluation is available1 .
III. ROBOT D ESIGNThis work demonstrates and evaluates the R2S2R pipelineon the untethered 3-bar prismatic tensegrity robot shown in
Fig. 1. The tensegrity robot has a rod length of 36 cm, and it
is driven by motors that extend and contract its cables to shift
its center of mass. The six short cables (three on each side)
are actuated by the motors while the three longer tendonsin the middle are passive elastic elements. These passive
tendons double as stretch sensors, and there are also six
sensor tendons in parallel with the six actuated cables. The
design and characterization of the stretch sensors are detailed
in previous work [35]. Each sensor is calibrated individuallyby fitting a linear model to map capacitance measurements tocorresponding lengths [36]. The stretch sensors are used bothfor feedback control and for pose estimation, as described in
section IV-B.
IV. R EAL 2S IM 2R EAL P IPELINE
The following discussion describes the various components of the proposed R2S2R pipeline as outlined in Fig. 2.
1 An appendix with addtional material and accompanying multimedia canbe found at: https://sites.google.com/view/sim2realFig. 2: R2S2R pipeline. A randomly generated policy is executed
first on the robot. An overhead RGB-D camera captures the
trajectory to estimate the robot’s state at each frame. Internal robot
parameters are then identified with a differentiable physics engine
given robot states and offline measurements. Given the identified
model, new policies are generated and executed on the robot. The
process can be repeated to further close the sim2real gap. Red
blocks correspond to real-robot tasks; blue blocks correspond tosimulation tasks. Labels correspond to sections in the text.

A. Random Policy and Real World Execution
The first step in the R2S2R pipeline is to generate a
random policy and execute it on the real robot. The policy
is defined as a list of robot cable lengths (0 means fully
contracted and 1 means fully extended) that represent a seriesof target robot shapes. To transition between these high-level
shapes, a PID controller extends or contracts the actuatedcables by sending low-level control signals based on the
feedback from the robot cable length sensors until the robot
reaches the next target shape. All policies start from the robot
rest state where the robot sits on the ground with the six
actuated cables fully extended, as shown in Fig. 1.
B. State Estimation
The next step is to estimate the robot states, i.e., theposition and orientation of each rod, at each time frame. We
adopt a state estimation method [37] designed for tensegrity
robots. This method tracks the 6-DoF pose of each rod given
a multi-modal input sequence of RGB images, depth images,
and measured cable lengths from the onboard stretch sensors.
The tracking method incorporates a variety of physical constraints, and it performs state estimation with high accuracy
and robustness to self-occlusions. Further, RANSAC [38] isused for ground detection so that contacts between end caps
and the ground could be found(Section IV-D).

Fig. 3: (Left) Measured physical parameters. Mass and length are
in grams and mm, respectively. (Right) Force-displacement curves
for long tendons (N = 11). A third-order polynomial was fit to eachcurve, and the average was used in the simulator.

C. Offline Robot Measurements
Physical parameters inside a simulator (e.g., mass, radiusof the end caps, lengths of the rods, inertia tensor, stiffness
coefficient, coefficient of friction, etc.) are key to modelinga real robot with high fidelity. However, identifying all ofthese parameters in one optimization is challenging. Instead,

2535
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

we measure some of these parameters offline and use themas inputs to the simulator. The measured mass and physicaldimensions of the robot’s components are listed in Fig. 3. We
also collected stiffness data for the long, passive tendons. A
batch of 11 sensor tendons was measured on a materials
testing system (Instron 3345) and fit with a third-orderpolynomial; we averaged the curves to input a single model
into the simulator. The short, actuated cables are inelastic
and modeled with high stiffness in the simulation.
D. System Identification
Physical parameters like the coefficient of friction and the
speed of the robot’s motors are difficult to measure offline.
We have previously introduced a differentiable physics engine that uses gradient descent to identify system parametersfrom ground truth data [7]–[9]. However, this engine has
been demonstrated only in simulation. This section describes
the key differences that are necessary for system identification when a real robot is the target platform.
Some parameters, such as the cable attachment points,
are not measured on the real robot but are crucial forthe success of the simulation. The benchmark tensegrity
simulator, NTRT [32], attaches cables at the ends of eachrod, which simplifies the modeling process. However, this
attachment strategy can lead the simulated robot to collapse
and get stuck, unable to recover because all the actuatedcables are coplanar, as shown in Figure 4 (Left). Although,this never happens to the real robot. Instead, we put thecable attachment points on the surface of the end caps. This
setup prevents the collapsing and results in a more stable
simulation, as shown in Figure 4 (Right). Note that these
attachment points are not available in the observations; we
implement a heuristic to infer their positions.
Each rod has six attachment points that are distributed
symmetrically and evenly on two end caps. Attachment
points are placed evenly at robot construction every 2⇡/3along the disk that is perpendicular to the rod. So it ispossible to infer all attachment points from only one of them.
At the robot rest state, one rod is in the center and the othertwo rods are on the side. Referring to Fig. 5, we first compute
point A directly, which is on top of the end cap. Then B, C,
can be inferred from it. Since the segments AD, BE, CF
are parallel to the rod, points D, E, F on the other end arecomputed based on points A, B, C. Points G, H are closest
to points B, C.

Fig. 4: (Left) After certain controls, the robot collapses onto theground if all cables are attached to the end of the rods. (Right)
With the same control sequences, the robot does not collapse if thecables are attached on the surface of the end caps.

Before each trajectory, the real robot resets itself to the rest
state. We also reset the simulator to align with the real robot’s
initial state using the data collected at time t = 0, including
both cable lengths and end cap positions. The initial pose

Fig. 5: Cable attachment points on the end caps in the rest state.

generated by the state estimation algorithm (Section IV-B)
may not have the robot perfectly resting on the ground due tothe limitations of the sensors. Thus, we added gravity to thesimulation to force the lowest end caps to come in contact
with the ground.
After initializing the robot in simulation, we use the differentiable physics engine to identify the remaining physicalparameters by performing gradient descent over a trajectory.
To accomplish this task, we introduce a detachment method
to compute nonzero gradients at contact points in order toefficiently learn contact parameters. Furthermore, the longhorizon trajectories and noisy observations from the realrobot can lead to conflicts in the gradient direction. To
combat these conflicts, we introduce the Key Frame Loss
(KFL) function to segment the trajectories and compute
losses at gait transitions. The detachment method and KFL
are described in Sections IV-F and IV-G, respectively.
E. Developing Policies with Symmetry Reduction
Policy search on tensegrity robots is challenging due tocombinatorial explosion since a sequence of control signals
for each actuation cable with unknown time horizon are need
to be discovered. Symmetry reduction has been developedfor 6-bar tensegrity robots [31], [39] to reduce the policy
search space. Here, we apply symmetry reduction to our
3-bar tensegrity robot and combine it with hybrid control,
where a high-level planner outputs the next target state anda low-level PID controller commands the cable lengths.

Fig. 6: (Left) The robot principle axis (yellow) is the average
direction of the three rods. The forward direction is indicated by thecyan arrow; the backward direction is indicated by the orange arrow.
(Right) Two robot poses, with different types of bottom triangles,
are mapped after rotating ⇡ radians around the ground normal.

To reduce the state space via symmetry reduction, we
use a relabeling scheme defined with respect to the support
triangle. We define the robot’s state in Fig. 6 (Left) as thereference state, from which we can rotate the robot about
its principle axis (the yellow line in Fig. 6) by intervals of⇡/6 to get 2 types of support triangles with opposite forward
and backward directions. To map from one type of support
triangle to the other, we rotate the robot by ⇡ around theground normal. After all rotations, we identify the mappingby computing the closest end caps to the reference state.

2536
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

To generate gaits, i.e., time-ordered sequences of target
poses, we conduct a graph search starting from the rest pose.
These short gaits are mapped to long control policies via
symmetry expansion. The nodes of the graph are robot poses
represented by binary cable lengths (0 is retracted and 1
is extended), and the edges of the graph are cable controls(retract, extend, or hold). To limit the search space, we search
to a depth of 4. The root node and leaf nodes are in thesame rest pose where all cables are fully extended. We use
the center of mass displacement as the reward function to
generate one forward and one backward rolling gait. We use
principal axis rotation as the reward function to generateclockwise and counterclockwise turning gaits.
F. Detachment Approach for Contact Gradients
Impulse-based dynamics can’t be directly applied to differentiable simulations due to a zero gradient issue [40]. We
propose a “detachment” method to obtain nonzero gradientsfor contact parameters e, µ. Passive forces, such as restitution
and friction generate equal gradients in opposite directions,and gradients of these forces should not be backpropagated.
The detach() function [41] returns a new tensor detachedfrom the computational graph, truncating the gradient propagation path.
Our detachment method can be applied to learn therestitution e and the force F = mg N , which is shownin Algorithm 1. The difference of speed ( v) and position( x) at time t + 1 and time t are detached to stop the passive
force gradient from backpropagating. Otherwise, the gradientwould be dxt+1 /de = 0.
Algorithm 1 Detachment Method for Restitution
0
vt+1= vt g t + F/m t
0v = (1 + e)vt+1.detach()0
vt+1 = vt+1 + v
if xt < ground thenx = ground xt
xt+1 = xt + vt+1 t + x.detach()else
xt+1 = xt + vt+1 t
end if

The detachment approach is also used for learning friction
parameters, as shown in Algorithm 2. We detach v because
the friction force is a passive force. We add the term
pf +pf .detach() to pass the gradient to µ in the case where
there is only static friction. This term guarantees that µ canalways be updated by the gradient, no matter if the initial µis higher or lower than the actual one. The evaluation of thecontact gradients and integration with Time of Impact (ToI)
computations can be found in the accompanying appendix
available online1 .
G. Key Frame Loss
System identification with long trajectories is difficult due
to sensing noise and the non-monotonicity of trajectories.
As shown in Fig. 7, the later observations are preferable
to reduce the fitting error. Consider a simple example of a
1D trajectory from x = vt with length T . We are given two

Algorithm 2 Detachment Method in Friction Computation
0
vt+1= vt + F/m t
0v = vt+1.detach()pf = µN t/m
if | v| > pf then v = v ⇤ pf /| v|else v = v
pf + pf .detach()end ifvt+1 = vt + F/m t + v
xt+1 = xt + vt+1 t

Fig. 7: Fitting the ground truth line (green) with two noisy
observations, pi and pj . Although pi and pj have the same sensing
error, the fitted line (red) by pj is better than that by pi . x = vt isa linear function (Left). y = f (t) is a non-convex function (Right).

data points, pi = (ti , xi ) and pj = (tj , xj ). ti is close to 0
and tj is close to T . xi and xj have the same observation
error relative to the ground truth. v is the parameter toestimate. Fig. 7 (Left) shows that the fitted line by pj isbetter than that by pi . Moreover, if x = f (t) is a nonmonotonic function, which is closer to reality for tensegrity
robot trajectories, the fitted line by pi is worse as shownin Fig. 7 (Right). The trajectory of an end cap is noisieraround time t = 0 because of the sudden motor activation.
Note that the loss landscape may be non-convex due tocontradictions in gradient directions, which can interfere withthe optimization step. As shown in Fig. 8, three losses yield
opposite gradient directions.

Fig. 8: The ground truth trajectory (green) includes three targets,
x1 , x2 and x3 . The unidentified engine (red) has a lower speed andtakes more time to reach the targets. Three losses are computed atthese targets. The gradients at loss1 and loss3 pull up the red line;
however, the gradient at loss2 drags it down.

The multiple-shooting (MS) method has been applied
for system identification with long trajectories [42]. MS,
however, needs full robot observations (i.e., positions andvelocities of each end cap) to initialize the simulator. The
defects may lead to contradicting gradient directions.
We introduce the Key Frame Loss (KFL) function, which
splits the robot trajectory into a list of monotonic segments
using unidirectional control intervals. We only consider theloss from the last frame in each segment. There are four

2537
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

Fig. 9: (Left) The multiple-shooting (MS) method needs both positions and velocities at each time step so the simulator can align withthe ground truth at intermediate time steps. It uses the entire length of a time window to compute defects between the ground truth andthe simulator. The window size is empirical, and the defects may provide contradicting gradient directions. (Right) The KFL method only
needs positions at each time step. The trajectory is split into segments based on unidirectional control intervals. The time step at the endof the control signal is the key frame where the defects are computed. The window size is the same size as the control interval, and thedefects provide consistent gradient directions.

departures of KFL from MS: 1) KFL only needs partial
observation, i.e., it only requires end cap positions, not theirvelocities; 2) In the intermediate time steps, we do not aim torecover the simulator from the partially observed state since
we do not know velocities; 3) The observation frequency
does not have to be constant; 4) The shooting time window
size is not fixed. Fig. 9 provides a comparison between MS
and KFL.
The benefits of KFL are that it 1) eliminates the factor oftime, 2) makes system identification robust to observationnoise, and 3) ensures the correct gradient direction. This
strategy can be generalized to other optimization problems
using trajectories as ground truth data: trajectories can be
split into monotonic segments, and the KFL can be appliedat the final frame of each segment like in this work.

Fig. 10: (Above) The ground truth (GT) and engine-predictedtrajectory are split into three gaits based on the control intervals.
(Below) The gaits are aligned at the starting points and their
key frames (KF0, KF1, KF2) are the last time frames of eachintersection. The predicted trajectory takes less time in gait 0 and 2,
but more time in 1, which leads to contradicting gradient directions.

To get the key frames, we split the trajectory into gaits
for monotonic segmentation, as in Fig. 10. In each gait, eachcable follows only one of the possible controls: retracting,holding, or extending. These controls form a convex trajectory segment, avoiding the zig-zag trajectory that leads
to contradicting gradients shown in Fig. 8. The key frame(KF) is the last time step of each gait. Consider an example
trajectory with three gaits, as shown in Fig. 10. Both theground truth (red) and the engine-predicted trajectory are
split into three gaits. The corresponding gaits are remapped,
and the KFs are the last time frames of their intersections.
The KFL is the system difference, i.e., the sum of thedifferences in the positions of the end caps at the KF. We
select the last time step because 1) the observation noisearound the start of each gait may lead to worse fitting as

shown in Fig. 7 and 2) the gait transition point is where
one gait is finished; however, whether the gait execution
is complete is determined by measurements from the cable
length sensors, so it is possible the gait transition occurs
earlier or later than predicted, as shown by the gaps in
Fig. 10.
Losses from different gaits may have contradicting gradients. In Fig. 10, the predicted trajectory takes less time
for gait 0 and 2, but it takes more time for gait 1. These
contradicting gradients can confuse the optimizer, as it will
not know whether to speed up or slow down. Some possible
reasons for these time differences are 1) The ground truth
timestamp is noisy due to the limitations of the sensors;2) The noisy cable length sensor readings lead to early or
later gait transitions; 3) The starting states are not perfectlyaligned; 4) There are gaps between the simplified physics
engine and the real robot. To solve the problem, we adopt
a mask filter to handle the contradicting gradients. Our
heuristic states that the identified engine should execute thesame gaits on the robot with similar time and behavior. Then,
we take the execution time of the whole trajectory as anindicator to filter out the opposing gradients. For example,
in Fig. 10, since the predicted trajectory takes a shorter lengthof time than the ground truth, we only consider key frame
losses of gait 0 and gait 2 to slow down the engine, and we
ignore the loss from gait 1. Additional evaluation about theKFL can be found in the link to the appendix1 .
V. E XPERIMENTAL R ESULTS
Our experiments evaluate the R2S2R pipeline both insimulation and on the real 3-bar tensegrity robot. Section VA evaluates key components of the proposed process, i.e.,the detachment strategy and the Key Frame Loss (KFL),
using synthetic data. Section V-B evaluates the full R2S2R
pipeline on the 3-bar tensegrity robot shown in Fig. 1 withthe dimensions in Fig. 3. Section V-C shows the additional
improvement when a second iteration of the R2S2R pipeline
is performed.
A. System Identification with Synthetic Trajectories
We evaluate KFL by comparing three different loss functions: 1) The average defects from all time steps (All Step)in each window; 2) The average defects of the last time step
(Last Step) in each window; 3) Our method (Ours) with KFL,

2538
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

which averages defects of the last time step in each window
and detaches the observations between each window. The
difference between these three methods is shown in Fig. 11.

observe greater variability in the CCW Turning gait. In thesimulator, this gait capitalizes on uniform friction to rotate
the robot’s principal axis; however, in the real world, thefriction between the ground and the end caps is nonuniform.
This discrepancy explains the higher variability.
TABLE I: Real Robot Gait Execution Repeatability Test
MetricFig. 11: A trajectory starts from the origin, moves along x, and thenreturns to the origin. This trajectory can be split into 2 windows. The
ground truth trajectory (green) takes less time than the estimated one(red). The second window of the ground truth is shifted to the rightto compute defects. (Left) The All Step loss function averages thedefects of all time steps in each window. (Right) The Last Step loss
function averages defects from the last time step in each window.
KFL also detaches the system state at the transition point (cyan) of
2 windows [41].

To simplify the problem, only motor speed is identified inthis task (in addition to motor speed, the friction coefficientis also identified in V-B). A synthetic trajectory where themotor speed is set to 0.5 is used as ground truth. The defects
are the differences in end cap positions. An Adam optimizer
with learning rate 0.1 is applied. Fig. 12 shows that our
method converges fast, smooth and stable. The curve of
the Last Step method fluctuates more due to the gradientpropagation across windows, even if the motor speed is close
to the target in later iterations. This shows the neeed for
detaching. The multiple windows approach considering alltime steps performs worse.

Fig. 12: To identify the motor speed parameter, three loss functions
are considered: All Step (blue), Last Step (orange), and KFL loss
with state detachment (green). KFL loss converges better.

B. Complete Evaluation of the R2S2R Pipeline
The R2S2R pipeline has been applied twice to show
continuous improvement each iteration. In the first iteration, a single robot trajectory given random controls isused to identify the physics engine parameters, including
motor speed and friction coefficient. From the identified
engine, we generate three policies corresponding to “forward
rolling,” “backward rolling,” and “counterclockwise turning”
behaviors. These policies are mapped to two long, openloop gaits with symmetry reduction: a straight gait (Straight)
composed of forward and backward rolling policies anda counterclockwise turning gait (CCW Turning) composedof “counterclockwise turning” polices. These two gaits are
executed in simulation and on the real robot, and these
executions are shown as blue and orange lines, respectively,
in Figure 13. We also execute these gaits starting fromdifferent positions on the ground to test the repeatability on
the real platform (Table I). The Center of Mass (CoM) andorientation are measured at the end of each trajectory. We

End CoM Position (m)End Orientation (radian)Trajectory

Straight

CCW Turning

Straight

CCW Turning

Mean ± std[-1.14± 0.02, 0.29±0.05][0.01±0.02, 0.29±0.02]

0.136±0.07

2.7± 0.23

Fig. 13: Trajectories in simulation and on the real robot for thesame policy. The open-loop trajectories start at (0, 0). The robot
trajectory is observed in order to generate new data to further refine
the simulator. (Left) The output for a straight-line trajectory of
the center of mass. The policy executes forward and backward
motion in the symmetry-reduction frame. (Right) The output froma counterclockwise turning policy.

C. Continuous Improvement with R2S2R

The second iteration re-identifies the engine using the tworeal robot trajectories from the first iteration, and then we
execute these policies again in simulation to show how theR2S2R pipeline reduces the sim2real gap. The trajectories
sampled from the refined simulation are plotted as green lines
in Fig. 13. The robot position and orientation at the end of the
trajectories are compared in Table II. Generally, the CoM andorientation errors go down in the refined simulation; however, the slightly increased CCW Turning orientation error
shows the limitations of simulating nonuniform environment
friction with a uniform friction coefficient.
TABLE II: Iterative Refined Simulation with Real Data
End CoM Position (m)End Orientation (radian)Trajectory

MetricStraight

CCW Turning

Straight

CCW Turning

Initial Simulation
Real Robot Execution
Refined Simulation[-0.97, 0.40][-1.11, 0.32][-1.11, 0.29][-0.02, 0.22][0.02, 0.29][0.05, 0.30]

0.31
0.02
0.12

2.36
1.78
2.45

Initial Error
Refined Error

0.16
0.03

0.26
0.06

0.09
0.03

0.04
0.13

Qualitatively, the trajectory executed on the robot has thesame behavior as the initial simulation. After the second
iteration, the simulation trajectory is even more similar to
the real robot trajectory, as shown in Fig 15 and 16.
After the second iteration of system identification, two newclockwise turning polices are generated, and these policies
are mapped to two long, open-loop gaits (Figure 14). The
deviations are larger for the arching gait (Figure 14 Left)because of the nonuniform friction between the end caps
and the ground. This gait covers a larger area, and thecoefficient of friction is not uniform everywhere on the floor.

2539
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

Fig. 14: Two new clockwise turning trajectories are executed insimulation and on the real robot. The open-loop trajectories startat (0, 0). (Left) The arch turning trajectory. (Right) The in-placeturning trajectory.

Fig. 16: The frame-by-frame comparison of the counterclockwise
turning trajectory execution in simulation and on the real robot.

Fig. 15: The frame-by-frame comparison of the straight trajectory
execution in simulation and on the real robot.

We observe smaller deviations for the in-place turning gait
(Figure 14 Right) where the robot is confined to a muchsmaller region. Qualitative visualization is also avaliable inFig 17 and Fig 18.
VI. C ONCLUSION
This paper demonstrates an R2S2R pipeline that mitigatesthe sim2real gap for tensegrity robots and develops locomotion policies that can be seamlessly transferred to real robots.
The sim2real gap is reduced by using a differentiable physics
engine that learns system parameters from real robot data
via gradient descent. The simulation can then generate newlocomotion policies and extend them for long trajectories
via symmetry reduction. After transferring these policies tothe real robot, the simulation can be continuously improvedby re-identifying the system parameters from the recorded
trajectories. To enable efficient system identification, we
introduce and experimentally validate the detachment approach for computing contact gradients and the Key Frame
Loss with a trajectory segmentation strategy. In future work,

Fig. 17: The frame by frame comparison of the clockwise arching
trajectory execution in simulation and on the real robot.

the accuracy of the physics engine could be improved byallowing the system to identify the physical parameters ofeach component (e.g., the stiffness of each sensor tendon)
and adapt online as these parameters change over time.
R EFERENCES[1] S. Lessard, D. Castro, W. Asper, S. D. Chopra, L. B. BaltaxeAdmony, M. Teodorescu, V. SunSpiral, and A. Agogino, “A bioinspired tensegrity manipulator with multi-dof, structurally compliant
joints,” in IROS. IEEE, 2016, pp. 5515–5520.
[2] A. P. Sabelhaus, L. J. van Vuuren, A. Joshi, E. Zhu, H. J. Garnier,K. A. Sover, J. Navarro, A. K. Agogino, and A. M. Agogino, “Design,
simulation, and testing of a flexible actuated spine for quadruped
robots,” arXiv preprint arXiv:1804.06527, 2018.
[3] M. Chen, J. Liu, and R. E. Skelton, “Design and control of tensegrity
morphing airfoils,” Mechanics Research Communications, vol. 103, p.
103480, 2020.
[4] J. Bruce, A. P. Sabelhaus, Y. Chen, D. Lu, K. Morse, S. Milam,
K. Caluwaerts, A. M. Agogino, and V. SunSpiral, “Superball: Exploring tensegrities for planetary probes,” 12th International Symposium

2540
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.

Fig. 18: The frame by frame comparison of the in-place clockwise
turning trajectory execution in simulation and on the real robot.
on Artificial Intelligence, Robotics, and Automation in Space (iSAIRAS), 2014.
[5] D. S. Shah, J. W. Booth, R. L. Baines, K. Wang, M. Vespignani, K. Bekris, and R. Kramer-Bottiglio, “Tensegrity robotics,” Soft
Robotics, vol. 9, no. 4, pp. 639–656, 2022.
[6] D. A. Surovik, K. Wang, M. Vespignani, J. Bruce, and K. E. Bekris,“Adaptive tensegrity locomotion: Controlling a compliant icosahedron
with symmetry-reduced reinforcement learning,” The International
Journal of Robotics Research, vol. 40, pp. 375 – 396, 2021.
[7] K. Wang, M. Aanjaneya, and K. Bekris, “Sim2sim evaluation of anovel data-efficient differentiable physics engine for tensegrity robots,”
in IEEE/RSJ International Conference on Intelligent Robots andSystems (IROS), 2021, pp. 1–8.
[8] ——, “A first principles approach for data-efficient system identification of spring-rod systems via differentiable physics engines,” in
L4DC, vol. 120. PMLR, 10–11 Jun 2020, pp. 651–665.
[9] ——, “A recurrent differentiable engine for modeling tensegrity robots
trainable with low-frequency data,” in ICRA, 2022.
[10] K. Bousmalis and S. Levine, “Closing the simulation-to-reality gap fordeep robotic learning (2019),” Google AI Blog http://ai. googleblog.
com/2017/10/closing-simulation-to-reality-gap-for. html, 2019.
[11] P. Martinez-Gonzalez, S. Oprea, A. Garcia-Garcia, A. Jover-Alvarez,
S. Orts-Escolano, and J. Garcia-Rodriguez, “Unrealrox: an extremely
photorealistic virtual reality environment for robotics simulations andsynthetic data generation,” Virtual Reality, vol. 24, no. 2, pp. 271–288,
2020.
[12] J. K. Murthy, M. Macklin, F. Golemo, V. Voleti, L. Petrini, M. Weiss,
B. Considine, J. Parent-Lévesque, K. Xie, K. Erleben, et al., “"gradsim: Differentiable simulation for system identification and visuomotor
control",” in ICLR, 2020.
[13] J. Collins, D. Howard, and J. Leitner, “Quantifying the reality gap
in robotic manipulation tasks,” in 2019 International Conference on
Robotics and Automation (ICRA). IEEE, 2019, pp. 6706–6712.
[14] F. Ramos, R. Possas, and D. Fox, “Bayessim: adaptive domain
randomization via probabilistic inference for robotics simulators,”in Robotics: Science and Systems (RSS), 2019. [Online]. Available:
https://arxiv.org/abs/1906.01728[15] F. Muratore, T. Gruner, F. Wiese, B. Belousov, M. Gienger, and
J. Peters, “Neural posterior domain randomization,” in Conference on
Robot Learning. PMLR, 2022, pp. 1532–1542.
[16] A. Zeng, S. Song, J. Lee, A. Rodriguez, and T. Funkhouser, “Tossingbot: Learning to throw arbitrary objects with residual physics,” IEEE
Transactions on Robotics, vol. 36, no. 4, pp. 1307–1319, 2020.
[17] V. Lim, H. Huang, L. Y. Chen, J. Wang, J. Ichnowski, D. Seita,
M. Laskey, and K. Goldberg, “"real2sim2real: Self-supervised learningof physical single-step dynamic actions for planar robot casting",” in
ICRA. IEEE Press, 2022, p. 8282–8289.
[18] A. Sethuraman and K. A. Skinner, “Towards sim2real for shipwreck
detection in side scan sonar imagery,” 3rd Workshop on Closing theReality Gap in Sim2Real Transfer for Robotics, 2022.
[19] M. Navardi, P. Dixit, T. Manjunath, N. R. Waytowich, T. Mohsenin,
and T. Oates, “Toward real-world implementation of deep reinforcement learning for vision-based autonomous drone navigation withmission,” UMBC Student Collection, 2022.
[20] A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, and
D. Scaramuzza, “Deep drone racing: From simulation to reality withdomain randomization,” IEEE TRO, vol. 36, no. 1, pp. 1–14, 2019.
[21] A. Sena, H. Kavianirad, S. Endo, E. Burdet, and S. Hirche, “The gap infunctional electrical stimulation simulation,” 3rd Workshop on Closing
the Reality Gap in Sim2Real Transfer for Robotics, 2022.

[22] J. Jabbour, “Closing the sim-to-real gap for ultra-low-cost, resourceconstrained, quadruped robot platforms,” 3rd Workshop on Closing theReality Gap in Sim2Real Transfer for Robotics, 2022.
[23] W. Huang, X. Huang, C. Majidi, and M. K. Jawed, “Dynamic
simulation of articulated soft robots,” Nature Communications, vol. 11,
no. 1, pp. 1–9, 2020.
[24] X. Huang, W. Huang, Z. Patterson, Z. Ren, M. K. Jawed, andC. Majidi, “Numerical simulation of an untethered omni-directional
star-shaped swimming robot,” in ICRA. IEEE, 2021, pp. 11 884–
11 890.
[25] N. N. Goldberg, X. Huang, C. Majidi, A. Novelia, O. M. O’Reilly,
D. A. Paley, and W. L. Scott, “On planar discrete elastic rod models
for the locomotion of soft robots,” Soft Robotics, vol. 6, no. 5, pp.
595–610, 2019.
[26] B. Wen, W. Lian, K. Bekris, and S. Schaal, “Catgrasp: Learning
category-level task-relevant grasping in clutter from simulation,” in
ICRA. IEEE, 2022, pp. 6401–6408.
[27] F. de Avila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z.
Kolter, “End-to-end differentiable physics for learning and control,” inAdvances in neural information processing systems, 2018, pp. 7178–
7189.
[28] J. Degrave, M. Hermans, J. Dambre, et al., “A differentiable physics
engine for deep learning in robotics,” Frontiers in neurorobotics,
vol. 13, p. 6, 2019.
[29] M. Zhang, X. Geng, J. Bruce, K. Caluwaerts, M. Vespignani, V. SunSpiral, P. Abbeel, and S. Levine, “Deep reinforcement learning fortensegrity robot locomotion,” in ICRA. IEEE, 2017, pp. 634–641.
[30] J. Luo, R. Edmunds, F. Rice, and A. M. Agogino, “Tensegrity
robot locomotion under limited sensory inputs via deep reinforcement
learning,” in ICRA. IEEE, 2018, pp. 6260–6267.
[31] D. Surovik, K. Wang, M. Vespignani, J. Bruce, and K. E. Bekris,“Adaptive Tensegrity Locomotion: Controlling a Compliant Icosahedron with Symmetry-Reduced Reinforcement Learning,” IJRR, 2019.
[32] NASA, “NASA Tensegrity Robotics Toolkit,” Accessed 2020, https://github.com/NASA-Tensegrity-Robotics-Toolkit/NTRTsim.
[33] B. T. Mirletz, I.-W. Park, R. D. Quinn, and V. SunSpiral, “Towards
bridging the reality gap between tensegrity simulation and robotic
hardware,” in IROS. IEEE, 2015, pp. 5357–5363.
[34] K. Caluwaerts, J. Despraz, A. Işçen, A. P. Sabelhaus, J. Bruce,
B. Schrauwen, and V. SunSpiral, “Design and control of compliant
tensegrity robots through simulation and hardware validation,” Journalof the royal society interface, vol. 11, no. 98, p. 20140520, 2014.
[35] W. R. Johnson, A. Agrawala, X. Huang, J. Booth, and R. KramerBottiglio, “Sensor tendons for soft robot shape estimation,” in IEEE
Sensors. IEEE, 2022, pp. 1–4.
[36] W. R. Johnson, J. Booth, and R. Kramer-Bottiglio, “Integrated sensing
in robotic skin modules,” in 2021 IEEE Sensors. IEEE, 2021, pp.
1–4.
[37] S. Lu, W. R. Johnson III, K. Wang, X. Huang, J. Booth, R. KramerBottiglio, and K. Bekris, “6n-dof pose tracking for tensegrity robots,”
arXiv preprint arXiv:2205.14764, 2022.
[38] M. A. Fischler and R. C. Bolles, “Random sample consensus: aparadigm for model fitting with applications to image analysis andautomated cartography,” Communications of the ACM, vol. 24, no. 6,
pp. 381–395, 1981.
[39] D. Surovik, J. Bruce, K. Wang, M. Vespignani, and K. E. Bekris,“Any-axis tensegrity rolling via bootstrapped learning and symmetry
reduction,” in ISER, Buenos Aires, Argentina, 11/2018 2018.
[40] K. Werling, D. Omens, J. Lee, I. Exarchos, and C. K. Liu, “Fastand feature-complete differentiable physics engine for articulated rigid
bodies with contact constraints,” in RSS, 2021.
[41] Pytorch, “Pytorch Detach Method,” Accessed 2022, https://pytorch.
org/docs/stable/generated/torch.Tensor.detach.html.
[42] E. Heiden, C. E. Denniston, D. Millard, F. Ramos, and G. S. Sukhatme,“Probabilistic inference of simulation parameters via parallel differentiable simulation,” ICRA, 2022.
[43] Y. Hu, L. Anderson, T.-M. Li, Q. Sun, N. Carr, J. Ragan-Kelley,
and F. Durand, “Difftaichi: Differentiable programming for physicalsimulation,” ICLR, 2020.

2541
Authorized licensed use limited to: Yale University. Downloaded on January 23,2024 at 19:05:06 UTC from IEEE Xplore. Restrictions apply.