Learning Differentiable Tensegrity Dynamics using Graph Neural Networks
Authors: Nelson Chen
Kun Wang
William R. Johson III
Rebecca Kramer-Bottiglio
Kostas Bekris
Mridul Aanjaneya
Venue: CoRL 2024 (2024)
Keywords: graph neural networks
differentiable simulation
tensegrity robots
TL;DR: This paper describes a graph neural network method to learn Tensegrity robot dynamics.

Abstract: Tensegrity robots are composed of rigid struts and flexible cables. They constitute an emerging class of hybrid rigid-soft robotic systems and are promising systems for a wide array of applications, ranging from locomotion to assembly. They are difficult to control and model accurately, however, due to their compliance and high number of degrees of freedom. To address this issue, prior work has introduced a differentiable physics engine designed for tensegrity robots based on first principles. In contrast, this work proposes the use of graph neural networks to model contact dynamics over a graph representation of tensegrity robots, which leverages their natural graph-like cable connec- tivity between end caps of rigid rods. This learned simulator can accurately model 3-bar and 6-bar tensegrity robot dynamics in simulation-to-simulation experiments where MuJoCo is used as the ground truth. It can also achieve higher accuracy than the previous differentiable engine for a real 3-bar tensegrity robot, for which the robot state is only partially observable. When compared against direct applications of recent mesh-based graph neural network simulators, the proposed approach is computationally more efficient, both for training and inference, while achieving higher accuracy. Code and data are available at https://github.com/nchen9191/tensegrity_gnn_simulator_public

Learning Differentiable Tensegrity Dynamics using Graph Neural Networks
Nelson Chen', Kun Wang', William R. Johnson III;, Rebecca "ramer-Bottiglion Kostas Bekris!, Mridul Aanjaneya! RRtgeers University, 2 AAmazon Robotics, Yale University
Abstract: Tensegrity robots are composed of rigid struts and flexible cables. They constitute an emerging class of hybrid rigid-soft robotic systems and are promising systems for wide array of applications, ranging from locomotion assembly. They are difficult control and model accurately, however, due their compliance and high number of degrees of freedom. To address this issue, prior work has introduced differentiable physics engine designed for tenseg- rity robots based on first principles. In contrast, this work proposes the use graph neural networks to model contact dynamics over a graph representa- tion tensegrity robots, which leverages their natural graph-like cable connec- tivity between end caps of rigid rods. This learned simulator can accurately model 3-bar and 6-bar tensegrity robot dynamics in comulation-to-simulle experiments where MuJoCo is used as the ground truth. It can also achieve higher accuracy than the previous differentiable engine for real 3-bar tenseg- rity robot, for which the robot state is only partially observable. When compared against direct applications of recent mesh-based graph neural network simula- tors, the proposed approach is computationally more efficient, both for training and inference, while achieving higher accuracy. Code and data are available https:/ //gith/bttt/t com httm//hhen991/ttesggtiii staerrttyyysssssllttrrrr.r simmlatoo___uuliicccc Keywords: tensegrity robots, graph neural networks, differentiable simulation
Figure 1: (Left) real 3-bar tensegrity robot and the graphical representation superimposed. (Right) A simulated 6-bar tensegrity robot in MuJoCo and the graphical representation superimposed. For both platforms, the graph consists of nodes (white) along the rods' axes connected with body edges (blue). Cable edges (yellow) connect two nodes on end caps of different rods. A special ground node (red G node) has contact edges (red) connected to body nodes close the ground.
1 Introduction
Tensegrity robots consist of rigid struts (rods) and flexible elements (cables). This allows them to be lightweight while exhibiting both compliance and rigidity. They can locomote by changing their shape actuation of their cables. Tensegrity robots are attracting interest due to their wide array of possible applications, such manipulation [1], locomotion [2]. morphing airfoils [3], and spacecraft landing [4]. Recent work [5] has also explored the assembly of multiple tensegrity robots to form and perform complex structures and tasks. Tensegrity robots, however, are difficult to accurately model and control due to their high number of degrees of freedom, significant nonlinearities, and complex dynamics, involving oscillatory behaviors and compliant mechanisms.
This work proposes the use of graph neural networks (GNNs) to improve on previous work on the modeling of tensegrity robots using differentiable engines [6, 8], the current state of the art
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Graph Gen
UO
U²
T-1
Feat
Feat
Feat Gen
T+J
Gen
Gen
x
x
SIN
SIM
SIM
SIM
Figure 2: A trajectory rollout: Upon initialization, the tensegrity graph structure is generated and then used as input at every simulation step a state X' and controls U are passed to a feature generation module that updates the graph structure with contact edges and generates feature vectors for all nodes } as well as edges {Eiz) }- These are passed to the simulator to predict X
in matching simulation to reality for tensegrity robots. Inspired by recent efforts that demonstrate GNNs success in modeling discontinuous, rigid-body contact [9, 10, 11], a GNN-based framework proposed for learning contact mechanics for tensegrity robots. The method injects structural prior by representing tensegrities as graphs. These graphs are constructed by leveraging the robots' natural graph-like connectivity with cables between rod end caps, well decomposition of the rigid rods smaller primitive objects (c.g., spheres and cylinders), which constitute object-level nodes (Fig. 1). This graph representation has orders of magnitude fewer nodes and edges than the mesh-based representation introduced in prior GNN work for rigid- simulators [9], facilitating faster training and inference with less hardware and fewer computational resources, while achieving higher accuracy.
The accompanying experiments evaluate the method and compare it against the first-principles dif- ferentiable physics engine in simulation and for real tensegrity robot. In simulation, MuJoCo [12] provides the ground-truth system. The comparison shows that the learned GNN model performs bet- ter than the baseline, for both the 3-bar and 6-bar tensegrity variants, when it has full observability of the robot's state. The available data for real robot correspond to captured trajectories on a 3-bar tensegrity using a vision-based solution [13]. The real data, however, only include end cap positions of the rods, but without instantaneous velocities each time step, which are critical for simulating active motion. Due to this partial observability, the first-principles differentiable engine [8] can only be trained with the whole trajectory where the initial state is at rest. In contrast, the GNN model does not require instantaneous velocity information and can improve upon the baseline by warm-starting with simulation data first and then fine-tuning given the real data. The evaluation includes mea- suring the computational requirements of mesh and surface-based versions of GNNs used in prior work [9, 10, 11] compared to the proposed sparser representation. The proposed method requires less computational resources for both training and inference. Finally, an ablation study evaluates the different choices of the proposed pipeline. In summary, the key contributions are:
simple and computationally efficient graph representation for tensegrity systems using object- level nodes and edges that can be used for modeling them using a GNN.
learning pipeline that combines both analytical physics components and GNN train models
that are predictive and stable over long rollouts.
Evaluations, in simulation and reality, of the GNN in modeling tensegrity robot dynamics.
2 Related Work
One family of tensegrity simulators based on solving systems of analytical differential equations representing tensegrity dynamics and structures. These include TensgrityMATTLABObjcc. [14], Software for Tensegrity Dynamics [15], and Models of Tensegrity Structures [16). These simulators

---

are built using MATLAB and only support frictionless contact. Other simulators support frictional contact and are based on traditional engines where governing equations from physics, analytical models, and numerical solvers are used. Paul et al. [17] built a simulator on top of ODE [18), which supports massless and volumeless virtual cables but has no contact. The open-source NASA tensegrity robot toolkit [19] and Caliper [20] are built on top of Bullet [21]. These simulators can still suffer from large simulation-to- -totrlttttttorrllitty gap due to unknown system parameters or unmodeled physics. To address this, Wang et al. developed a differentiable physics engine for tensegrity robots
[6, 7, 8), which serves as the baseline for this work.
Differentiable physics engines have been gaining momentum in robotics. They are able to identify and learn model parameters via gradient- cradient-based optimization, often resulting in faster training and better data efficiency. These simulators range from fully first-piicipled [22, 23, 24, 25, 26] to purely data-driven [27, 28, 29, 30, 31], well as hybrid setups [32, 33, 34].
One emerging data-driven modeling choice is graph neural networks (GNNs). GNNs are able to model particle-basee physics [35, 36, 37, 38], mesh- based physics [39, 40, 41], and deformable rod dynamics [42]. Contrary to prior works [33, 43] that suggest that neural networks cannot learn dis- continuous, higidbboyy contact, recent work [9, 10, 11] demonstrates that GNNs can. Applications in robotics include GNNs modeling object-object and object-gripper interactions [44, 45, 46] and predicting the next states of soft gripper [47]. Beyond simulation, GNNs have also been used in a variety of other applications, such as modeling robot kinematics [48, 49], multi-robot coordina- tion [50]. and paannhinnnig [51]. This work uses hybrid model with analytical models of actuation and passive forces combined with data-driven GNN model for ground contact.
3 Approach
Tensegrity robots are typically treated as of rigid rods and a of cables that connect between the rods' end caps, referred their system topology. The robot state X includes the state of each rod at time step 1, X (P. Rt, V 22), consisting of the position P, orientation R linear velocity V and angular velocity S2 Then, the simulator can be seen as function FSIM (X', U)) that takes the current state X along with set of controls U", to predict the next state, i.e.,
(1) The proposed simulator FSTM uses a GNN to model the robot dynamics where the tensegrity robot represented by graph, G', with nodes Art and edges Et, at time step This results in a learnable simulator GNN that predicts changes in velocity from contact:
x = FsIM (X*, U*)
g' = (W", + F(X', U GO
(2)
AV'NN = GN'
(3)
where F()) is a feature generator function. In parallel, passive forces (cable and gravity) are com- puted analytically with function H.) and transformed to node-level velocity changes, Av'pp, via mapping function (-).
AViF M( U')
(4)
Finally, AV'NNN and Avip are integrated up with the semi-implicit Euler scheme to predict next node states (p(+1, vl+1) and mapped back rigid-body state X with M'(.)
r+l vf + Av + AV'NN p 1+1 = + At x Mp ,
(5)
(6)
This procedure serves as a single simulation step in our framework. Trajectory rollouts can then be generated by applying this process for K number of steps in an auto-regressive manner (X", x ..... XK), as shown in Fig. 2. The details of single step depicted in Fig.
The actuation and cable forces are computed analytically with linear models. For actuation, there are motors that act on cables to change their rest lengths. The cables are modeled as linear springs using Hooke's law, but they only allow tension and not compression. sim-to-sim experiments, these linear models are exact, but in reality, the cables and the motors can have non- non-linear components that the GNN can potentially compensate for.
should be noted that in
3

---

GNN Simulator
X
x
Feature Generator
Encoder
Node to Rigid-Body
Actuation
P0
Processor
Time Integrator
Passive
Decoder
Av
t+t GNN
Forces
Rigid-Body to Node
Liv
Ap
Figure 3: A single simulation step. The current rigid-body state X' and controls U are used to analytically compute the cable's rest length changes A and the passive force induced velocity changes A PF V AV yt++1 is mapped to node-space velocity changes Av X' and A are used to generate node and edge features. They are then encoded, processed through multiple message-
passing steps, and decoded to the predicted A 1+1 Finally, Av Ar141 pp and Av141 4+1 GNN are summed,
time-integrated, and mapped to the predicted next rigid-body state X
Graph structure The graph G composed of body nodes N [Nr}. indra-body edges gbody { {Eij {Ehodd), contact edges goon = ( ", and cable edges goable = { (Fggbble). Upon initialization, the topology graph G" is generated, containing only the graph structure and node body-frame positions {x;") J. At time step i, node and edge feature vectors (N₁, FXy)) and node states the world frame x (p/, v/) are computed based on (X*, U*) and are added the topology graph go to form G Each rod is represented by subgraph consisting of set of connected body nodes along its center axis formed by decomposing the rod into simpler primitive rigid bodies, as shown in Fig. 4. Rod subgraphs are connected with each other via cable edges at nodes corresponding to the cables be- tween pairs of rod end caps. A special ground body node is constructed with dynamically generated contact edges connecting the ground body node to rod body nodes that are within radius rg. All of these together form full graph that represents the tensegrity robot. Examples of the graph structures for 3-bar and 6-bar tensegrities can be seen in Fig. 1.
Figure 4: Graph structure for a sin- gle rod: (left) original rod; (mid- dle) rod decomposed to multiple primitive bodies; (right) rod nodes and edges.
Nodes The set N of nodes represents primitive rigid bodics that, together, compose the rods. For the GNN to be translation- equivariant, velocities, instead of positions, are used as input fea- tures. More detailed descriptions of the features can be found in Appendix In addition, the proposed approach only requires the average velocity v.""? which can be computed from positional data as (p/ - p 1//1)///,.. as shown in Fig. 5. Previous works in tra- ditional physics simulators [52, 23] that model contact typically compute time-of- of-impact within a time step and split the step into pre-contact and post-contact phase for accurate simulation. This computation requires accurate approximations of instantaneous velocities, something hard to infer given only positional data. In contrast, the GNN is able learn the average velocities and accurately incorporate contact dynamics in its predictions.
yFD
To
T
Figure 5: Difference between fiite-differrnce velocity v vs. instantaneous velocity v in single step where collision occurs.

---

The metrics are the average positional, rotational, and ground-penetration error Edges There are three types of edges: (i) body edges E (ii) cable edges Eighte and (iii) contact edges E The body edges b connect nodes belonging to the same rod. These edges learn rigid-body constraints, but do not strictly enforce them. This allows for small degree of softness that can aid in the learning process. The cable edges Eguble connect end cap nodes belonging to different rods with physical cable attachments. The contact edges Eij" connect body nodes the ground node when they are within a user-specified distance. While contact is often viewed constraint, this formulation models contact as unique edge type in the graph neural network due to the fact that contact introduces discontinuous motion with complex friction dynamics.
GNN architecture encoder The encoder is multi-layer perceptron (MLP), which receives the input feature vectors N₂ or Eiy and outputs latent vector embedded in a larger dimensional space. There is dedicated encoder for each type of node and edge of the graph. In our setup, there is one node encoder, MLP., and three edge encoders, MLPeae MLP MLP Ebady"
Scable
Econ'
GNN architecture processor The processor a sequence of message-passing steps that aims to learn the latent dynamics. At each message-passing step I, all latent edge vectors are updated from the current latent edge vectors and the two connecting latent nodes' vectors. Then, node vectors are updated from aggregating new edges and passed through another MLP. This process is repeated L times with L different MLPs that do not share weights.
f = MLP MP (N!- .N/-1 _ETT')
(8)
N MLP update (N!-1.) Σ Ebody.
Ely
Σ
Eig
E.aalee
EEgon
GNN architecture decoder The decoder is also MLP that takes the last latent node vectors and outputs the predicted velocity changes Auf per node, which are then used to integrate up to velocities and positions. Δuf are predicted instead of position or velocities so that the GNN's predictions would not violate the underlying governing differential equations.
Av AVANN, = MLP (N())
(10)
Loss The loss function is the mean squared error (MSE) per node between the predicted and (com- puted) ground truth (GT) velocity changes.
AV = ((P') - p')/At) - v
(11)
1 Σ (GNN -V 2
Av
++1 GT
(12)
B
As the experiments show, the loss must be defined at the node level instead of the rigid-body state level. loss at the state level does not drive the GNN to learn node dynamics since the node output is averaged when mapping back to the rigid-body state. In addition, Av; are used instead of positions or velocities as the acceleration errors are not functions of the time step. As time step decreases (often needed for accuracy and stability), the positional error magnitude becomes smaller, causing the update gradients to vanish.
4 Experimental Results
The approach is compared against the previous first-principles differentiable engine (DPE) in both simulation and reality. There is also an ablation on which parts of the engine should be computed with first-principles and which should be learned via GNN, as well as which quantities to com- pute the loss over. Lastly, the proposed approach is compared against the surface-based (mesh and meshless) GNN representations from the literature on learning rigid-body simulators.
Evaluation metrics:
over a trajectory rollout (x X ,..., x The positional error Cpos is measured as the absolute distance between the predicted and GT robot center of mass P normalized by the length of a rod Trrad- The rotational error Grot measured as the angular difference between the rods' predicted and GT center axes f. The ground penetration error epen is the absolute penetration distance into the ground beyond what is seen in the GT data and normalized by the length of the rod, where is the height of the point closest to the ground on the robot:
5

---

E
P
Σ COs (f
Opos
Lrock
K
Open
max
min(= 0),0)
Lrod
Sim-to-sim evaluation Two types of GT trajectories are generated MuloCo: (1) active trajectories with control signals, and (2) passive trajectories where the tensegrity robot is initialized with random height and linear velocity. This is done for both a 3-bar and 6-bar tensegrity robot. Forty-six trajectories were generated, amounting to 17 minutes of data. These data were split into 50% training. 25% validation, and 25% testing by trajectory. The models were trained for a fixed number of epochs, and the model with the lowest validation loss was saved and executed on the held-out test set. The baselines are the DPF, an MLP trained to predict rod dynamics, and an MLP trained to predict all rods simultancously.
Positional enoor rod length)
Baseline
Penetration error 0% rod length)
All-rods MLP
10"
Ours
Single-rod
10
Pemetrati ation error (mm)
105
10r
Rotational
40-
70
10-1
201
10a
Sbar
Sear
300r
obar
3bar
6ber
Figure 6: Sim-to-sim results for the 3-bar and 6-bar robots. The approach is compared against alternatives: the baseline differentiable engine, MLP for each rod, and an MLP for all rods.
In simulation where we have access to high-frequency. full state information, the proposed method strongly outperforms the baselines as highlighted in Fig. It improved upon the differentiable physics engine by 30% and 9a for the 3-bar tensegrity and 11% and 133 for the 6-bar tensegrity. The due increased complexity. The 6-bar robot has
6-bar tensegrity
difficult to learn smaller base of support than the width of the robot, which causes it to be unstable with small mo- mentum. For the 3-bar tensegrity, the base's width is equal to the robot's width, so it is more stable. The MLP variants performed poorly, especially with high penetration errors, reinforcing the results seen in previous works showing vanilla neural networks have difficulty learning discontinuous con- tact dynamics. Although the penetration error higher in the proposed method than the baseline, both the proposed model and bascline have low penetration errors, i.c., below 0.01% (0.036mm) in simulation results. Furthermore, the authors hypothesize that, with the graph formulation, the no-peentrationn constraint is "softened," and hence the contact dynamics smoothed and learning is eased.
rod length)
Baseline
Penetration rod length)
20
0.038
3.0
Perittral on efror (mmm)
Rotational enror (*)
2.5
15
15
0.006
10.
10
0.004
1.5
1.0
0.002
8.5
Figure 7: Physical 3-bar tensegrity experimental results. Compared our method with warm starting with either MuJoCo simulator data or DPE simulator data.
Real 3-bar tensegrity For the real 3-bar tensegrity robot, 16 trajectories were collected of 1.5 minutes each that include straight rolling, clockwise and counterclockwise rolling, crawling. and random cable actuation motion. This dataset is split eight trajectories for training, four for validation, and four for testing. The collected data only track the center of the end caps' positions. This means that the rotational component about the rod center axes and the instantancous velocities

---

Clockwise
Counterclockwise
Crawling
Straight Rolling
5.4
0.30
2.0
0.25
0.5
oui Mg. eirdst
0.30
Our femine narriiat
Y (m))
6.2
2.15
-0.10
0.3
0.2
2.30
-0.5
2.05
2.00
020
2.2
(m)
(m)
(m)
X (m)
Figure 8: Plots of tensegrity robot's center-of-mass starting from the origin, (0, 0), Trajectories
range from 45 to 75 seconds in length with a stepsize of 10ms. Four gaits in the test set comparing
the baseline (DPE) and the learned simulators (with MJC and baseline warmstarts) against ground truth. Rendering of these trajectories can be seen in the attached video.
are unavailable. Furthermore, the data are captured at lower frequency with non- con-unfform step sizes, including control signals. Due to these issues, additional steps to the current training and testing process are used to work with real data. A PID controller is included in the loop reflecting the operation a PID controller on the real platform that receives cable length targets and generates the low-level controls. This introduces temporal differences, and the evaluation criteria focus on measuring errors at the last frame per gait cycle. Finally, the engine is pre-trained over data gener- ated from simulator (MuJoCo and baseline), then fine-tuned over the entire trajectories of the real robot.
The GNN model can still learn over observations of a real tensegrity and improve upon it, as seen in Fig. 7. The improvements are more incremental relative to the full observability results with about 3% improvement over the baseline in positional error and about laddd 5over the baseline in rotational error. A potential reason is that fine-tuning over long trajectories may not be ideal. The long trajectories produce deep computational graphs, which can cause vanishing gradients. Secondly, the long trajectories also have noisy observations that produce conflicting gradients when trained in a single step. Lastly, the PID controller also influences the number of steps and input control signals, changing the internal structure of the computational graph and, subsequently, the loss function that the model is optimizing for. Thus, it is not necessarily the case that the gradients are still pointing in a downward direction for the changed loss function. Qualitatively, the GNN models trained in this fashion are still able exhibit improved predictive capabilities in terms of the test trajectories of Fig. 8. Similar to the simulation results, even though the GNN results have higher penetration errors than the baseline results, the GNN results are still below 0.008% (0.03mm).
Positional Error (%)
Rotational Error (*)
Penetration Error (% / mm)
Model
All analytical (baseline) All analytical + GNN residual
34.30
13.74
0.0028 0.01 0.0017 0.006
8.87
8.93
Analytical passive forces + GNN contact (ours)
4.03
4.69
0.0041 / 0.015
GNN passive forces + GNN contact
9.35
10.97
0.0032 / 0.01
Table 1: Ablation study that compares GNN in different parts of the simulation workflow.
Ablations Table evaluates (i) an all analytical model (DPE baseline), (ii) analytical with a GNN global residual, (iii) analytical passive forces (cable and gravity) computation and the GNN for contact (our chosen method), and (iv) single GNN that learns both passive forces and contact. The data used here are the 3-bar tensegrity data from MuJoCo also used for the above sim-to-sim experiments. All GNN variants outperformed the all analytical baseline, and the proposed hybrid approach showed the best results in positional and rotational error.
Table evaluates the choice of the loss function. Two alternatives to the proposed approach are considered that operate in rigid-body space: (i) a positional loss over the rod end cap positions, which combines positional and rotational errors into one metric; (ii) a 6l) pose loss comprising a Euclidean distance between centers of mass and an angular difference between center axes per rod. The end points' positional loss outperformed the 6D pose loss, but the proposed node-based loss strongly outperformed both rigid bbsed losses as it better enforces node dynamics.
7

---

Positional Error (%)
Rotational Error (*)
Penetration
Model
Error (% / mm)
Position orientation loss
126.64
32.95
0.0086 0.031 0,0050 0.018
End point positions loss
69.07
28.24
Node position loss (ours) 4.03
4.69
0.0041 0.015
Table 2: Ablation study that compares loss taken in rigid-body space and node space.
Mesh/Surface representations multi-object representations Table compares the proposed approach against alternative choices for the graphical representation for the GNN previously used for general rigid-body GNN simulators i.e., the general mesh [9, 10], and surface-based [11] repre- sentations. The table reports training wall-clock time, inference wall-clock time on the GPU, and inference wall-clock time on a single CPU.
Pos Error
Rot Error
# of Nodes
# of Edges
GPU Train (hours)
CPU Train (hours)
GPU
1 CPU Inference (s/step)
Model
Inference (s/step)
Mesh
38.41 %
67.51
603
7632
58.66
n/a
0.0228
0.1238
Surface
49.60%
90.53
600
1212
25.51
n/a
0.0171
0.0207
Object (ours) 8.78%
8.84
22
60
7.69
12.56
0.0237
0.0153
Table 3: Performance metrics for different graphical representations for the GNN.
The proposed object-baset representation is much more accurate as well as efficient. The mesh and surface-bel based representations did not converge to good rollout accuracy for the same epochs. They also need more time, model size, and node density to achieve reasonable accuracy, all of which increases the computational resources required. The proposed approach can be trained via CPU only with a small increase in training time. For inference over GPU, the proposed representation is mildly slower than the mesh and surface ones. When inference is performed over single CPU, the proposed object-based representation is faster than the alternatives.
Limitations
While the proposed engine outperforms the alternatives in data captured from real 3-bar tensegrity, it performs more effectively on data captured from simulation with full observability. This is due to several key differences between the two setups: (i) real sensing data only provide the center position of each end cap, providing only five out the six DoFs each rod (missing the orientation about its center axis); (ii) real state estimation not directly estimating instantaneous linear and angular velocities; (iii) real observations have lower frequency than what is possible in simulation and may be available at non-uniform time intervals; (iv) the real PID controller
black box that could not be perfectly modeled in simulation. A potential direction to overcome these partial observability limitations corresponds to applying trajectory smoothing and using additional sensing information to estimate each rod's orientation about its axis. Smoothing can help identify the set of states that best explain the entire observed trajectory so that it respects physical constraints.
Another limitation is that the proposed method only considers flat ground, step in modeling contact dynamics for tensegrity structures using GNNs. To deal with non-flat terrains and obstacles, the plan is to generalize the distance-to-ground feature to a signed-distane This would be nigh-dimensional feature, so the encoding of this information would need to be explored. Additionally, data diversity would increase, thus increasing data requirements.
field given the terrain.
Discussion
This work shows that representing tensegrity robots as object-based graphs allows GNNs to learn complex contact dynamics and improves simulation accuracy for differentiable engines. It also provides computational benefits over alternatives. This observation can have broader implications for modeling robotic platforms via GNNs, especially those that are difficult to model analytically and which exhibit a graphical structure. potential target adaptive hands [53] that are also cable- driven, have compliant joints, and can be graphically represented. The simulator can also be used for learning controllers that achieve more sophisticated gaits and skills.

---

Acknowledgments
N. C. and M. A. were supported in part by the National Science Foundation (NSF) under awards CCF-21 10861, IIS- IIS-2132972, IIS 12-2228,5 and CCF-2312220 well research gift from Red Hat, Inc. In addition, N. C. and K. B. were supported in part by the NSF under award IIS-1956027. W. R. J. and R. K. B. were supported by the NSF under grant no. IIS-1955225. Any opinions, findings and conclusions, or recommendations expressed in this material are those the authors and do not necessarily reflect the views of the NSF.
References
[1] Lessard, D. Castro, W. Asper, S. D. Chopra, L. B. Baltaxe-Admony, M. Teodorescu, V. Sun- Spiral, and A. Agogino. A bio-inspired tensegrity manipulator with multi-dof, structurally compliant joints. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), pages 5515-5520, 2016.
[2] A. P. Sabelhaus, I.. J. van Vuuren, A. Joshi, F. I.. Zhu, J. Garnier, K. A. Sover, J. Navarro, A. K. Agogino, and A. M. Agogino. Design, simulation, and testing of a flexible actuated spine for quadruped robots. arXiv: Robotics, 2018.
[3] M. Chen, J. Liu, and R. Skelton. Design and control of tensegrity morphing airfoils. Mechanics Research Communications, 103: 01 2020.
[4] Bruce, A. P. Sabelhaus, Y. zhen Chen, D. Lu, J. Morse, S. Milam, K. Caluwaerts, A. Agogino, and V. SunSpiral. Superball Exploring tensegrities for planetary probes. 2014.
[5] L. Zhao, Y. Wu, W. Yan, W. Zhan, X. Huang, J. Booth, A. Mchta, K. Bekris, R. Kramer- Bottiglio, and D. Balkcom. Starblocks: Soft actuated self-connecting blocks for building de- formable lattice structures. IEEE Robotics and Automation Letters, PP:1-8, 08 2023.
[6] K. Wang, M. Aanjaneya, and K. Bekris. Sim2sim evaluation novel data-efficient differen- tiable physics engine for tensegrity robots. pages 1694-1701. 09 2021.
[7] K. Wang, M. Aanjaneya, and K. Bekris. A recurrent differentiable engine for modeling tenseg- rity robots trainable with low-frequency data. 2022 International Conference on Robotics and Automation (ICRA), pages 3230-3237, 2022.
[8] K. Wang, W. R. Johnson, S. Lu, X. Huang, J. Booth, R. Kramer-Bottiglis M. Aanjaneya, and K. Bekris. Real2sim2real transfer for control of cable-driven robots via differentiable physics engine. In 2023 IEEE/SS International Conference on Intelligent Robots and Systems (IROS), pages 2534-2541, 2023.
[9] K. R. Allen, L. Guevara, Y. Rubanova, Stachenfeld, Sanchez- Gonzalzz, P. Battaglia, and T. Pfaff. Graph network simulators can learn discontinuous, rigid contact dynamics. In K. Liu, D. Kulic, and J. Ichnowski, editors, Proceedings of The 6th Conference on Robot Learn- ing. volume 205 of Proceedings of Machine Learning Research, pages 1157-1167. PMLR, 14-18 Dec 2023.
[10] K. R. Allen, Rubanova, Lopez-Guevara, Whitney, A. Sanchez-Gonzalez Battaglia, and Pfaff. Learning rigid dynamics with face interaction graph networks. ArXiv, abs/2212.03574 2022.
[11] T. LupezaGuevara, Y. Rubanova, W. F. Whitney, T. Pfaff, K. Stachenfeld, and K. R. Allen. Scaling face interaction graph networks to real world scenes, 2024.
[12] E. Todorov, T. Erez, and Tassa. Mujoco: A physies engine for model-based control. In 2012 JEEE/RSJ International Conference Intelligent Robots and Systems, pages 5026- 5033, 2012.

---

[13] S. Lu, W. R. Johnson, K. Wang, X. Huang, J. Booth, R. Kramer- Kramer-Bottigh Bottiglio, and K. Bekris. 6n- dof pose tracking for tensegrity robots. In Robotics Research, pages 136-152, Cham, 2023. Springer Nature Switzerland.
[14] J. M. Friesen. Tensegrity matlab objects. URL httpps://gthhab..cmm//eeeeeeeeee Batpea_MMAALA______
[15] V. Tadiparthi, S.-C. Hsu, and R. Bhattacharya. Stedy: Software for tensegrity dynamics. Jour- nal of Open Source Software, 4(33):1042, 2019.
[16] R. Goyal, M. Chen, M. Majji, and R. E. Skelton. Motes: Modeling of tensegrity structures. Journal Open Source Software, 4(42):1613, 2019.
[17] C. Paul, F. J. V. Cuevas, and H. Lipson. Design and control tensegrity robots for locomotion. IEEE Transactions on Robotics, 22:944-957, 2006.
[18] R. Smith. Open dynamics engine, 2008. URL http: //www. ode. org/
[19] K. Caluwaerts, Despraz, A. Iscen, A. Sabelhaus, J. Bruce, B. Schrauwen, and V. Sunspiral. Design and control of compliant tensegrity robots through simulation and hardware validation. Journal of the Royal Society, Interface the Royal Sociery, 11, 09 2014.
[20] S. Wittmeier, M. Jäntsch, K. Dalamagkidis, M. Rickert, H. G. Marques, and A. Knoll. Caliper: A universal robot simulation framework for tendon-driven robots. In 20// JEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, pages 1063-1068, 2011.
[21] E. Coumans. Bullet physics simulation. ACM SIGGRAPH 2015 Courses, 2015.
[22] F. de Avila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z. Kolter. End-to-end dif- ferentiable physics for learning and control. In S. Bengio, H. Wallach, H. Larochelle, K. Grau- man, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[23] J. Degrave, M. Hermans, J. Dambre, and F. wyffels. A differentiable physics engine for deep learning in robotics. Frontiers in Neurorobotics, 13, 2019.
[24] K. Werling, D. Omens, J. Lee, I. Exarchos, and K. Liu. Fast and feature-comme complee differentiable physics engine for articulated rigid bodies with contact constraints. 07 2021.
[25] A. H. Taylor, S. Le Cleac h. Z. Kolter, M. Schwager, and Z. Manchester. Dojo: A differentiable simulator for robotics. arXiv preprint arXiv:2203.00806, 2022.
[26] Y.-L. Qiao, Liang, Koltun, and M. C. Lin. Efficient differentiable simulation of articulated bodies. In Meila and Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8661- 8671. PMLR, 18-24 Jul 2021.
[27] M. Raissi, P. Perdikaris, and G. Karniadakis. Physics-informed neural networks: A deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019. ISSN 0021-9991.
[28] B. B. Juš Kocijan, Agathe Girard and R. Murray-Smith Dynamic systems identification with gaussian processes. Mathematical and Computer Modelling of Dynamical Systems, 11(4): 411-424, 2005.
[29] C. L. C. Mattos, A. Damianou, G. A. Barreto, and N. D. Lawrence. Latent autoregressive gaussian processes models for robust system identification. IPAC-PapersOnLine, 49(7):1121- 1126, 2016. ISSN 2405-8963. 1 11th IFAC Symposium Dynamics and Control of Process SystemsIncluding Biosystems DYCOPS-CAB 2016.
10

---

[30] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning 2019.
[31] S. Markidis. Physics-informed deep-learning for scientific computing. ArXiv, abs/2103.0655555,
2021.
[32] E. Heiden, D. Millard, E. Coumans, Y. Sheng, and G. S. Sukhatme. Neuralsim: Augment- ing differentiable simulators with neural networks. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9474-9481, 2020.
[33] S. Pfrommer, M. Halm, and M. Posa. Contactnets: Learning of discontinuous contact dynamics with smooth, implicit representations. Conference on Robot Learning, 2020.
[34] B. Bianchini, M. Halm, and M. Posa. Simultaneous learning of contact and continuous dy- namics. In Conference on Robot Learning (CoRL), Nov. 2023.
[35] A. Sanchez-Gonzales J. Godwin, T. Pfaff, R. Ying. J. Leskovee, and P. W. Battaglia. Learning to simulate complex physics with graph networks. ArXiv, abs/2002.09405, 2020.
[36] Y. Choi and K. Kumar. Graph neural network- weawork-bsseddee surrogate model for granular flows. Com- puters and Geotechnics, 166:1066 06015, 2024. ISSN 0266-352X.
[37] R. Bhattoo, S. Ranu, and N. M. A. Krishnan. Learning the dynamics of particle-based systems with lagrangian graph neural networks. Machine Learning: Science and Technology. 4, 2022.
[38] J. Shlomi, P. W. Battaglia, and J. R. Vlimant. Graph neural networks in particle physics. Machine Learning: Science and Technology, 2. 2020.
[39] T. Pfaff, M. Fortunato, A. Sanzhez-Gonzalez, and P. W. Battaglia. Learning mesh-based sim- ulation with graph networks. ArXiv, abs/2010.03409, 2020.
[40] M. Fortunato, T. Pfaff, P. Wimsberger, A. Pritzel, and P. W. Battaglia. Multiscale meshgraph- nets. ArXiv, 10.0//00 2022.
[41] J. Linkerhägner, N. Freymuth, P. M. Scheikl, F. Mathis- Ullrich and G. Neumann. Grounding graph network simulators using physical sensor observations. ArXiv, abs/2302. 11664, 2023.
[42] H. Shao, Kugelstadt, Hädrich, W. Palubicki, Bender, S. Pirk, and D. L.. Michels. Aceu- rately solving rod dynamics with graph learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 4829-4842. Curran Associates, Inc., 2021.
[43] M. Parmar, M. Halm, and Posa. Fundamental challenges in deep learning for stiff contact dynamics. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5181-5188 2021.
[44] H. Chen, Y. Niu, K. Hong, S. Liu, Y. Wang, Y. Li, and K. interactions with behavior primitives: An application in stowing tasks. Proceedings of Machine Learning Research, 229, 2023. ISSN 2640-3498.
Predicting object
[45] X. Lou, Y. Yang, and C. Choi. Learning object relations with graph neural networks for target- driven grasping in dense clutter. In 2022 International Conference on Robotics and Automation (ICRA), pages 742-748, 2022.
[46] Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. ArXiv, abs/1810.01 2018.
[47] Almeida, Schydlo, A. Dehban, and J. Santos-Victor. Sensorimotor graph: Action- conditioned graph neural network for learning robotic soft hand dynamics. pages 9569-9576, 09 2021.
11

---

[48] J. T. Kim, J. Park, S. Choi, and S. Ha. Learning robot structure and motion embeddings using graph neural networks. ArXiv, abs/2109.07543, 2021.
[49] 7. Zhang, L. Yang, C. Sun, W. Shang, and J. Pan. CafkNet: GNN-Empowered Forward Kine- matic Modeling for Cable-Driven Parallel Robots. arXiv e-prints, art. arXiv:2402.18420, Feb. 2024.
[50] E. Tolstaya, F. Gama, J. Paulos, G. Pappas, V. Kumar, and A. Ribeiro. Learning decentral- ized controllers for robot swarms with graph neural networks. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, Proceedings of the Conference Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 671 671-682. PMLR, 30 Oct-01 Nov 2020.
[51] Q. Li, Gama, A. Ribeiro, and A. Prorok. Graph neural networks for decentralized multi- robot path planning. In 2020 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 1 11785-11792, 2020.
[52] Y. Hu, L. Anderson, T.-M. Li, Q Sun, N. Carr, J. Ragan-Kelley, and Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint arXiv: 00935,, 2019.
[53] R. Ma. J. Belter, and A. Dollar. Hybrid deposition manufacturing: Design strategies for multi- material mechanisms via three-dimensional printing and material deposition. Journal of Mech- anisms and Robotics, 7:021002, 05 2015. doi: 11/11.142444000
[54] M. Müller, B. Heidelberger, M. Teschner, and M. Gross. Meshless deformations based on shape matching. ACM Trans. Graph., 24:471-478, jul 2005.
12

---

Supplementary Material
A Graph features
Body nodes Each node has the features N, = {m, ,I, v. dg), where m is the reciprocal of the mass, /,- is the inverse of the inertia tensor in the principal directions, v D is the first-order fmite-ddifference velocities defined as (P) p 1 At, and, lastly, dg is the distance from the ground. The finiteddifference velocities are not instantaneous as in traditional physics simulators. They are instead average velocities over the time step. Previous work on differentiable engines [52] computes time of impact within a time step for accurate simulation. The proposed approach only considers average velocity instead, without splitting the time step, as shown in Fig. The GNN is able to learn over average velocities and accurately incorporate the contact dynamics in its predictions.
Body edges b have features Enpoyy {dij. |d d' [|d] where dij is the displacement vector between two body nodes and d the displacement vector between two body nodes in the undeformed, body-frame state.
Kiy. (ig) where (diy, viy") are the displacement and relative velocity vectors between end cap nodes i and j. and (L4) Kij, Cij) are the cable's rest length, stiffness, and damping, respectively.
Cable edges Eggulle have features Ei Eggelee = {dij, d.j. Vij" Li
Contact edges Egg" have features E = {dg, n, vaa oter }. where dg is the minimum signed- distance to the ground, á is the contact normal unit vector, and veet tangential components of the relative velocity.
ynorm and oter are the normal and
B Training details
Hardware and implementation All methods are trained and tested on machine equipped with Nvidia RTX 4090 GPU and an AMD Ryzen 7950 16-core, 4.5 GHz CPU. All simulators were built and executed using PyTorch and PyGeometric.
Training strategy For models trained with simulation data, curriculum learning strategy is em- ployed where the trajectory rollout length progressively increased. Initially, the model trained to predict the immediate 1-step ahead state for 200 epochs. The model train loss and validation loss will start to flatten out and further training tends to see an increase in full trajectory error. Next, the model is trained to perform a 2-steps look ahead for 100 epochs, 4-steps ahead for 50 epochs, and finally, 8-steps ahead for 25 epochs. It is observed that further increase in rollout length does not decrease error. This procedure allows the model to incrementally experience the errors makes during rollout and to adjust them.
For the mesh and surface-based models, the models were trained based on the procedures described in prior work [9, 10, 11]. These models were only trained with 1-step roll-out lengths and with Gaussian random-walk noise of x 10-4 to node positions upon input to the model, for 800 epochs.
Training hyperparameters The models were trained with progressively decreasing learning rates of 10-5, 10-6 10-7, and 10-8, These learning rates correspond the n-steps look ahead curriculum phases detailed above. A mini-batch size of 128 was used during training. The Adam optimizer with weight decay of 10-2 was used.
Data Augmentation At the start of each training step, the ,, π] about the z-axis.
rotated by a random angle
13

---

C
Model details
Network architecture All the multi-layer perceptron (MLP) models have two layers, ReLU activa- tions, and residual connections. All MLPs, except the decoder, apply LayerNorm on their outputs. For the surface-based vs multi-object experiments, the width and latent vector dimension sizes were 64. For all other experiments, the dimension sizes were 128. Lastly, models for the 3-bar tensegrity had four passage-passingg steps while the models for the 6-bar tensegrity had 10 message-passinng steps due the larger graph. All inputs were normalized to zero-mean and unit variance.
Rigid-body state to node states Let T be the rigid-body transformation that takes the rigid body from its body frame to the world frame at time step t. To compute the node states, P v,, the same transformation is applied the node positions in the body frame:
= (T
Node states to rigid-body state Since the node positions are not strictly enforced to maintain rela- tive distance from one another, there can be shape violations. Hence, an approximation of the rod's pose needed. Pe is approximated as the average of the node positions. R is computed as minimal rotation needed to rotate the 7-axis unit vector rod's center axes é; approximated with the unit vector pointing from one rod endpoint the other. more principled method, such as shape matching [54], can be used if the graph is more complex. Subsequently, the rigid-body velocities are the first-order approximation of velocities needed to take [P', R*) to [p+t,R
Cable model The cable model a linear model following Hooke's law that allows for tension but not compression. computes the cable force F cablle at time based on stiffness component Fk and the damping component Ft:
( K [K(Teet - Ax")
if 1 rest Ar
Fk
0
otherwise
F² = =c(vvet x²)
F cable (Fk FF
where K is the cable stiffness, is the cable damping, yest is the rest length at time t, Arf is the distance between the cable attachment points at time t, vfet is the relative velocity between the cable attachment points, and & is the unit direction pointing from one attachment point to the other.
Actuation model The actuation model consists linear model motor that receives an input control signal u' between [-1, and acts on cable by changing the cable's rest length 1/ at time
w² 88mma
Alrest = 0.5(w* + +
(est = P-alt Alrest
where wf is the angular velocity of the motor time f, is input speed parameter [0, 1), Wmaz is the maximum angular velocity the motor can achieve, Alrest is the change in rest length due to the motor, Twinch is the winch radius of the motor, and At is the time step size.
D Tensegrity Robot Details
Table provides the values for physical parameters of the real tensegrity robot. These measure- ments were used in order to set the corresponding parameters in simulation to the same value. The parameters not learned by the differentiable physics engine.
14

---

Attribute
Measurement
inner rod length inner rod radius
0.325m
0.0016m
inner rod mass end cap radius
3.8g
0.0175m
end cap mass
10.5g
motor radius
0.0175m
motor length
0.045m
motor offset (center to center)
0.1175m
motor mass
35.3g
short cable stiffness
10
short cable damping ratio
10 s/m
long cable stiffness
10 N/kN/
long cable damping ratio
10 8
Table 4: Physical tensegrity robot measurements used simulation parameters
E
MuJoCo Setup
MuloCo was the simulator chosen to be the ground truth data in the sim-to-sim experiments. The data were generated using the following setup:
Rigid bodies and cables were set with the same mass, geometric properties, and stiffnesses the real robot shown in Table of the appendix:
Friction coefficients were set 0.5 for tangential, 0.005 for torsional, and 0.9 for rolling.
The time step size was 10 ms, and semi-explicit Euler time integration scheme was em- ployed.
The tendon model was adopted but modified to only apply tension no compression forces.
The authors executed the rolling and crawling controls developed for the real robot, and simulated PID controller, generate trajectories the training, validation, and testing data sets.
Additionally, the authors generated drop and throwing trajectories by initializing robots random heights and random linear velocities.
Multi-Layered Perceptron Baselines
MLPs were used the sim-to-sing experiments to show why a specialized architecture based on GNNs is beneficial. In order to best compare against the MLP baselines the GNN, the authors tried to keep the neural network size the same between the two. This resulted in the MLP comparison points having:
Layer widths of 128
(2 (number of message passes in GNN 4) + 2) number of layers
Residual connections every 2 layers
LayerNorms every layers, except at the very last layer
ReLU activation functions
Trained with the same number epochs (~~ 375) as the GNN
G
Real Robot Training Procedure
For the real 3-bar tensegrity robot experiments, the following steps were taken:
15

---

1. A simulator (DPE or MuJoCo) was tuned to best match the real training set
DPE learned via gradient descent;
MuJoCo contact parameters were searched random search
2. Dense, high-frequency, and fully observable data matching training trajectories were gen- erated in simulation
3. The proposed GNN model was then pretrained over the generated simulation training data using the training setup and strategy descibed in Appendix B.
4. Then, the pretrained GNN model was fine-tuned by training over the real training trajecto- ries, where single forward pass a full rollout. This model was trained over 10 epochs at relatively small learning rate of 10-$. Additionally, PID controller was included in the training loop.
16


Reviews:
[{"review_id": "0EqDeWDekR", "reviewer": "Reviewer_Luga", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work proposes a method for simulating tensegrity structures using graph-neural networks to aid in modeling contact dynamics. The approach is compared to traditional analytic rigid-body simulation and validated on a tensegrity robot.  The novelty of the work is in the explicit modeling of the tensegrity structure with a graph-neural-network (GNN) to aid in the computation of the contact dynamics, where other attempts either used a generalized formulation (mesh-primitives + GNNs) or relied on rigid-body simulators and first-principles.", "summary_of_recommendation": "The paper is a novel application, which provides support for the method of using graph-neural-networks as a technique for modeling complex physical systems. The claims on performance are nascent and would require a solid benchmark paper to be convincing that this technique was truly superior. My pause on giving a solid accept is that I have one lurking doubt about the validity of the problem formulation; classically a position-based euler-scheme would be too stiff and nearly impossible to achieve good results. A solid rebuttal from the authors would convert me to a solid accept.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 2, "quality_of_limitations": 3, "questions_for_rebuttal": "Please see review for suggestions and nits. I only have one fundamental issue that I would like addressed:\n\nWhy is the contact problem formulated via a state-transition? Can the authors please provide prose (math would be wonderful) explaining why the problem wasn't framed in terms of impulse-momentum or energy? And explain why the \"average velocity\" over time-steps does not violate causality (e.g. present-state has knowledge of future-state) or it doesn't matter?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DrqX3bhiWp", "reviewer": "Reviewer_z5sV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper introduces an approach to modeling the dynamics of tensegrity robots using Graph Neural Networks (GNNs). The paper's primary strength lies in its innovative application of GNNs to model the complex dynamics of tensegrity robots. In particular, they propose a framework that represents tensegrity robots as graphs, utilizing the graph-like structure formed by the robots' cables and rods. The approach aims to overcome challenges in modeling and controlling tensegrity robots due to their complex dynamics and high degrees of freedom. The method leverages GNNs to predict the contact dynamics and improves upon previous differentiable physics engines. The results show better accuracy and computational efficiency when compared to existing methods, demonstrating the potential of GNNs in this domain.", "summary_of_recommendation": "The paper presents an advancement in the modeling of tensegrity robot dynamics using GNNs.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "- Why was the specific GNN architecture chosen, and how does it compare to other possible architectures, such as those incorporating attention mechanisms or recurrent elements? Would alternative architectures potentially improve the model's performance, especially in handling temporal dynamics and varying contact conditions?\n\n- Given the assumption of flat ground in the current model, what modifications would be necessary to accommodate complex terrains or non-flat surfaces? Can you discuss the potential challenges and solutions for expanding the model to more diverse environments?\n\n- How does the model scale with increasing complexity in tensegrity structures or more extensive simulation environments?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jBM6Jy1myq", "reviewer": "Reviewer_mw3p", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work proposes a learning method using graph neural networks (GNNs) to improve tensegrity robots modeling. The graph representation leverages the natural-like cable connectivity between the rod end caps. The model is used for 3-bar and 6-bar tensegrity robots in simulation and 3-bar one in reality. The result shows higher accuracy and efficiency than previous work. This work shows a good application of GNN for modeling the dynamics of a hybrid rigid-soft mobile robot.", "summary_of_recommendation": "This paper shows the potential advantage of graph neural network (GNN) for applicaitons on hybrid rigid-soft robot, which inspires readers to model the dynamics to robot using GNN.", "technical_quality": 2, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "1. Please add more details on the setup of the simulation in MuJoCo.\n2. Please point out what DPE is used for comparison, and is it state-of-the-art?\n3. Please introduce the details on MLP, such as layer numbers, epoches, and can it be improved by adding more layers or other ways?\n4. Please add more details on the finetune during the real 3-bar tensegrity robot experiment.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y8APQQZBcz", "reviewer": "Reviewer_uba8", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes a graph neural network architecture to learn the discrete-time forward dynamics of tensegrity robots. Each joint of the robot is modeled as node in a Graph while linear rigid bodies are modeled as a sequence of nodes that are connected by edges. Contacts are modeled as edges between nodes and the ground which is assumed to be flat. The GNN is trained via supervised regression to predict changes in the node states. The utility of the proposed model is being demonstrated in simulation using a Mujoco model as baseline and hard-ware experiments.", "summary_of_recommendation": "Nice work that shows how to use structure from robot kinematics to predict dynamics using GNNs. The authors could discuss a bit more critically what design decisions they made to get the model and why. Also the experimental discussion requires some polishing. Albeit, I enjoyed reading this paper.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "- **Question - C1:** Do the authors feed to the GNN the position of each node? If yes, how do you ensure that the GNNs prediction stays invariant to translation in space?\n- **Question 2:** How do you tune the baseline model? Can you share the hyper-parameter search method and its results?\n- **Question 3 - C2:**  What is Fig. 8 showing? Is this simply a model rollout? If yes, for how long and what prediction FPS? If the roll-out is longer than 2 seconds, how do you ensure that the predicted state is not diverging?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: PRE REBUTTAL:

High level summary of reviews:

Strengths:

- All reviewers feel that the paper presents a novel application of GNNs to the complex dynamics of tensegrity robots.
- Many reviewers state that the paper is well-written and clear.
- Experiments are thorough (and on real world hardware) and the video included was high quality.
- Improvement over state of the art for tensegrity robots.
 
Weaknesses:

- Multiple reviewers question whether the experiments generalize well; for example, the models were only tested on flat ground (method assumption) and the real experiments were only conducted on a 3-bar robot.
- Some issues around the problem formulation were raised by multiple reviewers. For example a position-based euler-scheme would be too stiff (Reviewer Luga).
- Additional clarifications on method details were requested (e.g. Mujoco simulation setup, details on MLP, etc). See individual reviewer comments for details.
- The ablation studies are limited and more exploration into different architectures or representations could be beneficial.
- The penetration error is higher and this is not explained in the paper.
- Only incremental improvements on real-world data.

POST REBUTTAL:

The authors provided a very detailed rebuttal and made a number of clarifying updates to the manuscript. Many of the weaknesses raised by the reviewers were resolved. One remaining weakness is still the incremental improvements on real-world data, which will certainly limit the impact of this work within the broader community (as the community is very real-world results driven these days). The authors comment that this will effectively be fixed in another submission (with orthogonal contributions) is understood, but the decision to make these 2 papers rather than one strong paper definitely results in a weaker submission as a result. This paper has a number of novel contributions and is well written is clear, but is ultimately weakened by final results. With that said, the paper marginally crosses the threshold for acceptance. The simulation results might be of interest to the community and the algorithmic contributions are somewhat novel.
Decision: Accept