OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion
Abstract
Muscle-actuated control is a research topic of interest spanning different ﬁelds, in particular biomechanics, robotics and graphics.
This type of control is particularly challenging because models are often overactuated, and dynamics are delayed and non-linear.
It is however a very well tested and tuned actuation model that has undergone millions of years of evolution and that involves interesting properties exploiting passive forces of muscle-tendon units and efﬁcient energy storage and release.
To facilitate research on muscle-actuated simulation, we release a 3D musculoskeletal simulation of an ostrich based on the MuJoCo simulator.
Ostriches are one of the fastest bipeds on earth and are therefore an excellent model for studying muscle-actuated bipedal locomotion.
The model is based on CT scans and dissections used to gather actual muscle data such as insertion sites, lengths and pennation angles.
Along with this model, we also provide a set of reinforcement learning tasks, including reference motion tracking and a reaching task with the neck.
The reference motion data are based on motion capture clips of various behaviors which we pre-processed and adapted to our model.
This paper describes how the model was built and iteratively improved using the tasks.
We evaluate the accuracy of the muscle actuation patterns by comparing them to experimentally collected electromyographic data from locomoting birds.
We believe that this work can be a useful bridge between the biomechanics, reinforcement learning, graphics and robotics communities, by providing a fast and easy to use simulation.
Introduction and related work
Actuation means generating forces in order to create movement.
In nature, movement in vertebrates is produced by contracting muscles which pull on skeletal bones, creating torques at the joints.
Similarly, robots use motors, but motors and muscles have very different properties.
A review of several different actuation methods is given by Peng & van de Panne (2017).
Understanding muscles is of interest in many different ﬁelds.
From a biological point of view, we would like to understand how animals solve complex locomotor tasks.
Muscles are also relevant in computer graphics to obtain accurate skin deformations in virtual characters and to obtain more natural-looking gaits for ﬁlms and video games (Angles et al., 2019; Abdrashitov et al., 2021; Modi et al., 2021).
In robotics, with the exception of soft robots, muscles have been studied relatively little despite their interesting energetic properties and the sophisticated movements they enable (Wang et al., 2021; Cotton et al., 2012).
In sports medicine musculoskeletal simulations are used to understand sports injuries (Bulat et al., 2019).
Biomechanics is the study of how biological systems produce motion.
Biological motion is affected by many factors such as the musculoskeletal geometry, internal states of the muscles, and forcelength-velocity curves (discussed below).
Understanding how complex biological systems such as humans and other animals use their muscles to move is quite challenging.
Obtaining data from living animals is difﬁcult and not neutral to animal welfare.
To overcome this problem, biomechanics researchers often use computer-based simulations combined with numerical optimization to estimate how muscles are used.
However, most of the research that uses optimization techniques does not leverage the new available tools from machine learning, in particular those based on the reinforcement learning (RL) framework (Sutton & Barto, 2018).
These tools have proven to be a valuable alternative to traditional optimization techniques to solve complex tasks in continuous and high-dimensional state and action spaces (Lillicrap et al., 2015; Fujimoto et al., 2018).
The NeurIPS conference has been hosting recurrent competitions to bridge this gap 2 , where the goal was to make a musculoskeletal human model walk.
The challenge used the OpenSim (Delp et al., 2007; Seth et al., 2018) simulator.
However, as documented in many of the solutions (Pavlov et al., 2018; Zhou et al., 2019; Akimov, 2019;
Kolesnikov & Khrulkov, 2020), the engine was too slow to properly use traditional deep RL methods, which often require millions or billions of samples to ﬁnd good solutions.
Moreover, the model used in the latest challenge was in many ways quite simplistic, actuating only the legs with 22 muscles in total and removing the arms.
Some innovative research has used deep RL to solve complex locomotion tasks such as playing basketball (Liu & Hodgins, 2018), performing athletic jumps (Yin et al., 2021), boxing and fencing (Won et al., 2021).
These studies used motion capture data for a rich reward signal during training (Merel et al., 2017;
Hasenclever et al., 2020; Merel et al., 2018).
Wang et al.
(2012) was one of the ﬁrst studies in the graphics community to use biological actuators.
Geijtenbeek et al.
(2013) used evolution-based algorithms to control muscular bipeds and also optimized muscle routing.
Jiang et al.
(2019) tried to bridge torque-actuated models and muscle dynamics, using neural networks to map muscle activations from torque commands and achieved natural looking gaits using torque actuators.
Lee et al.
(2018) used volumetric muscles and trajectory optimization for juggling.
Lee et al.
(2019) combined RL and motion capture clips to control a human musculoskeletal model solving a variety of tasks from walking to weight lifting.
Simulating musculoskeletal animal models is not common outside the ﬁeld of biomechanics.
Most of the efforts are focused on humans because of available resources in anatomical atlases and interest from the entertainment industry and sports.
Available full-body animal musculoskeletal models include, for example the chimpanzee (Sellers et al., 2013) and dog (Stark et al., 2021).
Moreover, for human models, researchers tend to have access to much higher quality motion capture data than for animals, but animal research often have superior empirical measurements of physiological function.
Here we present a new ostrich musculoskeletal model that uses a fast physics engine with reinforcement learning tasks.
To the best of our knowledge, we are the ﬁrst to apply RL to such a realistic [Figure] animal model, solving locomotion tasks.
The main differences from prior human simulations are obviously the anatomy of the model but also the RL architecture; e.g. Lee et al.
(2019) separated muscle coordination from trajectory mimicking through two different networks with privileged information and an intermediate PD controller.
Ostriches are interesting to study because they are fast, economical bipedal runners (Alexander et al., 1979).
With our new computational tools, which we release along with this paper, we hope to open new opportunities for researchers interested in accurate and fast muscle simulation provided by MuJoCo combined with RL.
First, we describe how we built the model by assembling various types of data and how we used
MuJoCo’s muscle model.
Then, we detail the various tasks that we designed.
The results from one of those tasks in particular are then used to evaluate the accuracy of the muscle excitation patterns by comparing against empirical electromyography data.
Finally, we brieﬂy explain how the similarity between the Cassie robot and ostriches allowed us to adapt our motion capture tracking pipeline to a simulated model of Cassie.
Model 2.1 Anatomical data acquisition
Building an anatomically accurate 3D musculoskeletal model required gathering data from real ostriches.
In this subsection, we discuss how we collected the anatomical data necessary for our model.
Computed tomography scanning We ﬁrst acquired multiple Computed tomography (CT) scans of a 96.5kg adult male ostrich specimen, donated by a local farm.
The CT scans were then segmented (separating the bone from the rest of the x-ray slices) in Mimics (Materialise, Inc.; Leuven, Belgium) software to obtain 3D models (polygonal meshes) for each bone from the ribcage to the head, including the wings.
These 3D geometries were important for two reasons: ﬁrstly they provided an accurate representation of the geometry of the bones, vital for deﬁning muscle attachment points; and secondly they allowed inference of the inertia of body segments.
To obtain the inertia of body segments we followed the steps provided in Hutchinson et al.
; the idea is to wrap the skeleton parts within meshes representing ﬂesh and assume constant density (usually water density is used), from there, volumes, masses and the inertia tensors can be computed.
Muscle dissection Next, we performed a dissection in order to gather data for muscles of the neck and rib cage for our ﬁnal model, this data is needed to correctly model muscles peak forces and lengths.
Wings are not actuated in our model because air friction is not considered in our simulations and wings are mostly used when turning at high velocity.
Dissection is important to properly understand muscle routing, where the muscles start (origin) and where they end (insertion).
During the dissection, we used anatomical descriptions from Böhmer et al.
; Tsuihiji (2005, 2007) to identify the different muscles.
We also acquired muscle-speciﬁc data: muscle mass, pennation angle, tendon length and muscle ﬁber lengths, used when simulating the muscle-tendon dynamics, as explained below.
2.2 Physics simulators
Here we discuss two computer modelling and simulation software packages commonly used by the biomechanics and reinforcement learning communities.
The ﬁrst one is OpenSim (Delp et al., 2007; Seth et al., 2018), an open-sourced engine that provides benchmarked physics-based muscle simulations.
It is relevant to our study not only for its popularity but also because the legs of our ostrich model were originally modelled with this engine (Hutchinson et al., 2015), as we can see in ﬁgure 2.
The same model was also used to run some tracking (inverse) simulations to estimate the forces, activations and mechanical work of the muscles involved in solving locomotion tasks (Rankin et al., 2016).
OpenSim is quite slow in simulating each time step, as documented in previous competitions in NeurIPS that used it to simulate muscle dynamics (Kolesnikov & Khrulkov, 2020).
The second simulator, MuJoCo (Todorov et al., 2012) is an open-source engine that is used in multiple
RL domains such as dm_control (Tassa et al., 2020) and Gym (Brockman et al., 2016) because it provides fast and accurate rigid body dynamics.
Speciﬁcally, tendon routing in OpenSim uses an iterative algorithm, while MuJoCo’s tendon routing is closed-form, and therefore much faster.
In a comparison made by Ikkala & Hämäläinen (2020), when calculating the average run time overforward simulations MuJoCo is roughly 600 times faster than OpenSim.
Both software packages deﬁne their models in a hierarchical tree structure.
Muscles attach to at least two bones, using two sites at the ends, respectively called the origin and the insertion.
The muscle geometry can be more elaborate than just the origin and insertion sites; other sites called way points and wrapping geometries can also be speciﬁed.
Way points are sites the muscle must pass through, which are useful to maintain an anatomically realistic 3D path for the muscles.
Wrapping geometries are useful to prevent muscles from penetrating the bones, and are geometric primitives such as spheres or cylinders.
After converting the OpenSim leg model to MuJoCo’s format, we completed the ostrich model by adding the ribcage, wings, neck and head geometries.
We then actuated the neck by adding the muscles, matching the routing from the dissection; we also changed the feet morphology from being completely ﬂat to match a more realistic pose.
In our ﬁnal model, each leg contains 7 hinge joints:for the hip, 2 for the knee, 1 for the ankle, and 1 for the metatarsophalangeal (mtp), while the neck contains 34 hinge joints; 2 for each pair of vertebrae, as can be seen in the middle image in ﬁgure 3.
These 48 joints are actuated by 121 muscles, making this model very challenging to control.
In comparison, the OpenSim model used for the NeurIPS 2019: Learn to Move - Walk Around challenge had only 22 muscles.
The muscle dynamics are explained in detail in the Appendix A. After converting the OpenSim model to MuJoCo, we found that ﬁne tuning the muscle lengths was necessary to improve stability.
To ﬁnd the new muscle ranges, we randomized the model joint angles in their limits a large number of times and recorded the global minimum and maximum lengths for each muscle.
Tasks
The musculoskeletal model described above has been implemented using the MuJoCo physics engine (Todorov et al., 2012) which is used in a number of popular control domains for reinforcement learning research, in particular dm_control (Tassa et al., 2020) and
OpenAI Gym (Brockman et al., 2016).
Most of these domains use simplistic models, often inspired by animal morphologies, made up of basic geometric bodies such as capsules, and torque-controlled multiaxial joints.
While such models are fast and fairly easy to control, they do not provide a realistic basis for studying animal movement.
On the other hand, simulations used in biomechanics typically model animals more accurately (Sellers et al., 2013; Hutchinson et al., 2015; Stark et al., 2021) but are slow and often used in conjunction with relatively simple control techniques from the trajectory optimisation repertoire (e.g., direct collocation while minimising the sum of squared muscle activations).
While the previous sections described the proposed musculoskeletal ostrich model, this section describes a set of tasks implemented with dm_control (Tassa et al., 2020) that we used repeatedly throughout this work to test the model.
These tasks could be used by other researchers for their own research.
For example, new biomechanical data can be produced from the simulated behaviours, while RL algorithms can be tested on this unique and challenging set of environments.
We focused on two types of tasks in particular: locomotion and neck control.
We found these to be particularly useful when designing the model because they used the two main groups of muscles present in the model.
To obtain the policies we used the TD4 agent, a mixture of TD3 (Fujimoto et al., 2018), with its pair of critics, delayed actor training and target action noise, and D4PG (Barth-Maron et al., 2018), with its distributional value function (Bellemare et al., 2017) and n-step returns.
We also found that Ornstein Uhlenbeck exploration noise (Lillicrap et al., 2015) was signiﬁcantly better than Gaussian noise, probably due to the importance of correlated excitation when controlling muscles.
We chose to use the Tonic deep RL library (Pardo, 2020), because it provided us with this state of the art agent and the ﬂexibility we needed to quickly try, evaluate and visualize experiments while allowing custom dm_control tasks to be used.
Details about the training hyperparameters and the experiments can be found in the Appendix and the policies obtained with the different tasks can be visualized on the website 3 .
[Figure] 3.1 Run forward
The ﬁrst task we experimented with had a simple “move forward” objective.
It was constrained to a vertical planar space, used torque-actuated joints, and the neck was kept rigidly attached to the thorax of the model.
The goal was to ensure that the underlying model and simulation structure was working as expected before adding muscles.
The obtained gait was similar to the one obtained when training on the 2D walker tasks from dm_control and Gym.
After adding muscles to the legs, we quickly found that the torque-actuated planar agent was not able to ﬁnd satisfactory gaits anymore.
Even after trying a large number of hyperparameter values, we noticed that the same gait was repeatedly emerging: the toes were completely (unrealistically) plantarﬂexed and the ankles remained straight in an unnatural way.
This gait was deemed unsatisfactory because it was not optimal in terms of maximum possible speed but also because was obviously unnatural.
3.2 Motion capture tracking
A typical solution to better constrain the space of policies is to engineer a more sophisticated reward function.
Reference motion tracking is a particularly well-suited option that has been used with motion capture data in numerous studies to produce natural-looking movements (Liu et al., 2010;
Peng et al., 2018; Chentanez et al., 2018; Bergamin et al., 2019; Merel et al., 2018; Peng et al., 2019;
Hasenclever et al., 2020).
However, producing motion capture data of ostriches performing various movements was outside of the scope of this paper.
Instead, as a proof of concept we started with reference motion data that we labelled by hand, frame by frame, using videos of running ostriches found online.
The task was still planar and the quality of the results produced encouraged us to continue further in this direction.
3.2.1 Motion capture data
To move the simulated ostrich to a 3D space and to use more diverse behaviours, we started searching for proper motion capture data.
We did not ﬁnd any open source database but after contacting the authors of Jindrich et al.
(2007), we obtained the data originally used to study joint kinematics of ostriches performing cutting maneuvers.
The provided dataset was composed of 82 clips, recorded at 240 Hz, containing the 3D coordinates of 14 markers distributed as follows: 2 on the breasts, 1 on the spine, 2 on the hips, 2 on the knees, 2 on the ankles, 2 on the mtps, 2 on the toes and 1 on the head.
We selected the largest part of the clips where at least 10 markers were simultaneously present for at least 1 second, typically disregarding the beginning and the end of the clips when the ostrich was not in the ﬁeld of view of the cameras.
This selection left 35 clips.
However, even in those clips, we found that a large number of markers were periodically missing, probably due to feathers and limbs occluding them from the cameras.
To overcome this issue, we used a bidirectional LSTM trained in a similar way as a denoising autoencoder.
With a probability 0 .
1 , we randomly masked some markers and asked the model to predict their value given the context of a segment of 100 steps from the same clip.
We then used this model to predict the actual missing markers.
Since the model was trained and then used on the same, relatively small dataset, we anticipated some overﬁtting issues, but playing the clips with the predicted markers in place of the missing ones gave surprisingly good results (see
Figure 5).
The denoising is discussed in more details in the Appendix D.
After rescaling the marker coordinates to match the size of our model, the next step was to transform the set of marker coordinates to joint positions.
Fortunately, enough markers were present over the different parts of the ostrich body to limit the space of solutions, except for the neck where only one marker was present on the head.
We used gradient descent with the mean Euclidean distance between the reference markers and the ones produced by joint poses on two sets of parameters simultaneously.
The ﬁrst set described where to attach the markers on the body parts and is shared across all the clips.
This was needed because we did not have the exact location of the markers used during the motion capture acquisition and how they would translate to locations with respect to the bones.
The second set of parameters described the joint values at every keyframe.
To reduce the space of solutions for the neck shape, we added a regularisation term, encouraging the neck to follow an S shape.
We ﬁrst tried to use a ﬁnite difference approximation of the gradient but found this approach to be too slow to be practical.
We then decided to implement the kinematics function in a differentiable way, mapping joint values to body locations to create marker locations.
This function was implemented using differentiable operations from the TensorFlow library (Abadi et al., 2016).
We found this approach to be particularly fast thanks to the possibility to simultaneously optimise for all the steps across all the clips using the batch dimension.
3.2.2 Single clip tracking
During training, an initial time-step is uniformly sampled, the model is initialised in the pose corresponding to this step and a small amount of Gaussian noise is added to help diversify experiences.
Since the control frequency used in our experiments was 40 Hz, the task used every sixth data points of motion capture data (240 Hz) during tracking.
Also, to allow the policy to perform time-dependent behaviours and to deal with ﬁnite horizons, we provided the remaining number of steps in the clip to the observations following Pardo et al.
Every clip was tracked by a different policy but using more mocap data could allow the creation of a reusable controller (Merel et al., 2018; Peng et al., 2019; Hasenclever et al., 2020).
After tuning a number of task and agent hyperparameters, we managed to get very satisfactory results in most clips.
The tracking reward is deﬁned as a product of the quantity exp ( − w p e p − w r e r ) measuring how much a body part’s location and orientation match the reference.
Given a body part with center of mass coordinates p and orientation matrix of inertia R , and the corresponding references ¯ p and ¯ R , we use the Euclidean distance e p = (cid:107) ¯ p − p (cid:107) and the angle of the difference rotation e r = arccos (( tr ( ¯ RR T ) − 1) / 2) . The weights w p and w r control the wideness of the Gaussians, accounting for the different magnitudes and importance.
The reward is bounded in (0, 1] and the multiplicative nature ensures that if one of the component is too far from the reference the entire reward decays to 0 .
3.2.3 Cyclic running clip tracking
Considering that the unconstrained running task did not give satisfactory results, we decided to recreate a task with similar conditions but using mocap tracking.
We therefore created a completely cyclic mocap clip from the middle section of one of the clips, that we repeated several times and smoothed to remove discrepancies at the boundaries.
Similarly to the results obtained with the previous task, we found the tracking to be very satisfactory.
A great advantage of this task is that it helped us produce simulated biomechanical data for a cyclic gait that could be directly compared to muscle excitations and lengths measured on living birds (see Section 4).
3.3 Neck control
The neck is composed of 17 vertebrae, allowing a large range of shapes to be produced.
To reach a head position and orientation, an inﬁnite number of solutions exist and when running, the weight and pose of the neck should inﬂuence balance and vision to a great extent.
Controlling the neck with muscles that cross multiple vertebrae requires a very sophisticated policy.
To tune the neck part of the model, we found that full-body mocap tracking was not ideal due to the large number of moving pieces and the fact that the reference poses of the neck were artiﬁcial.
We therefore created another task whose objective was to reach random targets with the beak.
To sample feasible target locations, we used rejection sampling.
Points were repeatedly sampled in a sphere centered at the base of the neck with a radius slightly smaller than the length of the neck, and rejected when inside a second smaller sphere roughly representing the torso of the ostrich.
For this task, only the ribcage, the neck and the head were used and the ribcage was kept ﬁxed in space.
While efﬁcient policies were easily obtained, initially neck shapes were very irregular, with many unnatural, abrupt changes.
To reduce this phenomenon, we increased the stiffness of the neck joints until satisfactory smoothness was obtained.
For the joint limits, we initially used values from Dzemski & Christian (2007) but then realized that the limits had to be increased to allow tighter neck curves which ostriches can perform.
Electromyography comparison
To demonstrate the realism of the proposed simulation, we chose to compare the muscle excitations outputted by a policy trained on the cyclic mocap tracking task, to Electromyography (EMG) data.
One one side, EMG measures electrical activity in muscles, in response to a stimulation from motoneurons and on the other side, raw actions produced by a policy are used by MuJoCo to produce muscle activations
A.
Collecting EMG data in animals is challenging, because their skin might be too thick for electrodes and surgery might be needed to implant them directly into the muscles.
Since, no EMG data of ostriches have been collected yet, we decided to perform comparisons with data from emus, close relatives of ostriches.
The EMG data were originally acquired by Cuff et al.
(2019) to study muscle recruitment patterns across different avian species in order to understand neuromuscular control from an evolutionary point of view.
Their results showed that walking/running birds generally use their hindlimb muscles in very similar ways, so emu EMG data are justiﬁable for comparisons with ostrich simulations.
To compare the data, following standard conventions in locomotion research, we divide the gait into two phases: stance (the foot is on the ground) and swing (the foot is off the ground).
Once we divide the gait into these phases, we scale the timing accordingly between the two studies (e.g., the two birds might be moving at different speeds).
As show in the ﬁgure 6, we ﬁnd broadly similar excitation patterns, except for the GL (Gastrocnemius
Lateralis) which had an extra burst of swing phase excitation in these simulations; not usually found in any extant birds (Cuff et al., 2019).
The ITC (Iliotrochantericus Caudalis) also had simulated excitation throughout more of the swing phase than in the EMG data; reminiscent of ﬁndings for simulated ostriches using the OpenSim model in Rankin et al.
There are many more muscles than actual degrees of freedom in the legs, so there are many possible solutions (excitation patterns) that are able to match the kinematic data.
Moreover, we do not penalize actions in our reward signal, so it is expected to see more excitations than in experimental data.
Compatibility with Cassie
Cassie is a bipedal robot that was produced by Agility Robotics and extensive work has been done on it using trajectory optimization (Reher et al., 2019;
Li et al., 2020; Apgar et al., 2018) and reinforcement learning (Xie et al., 2018, 2020; Li et al., 2021)
Its morphology is very similar to that of ostrich legs with short thighs, however, since Cassie’s joints are actuated by electric motors, they are each limited to one degree of freedom.
To create multiple degrees of freedom, multiple joints have to be added in series and an offset is required between them for mechanical reasons.
This is a major difference between ball joints allowed by musculoskeletal bodies and traditional robots.
Furthermore, Cassie couples the knee and ankle joints together.
However, the similarity with the ostrich morphology seemed sufﬁcient to try applying the mocap tracking to this robot.
This provides an interesting opportunity to compare muscle and torque actuation with two different models of real world bodies with roughly similar morphologies.
We took the MuJoCo model of Cassie, provided by the Oregon State University Dynamic Robotics
Laboratory 4 .
The model had to be tuned to increase its stability and some constraints to the model had to be added to ensure that parts were correctly attached.
These constraints were equality constraints to ensure that the rods on the back of the legs are always in contact.
To satisfy these constraints we used MuJoCo’s Jacobian function when performing the gradient descent (Buss, 2004).
We then used the previously described mocap generation pipeline to obtain robot poses that corresponded to the mocap clips, excluding the neck.
Surprisingly, the morphology of Cassie is sufﬁciently close to the one of an ostrich to allow the clips to be tracked accurately.
This provides a set of interesting behaviors feasible with the Cassie robot, including more natural gaits than the one usually found in the literature and demos using this robot.
Conclusion
We presented a novel musculoskeletal ostrich model that we hope will be useful to researchers in the ﬁelds of reinforcement learning, neuroscience, biomechanics and computer graphics.
There are still some future questions we can answer using our ostrich model.
Firstly, we did not explore mechanical or metabolic efﬁciency.
Animals including humans are very power-efﬁcient when solving certain tasks, yet in our reward signal we did not have a penalty term for muscle usage and it is unclear how to insert it into the equation.
The best approach might be to model directly fatigue (Potvin &
Fuglevand, 2017) in the muscle simulation so that through time the muscle force output will decrease.
Furthermore, obtaining satisfactory results without using motion capture would be desirable to obtain movement predictions produced in a broader set of situations such as terrain adaptation.
Acknowledgments and Disclosure of Funding
The research presented in this paper has been supported by the Royal Veterinary College and Dyson
Technology Ltd.
A Muscle dynamics
Difference between excitation and activation In biomechanics and neuroscience, researchers separate the excitation and activation signals.
Neural excitation by the nervous system can be described as the discharge sequence of the efferent ﬁbres.
Neural excitation u is coupled with contraction via an intermediate state called activation a (Zajac, 1989).
This intermediate state converts an electrochemical signal to mechanical force output.
In MuJoCo, this conversion is modelled as a ﬁrst-order nonlinear ﬁlter whose input is the control signal:
[Equation]
where τ is deﬁned as:
[Equation]
τ a , τ d are time constants for activation and deactivation, equal by default to 10 ms and 40 ms , respectively.
Contraction dynamics
The contraction dynamics are implemented using the Hill-type model as shown in ﬁgure 8.
This mechanical model computes the total force generated by a muscle by adding together the active contractile and passive properties of muscles.
The contractile element models the muscle contraction force scaled by the activation signal a.
The passive properties are modeled like springs: if we stretch a muscle it will generate a passive force to go back to its rest position.
The active contraction dynamics can be summarized by the formula:
[Equation]
The functions f l , f p and f v are plotted in ﬁgure 8.
These functions are built into MuJoCo, capturing the contraction dynamics for basic purposes, and have been validated by the biomechanics community for some conditions (Millard et al., 2013).
All muscle-related quantities in MuJoCo are scaled by the muscle-tendon actuator’s resting length; the advantage of this representation is that all muscles behave similarly.
MuJoCo does not allow speciﬁcation of the muscle resting length L 0 and tendon slack length LT directly (in contrast to OpenSim); this is due to the fact that MuJoCo does not include a stateful elastic component in the muscle model.
This is a major compromise in MuJoCo when compared to OpenSim, that allows it to run faster.
Instead, the actuator length range LR needs to be speciﬁed, which is the minimum and maximum of the sum of muscle and tendon lengths, along with a range R in units of L 0 .
In other words, the actuator length range LR deﬁnes the interval of values that the actuator is allowed to use during the simulation and the range R is a percentage expressing how much the actuator (here simply called “muscle”) can shorten or extend.
At model compile time, MuJoCo solves these two equations where the unknowns are the muscle rest length L 0 and tendon length LT : ( [Equation]
[Equation]
R max
B Task details
Here we describe the various components of the tasks.
[Table]
[Table]
similarity
[Table]
reached
[Table]
[Table]
C Hyperparameters
Here we describe the hyperparameters used in the experiments.
[Table] [Table]
D Motion capture data completion
Here we provide a visualization of the amount of missing data in the original mocap and representation of the model we used to predict the missing data.