Variational Autoencoder with Differentiable Physics Engine for Human Gait Analysis and Synthesis
Abstract
We address the task of learning generative models of human gait.
As gait motion always follows the physical laws, a generative model should also produce outputs that comply with the physical laws, particularly rigid body dynamics with contact and friction.
We propose a deep generative model combined with a differentiable physics engine, which outputs physically plausible signals by construction.
The proposed model is also equipped with a policy network conditioned on each sample.
We show an example of the application of such a model to style transfer of gait
Introduction
Analysis and synthesis of human gait are prevalent issues in biomechanics [see, e.g., 40].
In this work, we aim to address them via learning generative models of human gait motion.
Generative models of gait can help us to analyze gait patterns by examining inferred quantities such as latent representations, as well as to synthesize gait patterns with desired properties.
They are useful, for example, in clinical decision making for treatment of pathological gait.
One of the challenges in learning generative model of gait is to ensure the physical validity of model’s outputs.
As motion of gait must always follow the physical laws, a generative model should also yield motions that comply with the physical laws.
However, purely data-driven generative models (e.g., ones only with deep neural networks) can often produce physically impossible or implausible motion patterns.
A quick remedy is to impose regularization that penalizes violation of the physical laws, but it does not guarantee the compliance of the physical laws outside the training data regime.
We suggest learning deep generative models built from a physics simulator as well as neural networks (see ﬁgure 1), so that the outputs comply by construction with the physical laws encoded in the simulator.
More speciﬁcally, we incorporate a differentiable simulator of articulated rigid body dynamics [see, e.g., 7, 5, 9, 10, 12, 14, 17, 27, 11, 28, 39] into the framework of variational autoencoders (VAEs) [22, 29].
We also consider a policy network for controlling the agent in the physics simulator and condition it with the latent representation to provide a sample-dependent control law.
We present an application of such hybrid generative models to style transfer of human gait.
Related work Combination of differentiable physics engines and machine learning models such as neural networks have been actively studied recently.
Many studies have been done in the context of prediction or system identiﬁcation [e.g., 7, 31, 19, 25, 8, 20, 32, 33, 37, 38, 6, 21, 41] and control or reinforcement learning [e.g., 7, 31, 16, 15, 34, 6, 18, 42].
Some researchers have investigated generative modeling or related methodologies combined with differentiable physics engines [7, 14, 17, 2, 35].
For example, de A. Belbute-Peres et al.
[7] suggested an autoencoding architecture with a differentiable physics simulator inside.
Takeishi and Kalousis (2021) [35] proposed a method to strike a balance between physics models and data-driven models in learning VAEs combined with physics models.
Our work is on this track of research but is more focused on gait modeling.
[Figure]
[Figure]
Our task combines features of related tasks (see table 1).
In human pose tracking (i.e., pose estimation from measurements such as videos and motion capture data)
[e.g., 3], consistency with the physical laws is often considered via inverse kinematics (IK), inverse dynamics (ID), and/or physics-informed regularization.
Reinforcement learning (RL) is at the intersection of machine learning, control, and physics simulation and has been applied to human motion [e.g., 23].
One of the strengths of VAE and its variants, which have been applied to gait as well [e.g., 4], is the capability of amortized inference, with which latent variables for new observations can be inferred quickly.
Our method is also notable in the sense that training of the model including a policy network is done with a single gradient descent loop only with the evidence lower bound (and some regularizers) as objective.
VAE with physics engine for gait 2.1 Target measurements
In this work, we deal with time-series of the three-dimensional position of markers attached to a subject, which can be measured by motion capture systems.
If there are m markers, each sequence is 3 m -dimensional multivariate time-series.
We suppose we have a collection of such sequences as data and would like to learn a generative model from them.
Simultaneous treatment of other signals that are often available in gait analysis, such as ground reaction force and electromyography measurements, is an extension to be addressed in the future.
2.2 Model architecture
The architecture of the proposed method, depicted in ﬁgure 1, follows the autoencoding structure.
Given a marker position sequence as input data x, encoder networks compute latent variable z. The decoder comprises a differentiable physics engine for rigid body dynamics simulation.
An agent in the physics engine is controlled with a neural network (i.e., policy network) conditioned on the latent variable z.
The physics engine should return a simulated marker position sequence, denoted by p in ﬁgure 1.
Finally, we match the scale of p with that of x to give the ﬁnal reconstruction ˆ x.
Let us formalize the idea.
Let x ∈ R (cid:96) × 3 m be an input marker position sequence of length (cid:96) .
The encoder networks, µ encoder : R (cid:96) × 3 m → R d z and σ 2encoder : R (cid:96) × 3 m → R > 0 , give the sufﬁcient statistics of the approximated posterior of the latent variable z ∈ R d z , that is,
[Equation]
Latent variable z is subsequently utilized in two ways.
Firstly, we compute from z the initial condition s 0 ∈ R d s fed into the physics simulator (because x is marker position while s include joint angles):
[Equation]
Secondly, we use z as an additional argument of the controller of the simulator’s agent, that is,
[Equation]
where s t ∈ R d s and u t ∈ R
d u are the simulator’s state and input signal (i.e., action) at time t , respectively.
We use the technique called positional encoding to vary z slightly at each time t .
Given such a conditioned controller, the physics engine runs the simulation of rigid body dynamics.
The core computation of the physics engine is the temporal transition of the state variable s t given action u t following the rigid body dynamics.
Namely, for each of t = 0 , . . .
, (cid:96) − 1 , it computes
[Equation]
The state variable s t comprises the generalized position and velocity of the agent, an articulated rigid body.
More speciﬁcally, in this work, it comprises the position, velocity, orientation, and angular velocity of the ﬂoating base of the agent, as well as the angle and angular velocity of the joints of the agent’s articulated body.
Consequently, the action u t is generalized force applied to each joint (i.e., torque).
We used a 21-degree-of-freedom model of the human musculoskeletal system as the agent, so
d s = 21 × 2 and d u = 21 − 6 as we do not directly control the ﬂoating base, which 6 dofs.
The physics engine should ﬁnally output a simulated position sequence of the markers attached to the agent.
We denote such a simulated sequence by p ∈ R (cid:96) × 3 m .
At the ﬁnal stage of the decoding process, we rescale p because the scale of the simulator’s agent is ﬁxed 1 , while the data may comprise subjects with different scales.
To this end, we compute the optimal scaling factors between p and x using differentiable convex optimization [1].
Let p t,i,j be the element of p corresponding to the position of the j -th marker along the i -axis at timestep t , for t = 1 , . . .
, (cid:96) , i =
X, Y, Z , and j = 1 , . . .
, m .
Let ˆ x ∈ R (cid:96) × 3 m be the ﬁnal reconstruction after scaling, and let ˆ x t,i,j be the element of ˆ x analogously to the case of p t,i,j .
Furthermore, let p t,i, fb denote the element of p corresponding to the ﬂoating base position along the i -axis at timestep t .
With these notions, the ﬁnal reconstruction ˆ x is given by the following linear scaling of p
[Equation]
where φ := { α i , β i , γ i } is the set of scaling factors.
They are computed via the following problem: min α
i
[Equation]
i
[Equation]
for i =
X, Y, Z .
Note that this convex optimization layer takes the original input x unlike ordinary autoencoder structures.
It is not problematic because it needs x only in training, and in a test phase, we can use arbitrary scaling factors for generating ˆ x (e.g., ones computed with some reference datapoint).
2.3 Learning
Learning is done by maximizing the evidence lower bound (ELBO) of the marginal log likelihood [see, e.g., 22], that is, E z ∼ p encoder [log p decoder ( x | z )] − D KL ( p encoder ( z ) (cid:107) p prior ( z )) , where p encoder is the distribution in equation 1, p decoder is a distribution such that the ﬁrst moment is given by equation 5, and p prior is some prior distribution of z. To reduce the variance, we use the path derivative ELBO [30].
Note that the policy network is also trained within this scheme altogether.
In addition to the evidence lower bound, we take several regularization terms into account.
First, we penalize the magnitude of the power by the agent’s action, i.e., the product of torque and angular velocity.
We also penalize the ﬁrst-order and second-order differences (along time) of u t to prevent implausible torque sequences.
In the case of conditional modeling, which will be introduced later in section 2.4, we consider independence-enforcing regularization for disentanglement of the latent variable.
We empirically found that when training the proposed model, it was essential to start training from short sequences and then feed longer sequences gradually.
2.4 Incorporating conditional variable for style transfer
We present an extension of the model in section 2.2.
We focus on the application of style transfer; given some x and some features c describing a “style” of the gait, we would like to generate new gait motion x (cid:48) having altered gait style c (cid:48) (cid:54) =
c .
To this end, we incorporate a conditional variable c into the model.
It particularly appears in the encoder part of the model; instead of computing z directly from x, we ﬁrst compute some intermediate quantity y ∈ R d y from x and c and then compute z from y and c (cid:48) , where c (cid:48) has the same value with c in training but has an arbitrary value in test.
During training, equation 1 is to be replaced by
[Equation]
where g , ˜ µ encoder , and ˜ σ 2encoder are neural networks.
In this two-stage computation, the intermediate variable y should capture the part of the information of x that is not described by c. Then, z is again encoded with c so that z retains all the information from x and c.
In other words, y should be a representation of x disentangled from c. Such semantics of y and z are not automatically obtained by simply maximizing the evidence lower bound.
We ensure the independence between c and y and the dependence between z and c by imposing regularization on the Hilbert–Schmidt independence criterion (HSIC) [13].
It is known that HSIC between two random variables becomes zero if and only if the variables are statistically independent [13], and when the variables are dependent, HSIC takes a positive value.
We minimize the following quantity as regularizer: λ 1 (cid:92) HSIC( c , y ) − λ 2 (cid:92) HSIC( c , z ) , where λ 1 and λ 2 are hyperparameters, and (cid:92) HSIC means an empirical estimation of HSIC.
We used the Gaussian kernel with width determined by the median trick.
We note that regularization of VAEs with HSIC was also studied in [26, 36].
[Figure]
In a test phase, we perform another branch of computation from
[Equation]
where c (cid:48) may have a value different from that of c. If y is successfully disentangled from c, this new z (cid:48) informed by c (cid:48) should attain information of c (cid:48)
(and not of c), which would enable style transfer of x into some ˆ x (cid:48) having the property of c (cid:48) .
Here, we assume that the condition variable c (or c (cid:48) ) does not contain the scale of the subject.
Such an assumption enables us to use the scaling factor φ computed with the original c even for ˆ x (cid:48) .
In ﬁgure 2, we show the computation ﬂow of the model with the original or altered conditional variable.
During training, only the upper part of ﬁgure 2 with the original c is run.
The lower part of ﬁgure 2 with the altered c (cid:48) works in applying style transfer.
Preliminary experiment 3.1 Conﬁguration
Dataset We used a public dataset of human locomotion [24].
We divided the 50 subjects of the dataset into training, validation, and test sets.
From the original dataset, we extracted the data of marker position measurements during walking and used them as x. We had n = 328 , 34 , andsequences for the training, validation, and test sets, respectively.
Each sequence contains one gait cycle (i.e., from a heel strike to the next heel strike).
We aligned the length of all the sequences to be (cid:96) =
500 with cubic interpolation.
We also used the information of the gait cadence as the conditional variable c, which varied from 40 to 170 [step/min] within the dataset.
Model As a differentiable physics engine in the proposed model, we used nimblephysics [39] library.
Other parts of the model were neural networks.
g comprised a multilayer perceptron (MLP) for computing features from x and c, a self-attention layer, and an average-pooling (along time) layer. ˜ µ encoder , f initializer , and f controller were MLPs.
3.2 Result
Figures 3a–3c show an example of the reconstruction by the proposed method, and ﬁgure 3d shows the corresponding motion of the agent in the physics simulator.
The result on a test sample without style transfer is displayed.
It successfully mimics gait motion.
For the whole test set, the average reconstruction root-mean-square error was 1 .
94 ± 0 .
36 [cm].
The error is relatively large for the later part of the sequence.
We again emphasize that the inferred motion, such as one in ﬁgure 3, inherently complies with the physical laws up to the ﬁdelity of the physics simulator.
We show an example of the style transfer, where we tried to change the cadence of gait.
We randomly picked 20 test samples and performed style transfer with the method presented in section 2.4. The
[Figure] [Figure] value of c (cid:48) is randomly drawn from a slow cadence range ( 40 ≤ c (cid:48) ≤ 66 [step/min]) or a fast cadence range ( 144 ≤ c (cid:48) ≤ 170 [step/min]).
Figure 4a shows the right hip ﬂexion angle computed by the simulator with the altered c (cid:48) given to the model.
In ﬁgure 4b, for comparison, we show the angle of the same joint inferred with inverse kinematics from some training samples with similar ranges of c. Note that such angles computed with inverse kinematics are never used as input to the proposed model.
Comparing the two plots of ﬁgure 4, we can ﬁnd that the generated signals successfully mimic the tendency that the minimum value of the angle comes at earlier gait cycle in fast gait.
Meanwhile, the generated signal show relatively small variability (especially at initial condition) partly because the test data contain a much smaller number of subjects than that of training data.
Conclusion
We proposed a deep generative model with a differentiable physics engine for modeling human gait, which can produce physically-consistent signals by construction.
We presented an example of the application to style transfer of gait cadence.
The presented work is preliminary, and we are working on a number of extensions, model variants, and applications. They include the use of some muscle models, style transfer based on physical / biological models instead of unstable disentanglement, and learning on large scale data of both healthy and pathological gait. Acknowledgments and Disclosure of Funding This work was supported by the Swiss National Science Foundation Sinergia project Modeling pathological gait resulting from motor impairments (CRSII5_177179).