Learning Physics Constrained Dynamics Using Autoencoders
Abstract
We consider the problem of estimating states ( e.g., position and velocity) and physical parameters ( e.g., friction, elasticity) from a sequence of observations when provided a dynamic equation that describes the behavior of the system.
The dynamic equation can arise from first principles ( e.g., Newton’s laws) and provide useful cues for learning, but its physical parameters are unknown.
To address this problem, we propose a model that estimates states and physical parameters of the system using two main components.
First, an autoencoder compresses a sequence of observations ( e.g., sensor measurements, pixel images) into a sequence for the state representation that is consistent with physics by including a simulation of the dynamic equation.
Second, an estimator is coupled with the autoencoder to predict the values of the physical parameters.
We also theoretically and empirically show that using Fourier feature mappings improves the generalization of the estimator in predicting physical parameters compared to raw state sequences when learning from high-frequency data.
In our experiments on three visual and one sensor measurement tasks, our model imposes interpretability on latent states and achieves improved generalization performance for long-term prediction of system dynamics over state-of-the-art baselines.
Introduction
Neural networks have become a core computational component in domains such as computer vision [1], natural language processing [2] and deep reinforcement learning [3].
Recent work has shown that neural networks can exhibit an inductive bias that is often introduced via designing specific structures [4].
This bias can be used to encode prior task knowledge that helps the network generalize to unseen data.
For example, convolution neural networks capture the translation invariance of key image features.
In this spirit, we develop a structured neural network model that leverages a dynamic equation to estimate both the state of a dynamical system ( e.g., position and velocity) and its physical parameters ( e.g., friction constants) from a sequence of partial observations.
Knowing the state and parameters of the system aids learning a control policy and tracking parameter value changes over time.
For instance, for a self-driving car, we want to learn a neural network that estimates the vehicle’s position and velocity from a sequence of egocentric camera images.
The estimated state can then be used in a control policy.
In addition, we want to track physical parameters over time, e.g., friction coefficients.
This is useful for vehicle maintenance and safety.
We expect that exploiting a neural network, regularized to follow a dynamic equation, will streamline the required data-intensive operations and yield improved performance.
Observations can be direct measurements of some system variables ( e.g., accelerations), or take a more complex form ( e.g., images).
We group such observations over specified time windows and also refer to these groupings as observations.
We obtain a compact representation for these observations that permits observation reconstruction using an autoencoder (see Fig. 1(a)).
{ h s } in Fig.
1(a) is a representation of an observation sequence { o s } over a specified time window, and { ˆ o s } is the reconstructed observation sequence.
Learning the autoencoder is both data and computationally intensive.
In addition, h s may not be physically interpretable and be a “system state.”
Motivated by these observations, we assume a simulator is provided that specifies the physical laws of the system.
This model can arise from first principles ( e.g., Newton’s laws), but its free parameters θ ( e.g., masses, lengths) remain to be specified.
Given an initial state and θ , the physics simulator generates a state trajectory { ˆ x s } consistent with the laws of physics.
To leverage this model, we require an estimator f ( · ) that maps a sequence of states { ˜ x s } to an estimate of θ .
We then couple the estimator f and the physics simulator with the autoencoder as shown in Fig. 1(b).
We train the autoencoder and f ( · ) to minimize the observation reconstruction loss (cid:80) s ∥
o s − ˆ o s ∥ 22 .
Within this process, we train the encoder h ( · ) to minimize the sum of squared state errors: (cid:80) s ∥
˜ x s − ˆ x s ∥ 22 .
The complete model (Fig. 1(b)) is called Autoencoder with Latent Physics (ALPS).
The paper’s contributions are three-fold.
(1) ALPS is the method that learns to identify the system parameters and the mappings between states and observations from data.
In contrast, conventional system identification ( e.g., tools in Matlab) requires knowing the function mapping and identifying the parameters online, which is costly to run.
(2) We show that one can learn periodic or vibrational behavior in this setting using a Fourier feature of states.
(3) We evaluate ALPS in three simulated and one real-world train dynamics.
ALPS can achieve up to 4.8x and 6.3x better physical parameter and state prediction accuracy, respectively, over prior approaches.
Related Work
Physics-informed neural networks.
There is growing interest in including a physics prior or algebraic and logical constraints into neural networks [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29].
For example, [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40] exploit Lagrangian or Hamiltonian mechanics to learn an energy-conserving system based on position, momentum, and the derivatives thereof along trajectories.
These works assume the physical parameters of the system are constant and need not be estimated.
In contrast, we estimate the physical parameters.
This is important for fault detection and localization, and for safety.
Papers [41, 42, 43, 44] learn a general physical simulation from data, but their model is required to have state information or a known forward rendering engine to map states to observations.
In contrast, we learn a physics-based autoencoder to estimate states in an unsupervised manner.
Paper [45] also uses an autoencoder with physics to predict parameters.
However, we show that learning state sequences as in [45] fails to generalize to unseen parameters when the system exhibits high-frequency behavior. Koopman-inspired neural networks.
Among many physics-informed neural networks developed in the past years, the recent effect has also considered applying deep learning in learning Koopman operators.
We highlight several papers that are similar to our approach here.
Koopman operators use an embedding to describe non-linear systems in a linear form. They are useful in analyzing the system dynamics but require domain knowledge to find the embedding. To overcome this challenge, the work [46, 47] use an autoencoder to identify Koopman eigenfunctions. In addition, the work [48, 49] also use an autoencoder for Koopman spectral analysis by learning Koopman invariant subspaces from data.
The use of autoencoder structure is similar to our approach, but we explicitly use a physics simulator for constraining the representation of the autoencoder.
Furthermore, the work [50] propose a method that optimizes neural networks by Koopman theory.
Fourier features for high-frequency data. [51, 52] show that using Fourier features helps neural networks learn high-frequency content in image regression tasks.
This approach requires specifying the Fourier series coefficients and the basis frequencies.
In contrast, the Fourier features in our model are computed from the states, which allows us to predict physical parameters.
In addition, the concurrent work [53] replaces the self-attention sublayers [54] with a Fourier transformation of the input word token in natural language processing.
They show that this method is sufficient to capture semantic relationships in several text classification tasks.
In this paper, we provide a theoretical justification for using Fourier features to learn system dynamics with high-frequency data. System identification.
The proposed approach uses the neural network to identify the system parameters from data, which is similar to conventional system identification methods such as linear grey-box model estimation (greyest) tool provided in Matlab [55].
However, ALPS does not require the mapping between the states and the observations, whereas the greyest method needs to provide full system dynamics.
This offers an advantage when the observation space is in high dimension. In addition, it is possible to use Matlab model compilation to speed up the estimation of system parameters, which is similar to the analogy of training ALPS first and then deploying it during test time. However, even with Matlab model compilation, ALPS is still different since ALPS uses a neural network to identify the system parameters.  Problem Formulation
We consider the continuous-time system
[Equation]
where A ∈ R n × n , B ∈ R n × p and x ∈ R n , u ∈ R p , o ∈ R q , denote the state, input, and observation, respectively.
The function g provides a partial observation o of x .
In most situations we have partial knowledge of A and B from physics.
This is true and reasonable in many large-scale industrial applications ranging from wind turbines to aircraft, where system designers use physics knowledge to design machines.
Hence we assume mappings A ( · ) :
Θ → R n × n and B ( · ) :
Θ → R n × p , from physical parameters θ ∈ Θ to the system matrices A ( θ ) , B ( θ ) , are given ( i.e., A and B obtain a specific sparse and parameteric form).
In addition, Eq.
(1) assumes the system dynamics to be linear, but our approach can be extended to other non-linear cases.
In practice, we only have observations at discrete points in time.
For simplicity, we assume these are equally spaced at times t = 0 , 1 , . . . .
At sample time t, we have the window of observations { o s } ts = t − τ +1 , where τ is the window length.
We assume the sampling rate satisfies the Nyquist rate.
Our problem can now be stated as follows.
Given functions A ( · ) :
Θ → R n × n , B ( · ) :
Θ → R n × p , we seek to learn a network that estimates a state sequence { x s } ts = t − τ +1 , physical parameters θ , and a mapping g ( · ) from a finite sequence of past observations { o s } ts = t − τ +1 and past known inputs { u s } ts = t − τ +1 .
Once trained, the network can predict states and has learned to adapt physical parameters to the context without re-training.
Note that even though the network operates with time sampled variables, the physics simulator can be used to predict states at any time.
This problem setup is distinct from that of HNN [33], HGN [36], or Symplectic ODE-Net [35].
In their setups, the model is only trained on data from a single physical parameter θ , and hence they need to re-train networks for every new set of physical parameters.
In addition, compared to conventional system identification approaches, we do not need to rerun the solver each time–we only need a forward pass of the network to get the estimation during deployment (testing time).
To make the learning problem well-defined, i.e., to ensure that we can uniquely identify the unknown system parameters, we make the following assumption.
Assumption 3.1.
(Identifiability [56]) :
For a sufficiently large τ, the sequence of states and inputs { ( x s , u s ) } ts = t − τ +1 uniquely specifies the unknown parameter θ .
Parameter identifiability is a well-studied problem in system identification [56, 57, 58, 59, 60, 61].
For example, the (continuous time) system ˙ x = (cid:20) − ( a + b ) b b − c (cid:21) x
+ (cid:20) 10 (cid:21) u with o =
[1 0] x , where θ =
[ a, b, c ] T and g ( · ) is a linear function, satisfies Assumption 3.1 when b ̸ = 0 .
Nework Architecture
The network in Fig. 1(b) is expanded in Fig.
2 into its four parts: an encoder network, a parameter estimator network, a physics simulator, and a decoder network.
The encoder network h ( · ) (from { o s } to { ˜ x s } ).
We consider two types of observations: (1) pixel images, or (2) direct measurements of some system variables.
For the first case, we use a convolution neural network to compress a pixel observation o into a compact vector embedding z ′ ∈ R d , which preserves image features.
For the second case, we use a feedforward network to project an observation o into some higher dimensional space with a vector embedding z ′ .
In addition, to estimating states from these vector embeddings, we need to aggregate { z ′ s } to extract the local and global context of the dynamics.
One approach is to use recurrent neural networks (RNN) ( e.g., Dreamer [62]).
However,
RNNs suffer from vanishing gradient problems and slow computation when processing long-term sequences.
Hence we use a self-attention network [54] to attend to z ′ of greatest importance to predict states and improve efficiency.
Furthermore, to inject a position signal of observations in the sequence, we add a positional encoding p ∈ R d to z ′
( z :=
z ′ + p ), where p are sine and cosine functions of different frequencies (see [54]).
Finally, we stack embeddings over τ steps to form a matrix Z ∈ R τ × d .
The self-attention module can be formulated as querying a dictionary with key-value pairs associated with learnable weight matrices W Q ∈ R d × d Q , W K ∈ R d × d K , and
W V ∈ R d × d V ( d Q = d K here):
Attention ( Q , K , V ) =
softmax ( QK T √ d ) V , where Q :=
ZW Q , K :=
ZW K , V :=
ZW V , and the softmax is taken over the sequence length τ.
To provide multiview of the embedding, we use the multihead variant of the attention by concatenating each attention head along the sequence axis Multihead := Concat ( head 1 , . .
, head i , . . , head I ) W O , head i := Attention ( Q , K , V ) , where W O ∈ R I
d V × d are learnable matrices, I is the number of heads, and each attention head i has its own learnable weight matrices W Qi ∈ R d × d Q , W Ki ∈ R d × d K , and W Vi ∈ R d × d V .
Finally, a feedforward network takes in the Multihead embedding (of size R τ × d ) and produces the parameters of the distribution for each state in the sequence.
The parameters are used to define a posterior distribution over the encoded state ˜ x s ∼ Q ( ·| ˜ o s ) with the prior P (˜ x s ) .
For a translational coordinate, the posterior distribution is a Gaussian distribution with a unit Gaussian prior.
Hence the network predicts a mean µ ∈ R n and a standard deviation σ ∈ R n of a Gaussian distribution.
In addition, for a rotational coordinate, the posterior distribution is a von Mises (vM) distribution with a unit vM prior.
Similar to a Gaussian distribution, a vM distribution is defined by two parameters: a mean µ ∈ R 2 , ∥ µ ∥ 2 = 1 (the angular position (cos φ, sin φ ) ), and a concentration κ ∈ R + around µ .
Such parameterization is found useful in practice, e.g., [63].
Appendix E provides details about self-attention networks for estimating states.
The parameter estimator network f ( · ) (from { ˜ x s } to θ ).
The parameter estimator network predicts physical parameters from state sequences { ˜ x s } .
However, for systems that involve periodic or vibrational behavior, prior work [51] has shown that neural networks fail to capture the highfrequency content in the data.
To improve generalization and reduce the effect of noise, we do a Fourier transform on each component j of state trajectories { ˜ x s ( j ) } ts = t − τ +1 to get { ˜ X ω ( j ) } tω = t − τ +1 :
˜ X ω ( j ) :
= (cid:80) tk = t − τ +
˜ x k
( j ) (cid:104)
cos (cid:16) 2 πτ ωk (cid:17) − i · sin (cid:16) 2 πτ ωk (cid:17)(cid:105) .
Note that for the systems that do not have periodic or vibrational behavior, using { ˜ x s ( j ) } ts = t − τ +1 would be enough.
One may use { ˜ X ω ( j ) } as features for the parameter estimator network to predict physical parameters.
However in Section 5 we will show that by using the neural tangent kernel (NTK) theory [64], which treats neural networks as a kernel regression, the resulting kernel matrix of { ˜ X ω ( j ) } does not preserve high-frequency components in the data.
To solve this, we will show that using the magnitude of the
Fourier features {| ˜ X ω ( j ) |} alleviates the issue.
Hence the parameter estimator network takes in a concatenation of {| ˜ X ω ( j ) |} from each component of the state and predicts physical parameters θ .
The physics simulator (from ˜ x t − τ +
and θ to { ˆ x s } ).
Given a start state ˜ x t − τ +1 (from the encoder’s first state prediction) and values for the physical parameters θ , we use the neural ordinary differential equation (ODE) [65], a differential ODE solver, to generate a simulated state trajectory { ˆ x s } ts = t − τ +1 : ˆ x t − τ +1 , . . .
ˆ x t − 1 , ˆ x t =
ODESolver(˜ x t − τ +1 , ˙ x
= A ( θ ) x + B ( θ ) u , τ, ∆) , where
ODESolver takes in a start state, an ODE, a window length, and a sampling time interval ∆ .
Note that ˆ x t − τ [-+
-]+1 =
˜ x
t − τ +1 .
In addition, using an ODE solver allows us to generate an accurate state trajectory compared to that of RNN as in [62].
The decoder network g ( · ) (from ˆ x s to ˆ o s ).
Finally, the decoder network is either a deconvolutional network (for image observations) or a feedforward network (for sensor measurements) that takes in each individual ODE-simulated state ˆ x s and generates a reconstructed observation ˆ o s . Discussion. Note that we can replace the decoder g with a differentiable rendering engine. Prior work [44] use a differentiable rendering engine to reconstruct the scene given the estimation of the system parameters and the states.
However, using a differentiable rendering engine may introduce a simulation overhead, and using a network here is more generalizable.
We think the future extension of ALPS to differentiable rendering engines is a valuable future research direction.
In addition, one may think that the estimator network can directly predict the system parameters given the input observation without taking the states. The reason to predict the states is that for some applications, it may be useful to know the state of the system. For example, for a self-driving car, we would like to know the speed of other vehicles by using observations from cameras. Knowing the speed of other vehicles ( i.e., , the state) allows the self-driving car to plan for a trajectory, which is important for the safe deployment of the system.
In addition, we would like to increase the interpretability of the model.
The inclusion of the state allows the system designer to ensure the representation learned by neural networks is informative.
The loss function. Given a sequence of τ observations, we minimize the following loss function:
[Equation]
The variational autoencoder (VAE) [66] loss is a variational bound on the marginal log-likelihood of the data.
It is used to train the encoder h, the estimator f, and the decoder g .
Using VAEs avoids learning degenerated solutions and provides stable training of the network over a deterministic network.
In addition, the observation reconstruction loss encourages reconstructed observations { ˆ o s } to match true observations { o s } , and the state reconstruction loss constrains encoded states { ˜ x s } to follow simulated states { ˆ x s } generated by physics.
The former is used to train h, f, and g, and the latter is used to train f and h. Both observation and state reconstruction losses are important for training.
We find that removing the state reconstruction term reduces the state prediction performance of the encoder since the network can predict arbitrary sequences.
In addition, removing the observation reconstruction term impedes the image reconstruction quality, which is vital for training the whole model.
In practice, we tune the weight for each loss term to accommodate different scales.
Note that for the larger loss term, we use a weight smaller than 1 to ensure learning stability.
Fourier Feature Mappings for Learning Periodic System Dynamics
An important feature of ALPS is that it uses a Fourier feature mapping of states to learn periodic system dynamics.
Prior works [67, 68] have empirically shown that using a Fourier feature can help with high frequency signals.
To lay the foundation for the analysis and justify the use of Fourier features, in Section 5.1 we first review the recent work that uses NTK theory [64, 51].
This allows us to control the training of our parameter estimator network as fully-connected networks.
Then in
Section 5.2 we use these tools to analyze the effects of using Fourier features, their magnitudes, and their phases when predicting system parameters that involve the periodic behavior .
5.1 Deep networks as a kernel regression
Consider a set of labelled training data { ( v i , y i ) } mi with v i ∈ R n , y i ∈ R , and i ∈ [1 : m ] .
Set y =
[ y 1 , . . .
, y m ] T ∈ R m .
Now bring in a feature map ϕ : R n → R r with kernel k ( v i , v j )
=
ϕ ( v i ) T ϕ ( v j ) .
Let K =
[ k ( v i , v j )]
∈ R m × m denote the kernel matrix for the training examples and k ( v ) =
[ k ( v i , v )]
∈ R m denote the vector of kernel evaluations k ( v i , v ) , i ∈ [1 , m ] , for a test sample v ∈ R n .
The resulting kernel regression predictor is ˆ y ( v ) =
y T K − 1 k ( v ) .
Now bring the concept of NTK proposed by [64].
The theory in [64] says that when the width of the layers of fully-connected deep networks with weights w initialized from a Gaussian distribution N tends to infinity, and the learning rate for stochastic gradient descent tends to zero, the neural network estimator ˆ y ( v ; w ) converges to the kernel regression solution using NTK.
The NTK is defined as
[Equation]
Under asymptotic conditions, a neural network’s output after t updates can be approximated as
[Equation]
where e M is the n by n matrix M given by the power series
e M = (cid:80) ∞
i =
1 i !
M i with M
= I .
Spectral bias in neural networks.
Now we want to compute the training error of a neural network after t times update.
Let K
= U Σ U T denote the eigendecomposition of the kernel matrix K which must be positive semidefinite (PSD).
Here U is an orthogonal matrix and Σ is a diagonal matrix whose entries are the nonnegative eigenvalues ordered by magnitude: λ 1 ≥ λ 2 ≥ · · · ≥ λ m ≥ 0 .
So the training error in terms of L 2 norm ∥·∥ 22 is (cid:13)(cid:13) ˆ y ( t ) − y (cid:13)(cid:13)
[Equation]
This shows the training convergence will decay exponentially at the rate ηλ i .
Hence the components in y will be learned faster if their corresponding eigenvalue is larger.
In Section 5.2 we will show that for a sequence of states without doing Fourier transform, the resulting NTK will have smaller eigenvalues at the high-frequency components.
This leads to a slower convergence in high-frequency components of data which are essential to identify parameters for the periodic behavior.
5.2 The effect of Fourier feature mapping
To understand the effect of the Fourier feature mapping, we first derive the kernel function of Fourier features, their magnitudes, and their phases.
Then we compare the spatial bias of the kernel matrices from these three kernels.
Kernels of Fourier feature mapping.
Consider a state trajectory v = [ x 0 , x 1 , . . .
, x τ − 1 ] T , where here we consider a 1D case for a scalar state x ∈ R although it can be extended to a vector state x ∈ R n .
The feature maps of the Fourier feature mapping, their magnitudes, and their phases are
[Equation]
where X [Equation] −
[Equation]
sin [Equation] kj [Equation]
Set C
k
= (cid:2) cos [Equation] ki [Equation]
kj
[Equation]
∈ R τ × τ , then the kernel functions of these mappings are [Equation] After mapping the input points into the Fourier features, we feed them into a neural network to obtain ˆ y ( ϕ ( v ); w ) .
Hence for DFT kernel function the resulting composed kernel of the neural network is k NTK ( ϕ DFT ( v 1 ) , ϕ DFT ( v 2 )) .
Similarly, we have the kernels for k MAG and k PHA .
Visualizing the composed NTK.
We generate time series data composed of different frequencies and magnitude of sine waves.
The length of the data is 200 samples, and we set τ = 100 ( i.e., v ∈ R 100 ) with the sampling rate being 100Hz.
We slide a window and hence have 101 instances in total.
Fig.shows the time series data, the effects of each kernel, and its spatial plot.
By construction, k MAG and k PHA have a slower decay in the high-frequency domain as shown in Fig. 3(b).
In addition, k DFT and no mapping have the same kernel matrix and a narrower kernel spatial.
This observation supports the idea of not using X ω when learning the system with periodic or vibrational behavior as | X ω | (magnitude) preserves high-frequency information, which is useful for parameter estimations.
Note that for the system that does not have high-frequency signals, both X ω and | X ω | are equally well.
Comparison to [51].
Our work is inspired by [51], which also uses Fourier feature mappings.
However, there are a few key differences.
(1) Setup.
The Fourier feature mapping in [51] transforms the low-dimensional x-y coordinates into high-dimensional Fourier features, which projects data into a high-dimensional space.
In contrast, we use Fourier feature mapping of raw time series data, which compresses data into more compact representations.
(2) Analysis .
We discuss the difference of
Fourier feature mappings, their magnitudes, and their phases to understand the effect of each mapping for predicting system parameters.
In contrast, [51] does not have such analysis.
Simulations
We study the following questions: (1) How does ALPS perform compared to other baseline methods without the Fourier feature mapping and physics-in-the-loop?
(2) What is the effect of self-attention networks in ALPS?
(3) How does ALPS perform in a full-scale train wheel system dynamics?
Visual dataset.
We generate three visual datasets: pendulum, mass-spring-damper (MSD), and a two-body system to compare the performance of ALPS to that of the baseline in the literature.
These baselines are very commonly used in the prior work.
We first randomly sample an initial state and physical parameters, and then generate a 125 step rollout following the true system dynamics, and render corresponding 64 by 64 by 3 pixel observation snapshots.
The sampling rate is 20Hz in the pendulum, 100Hz in MSD, and 6Hz in two-body systems.
The observation length τ is 100.
In total, we generate 500 training and 500 test trajectories, resulting in 13,000 training and test sequences.
(1) Pendulum.
The dynamics is
[Equation]
, where φ is an angle, ˙ φ is an angular velocity, β is a friction coefficient, G is a gravitational constant, and L is the length of the pendulum.
We fix G = 10 , L = 1 , then sample β from a uniform distribution β ∼ U (0 . 1 , 1) , an initial angle from a uniform distribution φ 0 ∼ U ( − π, + π ) , and an initial angular velocity from a uniform distribution ˙
φ 0 ∼ U (0 . 5 , 4) .
We predict θ =
[ β ] in this task. (2) Mass-spring-damper.
The dynamics of double mass-spring-damper system is
[Equation]
where x (1) , x (3) are the displacement and velocity of the primary spring; x (2) , x (4) are the displacement and velocity of the secondary spring; u (1) , u (2) are the inputs; α (2) , α (4) are the stiffness and damping ratio of the primary spring and damper; α (1) , α (3) are the stiffness and damping ratio of the secondary spring and damper, and m (1) , m (2) are the mass of the primary and secondary spring.
We fix all the parameters and the initial state except for α (2) , which is sampled from a uniform distribution α (2) ∼ U (4 × 10 3 , 4 × 10 6 ) .
The input excitations are sampled from a unit Gaussian distribution u ∼ N (0 , 0 . 05)
× N (0 , 0 . 05) ( i.e., white noise).
We predict θ =
[ α (2)] in this task. (3) Two-body.
In this system two particles interact with each other via an attractive force.
The dynamics is a Hamiltonian H =
∥ p (1) ∥ 22 2 m (1) + ∥ p (2) ∥ 22 2 m (2) + Gm (1) m (2) ∥
q (1) − q (2) ∥ 2 , where p (1) , p (2) are the positions of the particles; q (1) , q (2) are the momentum of the particles; m (1) , m (2) are the masses of the particles, and G is a gravitational constant.
We fix G = 10 , m (1) = m (2) = 1 , then sample H from a uniform distribution H ∼ U (0 . 4 , 1) .
We predict θ =
[ H ] in this task.
Time series dataset.
In addition, we obtain an MSD system dataset with state measurements.
(4) MSD time series data from full-scale train wheel suspension system.
To show the applicability of ALPS, we conduct experiments with the data that represent the situation when sensors are installed in the real train wheel suspension system.
We use full-scale dynamics of the train wheel suspension system with 18 state elements, 12 observations, 16 input excitations, and 24 system parameters.
We want to identify the stiffness and damping ratio of the 12 spring-dampers.
The dynamics are complex due to the interactions of multiple springs and dampers.
In addition, this problem is challenging as the data contain noise.
To align with the real world scenario, we assume that the system parameters of the system slowly change over time.
As a result, the data has the same set of system parameters but with different initial conditions ( e.g., initial vibration speed).
The dataset contains 20 trajectories with 500 steps sampled by 100Hz.
We use a 50-50 split to get the training and test datasets with τ = 100 .
Here we do not use the self-attention and decoder network in ALPS–the estimator takes in state measurements directly.
Baselines.
We select the following baselines that are commonly used in the literature. (1) Context-aware dynamics model (CDM) [69].
CDM is the state-of-the-art method to learn the system dynamics into two stages: it first learns a context vector that captures the local dynamics and then predicts the next state based on the context vector and the current state and input.
CDM does not consider physics prior nor Fourier feature mapping.
This is to show that using physics priors improves performance and benchmark the results. (2) Autoencoder.
We remove the estimator and the physics simulator to benchmark the mismatch between the true state and the latent representation (equal dimension) of the autoencoder in Fig. 1(a) to show the interpretability of ALPS.
(3) ALPS w/o the Fourier feature mapping.
We consider a variant by replacing the Fourier feature mapping with raw encoded state trajectories { ˜ x s } .
This method is similar to [45], which also uses time series data to learn system dynamics. (4) ALPS w/o self-attention networks.
We consider a variant by replacing the self-attention network with a simple MLP to predict the position of the system, followed by a first-order finite-difference estimator to estimate the velocity.
This uses the same approach as in [63] to estimate states. Finally, Pendulum
Mass-Spring-Damper
[Table]
Autoencoder [Table] ALPS achieves competitive performance in predicting physical parameters and states. we ensure that all baselines are comparable in terms of representation power and the number of parameters.
The loss functions used for these baselines are included in Appendix D.
Evaluation metrics.
We use a mean squared error (MSE) between simulated { ˆ x s } and true states { x s } to evaluate the performance: SE :
= 1 N (cid:80) Ni =1 (cid:80) ts = t − τ +
∥
ˆ x s − x s ∥ 22 , where N is the number of test data.
We also compute MSE between reconstructed { ˆ o t } and true observations { o t } : OE :=
N (cid:80) Ni =1 (cid:80) ts = t − τ [-+
-]+1 ∥
ˆ o s − o s ∥ 22 .
Moreover, we compute the absolute value difference between estimated ˆ θ and true parameters θ : PE :
= 1 N (cid:80) Ni {+=
+}
(cid:13)(cid:13)(cid:13) ˆ θ − θ (cid:13)(cid:13)(cid:13) 1 .
Results in the visual tasks.
Fig.
4 shows the results.
Note that we let the size of the latent representation of the autoencoder be the same as the size of the true states.
The reason is that we would like to know its state error when there is no constraint imposed so that we can quantify the difference from the one with imposing physics constraints.
We see that (1) ALPS achieves the best performance in predicting physical parameters in all cases, with at the most 4.8x lower error in the pendulum task.
On the other hand, the Fourier feature mapping does not have much effect in predicting the physical parameter in the two-body system.
This verifies the analysis of the Fourier feature mapping: for the task with a wider frequency spectrum ( e.g., 100Hz in the MSD), the Fourier feature improves the prediction due to higher eigenvalues in the high-frequency content, whereas for the task with only a low-frequency spectrum ( e.g., 6Hz in the two-body), raw state sequences can already capture low-frequency content.
This supports the idea of using Fourier feature mappings for high-frequency data.
(2) ALPS achieves competitive results in predicting the states, with at the most 6.3x lower error in the MSD task.
We further visualize the state prediction results in the pudendum task for ALPS, which shows that ALPS can accurately track the true states.
Without self-attention networks, the network has a substantial SE due to a large error in computing the velocity, as in the two-body task.
The smaller the sampling rate is, the greater the velocity estimation error is.
In addition, the high SE in autoencoder suggests that its latent representation is uninterpretable.
This verifies the idea of using physics to constrain the representation of the autoencoder to estimate states.
In addition, (3) ALPS achieves comparable results in reconstructing observations.
The low error for
CDM in the MSD and two-body tasks is due to degenerated solutions as shown in the Fig.
This implies that using physics stabilizes the training and improves the reconstruction of observations.
(4) Finally, the autoencoder baseline has higher reconstruction loss in some tasks.
This is because that it learns a degenerated solution, producing blur images due to high-frequency movements of objects on the scene.
Fig.
8 in Appendix E provides more qualitative visualizations of reconstructed images.
Specifically, we find that CDM fails to track the state and resulting blur images in the mass-spring-damper system or duplicated objects in the two-body system.
In contrast, ALPS can precisely track the state and reconstruct the image well.
Our remaining simulations explore ALPS’s ability in predicting physical parameters from raw state measurements.
[Figure]
Our method achieves the best performance.
Results in MSD time series data from full-scale train wheel suspension system.
To scale the proposed approach to a more complex system, we apply the approach in a system with a full-scale train wheel suspension system.
The system containsstate elements, 12 observations, 16 input excitations, and 24 system parameters.
Fig.
5 shows the results.
Here ALPS does not use the encoder nor decoder network–the estimator takes in state measurements directly, as the system is fully-observable in the dataset.
Overall we see that (1) ALPS achieves the best performance.
(2) CDM has worse SE since an MLP cannot learn well from data with high-frequency content.
In addition, we find that the average prediction error rate ( i.e., ˆ θ − θθ ) for these 24 parameters is 0.42 % on the complex system.
This also implies that by providing physics to the network, we can improve SE.
These observations show ALPS can robustly identify parameters simultaneously from state measurements, and support the theory of using Fourier features to learn periodic dynamics.
Moreover, this result shows ALPS can be deployed in real prognostic applications for tracking physical parameters to enhance railroad safety.
Our result is significant for system designers to diagnose the system and hence schedule maintenance.
Conclusion
We addressed the problem of predicting states and physical parameters of a system from observations with dynamic equations.
We showed that the latent representation of the autoencoder is uninterpretable.
We then use dynamic equations to constrain the latent representation of the autoencoder to be consistent with the laws of physics.
We analyzed the effect of Fourier features for estimating physical parameters.
The results showed that ALPS achieves competitive performance in the visual tasks and the time series dataset with up to 24 parameter predictions at the same time.
This shows that our method scales to high-dimensional systems and it is useful for real-world applications.
Limitation and future work. Future work could improve ALPS in several ways.
For instance, the understanding of underlying mechanisms in self-attention networks for estimating states from observations can be advanced. One path is to probe the network by using approaches in natural language processing.
In addition, we require to have dynamic equations and assume that the system is linear and exhibits periodic or vibrational behaviors.
These make it challenging to generalize to more complicated settings such as contact dynamics. One solution is to combine known physics with neural models ( e.g., interaction networks [13]) to compensate for modeling error, and use time-domain and frequency-domain features together to predict physical parameters.
Moreover, it would be interesting to explore other more complex tasks ( e.g., non-linear systems, fluid dynamics, molecular simulation).
Another limitation of ALPS is the overhead caused by simulating trajectories. For the training of ALPS, the computation cost mostly comes from running the differential solver.
For example, under the same batch size of 10 and one gradient update, on average ALPS requires 498 seconds whereas the autoencoder requires 317 seconds, which is 57% slower.
The training time for CDM is also similar to the autoencoder.
A method to overcome this will be valuable.
In addition to the challenge of computation cost, the further investigation of the full system with higher dimensional system parameters ( e.g., large-scale machines with a hundred system parameters) and noise observation is an important future research direction.
Finally, incorporating the ideas from the literature on the system identification [59, 60, 61] to ALPS will further enhance the performance of ALPS. Acknowledgments and Disclosure of Funding
The authors would like to thank the anonymous reviewers and the area chair for their insightful and helpful comments and feedback.
Tsung-Yen Yang thanks Siemens Corporation, Corporate
Technology for their support.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] We use an autoencoder to estimate states from observations with physics-in-the loop, and show that using the Fourier feature can improve generalization.
Our experiments support this idea.
(b) Did you describe the limitations of your work?
[Yes] See Section 7 and Section A in the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] We do not see any potential negative societal impacts of the work.
We show one example of applying the proposed model in a real engineering application in Section 6, and discuss the potential impact and limitations in Section A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sectionand Section 5.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Section B and Section C in the supplementary material.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section D in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 6 and Section D in the supplementary material. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [N/A] We follow the same style of machine learning papers to report the results.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section D in the supplementary material. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] We develop our experiments for the visual tasks by modifying the code from [63].
In addition, we develop our own code for the MSD time series data.
(b) Did you mention the license of the assets? [Yes] The code in [63] is open-sourced. (c) Did you include any new assets either in the supplemental material or as a URL? [Yes] The URL can be bound in [63].
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] We do not use the data from other people. We generated the data by our own. (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
The data we use only contain measurements of the physical systems. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] We do not use crowdsourcing.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A]
We do not use crowdsourcing.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A] We do not use crowdsourcing.
Supplementary Material
Outline.
Supplementary material is outlined as follows.
Section A discusses the impact of the proposed model.
Section B details the proof of the Fourier feature kernels.
Section C gives the derivation of the training loss of an NTK.
Section D provides details of the training pipeline and the model architecture.
Finally, Section E assembles the additional experimental results.
These include:
(1) visualization of encoded states estimated from the self-attention network of ALPS and its attention maps in the pendulum task, (2) visualization of reconstructed observations and prediction of system parameters for each task, and (3) analysis of self-attention networks in predicting states from observations.
A Impact of ALPS
Many control systems such as self-driving cars and autonomous robots are complex.
In order to deal with this complexity, researchers are increasingly using reinforcement learning (RL) to design control policies.
One approach is to design a model-based RL algorithm, in which we first learn a model of the system dynamics from trajectory data, and then use this model to learn or synthesize a policy.
Compared to the approach that uses neural networks to model the dynamics, ALPS can exploit prior knowledge about the system dynamics to have better learning efficiency and generalization.
In addition, using continuous-time system dynamics allows us to better capture the behavior of the system, and incorporate other control properties into the model such as stability and controllability of the system.
One assumption of ALPS is that we need to have prior knowledge of the system.
This assumption is reasonable since in many practical applications, system designers can use the law of physics to design models.
For example, we know Newton’s laws of motion are still true on Mars, but its gravitational constant is different.
However, for some systems such as contact dynamics, we do not have good knowledge about how to design models.
We make assumptions that ALPS requires knowing the equation of the system dynamics and having a differentiable solver for it.
These assumptions are too restrictive to make this useful in practice.
A potential way to overcome this is to develop a hybrid approach, in which we use positions and velocities that follow the law of physics as latent state representations, followed by network networks that capture complex dynamics to compensate for unmodeled parts of the system.
This raises several questions: What kind of neural architecture is needed to capture the dynamics of the system?
How to applied ALPS in more complicated settings or unseen objects during testing?
We believe this paper will encourage future work to develop rigorous design and analysis tools for learning system dynamics.
B Proof of the Kernel Functions
We now derive the kernel of the Fourier feature mapping.
We review the setup here.
Consider a state trajectory v = [ x 0 , x 1 , . .
, x τ − 1 ] T , where here we consider a 1D case for a scalar state x ∈ R although it can be easily extended to a vector state x ∈ R n .
The feature maps of the Fourier feature mapping, their magnitudes
[Equation]
and their phases are
[Equation]
Set C
[Equation]
ki
[Equation]
sin
[Equation]
∈ R τ × τ , then the kernel functions of these mappings are
[Equation]
Proof: (1) k DFT ( v 1 , v 2 ) : Let v 1 =   x 1 , 1 , .
.. , x τ − 1 ,
 and v 2 =   x 1 , 2 , .
.. , x τ − 1 ,
 .
Then k
[Equation]
[Equation]
[Equation]
ϕ
[Equation]

[Equation]
which completes the proof of k DFT ( v 1 , v 2 ) .
[Table]
2) k MAG ( v 1 , v 2 ) :
Let c k = 

[Equation]

[Equation]
 
[Equation]
 
[Equation]
Set a k = v T 1 s k , b k = v T 2 c k , c
k = v T 1 c k , and d k = v T 2 s k .
Then (2) =
[Equation]
which completes the proof of k MAG ( v 1 , v 2 ) .
C Derivation of the Training Loss of NTK
We now provide more details on the derivation of the training error of a neural network after t times update.
To be complete, we review the notations here.
Let K
= U Σ U T denote the eigendecomposition of the kernel matrix K which must be positive semidefinite (PSD).
Here U is an orthogonal matrix and Σ is a diagonal matrix whose entries are the nonnegative eigenvalues ordered by magnitude:
λ 1 ≥ λ 2 ≥ · · · ≥ λ m ≥ 0 .
So the training error in terms of L 2 norm ∥·∥ 22 is
[Equation]
where the last two equations follows the fact that e − η K t
= U e − η Σ t U T .
This completes the derivation.
D Experimental Details D.1 Training details
Setup.
For all the tasks we train our model and the baselines using Adam [70] with maximum of 10,000 epochs and a learning rate of 3 × 10 − 3 .
{+The batch size is 32 ( i.e., 32 rollouts), and we use a gradient clipping of 1 . 
+}
in terms of 2-norm to stabilize training.
For the physics simulator, we use the four-step Runge-Kutta (RK4) as the numerical integration scheme in neural ODE.
For computational resources, we train our model and the baselines in two machines: machine A has an Intel i9-9980HK
CPU, and machine B has an Intel i7-6850K CPU with 4 NVIDIA GTX 1080 GPUs.
In general, it takes about 8 hours to finish training.
Fig.
6 plots the training curves of the four tasks.
Instructions needed to reproduce the results.
The code can be found in the same zip file.
D.2 Implementation and model architectures
We use the same architecture over our model in the visual tasks.
A grid search is conducted to find the hyper-parameters of the model ( i.e., size of the networks, types of the activation function).
We now describe each component in ALPS.
The encoder.
To process pixel observations, the encoder uses a convolution neural network (CNN) with two convolutions and 2 by 2 max-poolings–the first convolution has a kernel size of 3 and a channel size of 3, and the second one has a kernel size of 4 and a channel size of 4, followed by vectorization to get a vector representation of an image.
To provide the position information to this image representation, we obtain a positional embedding with the same size as the image representation using sine and cosine functions.
We then concatenate these two vectors and feed them into a self-attention network, where the network has 10 attention heads and a relu activation function.
Finally, the network outputs state estimations along the time horizon.
Specifically, we only use the encoder part of the Transformer [54] without a dropout operation.
We find that removing a dropout operation and using a relu activation function achieve the best result.
The estimator.
For each element of the state predicted from the encoder, we construct a Fourier feature mapping.
For example, the state of the pendulum consists of an angle and an angular velocity–we will have two vectors of the Fourier feature mapping here.
Then, for each Fourier feature mapping, we use a residual network [1] with two residual blocks and a tanh activation function to get a representation.
Finally, we concatenate these representations and feed them into an MLP to predict a system parameter.
We find that using residual networks and a tanh activation function makes training faster and converges to a good result.
The simulator.
The ODE solver takes in the predicted system parameter, and then rollouts state trajectories given the initial state predicted from the encoder.
The decoder.
We use a two-layer MLP with a relu activation function and the size of 400, 400 to take in the simulated states and reconstruct the observations.
For the real MSD task, we remove the encoder and decoder in ALPS since we get noisy measurements of states in train wheel systems.
For ALPS without Fourier features, we use the same architecture but the estimator becomes a single residual network that takes in an entire state trajectory in the time domain.
For ALPS without self-attention networks, we use the same architecture but the encoder becomes a two-layer MLP with a relu activation that takes in image representations and predicts the position of an object.
To estimate the velocity, we follow the procedure in [63], which uses a finite difference method.
Finally, for the baseline model CDM, we use the same model in [69], which consists of a context network (an MLP with the size of 400, 400), a forward network (an MLP with the size of 20, 20), and a backward network (an MLP with the size of 20, 20).
Note that CDM is trained with the loss function used in [69]; Autoencoder is trained with the VAE loss and observation reconstruction loss terms; the ablations of ALPS are trained with the same loss functions as the full model.
E Additional Results E.1 Visualization of encoded states prediction and attention map of the self-attention network
An important feature of ALPS is that it uses self-attention networks to infer states from pixel observations.
Fig.
7 shows the encoded states { ˜ x s } , the ODE-simulated states { ˆ x s } , and the true states { x s } over time horizons τ as well as its attention map in the pendulum task.
We see that the self-attention network is able to track the state evolution of the system from pixel observations.
In addition, we observe that there is a small delay between the true states and the ODE-simulated states.
This is due to a small prediction error in the initial state and the system parameter, which then accumulates over the time horizon.
For the attention map, the summation of each row is equal to one.
This map shows that for each time step how much information of each observation in the entire trajectory is needed to predict the state at that time step.
Here, we use a mask with the size of 15 to limit the attention width of the network.
We see that the network mostly uses the observation in the beginning and the end of the attention mask to infer states.
This suggests that the network learns to predict an average velocity.
In addition, we see that there are vertical attention patterns, which happens to be at the peak of the cosine wave.
We suspect that the network uses this information as an anchor to capture the periodic behavior of the pendulum.
We leave this as a future work to understand the network.
E.2 Visualization of reconstructed observations and prediction of system parameters
ALPS uses an autoencoder to learn system dynamics.
Fig.
8 shows reconstructed observations of
ALPS and the baseline in three visual tasks.
We see that ALPS is able to accurately reconstruct the observations, whereas CDM fails to predict observations with duplicated particles in the Two-Body system and blur images in the MSD system.
This implies that using physics in the loop can help to converge to a good solution.
In addition, Fig.
9 shows the system parameter prediction performance of ALPS over three visual tasks.
Please see the caption for more details about the figure.
We see that the model is able to reliably predict the system parameters in most cases.
However, we also see there are some outliers for each group.
We leave this as a future improvement.
E.3 Analysis of self-attention networks
An important feature of ALPS is the use of self-attention networks for estimating states from observations.
To provide the intuition about the mechanism of the self-attention network, we regress the self-attention network with true states of the system and visualize its attention weights on a simple vehicle steering dynamics:
[Equation]
where the first element in x = [ x p , x r ] T represents the lateral path deviation ( i.e., the distance between the vehicle and the center of the road), the second element represents turning rate, and u is the input force to the steering wheel.
In this example, we want to estimate either x p or x r by observing x r or x p , respectively.
Intuitively, in the first case (estimate x p from x r ), a self-attention network should perform derivative operation, whereas in the second case (estimate x r from x p ), a self-attention network should perform integration operation.
Fig.
10 shows the result of the two cases.
Here we use a single head self-attention network with a tanh activation function to regress the supervised data, and test on the testing data.
In the first case the network attends to the neighboring observations at the current time ( i.e., diagonal pattern), whereas in the second case the network attends to the observations from beginning to the current time ( i.e., lower triangular pattern, global context).
This observation suggests the network does exploit the local context to compute the gradient ( i.e., x rt ≈ x pt +1 − x pt ∆ t ), and use the global context to obtain the integral ( i.e., x pt ≈ (cid:80) t − 1 t ′ =
0 x rt ′ ).