Learning Physics Constrained Dynamics Using Autoencoders
Anonymous Author(s)
AffiliationAddress email
Abstract
We consider the problem of estimating states ( e.g., position and velocity) andphysical parameters ( e.g., friction, elasticity) from a sequence of observationswhen provided a dynamic equation that describes the behavior of the system.
Thedynamic equation can arise from first principles ( e.g., Newton’s laws) and provideuseful cues for learning, but its physical parameters are unknown.
To address thisproblem, we propose a model that estimates states and physical parameters of thesystem using two main components.
First, an autoencoder compresses a sequenceof observations ( e.g., sensor measurements, pixel images) into a sequence for thestate representation that is consistent with physics by including a simulation of thedynamic equation.
Second, an estimator is coupled with the autoencoder to predictthe values of the physical parameters.
We also theoretically and empirically showthat using Fourier feature mappings improves the generalization of the estimatorin predicting physical parameters compared to raw state sequences when learningfrom high-frequency data.
In our experiments on three visual and one sensormeasurement tasks, our model imposes interpretability on latent states and achievesimproved generalization performance for long-term prediction of system dynamicsover state-of-the-art baselines.
Introduction
Neural networks have become a core computational component in domains such as computer vision[1], natural language processing [2] and deep reinforcement learning [3].
Recent work has shownthat neural networks can exhibit an inductive bias that is often introduced via designing specificstructures [4].
This bias can be used to encode prior task knowledge that helps the network generalizeto unseen data.
For example, convolution neural networks capture the translation invariance of keyimage features.
In this spirit, we develop a structured neural network model that leverages a dynamicequation to estimate both the state of a dynamical system ( e.g., position and velocity) and its physicalparameters ( e.g., friction constants) from a sequence of partial observations.
Knowing the stateand parameters of the system aids learning a control policy and tracking parameter value changes over time.
For instance, for a self-driving car, we want to learn a neural network that estimates the vehicle’s position and velocity from a sequence of egocentric camera images.
The estimated state can then be used in a control policy.
In addition, we want to track physical parameters over time, e.g.,friction coefficients.
This is useful for vehicle maintenance and safety.
We expect that exploiting a neural network, regularized to follow a dynamic equation, will streamline the required data-intensiveoperations and yield improved performance.
Observations can be direct measurements of some system variables ( e.g., accelerations), or take a more complex form ( e.g., images).
We group such observations over specified time windows and also refer to these groupings as observations.
We obtain a compact representation for these observations that permits observation reconstruction using an autoencoder (see Fig. 1(a))
h s } in Fig.
1(a) is a representation of an observation sequence { o s } over a specified time window, and { ˆ o s } is thereconstructed observation sequence.
Learning the autoencoder is both data and computationallyintensive.
In addition, h s may not be physically interpretable and is not guaranteed to have the Markov property, i.e., be a “system state.”
Motivated by these observations, we assume a simulator is provided that specifies the physical lawsof the system.
This model can arise from first principles ( e.g., Newton’s laws), but its free parametersθ ( e.g., masses, lengths) remain to be specified.
Given an initial state and θ , the physics simulatorgenerates a state trajectory { ˆ x s } consistent withthe laws of physics.
To leverage this model, werequire an estimator f ( · ) that maps a sequence ofstates { ˜ x s } to an estimate of θ .
We then couplethe estimator f and the physics simulator with theautoencoder as shown in Fig. 1(b).
We train theautoencoder and f ( · ) to minimize the observationreconstruction loss (cid:80) s ∥
o s − ˆ o s ∥ 22 .
Within thisprocess, we train the encoder h ( · ) to minimize thesum of squared state errors: (cid:80) s ∥
˜ x s − ˆ x s ∥ 22 .
Thecomplete model (Fig. 1(b)) is called Autoencoderwith Latent Physics (ALPS).
The paper’s contributions are three-fold.
(1) Wepropose and explore a model that integrates physicsinto an autoencoder to perform unsupervised stateand physical parameter predictions.
(2) We showthat one can learn periodic or vibrational behaviorin this setting using a Fourier feature of states.
(3) We evaluate the proposed model in four tasks: ( a ) pendulum with friction observed through images, ( b ) mass-spring-damper system with stiffnessobserved through images, ( c ) two-body systems with potential energy observed through images,and ( d ) real-world vehicle wheel suspension system with accelerations observed through sensormeasurements.
The proposed approach can achieve up to 4.8x and 6.3x better physical parameter andstate prediction accuracy, respectively, over prior approaches.
Related Work
Physics-informed neural networks.
There is growing interest in including a physics prior oralgebraic and logical constraints into neural networks [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29].
For example, [30, 31, 32, 33, 34, 35, 36, 37, 38,39 , 40] exploit Lagrangian or Hamiltonian mechanics to learn an energy-conserving system basedon position, momentum, and the derivatives thereof along trajectories.
These works assume thephysical parameters of the system are constant and need not be estimated.
In contrast, we estimatethe physical parameters.
This is important for fault detection and localization, and for safety.
Papers[41, 42, 43, 44] learn a general physical simulation from data, but their model is required to havestate information or a known forward rendering engine to map states to observations.
In contrast, welearn a physics-based autoencoder to estimate states in an unsupervised manner.
Paper [45] also usesan autoencoder with physics to predict parameters.
However, we show that learning state sequencesas in [45] fails to generalize to unseen parameters when the system exhibits high-frequency behavior.
Fourier features for high-frequency data.
[46, 47] show that using Fourier features helps neuralnetworks learn high-frequency content in image regression tasks.
This approach requires specifyingthe Fourier series coefficients and the basis frequencies.
In contrast, the Fourier features in ourmodel are computed from the states, which allows us to predict physical parameters.
In addition, theconcurrent work [48] replaces the self-attention sublayers [49] with a Fourier transformation of theinput word token in natural language processing.
They show that this method is sufficient to capturesemantic relationships in several text classification tasks.
In this paper, we provide a theoreticaljustification for using Fourier features to learn system dynamics with high-frequency data.
Problem Formulation
We consider the continuous-time system
[Equation]
) where A ∈ R n × n , B ∈ R n × p and x ∈ R n , u ∈ R p , o ∈ R q , denote the state, input, and observation,respectively.
The function g provides a partial observation o of x .
In most situations we have partialknowledge of A and B from physics.
This is true and reasonable in many large-scale industrialapplications ranging from wind turbines to aircraft, where system designers use physics knowledgeto design machines.
Hence we assume mappings A ( · ) :
Θ → R n × n and B ( · ) :
Θ → R n × p , fromphysical parameters θ ∈ Θ to the system matrices
A ( θ ) , B ( θ ) , are given ( i.e., A and B obtain aspecific sparse and parameteric form).
In addition, Eq.
(1) assumes the system dynamics to be linear,but our approach can be extended to other non-linear cases.
In practice, we only have observations at discrete points in time.
For simplicity, we assume theseare equally spaced at times t = 0 , 1 , . . . .
At sample time t, we have the window of observations{ o s } ts = t − τ +1 , where τ is the window length.
We assume the sampling rate satisfies the Nyquist rate.
Our problem can now be stated as follows.
Given functions A ( · ) :
Θ → R n × n , B ( · ) :
Θ → R n × p ,we seek to learn a network that estimates a state sequence { x s } ts = t − τ +1 , physical parameters θ ,and a mapping g ( · ) from a finite sequence of past observations { o s } ts = t − τ +1 and past knowninputs { u s } ts = t − τ +1 .
Once trained, the network can predict states and has learned to adapt physicalparameters to the context without re-training.
Note that even though the network operates with timesampled variables, the physics simulator can be used to predict states at any time.
This problem setupis distinct from that of HNN [33], HGN [36], or Symplectic ODE-Net [35].
In their setups, the modelis only trained on data from a single physical parameter θ , and hence they need to re-train networksfor every new set of physical parameters.
In addition, compared to conventional system identificationapproaches, we do not need to rerun the solver each time–we only need a forward pass of the networkto get the estimation during deployment (testing time).
To make the learning problem well-defined, i.e., to ensure that we can uniquely identify the unknownsystem parameters, we make the following assumption.
Assumption 3.1.
(Identifiability [50]) :
For a sufficiently large τ, the sequence of states and inputs{ ( x s , u s ) } ts = t − τ +1 uniquely specifies the unknown parameter θ .
Parameter identifiability is a well-studied problem in system identification [50, 51, 52].
For example,the (continuous time) system ˙ x = (cid:20) − ( a + b ) b b − c (cid:21) x
+ (cid:20) 10 (cid:21) u with o =
[1 0] x , where θ =[ a, b, c ] T and g ( · ) is a linear function, satisfies Assumption 3.1 when b ̸ = 0 .
Nework Architecture
The network in Fig. 1(b) is expanded in Fig.
2 into its four parts: an encoder network, a parameterestimator network, a physics simulator, and a decoder network.
The encoder network h ( · ) (from { o s } to { ˜ x s } ).
We consider two types of observations: (1) pixelimages, or (2) direct measurements of some system variables.
For the first case, we use a convolutionneural network to compress a pixel observation o into a compact vector embedding z ′ ∈ R d , whichpreserves image features.
For the second case, we use a feedforward network to project an observationo into some higher dimensional space with a vector embedding z ′ .
In addition, to estimating statesfrom these vector embeddings, we need to aggregate { z ′ s } to extract the local and global context of thedynamics.
One approach is to use recurrent neural networks (RNN) ( e.g., Dreamer [53]).
However,
RNNs suffer from vanishing gradient problems and slow computation when processing long-termsequences.
Hence we use a self-attention network [49] to attend to z ′ of greatest importance topredict states and improve efficiency.
Furthermore, to inject a position signal of observations in the sequence, we add a positional encodingp ∈ R d to z ′
( z :=
z ′ + p ), where p are sine and cosine functions of different frequencies (see [49]).
Finally, we stack embeddings over τ steps to form a matrix Z ∈ R τ × d .
The self-attention module can be formulated as querying a dictionary with key-value pairs associatedwith learnable weight matrices W Q ∈ R d × d Q , W K ∈ R d × d K , and
W V ∈ R d × d V ( d Q = d K here):
Attention ( Q , K , V ) =
softmax ( QK T √ d ) V , where Q :=
ZW Q , K :=
ZW K , V :=
ZW V , andthe softmax is taken over the sequence length τ.
To provide multiview of the embedding subspace,we use the multihead variant of the attention module by concatenating each attention head along thesequence axis Multihead := Concat ( head 1 , . .
, head i , . . , head I ) W O , head i := Attention ( Q , K , V ) , where W O ∈ R I
d V × d are learnable matrices, I is the number of heads, and each attention head i hasits own learnable weight matrices W Qi ∈ R d × d Q , W Ki ∈ R d × d K , and W Vi ∈ R d × d V .
Finally, a feedforward network takes in the Multihead embedding (of size R τ × d ) and produces theparameters of the distribution for each state in the sequence.
The parameters are used to define aposterior distribution over the encoded state ˜ x s ∼ Q ( ·| ˜ o s ) with the prior P (˜ x s ) .
For a translationalcoordinate, the posterior distribution is a Gaussian distribution with a unit Gaussian prior.
Hence thenetwork predicts a mean µ ∈ R n and a standard deviation σ ∈ R n of a Gaussian distribution.
Inaddition, for a rotational coordinate, the posterior distribution is a von Mises (vM) distribution with aunit vM prior.
Similar to a Gaussian distribution, a vM distribution is defined by two parameters: amean µ ∈ R 2 , ∥ µ ∥ 2 = 1 (the angular position (cos φ, sin φ ) ), and a concentration κ ∈ R + aroundµ .
Such parameterization is found useful in practice, e.g., [54].
Appendix E provides details aboutself-attention networks for estimating states.
The parameter estimator network f ( · ) (from { ˜ x s } to θ ).
The parameter estimator networkpredicts physical parameters from state sequences { ˜ x s } .
However, for systems that involve periodic or vibrational behavior, prior work [46] has shown that neural networks fail to capture thehigh-frequency content in the data.
In addition, real-world data often contain noise.
To improvegeneralization and reduce the effect of noise, we do a Fourier transform on each component j of statetrajectories { ˜ x s ( j ) } ts = t − τ +1 to get { ˜ X ω ( j ) } tω = t − τ +1 :
˜ X ω ( j ) :
= (cid:80) tk = t − τ +
˜ x k
( j ) (cid:104)
cos (cid:16) 2 πτ ωk (cid:17) −i · sin (cid:16) 2 πτ ωk (cid:17)(cid:105) .
Note that for the systems that do not have periodic or vibrational behavior, using{ ˜ x s ( j ) } ts = t − τ +1 would be enough.
One may use { ˜ X ω ( j ) } as features for the parameter estimator network to predict physical parameters.
However in Section 5 we will show that by using the neural tangent kernel (NTK) theory [55], whichtreats neural networks as a kernel regression, the resulting kernel matrix of { ˜ X ω ( j ) } does not preservehigh-frequency components in the data.
To solve this, we will show that using the magnitude of the
Fourier features {| ˜ X ω ( j ) |} alleviates the issue.
Hence the parameter estimator network takes in aconcatenation of {| ˜ X ω ( j ) |} from each component of the state and predicts physical parameters θ .
The physics simulator (from ˜ x t − τ +
and θ to { ˆ x s } ).
Given a start state ˜ x t − τ +1 (from theencoder’s first state prediction) and values for the physical parameters θ , we use the neural ordinary differential equation (ODE) [56], a differential ODE solver, to generate a simulated state trajectory{ ˆ x [Equation] } ts = t − [Equation]
+1 : ˆ x t − τ +1 , . . . ˆ x t − 1 , ˆ x t =
ODESolver(˜ x t − τ +1 , ˙ x
= A ( θ ) x + B ( θ ) u , τ, ∆) , where
ODESolver takes in a start state, an ODE, a window length, and a sampling time interval ∆ .
Notethat
ˆ x t − τ +
=
˜ x
t − τ +1 .
In addition, using an ODE solver allows us to generate an accurate statetrajectory compared to that of RNN as in [53].
The decoder network g ( · ) (from ˆ x s to ˆ o s ).
Finally, the decoder network is either a deconvolutionalnetwork (for image observations) or a feedforward network (for sensor measurements) that takes ineach individual ODE-simulated state ˆ x s and generates a reconstructed observation ˆ o s .
The loss function.
Given a sequence of τ observations, we minimize the following loss function:
[Equation] [Table]
The variational autoencoder (VAE) [57] loss is a variational bound on the marginal log-likelihoodof the data.
It is used to train the encoder h, the estimator f, and the decoder g .
Using VAEsavoids learning degenerated solutions and provides stable training of the network over a deterministicnetwork.
In addition, the observation reconstruction loss encourages reconstructed observations{ ˆ o s } to match true observations { o s } , and the state reconstruction loss constrains encoded states{ ˜ x s } to follow simulated states { ˆ x s } generated by physics.
The former is used to train h, f,and g, and the latter is used to train f and h. Both observation and state reconstruction losses areimportant for training.
We find that removing the state reconstruction term reduces the state predictionperformance of the encoder since the network can predict arbitrary sequences.
In addition, removingthe observation reconstruction term impedes the image reconstruction quality, which is vital fortraining the whole model.
In practice we find using the same weight for each loss term works well.
Fourier Feature Mappings for Learning Periodic System Dynamics
An important feature of ALPS is that it uses a Fourier feature mapping of states to learn periodicsystem dynamics.
Prior works [58, 59] have empirically shown that using a Fourier feature can helpwith high frequency signals.
To lay the foundation for the analysis and justify the use of Fourierfeatures, in Section 5.1 we first review the recent work that uses NTK theory [55, 46].
This allowsus to control the training of our parameter estimator network as fully-connected networks.
Then in
Section 5.2 we use these tools to analyze the effects of using Fourier features, their magnitudes, andtheir phases when predicting system parameters that involve the periodic behavior .
5.1 Deep networks as a kernel regression
Consider a set of labelled training data { ( v i , y i ) } mi with v i ∈ R n , y i ∈ R , and i ∈ [1 : m ] .
Sety =
[ y 1 , . . .
, y m ] T ∈ R m .
Now bring in a feature map ϕ : R n → R r with kernel k ( v i , v j )
=
ϕ ( v i ) T ϕ ( v j ) .
Let K =
[ k ( v i , v j )]
∈ R m × m denote the kernel matrix for the training examplesand k
( v ) =
[ k ( v i , v )]
∈ R m denote the vector of kernel evaluations k ( v i , v ) , i ∈ [1 , m ] , for a testsample v ∈ R n .
The resulting kernel regression predictor is ˆ y ( v ) =
y T K − 1 k ( v ) .
Now bring the concept of NTK proposed by [55].
The theory in [55] says that when the width of thelayers of fully-connected deep networks with weights w initialized from a Gaussian distribution N tends to infinity, and the learning rate for stochastic gradient descent tends to zero, the neural networkestimator ˆ y ( v ; w ) converges to the kernel regression solution using NTK.
The NTK is defined as
[Equation]
Under asymptotic conditions, a neural network’s output after t updates can be approximated as
[Equation]
where e M is the n by n matrix M given by the power series
e M = (cid:80) ∞
i =
1 i !
M i with M
= I .
Spectral bias in neural networks.
Now we want to compute the training error of a neural networkafter t times update.
Let K
= U Σ U T denote the eigendecomposition of the kernel matrix K whichmust be positive semidefinite (PSD).
Here U is an orthogonal matrix and Σ is a diagonal matrix whoseentries are the nonnegative eigenvalues ordered by magnitude: λ 1 ≥ λ 2 ≥ · · · ≥ λ m ≥ 0 .
So thetraining error in terms of L 2 norm ∥·∥ 22 is (cid:13)(cid:13)
ˆ [Equation]
This shows the training convergence will decay exponentially at the rate ηλ i .
Hence the componentsin y will be learned faster if their corresponding eigenvalue is larger.
In Section 5.2 we will showthat for a sequence of states without doing Fourier transform, the resulting NTK will have smallereigenvalues at the high-frequency components.
This leads to a slower convergence in high-frequencycomponents of data which are essential to identify parameters for the periodic behavior.
5.2 The effect of Fourier feature mapping
To understand the effect of the Fourier feature mapping, we first derive the kernel function of Fourierfeatures, their magnitudes, and their phases.
Then we compare the spatial bias of the kernel matricesfrom these three kernels.
Kernels of Fourier feature mapping.
Consider a state trajectory v = [ x 0 , x 1 , . . .
, x τ − 1 ] T , wherehere we consider a 1D case for a scalar state x ∈ R although it can be extended to a vector statex ∈ R n .
The feature maps of the Fourier feature mapping, their magnitudes, and their phases are
[Equation]
where X ω = (cid:80) τ − 1 j =0 x j (cid:104) cos (cid:16) 2 πτ kj (cid:17) − i sin (cid:16) 2 πτ kj (cid:17)(cid:105) .
Set C
k
= (cid:2) cos
[Equation] πτ ki [Equation] − sin (cid:0) 2 πτ kj (cid:1)(cid:3) ∈
R τ × τ , then the kernel functions of these mappings are
[Equation]
) T ϕ PHA ( v 2 ) . After mapping the input points into the Fourier features, we feed them into a neural network to obtainˆ y ( ϕ ( v ); w ) .
Hence for DFT kernel function the resulting composed kernel of the neural network isk NTK ( ϕ DFT ( v 1 ) , ϕ DFT ( v 2 )) .
Similarly, we have the kernels for k MAG and k PHA .
Visualizing the composed NTK.
We generate time series data composed of different frequencies andmagnitude of sine waves.
The length of the data is 200 samples, and we set τ = 100 ( i.e., v ∈ R 100 )with the sampling rate being 100Hz.
We slide a window and hence have 101 instances in total.
Fig.
3shows the time series data, the effects of each kernel, and its spatial plot.
By construction, k MAG andk PHA have a slower decay in the high-frequency domain as shown in Fig. 3(b).
In addition, k DFTand no mapping have the same kernel matrix and a narrower kernel spatial.
This observation supportsthe idea of not using X ω when learning the system with periodic or vibrational behavior as | X ω |(magnitude) preserves high-frequency information, which is useful for parameter estimations.
Notethat for the system that does not have high-frequency signals, both X ω and | X ω | are equally well.
Comparison to [46].
Our work is inspired by [46], which also uses Fourier feature mappings.
However, there are a few key differences.
(1) Setup.
The Fourier feature mapping in [46] transformsthe low-dimensional x-y coordinates into high-dimensional Fourier features, which projects datainto a high-dimensional space.
In contrast, we use Fourier feature mapping of raw time series data,which compresses data into more compact representations.
(2) Analysis .
We discuss the difference of
Fourier feature mappings, their magnitudes, and their phases to understand the effect of each mappingfor predicting system parameters.
In contrast, [46] does not have such analysis.
Simulations
We study the following three questions: (1) How does ALPS perform compared to other baselinemethods without the Fourier feature mapping and physics-in-the-loop?
(2) What is the effect ofself-attention networks in ALPS?
(3) How does ALPS perform in a real-world time series data?
Visual dataset.
We generate three visual datasets: pendulum, mass-spring-damper (MSD), and atwo-body system to compare the performance of ALPS to that of the baseline in the literature.
Thesebaselines are very commonly used in the prior work.
We first randomly sample an initial state andphysical parameters, and then generate a 125 step rollout following the true system dynamics, andrender corresponding 64 by 64 by 3 pixel observation snapshots.
The sampling rate is 20Hz in thependulum, 100Hz in MSD, and 6Hz in two-body systems.
The observation length τ is 100.
In total,we generate 500 training and 500 test trajectories, resulting in 13,000 training and test sequences.(1)
Pendulum.
The dynamics is
[Equation]
, where φ is an angle, ˙ φ is anangular velocity, β is a friction coefficient, G is a gravitational constant, and L is the length of thependulum.
We fix G = 10 , L = 1 , then sample β from a uniform distribution β ∼ U (0 . 1 , 1) , aninitial angle from a uniform distribution φ 0 ∼ U ( − π, + π ) , and an initial angular velocity from auniform distribution ˙
φ 0 ∼ U (0 . 5 , 4) .
We predict θ =
[ β ] in this task.(2) Mass-spring-damper.
The dynamics of double mass-spring-damper system is
[Equation]
[Table]
,where x (1) , x (3) are the displacement and velocity of the primary spring; x (2) , x (4) are the displacement and velocity of the secondary spring; u (1) , u (2) are the inputs; α (2) , α (4) are the stiffness anddamping ratio of the primary spring and damper; α (1) , α (3) are the stiffness and damping ratio ofthe secondary spring and damper, and m (1) , m (2) are the mass of the primary and secondary spring.
We fix all the parameters and the initial state except for α (2) , which is sampled from a uniformdistribution α (2) ∼ U (4 × 10 3 , 4 × 10 6 ) .
The input excitations are sampled from a unit Gaussiandistribution u ∼ N (0 , 0 . 05)
× N (0 , 0 . 05) ( i.e., white noise).
We predict θ =
[ α (2)] in this task.(3) Two-body.
In this system two particles interact with each other via an attractive force.
Thedynamics is a Hamiltonian H =
∥ p (1) ∥ 22 2 m (1) + ∥ p (2) ∥ 22 2 m (2) + Gm (1) m (2) ∥
q (1) − q (2) ∥ 2 , where p (1) , p (2) are the positionsof the particles; q (1) , q (2) are the momentum of the particles; m (1) , m (2) are the masses of theparticles, and G is a gravitational constant.
We fix G = 10 , m (1) = m (2) = 1 , then sample H froma uniform distribution H ∼ U (0 . 4 , 1) .
We predict θ =
[ H ] in this task.
Time series dataset.
In addition, we obtain an MSD system dataset with state measurements.(4)
Real MSD time series data (wheel suspension system).
To show the applicability of ALPS,we conduct real-world experiments with the data obtained from sensors installed in the train wheelsuspension system by a team of engineers.
We use full-scale dynamics with 18 state elements, 12observations, 16 input excitations, and 24 system parameters.
We want to identify the stiffnessand damping ratio of the 12 spring-dampers.
The dynamics are complex due to the interactions ofmultiple springs and dampers.
In addition, this problem is challenging as the data contain noise.
Thedataset contains 20 trajectories with 500 steps sampled by 100Hz.
We use a 50-50 split to get thetraining and test datasets with τ = 100 .
Here we do not use the self-attention and decoder network in
ALPS–the estimator takes in state measurements directly.
Baselines.
We select the following baselines that are commonly used in the literature.(1) Context-aware dynamics model (CDM) [60].
CDM is the state-of-the-art method to learn thesystem dynamics into two stages: it first learns a context vector that captures the local dynamics, andthen predicts the next state based on the context vector and the current state and input.
CDM doesnot consider physics prior nor Fourier feature mapping.
This is to show that using physics priorsimproves performance and benchmark the results.
Pendulum
Mass-Spring-Damper
[Table]
competitive performance in predicting physical parameters and (2) Autoencoder.
We remove the estimator and the physics simulator to benchmark the mismatchbetween the true state and the latent representation (equal dimension) of the autoencoder in Fig. 1(a) to show the interpretability of ALPS.(3) ALPS w/o the Fourier feature mapping.
We consider a variant by replacing the Fourier featuremapping with raw encoded state trajectories { ˜ x s } .
This method is similar to [45], which also usestime series data to learn system dynamics.(4) ALPS w/o self-attention networks.
We consider a variant by replacing the self-attention networkwith a simple MLP to predict the position of the system, followed by a first-order finite-differenceestimator to estimate the velocity.
This uses the same approach as in [54] to estimate states.
Finally,we ensure that all baselines are comparable in terms of representation power and the number ofparameters.
The loss functions used for these baselines are included in Appendix D.
Evaluation metrics.
We use a mean squared error (MSE) between simulated { ˆ x s } and true states{ x s } to evaluate the performance: SE :
= 1 N (cid:80) N i =1 (cid:80) t s = t − τ [-+
-]+1 ∥
ˆ x s − x s ∥ 2 2 , where N is the numberof test data.
We also compute MSE between reconstructed { ˆ o t } and true observations { o t } : OE :=1 N (cid:80) Ni
=1 (cid:80) ts = t − τ +
∥
ˆ o s − o s ∥ 22 .
Moreover, we compute the absolute value difference betweenestimated ˆ θ and true parameters θ : PE :
= 1 N (cid:80) Ni {+=
+}
(cid:13)(cid:13)(cid:13) ˆ θ − θ (cid:13)(cid:13)(cid:13) 1 .
Results in the visual tasks.
Fig.
4 shows the results.
We see that (1) ALPS achieves the bestperformance in predicting physical parameters in all cases, with at the most 4.8x lower error inthe pendulum task.
On the other hand, the Fourier feature mapping does not have much effect inpredicting the physical parameter in the two-body system.
This verifies the analysis of the Fourierfeature mapping: for the task with a wider frequency spectrum ( e.g., 100Hz in the MSD), the Fourierfeature improves the prediction due to higher eigenvalues in the high-frequency content, whereas forthe task with only a low-frequency spectrum ( e.g., 6Hz in the two-body), raw state sequences canalready capture low-frequency content.
This supports the idea of using Fourier feature mappings forhigh-frequency data.
(2) ALPS achieves competitive results in predicting the states, with at the most6.3x lower error in the MSD task.
We further visualize the state prediction results in the pudendumtask for ALPS, which shows that ALPS can accurately track the true states.
Without self-attentionnetworks, the network has a substantial SE due to a large error in computing the velocity, as in the [Figure] two-body task.
The smaller the sampling rate is, the greater the velocity estimation error is.
Inaddition, the high SE in autoencoder suggests that its latent representation is uninterpretable.
Thisverifies the idea of using physics to constrain the representation of the autoencoder to estimate states.
In addition, (3) ALPS achieves comparable results in reconstructing observations.
The low error for
CDM in the MSD and two-body tasks is due to degenerated solutions as shown in the Fig.
Thisimplies that using physics stabilizes the training and improves the reconstruction of observations.(4)
Finally, the autoencoder baseline has higher reconstruction loss in some tasks.
This is becausethat it learns a degenerated solution, producing blur images due to high-frequency movements ofobjects on the scene.
Fig.
8 in Appendix E provides more qualitative visualizations of reconstructedimages.
Specifically, we find that CDM fails to track the state and resulting blur images in themass-spring-damper system or duplicated objects in the two-body system.
In contrast, ALPS canprecisely track the state and reconstruct the image well.
Our remaining simulations explore ALPS’sability in predicting physical parameters from raw state measurements.
Results in real MSD time series data.
To scale the proposed approach to a more complex system,we apply the approach in a system with a full-scale train wheel suspension system.
The systemcontains 18 state elements, 12 observations, 16 input excitations, and 24 system parameters (see Fig. 5(a) for the illustration).
Fig.
5(b) shows the results.
Here ALPS does not use the encoder nordecoder network–the estimator takes in state measurements directly, as the system is fully-observablein the dataset.
Overall we see that (1) ALPS achieves the best performance.
Without using the Fourierfeature mapping, the network cannot learn to identify physical parameters.
(2) CDM has worse SEsince an MLP cannot learn well from data with high-frequency content.
In addition, we find that theaverage prediction error rate ( i.e., ˆ θ − θθ ) for these 24 parameters is 0.42 % on the real world complexsystem.
This also implies that by providing physics to the network, we can improve SE.
Theseobservations show ALPS can robustly identify parameters simultaneously from state measurements,and support the theory of using Fourier features to learn periodic dynamics.
Moreover, this resultshows ALPS can be deployed in real prognostic applications for tracking physical parameters toenhance railroad safety.
Our result is significant for system designers to diagnose the system andhence schedule maintenance.
Appendix A provides discussions of the real-world impact.
Conclusion and Future Work
We addressed the problem of predicting states and physical parameters of a system from observationswith dynamic equations.
We showed that the latent representation of the autoencoder is uninterpretable.
We then use dynamic equations to constrain the latent representation of the autoencoderto be consistent with the laws of physics.
We analyzed the effect of Fourier features for estimatingphysical parameters.
The results showed that ALPS achieves competitive performance in the visualtasks and the real raw time series dataset with up to 24 parameter predictions at the same time.
Thisshows that our method scales to high-dimensional systems and it is useful for real-world applications.
Future work could improve ALPS in several ways.
For instance, the understanding of underlyingmechanisms in self-attention networks for estimating states from observations can be advanced.
Inaddition, we require to have dynamic equations and assume that the system is linear and exhibitsperiodic or vibrational behaviors.
One solution is to combine known physics with neural models ( e.g.,interaction networks [13]) to compensate for modeling error, and use time-domain and frequencydomain features together to predict physical parameters.
Moreover, it would be interesting to exploreother more complex tasks ( e.g., non-linear systems, fluid dynamics, molecular simulation).
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] We use an autoencoder to estimate states from observations with physics-in-the loop, and show that using the Fourier feature can improvegeneralization.
Our experiments in Table 4 and Fig.
5 support this idea.
(b) Did you describe the limitations of your work?
[Yes] See Section 7 and Section A in
the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] We do
not see any potential negative societal impacts of the work.
We show one example of
applying the proposed model in a real engineering application in Section 6, and discussthe potential impact and limitations in Section A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section and Section 5.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Section B and
Section C in the supplementary material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)? [Yes] See Section D in the supplementary material. (b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they were chosen)?
[Yes] See Section 6 and Section D in the supplementary material.
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[N/A] We follow the same style of machine learning papers to
report the results. (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)? [Yes] See Section D in the supplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We develop ourexperiments for the visual tasks by modifying the code from [54].
In addition, wedevelop our own code for the MSD time series data.
[Table]
(b) Did you mention the license of the assets?
[Yes] The code in [54] is open-sourced.(c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]The URL can be bound in [54].
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A] We do not use the data from other people.
We generated thedata by our own.
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[N/A]
The data we use only contain measurementsof the physical systems.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
We do not use crowdsourcing.(b)
Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
We do not use crowdsourcing.
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A] We do not use crowdsourcing.
[Table]
Supplementary Material
Outline.
Supplementary material is outlined as follows.
Section A discusses the impact of theproposed model.
Section B details the proof of the Fourier feature kernels.
Section C gives thederivation of the training loss of an NTK.
Section D provides details of the training pipeline and themodel architecture.
Finally, Section E assembles the additional experimental results.
These include:
(1) visualization of encoded states estimated from the self-attention network of ALPS and itsattention maps in the pendulum task,
(2) visualization of reconstructed observations and prediction of system parameters for eachtask, and(3) analysis of self-attention networks in predicting states from observations.
A Impact of ALPS
Many control systems such as self-driving cars and autonomous robots are complex.
In order todeal with this complexity, researchers are increasingly using reinforcement learning (RL) to designcontrol policies.
One approach is to design a model-based RL algorithm, in which we first learn amodel of the system dynamics from trajectory data, and then use this model to learn or synthesize apolicy.
Compared to the approach that uses neural networks to model the dynamics, ALPS can exploitprior knowledge about the system dynamics to have better learning efficiency and generalization.
In addition, using continuous-time system dynamics allows us to better capture the behavior of thesystem, and incorporate other control properties into the model such as stability and controllability ofthe system.
One assumption of ALPS is that we need to have prior knowledge of the system.
This assumption isreasonable since in many practical applications, system designers can use the law of physics to designmodels.
For example, we know Newton’s laws of motion are still true on Mars, but its gravitationalconstant is different.
However, for some systems such as contact dynamics, we do not have goodknowledge about how to design models.
We make assumptions that ALPS requires knowing theequation of the system dynamics and having a differentiable solver for it.
These assumptions are toorestrictive to make this useful in practice.
A potential way to overcome this is to develop a hybridapproach, in which we use positions and velocities that follow the law of physics as latent staterepresentations, followed by network networks that capture complex dynamics to compensate forunmodeled parts of the system.
This raises several questions: What kind of neural architecture is needed to capture the dynamics ofthe system?
How to applied ALPS in more complicated settings or unseen objects during testing?
We believe this paper will encourage future work to develop rigorous design and analysis tools forlearning system dynamics.
B Proof of the Kernel Functions
We now derive the kernel of the Fourier feature mapping.
We review the setup here.
Consider astate trajectory v =
[ x 0 , x 1 , . . .
, x τ − 1 ] T , where here we consider a 1D case for a scalar state x ∈ Ralthough it can be easily extended to a vector state x ∈ R n .
The feature maps of the Fourier featuremapping, their magnitudes, and their phases are
[Equation]
Set C
[Equation]
sin
[Equation]
∈ R τ × τ , then the kernel functions of these mappings are
[Equation]
Proof:(1)
k DFT ( v 1 , v 2 ) : Let v 1 =   x 1 , 1 , .
.. , x τ − 1 ,
 and v 2 =   x 1 , 2 , .
.. , x τ − 1 ,
 .
Thenk
[Equation]
v 1 , v 2 )
[Equation]
[Equation]
ϕ
[Equation]
[Equation]
 
[Equation]
[Equation]

[Equation]

[Equation]

[Equation]

[Equation]
which completes the proof of k DFT ( v 1 , v 2 ) .
[Equation]
2) k MAG ( v 1 , v 2 ) : Let c k =
[Equation]

[Equation]
 
[Equation]
 
[Equation]
[Equation]
Set a k = v T 1 s k , b k = v T 2 c k , c
k = v T 1 c k , and d k = v T 2 s k .
Then(2) =
[Equation]
which completes the proof of k MAG ( v 1 , v 2 ) .
C Derivation of the Training Loss of NTK
We now provide more details on the derivation of the training error of a neural network aftert times update.
To be complete, we review the notations here.
Let K
= U Σ U T denote theeigendecomposition of the kernel matrix K which must be positive semidefinite (PSD).
Here U is anorthogonal matrix and Σ is a diagonal matrix whose entries are the nonnegative eigenvalues ordered by magnitude:
λ 1 ≥ λ 2 ≥ · · · ≥ λ m ≥ 0 .
So the training error in terms of L 2 norm ∥·∥ 22 is
[Equation]
where the last two equations follows the fact that e − η K t
= U e − η Σ t U T .
This completes the derivation.
D Experimental Details
D.1 Training details
Setup.
For all the tasks we train our model and the baselines using Adam [61] with maximum of10,000 epochs and a learning rate of 3 × 10 − 3 .
{+The batch size is 32 ( i.e., 32 rollouts), and we use agradient clipping of 1 . 
+}
in terms of 2-norm to stabilize training.
For the physics simulator, we use thefour-step Runge-Kutta (RK4) as the numerical integration scheme in neural ODE.
For computationalresources, we train our model and the baselines in two machines: machine A has an Intel i9-9980HK
CPU, and machine B has an Intel i7-6850K CPU with 4 NVIDIA GTX 1080 GPUs.
In general, ittakes about 8 hours to finish training.
Fig.
6 plots the training curves of the four tasks.
Instructions needed to reproduce the results.
The code can be found in the same zip file.
D.2 Implementation and model architectures
We use the same architecture over our model in the visual tasks.
A grid search is conducted to findthe hyper-parameters of the model ( i.e., size of the networks, types of the activation function).
Wenow describe each component in ALPS.
The encoder.
To process pixel observations, the encoder uses a convolution neural network (CNN)with two convolutions and 2 by 2 max-poolings–the first convolution has a kernel size of 3 anda channel size of 3, and the second one has a kernel size of 4 and a channel size of 4, followedby vectorization to get a vector representation of an image.
To provide the position informationto this image representation, we obtain a positional embedding with the same size as the imagerepresentation using sine and cosine functions.
We then concatenate these two vectors and feed theminto a self-attention network, where the network has 10 attention heads and a relu activation function.
Finally, the network outputs state estimations along the time horizon.
Specifically, we only use theencoder part of the Transformer [49] without a dropout operation.
We find that removing a dropoutoperation and using a relu activation function achieve the best result.
The estimator.
For each element of the state predicted from the encoder, we construct a Fourierfeature mapping.
For example, the state of the pendulum consists of an angle and an angularvelocity–we will have two vectors of the Fourier feature mapping here.
Then, for each Fourier featuremapping, we use a residual network [1] with two residual blocks and a tanh activation function toget a representation.
Finally, we concatenate these representations and feed them into an MLP topredict a system parameter.
We find that using residual networks and a tanh activation function makestraining faster and converges to a good result.
The simulator.
The ODE solver takes in the predicted system parameter, and then rollouts statetrajectories given the initial state predicted from the encoder.
The decoder.
We use a two-layer MLP with a relu activation function and the size of 400, 400 totake in the simulated states and reconstruct the observations.
For the real MSD task, we remove the encoder and decoder in ALPS since we get noisy measurementsof states in train wheel systems.
For ALPS without Fourier features, we use the same architecture butthe estimator becomes a single residual network that takes in an entire state trajectory in the timedomain.
For ALPS without self-attention networks, we use the same architecture but the encoderbecomes a two-layer MLP with a relu activation that takes in image representations and predicts theposition of an object.
To estimate the velocity, we follow the procedure in [54], which uses a finitedifference method.
Finally, for the baseline model CDM, we use the same model in [60], which consists of a contextnetwork (an MLP with the size of 400, 400), a forward network (an MLP with the size of 20, 20),and a backward network (an MLP with the size of 20, 20).
Note that CDM is trained with the loss function used in [60]; Autoencoder is trained with the VAEloss and observation reconstruction loss terms; the ablations of ALPS are trained with the same lossfunctions as the full model.
E Additional ResultsE.1 Visualization of encoded states prediction and attention map of the self-attention network
An important feature of ALPS is that it uses self-attention networks to infer states from pixelobservations.
Fig.
7 shows the encoded states { ˜ x s } , the ODE-simulated states { ˆ x s } , and the truestates { x s } over time horizons τ as well as its attention map in the pendulum task.
We see thatthe self-attention network is able to track the state evolution of the system from pixel observations.
In addition, we observe that there is a small delay between the true states and the ODE-simulatedstates.
This is due to a small prediction error in the initial state and the system parameter, which thenaccumulates over the time horizon.
For the attention map, the summation of each row is equal to one.
This map shows that for each timestep how much information of each observation in the entire trajectory is needed to predict the stateat that time step.
Here, we use a mask with the size of 15 to limit the attention width of the network.
We see that the network mostly uses the observation in the beginning and the end of the attentionmask to infer states.
This suggests that the network learns to predict an average velocity.
In addition,we see that there are vertical attention patterns, which happens to be at the peak of the cosine wave.
We suspect that the network uses this information as an anchor to capture the periodic behavior of thependulum.
We leave this as a future work to understand the network.
E.2 Visualization of reconstructed observations and prediction of system parameters
ALPS uses an autoencoder to learn system dynamics.
Fig.
8 shows reconstructed observations of
ALPS and the baseline in three visual tasks.
We see that ALPS is able to accurately reconstruct theobservations, whereas CDM fails to predict observations with duplicated particles in the Two-Bodysystem and blur images in the MSD system.
This implies that using physics in the loop can help toconverge to a good solution.
In addition, Fig.
9 shows the system parameter prediction performance of ALPS over three visualtasks.
Please see the caption for more details about the figure.
We see that the model is able to reliablypredict the system parameters in most cases.
However, we also see there are some outliers for eachgroup.
We leave this as a future improvement.
E.3 Analysis of self-attention networks
An important feature of ALPS is the use of self-attention networks for estimating states fromobservations.
To provide the intuition about the mechanism of the self-attention network, we regressthe self-attention network with true states of the system and visualize its attention weights on a simple vehicle steering dynamics:
[Equation]
where the first element in x = [ x p , x r ] T represents the lateral path deviation ( i.e., the distancebetween the vehicle and the center of the road), the second element represents turning rate, and uis the input force to the steering wheel.
In this example, we want to estimate either x p or x r byobserving x r or x p , respectively.
Intuitively, in the first case (estimate x p from x r ), a self-attentionnetwork should perform derivative operation, whereas in the second case (estimate x r from x p ), aself-attention network should perform integration operation.
Fig.
10 shows the result of the twocases.
Here we use a single head self-attention network with a tanh activation function to regress thesupervised data, and test on the testing data.
In the first case the network attends to the neighboringobservations at the current time ( i.e., diagonal pattern), whereas in the second case the networkattends to the observations from beginning to the current time ( i.e., lower triangular pattern, globalcontext).
This observation suggests the network does exploit the local context to compute the gradient( i.e., x rt ≈ x pt +1 − x pt ∆ t ), and use the global context to obtain the integral ( i.e., x pt ≈ (cid:80) t − 1 t ′ =
0 x rt ′ ).