Physics-Guided Approach with Transfer Learning
in Vehicle Lateral Dynamics

Fabien Lionti¹*, Nicolas Gutowski²†, Sébastien Aubin³†,
Philippe Martinet¹†

¹*Université Côte d'Azur, INRIA, 2004 Rte des Lucioles, Valbonne, 06902, France.

²Université d'Angers, LERIA, 2 Bd de Lavoisier, Angers, 49000, France.

³Direction Générale de l'Armement, Techniques Terrestres, Rue de la Chédditière, Montreuil-Juigné, 49460, France.

*Corresponding author(s). E-mail(s): fabien.lionti@inria.fr;
Contributing authors: nicolas.gutowski@univ-angers.fr;
sebastien.aubin@intradef.gouv.fr; Philippe.Martinet@inria.fr;
†These authors contributed equally to this work.

**Abstract**

Simulating long-horizon state variable trajectories using neural network models is a challenging task due to error accumulation from auto-regressive predictions. Poor generalization exacerbates this issue, particularly under noisy and disturbed data conditions. This paper addresses these challenges by introducing a robust methodology that combines physics-based modeling with data-driven approaches to improve long-horizon simulation accuracy. Specifically, we propose a custom loss function designed to minimize auto-regressive prediction errors across extended time horizons, ensuring alignment between predicted trajectories and actual state measurements. Additionally, we leverage transfer learning by pretraining on simulated data generated from a dynamical model, where parameters are estimated using Bayesian inference, to incorporate prior physical knowledge into the learning process. Our methodology is demonstrated on the lateral dynamics of a vehicle using real-world noisy measurements, achieving competitive performance compared to traditional methods. This work highlights the effectiveness of combining custom loss design and transfer learning to enhance long-term simulation robustness

**Keywords:** Bayesian Inference, Vehicle Dynamics, Parameter Estimation, Lateral Dynamics, Nonlinear Systems
# 1 Introduction

Accurate dynamical models are essential for performing numerical simulations, which are crucial tools in various engineering tasks. Numerical simulations enable engineers to predict and analyze the behavior of complex systems under different conditions without the need for extensive real world experiments. This capability is essential for designing and optimizing engineering systems, performing safety assessments, and developing control strategies (Hu et al., 2021). However, obtaining accurate dynamical models that can provide reliable long-horizon simulations remains a challenging problem (Jiménez-Guarneros et al., 2017). The complexity of real-world systems, coupled with uncertainties in model parameters and external disturbances, often leads to significant deviations between simulated and actual system behaviors over extended time horizon due to poor generalization capabilities and increasing error propagation over time. Physics-based approaches to dynamical system modeling involve deriving equations that describe system behavior based on first principles. However, designing these models manually can be tedious, as it requires a deep understanding of the involved physical principles. To make the problem tractable, these models often rely on simplifying assumptions that introduce biases (Karniadakis et al., 2021), thereby limiting their accuracy and applicability in real-world scenarios, particularly for long-horizon simulations in the context of vehicle dynamics (Pacejka, 2006). Despite these limitations, Physic-based approaches have the advantage of relying on small amounts of data to estimate a limited number of parameters that are clear, interpretable, and fully describe the model behavior. On the other hand, learning-based approaches represent a general-purpose method for modeling dynamical systems. These methods have high modeling capabilities and can capture complex system behaviors with relatively few assumptions. This flexibility allows them to adapt to a wide range of applications and system dynamics. However, this flexibility also comes with a significant risk of overfitting (Vapnik, 1999) and generating solutions that do not adhere to physical principles outside the training dataset domain. Obtaining such models requires datasets that cover the entire state and control space of the system. This dependence on exhaustive datasets can be a limitation in scenarios where data is scarce or expensive to obtain. The field of physics-guided machine learning (Karniadakis et al., 2021) offers a promising approach by combining the strengths of both physics-based modeling and data-driven methods. This synergy aims to achieve accurate dynamical models, offering improved performance in scenarios with small data or unbalanced distributions. However, efficiently merging both sources of information can be challenging, as significant biases in prior knowledge can contribute to degrade the overall performance of the final model. Each previously mentioned modeling method relies on data that, in a real-world context, involves dealing with noisy, disturbed, and unevenly spread measurements. These challenges make it difficult to capture accurate system behavior. Consequently, the scope of this paper is twofold: first, to propose a robust methodology for training a neural networks architecture on noisy and disturbed data acquisition effectively; and second, to ensure enhanced generalization capabilities for long-horizon simulations by leveraging prior knowledge about the underlying physics in context of the vehicle lateral dynamic. The contributions presented in this paper are as follows:
1. An approach is introduced to integrate prior physical knowledge into the neural network training process using transfer learning techniques. This enables the transfer of knowledge from a source dataset, obtained from simulations of a known physical model, to a target dataset comprising real-world data.

2. A fully-differentiable auto-regressive loss function is proposed during the fine-tuning process to ensure robust training of the neural network.

3. Simulation data is generated using a physics-based model with parameters previously estimated from real-world measurements. Bayesian inference techniques are employed to incorporate prior knowledge at the parameter level of the physical model during estimation.

4. A comprehensive evaluation is provided demonstrating that the proposed method outperforms existing approaches in long-term simulation.

## 2 Related work

System identification is a longstanding problem in the field of systems engineering. In the case of nonlinear dynamical systems, it involves solving ill-posed inverse problems due to the multiplicity of likely parameter sets for physical models (Ljung, 1999). Identifying these parameters from data often involves optimizing least squares problems (Stortelder, 1998), which is equivalent to finding the set of parameters that maximizes the likelihood of aligning predicted state variable trajectories with system observations under the Gaussian error. In the case of poorly informative data, the multiplicity of solutions can lead to parameter estimates that do not extend beyond the range of measurements. To address this challenge, Bayesian inference methods (Bretó et al., 2006) have been widely used, as they allow the incorporation of prior knowledge about parameter distributions and act as constraints to maintain physically meaningful estimates in limited initial data. Approximate Bayesian inference methods, such as the Approximate Bayesian Computation (ABC) algorithm (Marin et al., 2012), are likelihood-free techniques that provide a straightforward way to estimate posterior probabilities of model parameters when dealing with intractable likelihood functions due to nonlinear dynamical systems and non-Gaussian noise uncertainties. These methods have demonstrated their efficacy in various parameter estimation problems, ranging from modeling the lateral dynamics of vehicles (Lionti et al., 2024) to cases of calibrating image simulations in cosmology (Akeret et al., 2015). These Bayesian methods are of particular interest to us, as one of them, which we developed in (Lionti et al., 2024), serves as a component of both the first step and the second step (see Figure 1) of the hybrid method we propose in subsection 3.3. It enables the generation of source training data that represents prior physical knowledge.

Learning-based methods offer a flexible approach to directly learn complex dynamical phenomena with minimal assumptions about the underlying process. MLPs are known as universal function approximators (Pinkus, 1999) and have been successfully used for system dynamics identification (Ghahramani and Roweis, 1998). However, this flexibility requires a sufficient quantity of data covering all state and control spaces of the system to ensure generalization capabilities. Recently, ResNet architectures
(He et al., 2016) have been recognized for their similarity to Euler forward numeri-
cal integration methods for differential equations (W., 2017). This analogy offers an
opportunity to optimize neural network architectures over multi-step time integrations
using automatic differentiation frameworks.

To improve the generalization performance of learning-based methods, physics-
guided machine learning aims to incorporate prior physical knowledge into machine
learning models through various mechanisms. One classical approach is to directly
learn a model of discrepancies between handcrafted models and system measure-
ments (Brynjarsdóttir and O'Hagan, 2014). This approach enhances performance by
learning the missing physics corresponding to the discrepancies. Other methods rely
on physics-informed loss functions that constrain the learning process to adhere to
physical principles, such as energy conservation, as demonstrated in Hamiltonian neu-
ral networks (Dzamba et al., 2019). Direct incorporation of physical knowledge into
neural networks has been investigated in (Cranmer et al., 2020), where the form of
the Lagrangian equation is integrated into the neural network architecture, show-
ing improved performance over standard architectures. In (Djeumou et al., 2021)
they focus on mechanical and robotic dynamical systems governed by ordinary dif-
ferential equations, where accurate modeling is essential for control and prediction.
Their framework is applied to rigid-body systems, such as the double pendulum,
which is characterized by nonlinear and chaotic dynamics, as well as multi-body
robotic systems. These systems are modeled by integrating physics-based constraints,
which improve generalization and stability, particularly in scenarios with limited train-
ing data. The results demonstrate the effectiveness of this approach in enhancing
prediction accuracy and constraint enforcement.

In a physic-informed context, we pre-train a *MLP* on the source data generated
by our Bayesian inference method. Transfer learning (Weiss et al., 2016) involves
leveraging knowledge gained from one task learned from a source dataset to enhance
learning on a related task with a target dataset. This typically entails reusing features
or representations learned from the source task to aid learning on the target task,
which may have less data available but related characteristics. This approach has
been investigated in the context of robotics to adapt simulation domains to real-world
applications (Bousmalis et al., 2018) showing improved learning speed in context of
robotic haptic problems. This latter approach interests us as it constitutes the core of
our contribution. Once the *MLP* is pre-trained, we fine-tune it using an autoregressive
formulation during the final part of step 3 (see Figure 1), with real measurement data
collected from a *Renault Premium* truck.

3 Proposed method

3.1 Overview

In the context of Transfer Learning, the method proposed in this paper leverages a
source dataset $D_{src}$ and a target dataset $D$. The approach transfers knowledge from
simulation data $D_{src}$, generated using a nonlinear dynamical bicycle model, to a target
task of learning lateral vehicle dynamics based on real-world data $D$. This is achieved
through a robust training formulation, comprising three key steps:
* The first step involves estimating parameters $\theta$ from the bicycle model $f(x_t, u_t, \theta)$, where $x_t \in \mathbb{R}^n$ denotes the state and $u_t \in \mathbb{R}^m$ denotes the control applied at a time step $t$. $\theta$ represents a set of fixed parameters that need to be estimated, but prior beliefs about their distribution are available. This parameter estimation is performed using Bayesian inference, leveraging the system measurement dataset $\mathcal{D}$ and the prior physical model of the lateral dynamics $f$ along with its associated parameter range beliefs $P(\theta)$. In practice, to perform this Bayesian estimation, we use a sampling method: ABC-SMC (as done in (Lionti et al., 2024)). The advantage of ABC-SMC as an iterative approach is that it allows for the gradual refinement of parameter estimation. To achieve this, it samples sets of parameters and compares the model predictions with the actual data, while taking into account the uncertainties associated with both the parameters and the measurements.

* The second step involves generating simulation data $\mathcal{D}_{src}$ using $f$ with the set of parameters that maximize the posterior $P(\theta|\mathcal{D})$.

* This dataset $\mathcal{D}_{src}$ is then used to pretrain a MLP which is then fine-tuned on measurement data $\mathcal{D}$ using an autoregressive training formulation.

<figure><img src="image_1.png" /><figcaption>Fig. 1 Description of the proposed method</figcaption></figure>

## 3.2 Physical model of the vehicle lateral dynamic

The non-linear dynamical bicycle model $f(x_t, u_t, \theta)$ (Pacejka, 2006) depends on a state vector $x_t = [v_y, \dot{\psi}]^T$ composed of the lateral speed $v_y$ and the yaw rate $\dot{\psi}$. The control vector $u_t = [v_x, \delta]^T$ includes the longitudinal speed $v_x$ and the front steering angle $\delta$. The model behavior depends on a set of parameters $\theta$, which includes variables such as the vehicle's mass $m$ expressed in kg, the center of gravity position defined by the front axle $l_f$ and rear axle $l_r$ distances in meters, the moment of inertia $I_z$ expressed in kg.m$^2$, and the unitless Pacejka coefficients for the front $\{B_f, C_f, D_f, E_f\}$ and rear $\{B_r, C_r, D_r, E_r\}$ tires (Bakker et al., 1989). This model, depicted in Figure 2, assumes that the vehicle is symmetric and operates under steady-state conditions with no load transfers during maneuvers and at a constant velocity $v_x$. It also assumes that the car body motion is static, neglecting pitch and roll dynamics. The following set of
differential equations describes the rate of change of the state $\dot{x}_t$ under the influence of $u_t$:

$$ \dot{v}_y = \frac{1}{m} (F_{yf} \cos(\delta) + F_{yr} - mv_x \dot{\psi}) \quad (1) $$

$$ \ddot{\psi} = \frac{1}{I_z} (l_f F_{yf} \cos(\delta) - l_r F_r) \quad (2) $$

Tire forces $F_{yf}$ with index $k \in \{f, r\}$ corresponding to front and rear wheel are provided by the Pacejka tire model:

$$ F_{yk} = D_k \sin(C_k \tan^{-1}(B_k \alpha_k - E_k(B_k \alpha_k - \tan^{-1}(B_k \alpha_k)))) \quad (3) $$

The tire model input depends on the tire slip angle $\alpha_f$ and $\alpha_r$. Tire slip angles are computed with the following set of equations:

$$ \alpha_f = \delta - \tan^{-1} \left( \frac{v_y + l_f \dot{\psi}}{v_x} \right) \quad (4) $$

$$ \alpha_r = -\tan^{-1} \left( \frac{v_y - l_r \dot{\psi}}{v_x} \right) \quad (5) $$

In Figure 2, the response of each tire is expressed in its own local basis. For the front tire, the forces are rotated by the steering angle $\delta$, leading to $F_y$ being the projection of $F_{yf}$ on the $v_y$ axis. For the rear tire, since there is no steering, $F_r$ is directly depicted and is equivalent to $F_{yr}$.

### 3.3 Parameter identification

The first step of the proposed method involves identifying the parameters $\theta$ of the bicycle model using Bayesian inference. This approach allows to combine prior information $P(\theta)$ about the parameters with a likelihood function $P(D|\theta)$, which represents the probability of observing the dataset $D$ given the parameters $\theta$. This combination yields the posterior distribution $P(\theta|D)$, which reflects updated belief about the parameters after taking the data into account.

$$ P(\theta|D) = \frac{P(D|\theta)P(\theta)}{P(D)} \quad (6) $$

To perform this inference task, a variant of the Approximate Bayesian Computation (ABC) method known as ABC-SMC (Sequential Monte Carlo) (Toni et al., 2009) is employed. This method is based on an iterative refinement of a population of particles representing system parameter. The initial population is sampled from the prior distribution of the parameters and iteratively filtered based on a distance criterion $\rho$. At each iteration $p$, a new set of particles is obtained by resampling accepted particles from the previous iteration using an importance sampling scheme. New particles
<figure><img src="image_2.png" /><figcaption>Fig. 2 Non-linear dynamical bicycle model illustrating the forces, states, and control inputs that define the governing set of differential equations</figcaption></figure>

are then obtained by randomly perturbing the resampled particles using a kernel distribution. Following this principle, the particles progressively converge towards an approximation of the posterior distribution. The role of the distance $\rho$ is to iteratively select parameters that are likely to correspond to parameters from the posterior. For small $\epsilon$, the sets of parameters generate simulation data $D_{\theta}$ with the model $f$ that is close to $D$, approximating the posterior:

$$P(\theta|D) = \lim_{\epsilon \to 0} P(\rho(D, D_{\theta}) < \epsilon | \theta) \quad (7)$$

where $\epsilon$ represents the final threshold in a sequence of $P$ decreasing thresholds, denoted as $\{\epsilon_0, \epsilon_1, \dots, \epsilon_{P-1}, \epsilon\}$. At each iteration, these thresholds are applied to guide the selection of parameter sets. Here, $D_{\theta}$ refers to the set of predicted trajectories $\{\hat{x}_0^n, \hat{x}_1^n, \dots, \hat{x}_T^n\}$, while $D$ represents the corresponding set of measured state variable trajectories $\{x_0^n, x_1^n, \dots, x_T^n\}$. The index $n$ denotes different state variable trajectories, each spanning $T$ time steps generated using the forward Euler integration scheme.
The constant $\Delta_t$ is the time step size used in the integration :

$$ \hat{x}_{t+1}^n = \begin{cases} \hat{x}_t^n + f(\hat{x}_t^n, u_t^n, \theta)\Delta_t, & \text{for } t > 0, \\ x_0^n, & \text{for } t = 0. \end{cases} $$

The distance metric $\rho$ is defined as the Root Mean Squared Error (RMSE) between predictions and observations over $T$ steps of trajectories, averaged over all $N$ trajectories:

$$ \rho(\mathcal{D}, \mathcal{D}_{\theta}) = \frac{1}{N} \sqrt{\frac{1}{T} \sum_{t=0}^{T} \left( \frac{\hat{x}_t^n - x_t^n}{\sigma_x} \right)^2} \quad (8) $$

To avoid favoring any particular state variable, the distance is standardized by $\sigma_x$, the standard deviation computed over each state variable of $\mathcal{D}$.

## 3.4 Transfer Learning

We define the MLP architecture as follows:

$$ \dot{x}_t = \phi(x_t, u_t, \beta) \quad (9) $$

where $\beta$ represents the parameters of the MLP model. To generate simulation data for the pre-training phase, fifteen particles corresponding to the last population $P$ of the ABC-SMC that minimize $\rho$ are retained. Those particles corresponds to the subset of parameters $\theta_{MAP}$ that maximizes the posterior distribution $P(\theta|\mathcal{D})$. Using these selected particles a simulation dataset $\mathcal{D}_{src}$ is generated. This is done by randomly sampling initial state variables $x_t$ and control variables $u_t$ from $\mathcal{D}$, and then generating one step ahead predictions $x_{t+1}$ using the function $f$ with a randomly sampled $\theta$ from the set $\theta_{MAP}$. The dataset $\mathcal{D}_{src}$ thus captures the variability in predictions from $f$ that corresponds to uncertainty in maximum a posteriori estimate.

During both pre-training and fine-tuning of the MLP architecture, the optimization is based on the following loss function:

$$ \mathcal{L}(\beta) = \frac{1}{T} \sum_{t=0}^{T} (\hat{x}_t - x_t)^2 \quad (10) $$

In the pre-training phase, as $\mathcal{D}_{src}$ does not include disturbances the MLP is optimized for a single time step by setting $T = 1$. In the fine-tuning phase, the MLP starts from the parameterization obtained from the pre-training phase. During this phase, the training process involves optimizing the auto-regressive trajectory generation using $\phi$ based on the forward Euler integration scheme over multiple time steps $T$. This process leverages automatic differentiation over involved numerical integration operations as described in (W., 2017).

# 4 Experimental setting
## 4.1 Data acquisition

Measurement dataset $\mathcal{D}$ were acquired from a Renault Premium truck using Corre- vit S-Motion DTI optical sensors from Kistler to measure longitudinal speed $v_x$ and lateral speed $v_y$. Yaw rate $\dot{\psi}$ was measured using an Alma inertial measurement unit. Data were collected at various speeds ranging from 15 to 80 km/h on a test track by executing different maneuvers, including slalom, avoidance maneuvers, and constant radius turns. The total duration of the recorded data is approximately 20 minutes.

## 4.2 Comparison and parameter setting

Comparisons are provided between the transfer-learning method, the bicycle model, an MLP trained solely on the training measurements, and an MLP that learned the discrepancy of the bicycle model as presented in (Chen and Xiu, 2021). While direct loss regularization (such as PINNs (Karniadakis et al., 2019)) is commonly used, we opted against it due to the bicycle model’s inherent bias, which introduced conflict- ing objectives when combined with empirical data. A more detailed discussion on this choice is provided in Section 6. The dataset $\mathcal{D}$ consists of 50% of the available measurements allocated for training and estimation for each method. The remaining measurements are divided into 30% for validation and 20% for testing. Each set was created by randomly selecting samples. Each method is evaluated using different com- binations of parameters, including MLP complexities defined by the MLP depth $\times$ number of neurons per hidden layer, specifically $\{4 \times 8, 4 \times 16, 4 \times 24\}$. Additionally, various integration time in secondes for the training loss are considered $\{0.05, 0.5, 1\}$. For all MLP training, the Adam optimizer is used with a learning rate of $10^{-3}$.

## 4.3 Evaluation metric

Evaluation of each compared methods involves computation of the RMSE and MAE between measured and predicted state variable trajectories over a ten-second time horizon. Predicted trajectories are generated autoregressively with a time step size $\Delta_t = 0.05$, corresponding to $T = 200$ forward Euler steps. To compute those metrics, predictions of the state variables start from an initial state $x_0$, which corresponds to the initially measured state of the test set sequence. Predictions are then produced using the same control commands $u_t$ as the one in the test set measurements, the RMSE and MAE are defined as:

$$ \text{RMSE} = \frac{1}{M} \sqrt{\frac{1}{T} \sum_{t=0}^{T} (\hat{x}_t^m - x_t^m)^2} \qquad (11) $$

$$ \text{MAE} = \frac{1}{M} \frac{1}{T} \sum_{t=0}^{T} |\hat{x}_t^m - x_t^m| \qquad (12) $$

$M$ is set to 20 and represents the number of trajectory in the test set.
## 4.4 Prior parameter setting

For bicycle parameter estimation using ABC-SMC methods, the prior distribution for each parameter in $\theta$ is modeled as an independent Gaussian probability density function. Table 1 lists the mean $\mu$ and standard deviation $\sigma$ values associated with each Gaussian prior distribution, corresponding to each parameter in $\theta$ vector. The mass parameter $m = 18781kg$ has been directly measured using a scale. These parameters are based on domain expert knowledge in vehicle dynamics and are validated against external sources. For the Pacejka tire coefficients, the prior means are set as $B \ge 0$, $C \ge 0$, $D \ge 0$, and $E \le 0$ to avoid unrealistic tire model behavior, in line with the guidelines from (Pacejka, 2006). The prior for the mass parameter $m$ is determined from direct measurements of the vehicle. For the moment of inertia parameter $I_z$, the prior mean is calculated using the prism method, as described in (MacInnis et al., 1997).

<table><caption>**Table 1** Prior distribution parameters. The mean ($\mu$) and standard deviation ($\sigma$) are provided for each parameter.</caption><thead><tr><th>Parameter</th><th>I<sub>z</sub></th><th>l<sub>f</sub></th><th>l<sub>r</sub></th><th>B</th><th>C</th><th>D</th><th>E</th></tr></thead><tbody><tr><td>μ</td><td>150000</td><td>5</td><td>5</td><td>7.1</td><td>3</td><td>3</td><td>-2</td></tr><tr><td>σ</td><td>50000</td><td>1</td><td>1</td><td>2</td><td>2</td><td>2</td><td>1</td></tr></tbody></table>

# 5 Results

## 5.1 Bicycle model performance

The aim of this section is to evaluate the quality of the parameter estimate of the lateral dynamic bicycle model using the ABC-SMC algorithm with different integration time.

Table 2 presents the performance of the top fifteen parameter sets $\theta_{MAP}$, which maximize the posterior distribution $P(\theta|D)$. The performance of these parameters is evaluated in terms of the RMSE and MAE for each state variable, $v_y$ and $\dot{v}$, averaged over the fifteen parameter sets.

Focusing on RMSE, the results demonstrate that longer integration times used to compute the distance metric lead to a reduction in RMSE and greater stability, as evidenced by the decrease in the associated standard deviation. The MAE metric exhibits a similar tendency, showing a consistent decrease as integration time increases, further reinforcing the robustness of the approach.

Increasing the number of integration steps allows for comparing trajectories over longer time horizons using $\rho$, resulting in a distance metric that is less sensitive to noise fluctuations as the signal-to-noise ratio improves with longer trajectories. However, increasing the number of steps requires greater computing power to compute $\rho$ for each tested particle at each iteration of the ABC-SMC algorithm.
<table><caption>**Table 2** RMSE (top line) & MAE (bottom line) computed over 10-second trajectories from the test set for various training integration times. Metric values are provided for the lateral speed $v_y$ and yaw rate $\dot{\psi}$ state variables. Uncertainties are given as $\pm$ one standard deviation.</caption><thead><tr><th>Integration Time (s)</th><th><i>v</i><sub>y</sub></th><th><i>ψ</i></th></tr></thead><tbody><tr><td rowspan="2">0.05</td><td>0.1416 ± 0.0144</td><td>0.0116 ± 0.0022</td></tr><tr><td>0.11822 ± 0.01225</td><td>0.00905 ± 0.00186</td></tr><tr><td rowspan="2">0.50</td><td>0.1354 ± 0.0060</td><td>0.0101 ± 0.0011</td></tr><tr><td>0.11232 ± 0.00558</td><td>0.00784 ± 0.00089</td></tr><tr><td rowspan="2">1.00</td><td>0.1185 ± 0.0051</td><td>0.0083 ± 0.0009</td></tr><tr><td>0.09904 ± 0.00407</td><td>0.00643 ± 0.00074</td></tr></tbody></table>

## 5.2 Transfer learning performances

The previous section provides information about baseline results obtained with the non-linear bicycle model. These results are of interest because the final performance of both the transfer-learning and discrepancy modeling methods depends on the quality and variability of the predictions from this prior knowledge about the vehicle dynamics.

To ensure a fair evaluation of the hybrid methods compared, the uncertainty related to prior knowledge must be taken into account. In the context of the transfer-learning approach, this uncertainty is addressed by repeating the experiment fifteen times, each starting from different pretraining on data simulated from *f* using parameters from the set $\theta_{MAP}$.

<table><caption>**Table 3** RMSE (upper line) and MAE (lower line) computed over 10-second trajectories from the test set, for various training integration times (IT) and different MLP complexities (corresponding to the number of neurons per hidden layer). Metrics are provided for $v_y$ and $\dot{\psi}$ state variables for each compared method. Uncertainties are provided as $\pm$ one standard deviation.</caption><thead><tr><th rowspan="2">IT</th><th rowspan="2">Complexity</th><th colspan="2">Learning-based</th><th colspan="2">Discrepancy</th><th colspan="2">Transfer-learning</th></tr><tr><th><i>V</i><sub>y</sub></th><th><i>ψ</i></th><th><i>V</i><sub>y</sub></th><th><i>ψ</i></th><th><i>V</i><sub>y</sub></th><th><i>ψ</i></th></tr></thead><tbody><tr><td rowspan="4">0.05</td><td rowspan="2">8</td><td>0.07390 ± 0.00700</td><td>0.00825 ± 0.00035</td><td><strong>0.07115</strong> ± 0.00667</td><td><strong>0.00782</strong> ± 0.00043</td><td>0.07821 ± 0.02219</td><td>0.01074 ± 0.00678</td></tr><tr><td>0.060881 ± 0.00538</td><td>0.00628 ± 0.00024</td><td>0.06008 ± 0.00870</td><td><strong>0.00580</strong> ± 0.00024</td><td><strong>0.05774</strong> ± 0.00436</td><td>0.00697 ± 0.00159</td></tr><tr><td rowspan="2">16</td><td>0.20725 ± 0.23938</td><td>0.03773 ± 0.06155</td><td><strong>0.07510</strong> ± 0.02942</td><td><strong>0.01006</strong> ± 0.00853</td><td>0.27206 ± 0.34704</td><td>0.02854 ± 0.03610</td></tr><tr><td>0.12415 ± 0.12383</td><td>0.03427 ± 0.05741</td><td><strong>0.06480</strong> ± 0.03859</td><td><strong>0.00813</strong> ± 0.00909</td><td>0.13917 ± 0.24018</td><td>0.00976 ± 0.00965</td></tr><tr><td rowspan="2">24</td><td rowspan="2"></td><td>0.15261 ± 0.22186</td><td>0.02597 ± 0.04075</td><td><strong>0.07917</strong> ± 0.03554</td><td><strong>0.00976</strong> ± 0.00576</td><td>0.10662 ± 0.06833</td><td>0.01417 ± 0.01019</td></tr><tr><td>0.32931 ± 0.98362</td><td>0.06095 ± 0.19442</td><td>0.08554 ± 0.08792</td><td>0.01464 ± 0.02211</td><td><strong>0.05163</strong> ± 0.00852</td><td><strong>0.00554</strong> ± 0.00091</td></tr><tr><td rowspan="4">0.5</td><td rowspan="2">8</td><td>0.05778 ± 0.00293</td><td>0.00682 ± 0.00028</td><td>0.06174 ± 0.00478</td><td>0.00667 ± 0.00027</td><td><strong>0.05451</strong> ± 0.00235</td><td><strong>0.00634</strong> ± 0.00032</td></tr><tr><td>0.0469 ± 0.00177</td><td>0.00520 ± 0.00034</td><td>0.04960 ± 0.00251</td><td>0.00503 ± 0.00018</td><td><strong>0.04393</strong> ± 0.00151</td><td><strong>0.00489</strong> ± 0.00027</td></tr><tr><td rowspan="2">16</td><td>0.05424 ± 0.00105</td><td>0.00626 ± 0.00025</td><td>0.05978 ± 0.00400</td><td>0.00643 ± 0.00022</td><td><strong>0.05265</strong> ± 0.00085</td><td><strong>0.00578</strong> ± 0.00019</td></tr><tr><td>0.04386 ± 0.00125</td><td>0.00485 ± 0.00019</td><td>0.05023 ± 0.00362</td><td>0.00487 ± 0.00014</td><td><strong>0.04181</strong> ± 0.00098</td><td><strong>0.00444</strong> ± 0.00017</td></tr><tr><td rowspan="2">24</td><td rowspan="2"></td><td>0.05358 ± 0.00165</td><td>0.00609 ± 0.00031</td><td>0.05504 ± 0.00175</td><td>0.00641 ± 0.00036</td><td><strong>0.05142</strong> ± 0.00148</td><td><strong>0.00589</strong> ± 0.00030</td></tr><tr><td>0.04281 ± 0.00177</td><td>0.00454 ± 0.00019</td><td>0.04672 ± 0.00512</td><td>0.00486 ± 0.00030</td><td><strong>0.04052</strong> ± 0.00172</td><td><strong>0.00451</strong> ± 0.00036</td></tr><tr><td rowspan="4">1.0</td><td rowspan="2">8</td><td>0.05361 ± 0.00127</td><td>0.00669 ± 0.00028</td><td>0.05378 ± 0.00167</td><td>0.00663 ± 0.00013</td><td><strong>0.05252</strong> ± 0.00204</td><td><strong>0.00637</strong> ± 0.00035</td></tr><tr><td>0.04314 ± 0.00028</td><td>0.00520 ± 0.00028</td><td>0.04295 ± 0.00216</td><td>0.00504 ± 0.00015</td><td><strong>0.04151</strong> ± 0.00129</td><td><strong>0.00473</strong> ± 0.00032</td></tr><tr><td rowspan="2">16</td><td>0.05109 ± 0.00082</td><td>0.00630 ± 0.00020</td><td>0.05483 ± 0.00133</td><td>0.00657 ± 0.00017</td><td><strong>0.05063</strong> ± 0.00084</td><td><strong>0.00566</strong> ± 0.00012</td></tr><tr><td>0.04118 ± 0.00118</td><td>0.00485 ± 0.00022</td><td>0.04468 ± 0.00170</td><td>0.00506 ± 0.00012</td><td><strong>0.04077</strong> ± 0.00093</td><td><strong>0.00442</strong> ± 0.00013</td></tr><tr><td rowspan="2">24</td><td rowspan="2"></td><td>0.05166 ± 0.00110</td><td>0.00618 ± 0.00023</td><td>0.05439 ± 0.00238</td><td>0.00638 ± 0.00025</td><td><strong>0.05004</strong> ± 0.00094</td><td><strong>0.00569</strong> ± 0.00016</td></tr><tr><td>0.04102 ± 0.00066</td><td>0.00473 ± 0.00023</td><td>0.04483 ± 0.00149</td><td>0.00513 ± 0.00028</td><td><strong>0.04027</strong> ± 0.00099</td><td><strong>0.00446</strong> ± 0.00017</td></tr></tbody></table>
For the discrepancy modeling, the experiments are repeated fifteen times, each using a different parameterization from the set $\theta_{MAP}$ associated with the bicycle model.

In this context, Table 3 shows the performance in terms of RMSE and MAE for each state variable, considering different integration times and MLP complexities. Focusing on RMSE, for each compared method, as in the previous section, increasing integration time is associated with a reduction in the RMSE and its variability, enhancing the robustness of the loss function against noise fluctuations. Increasing the integration time results in a loss function where the deterministic trajectory error dominates over noise fluctuations. Conversely, for smaller integration times, the signal-to-noise ratio favors noise, leading to higher uncertainty in MLP parameter estimation and increased long-term prediction errors.

Additionally, increasing MLP complexity from 8 to 16 and 24 neurons per hidden layer is associated with improved performance for each method. The transfer learning method shows a smaller RMSE and standard deviation over each state variable. Discrepancy modeling shows higher RMSE compared to the learning-based method, which is our baseline as it represents the case where no additional information other than dataset $\mathcal{D}$ is used. A hypothesis for this loss of performance is that the strong bias inherent in the non-linear bicycle model, particularly for the $v_y$ state variable, increases the complexity of the corrections that the MLP must learn to reduce this bias in contrast to cases where no prior knowledge is incorporated. Figure 4a clearly illustrates this significant discrepancy. For small integration time transfer-learning provide slightly lower performance. This is likely due to the limited benefit of domain adaptation over short horizons. Since the system dynamics do not diverge significantly within 0.05 of prediction horizon during training. In this case, the prior knowledge from the source domain may introduce subtle biases rather than providing substantial improvements. Additionally, transfer learning involves a physics-informed component that can introduce over-regularization, reducing flexibility in short-term predictions. While this regularization benefits long-term stability, it may slightly hinder short-horizon adaptability.

The Figure 3 shows the evolution of the averaged RMSE over each trajectory of the test set for each state variable, for increasing lengths of autoregressive prediction (integration time). This is done for the parameter set for each method that yielded the lowest RMSE for each state variable in Table 3.

For the learning-based, transfer learning, and discrepancy methods, we observe an increase in error at the beginning of the simulation, followed by a slight attenuation and stabilization over time of the RMSE on the yaw rate state variable. This pattern seems to indicate an error accumulation due to random fluctuations in the measured trajectory that cannot be learned by any method. Over time, the properties of these random fluctuations remain constant, and the predictions of each method are valid up to a constant prediction bias, implying RMSE stabilization over time. In the case of the lateral speed state variable, we observe a similar pattern, but the RMSE tends to be in slightly increase tendency at the end of the trajectory, indicating error accumulation over time. For the bicycle model, due to its inability to capture all phenomena present in the measurements, error accumulate without any
perceptible stabilization. Figure 4 presents the trajectory simulated over a temporal horizon of 10 seconds for each method, focusing on the $v_y$ and $\dot{\psi}$ state variables during a slalom maneuver. The bicycle model demonstrates degraded predictions for each state variable, with a significant bias in $v_y$. This degradation as observed in 3 is attributed to the model's lack of degrees of freedom, as it always assumes steady-state behavior, failing to account for lateral mass transfer. In contrast, our transfer learning method, which leverages knowledge from the bicycle model, shows improvement during the high-dynamic phases of the slalom trajectory, particularly in scenarios involving mass transfer, compared to purely learning-based and discrepancy modeling methods. The Table 4 shows the relative performance gains in percentage with respect to the learning-based method. The relative gain in performance is computed as follows for each state variable: $(1 - \frac{RMSE}{RMSE_{baseline}}) \times 100$, where $RMSE_{baseline}$ corresponds to the smallest obtained RMSE of the learning-based method in Table 3 in bold. The

<figure><img src="image_3.png" /><figcaption>**Fig. 3** This figure illustrates the evolution of the RMSE for each state variable: $v_y$ (left) and $\dot{\psi}$ (right), over 10-second trajectories for the compared methods. The methods are: (1) the calibrated bicycle model with the parameter set $\theta_{MAP}$, which maximizes the posterior distribution $P(\theta|D)$ obtained via the ABC-SMC algorithm (physics-based), (2) the learning-based model, trained solely on system observations $D$, (3) the transfer-learning model, which incorporates pretraining with simulation data generated by the bicycle model using $\theta_{MAP}$, and (4) the discrepancy-learning model, which is designed to capture the missing physics.</figcaption></figure>

<table><caption>**Table 4** Relative performance gains in percentage with respect to the learning-based method</caption><thead><tr><th></th><th>$v_y$</th><th>$\dot{\psi}$</th></tr></thead><tbody><tr><td>Transfer-learning</td><td>+2.00</td><td>+7.06</td></tr><tr><td>Discrepancy</td><td>-5.26</td><td>-4.76</td></tr><tr><td>Physic-based</td><td>-131.19</td><td>-36.28</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>**Fig. 4** Trajectory simulation of 10-second duration for a slalom path. Transparent areas correspond to ±0.1σ around the predicted mean.</figcaption></figure>

physics-based model shows significant performance loss with respect to the baseline
due to previously exposed limitations. Although its predictive accuracy is far outside
the range of the learning-based method, the integration of this imperfect knowledge
offers a way to significantly improve the performance of the transfer-learning method.
The gain in performance for transfer-learning is dependent on the quality of the predic-
tive performance of the physics-based method. For the state variable $v_y$, improvement
is less significant than for $\dot{\psi}$ due to the strong bias of the bicycle model on the lateral
speed. The discrepancy method shows a global decrease in performance for each state
variable with respect to the baseline, related to the difficulty of making this method
stable in an auto-regressive training context. Implementation of our method and data
are available at https://github.com/N9TT-9G0A-B7FQ-RANC/JIS

# 6 Discussion

Although the integration of prior knowledge through physical models and their associated parameter beliefs in a transfer-learning context allows for improved predictive accuracy over a long-term prediction horizon, we did not assess the sensitivity of this improvement when using more complex physical models with better predictive capabilities, which can have impact on final performances.

We acknowledge that PINN-based regularization is a widely used approach in scientific machine learning, we opted against it in our framework due to the inherent limitations of the bicycle model. Specifically, this model does not fully capture the necessary degrees of freedom to align with real-world observations, leading to a structured bias in its predictions. Incorporating this biased model directly into the loss function as a hard constraint resulted in a multi-objective optimization problem (Deb
et al., 2016), (Letard et al., 2024), where the conflicting objectives between the phys-
ical model and empirical data degraded performance. Instead, the transfer-learning
approach allows us to leverage prior knowledge from the bicycle model in a more
flexible manner, benefiting from its initial guidance while letting the data refine the
final model without imposing rigid constraints. This ensures better generalization and
improved long-term predictive stability.

On the other side, we restricted the scope of our experimentation to MLP mod-
els that are designed to take only the current state and control as input, assuming
a memoryless property of the underlying process. Recurrent neural network (RNN)
architectures (Salehinejad et al., 2018) have demonstrated wide range of success in
time series forecasting and could present an additional layer of performance improve-
ment due to the exploitation of more powerful inductive biases in conjunction with
the use of prior knowledge through the same transfer-learning procedure described in
this paper.

In this context, RNNs, according to (Mohajerin and Waslander, 2019), require a careful memory state initialization procedure to guarantee stable long-term predictions. This requirement arises from the need to ensure continuity between the hidden states of connected trajectories during the training process. Since hidden states are not directly observable, their estimation introduces an ill-posed optimization problem, as the initial conditions for hidden states must be inferred without direct measurements. If the hidden state continuity is not properly enforced, this can lead to instability or divergence. In contrast, memoryless models, such as MLPs, operate directly on the measured full-state of the system at each time step, avoiding the need for hidden state estimation. This property simplifies the training process and ensures convergence without requiring additional constraints to maintain consistency between trajectories. Given the additional layer of complexity associated with employing recurrent architectures, and despite their strong relevance to the scientific community, we consider this topic to be outside the scope of the issues addressed in this paper. However, it represents a promising direction for future research. While this study demonstrates the use of Bayesian methods for parameter estimation, it does not address the quantification of uncertainty in the predictions. Specifically, the problem of uncertainty quantification, particularly in relation to the simulation data utilized during the pre-training phase and its subsequent impact on the finetuning phase, remains unexplored. This limitation highlights an important opportunity to leverage Bayesian methods for providing insights into predictive reliability, which could further enhance the robustness and interpretability of the model. We consider this a promising avenue for future research and have outlined it in the conclusion and future work section.

7 Conclusion and Future Work

Accurate long-term simulation of dynamical systems is critical for studying, opti-
mizing, controlling, and monitoring of process. In this work, we developed a robust
methodology to address this challenge by leveraging both system measurements and
prior knowledge about the system's physical behavior. Our method is based on trans-
fer learning from a pretrained MLP on simulation data generated by a known physical
model to a source dataset corresponding to real-world measurements of state variable evolution. The physical model includes a set of parameters estimated using Bayesian inference techniques that leverage additional prior knowledge about parameter distributions. The fine-tuning process involves an autoregressive function that improves the prediction accuracy of the final model by optimizing the MLP over many numerical integration steps, enhancing the robustness of the training against noise and disturbances in the measurements. Our experimental results on real-world measurements related to lateral vehicle dynamics demonstrate that our proposed method significantly outperforms purely data-driven models and traditional discrepancy modeling frameworks.

# 8 Acknowledgements

This research is financially supported by the Ministry of the Armed Forces through the Defense Innovation Agency (AID) and by the French National Institute for Research in Computer Science and Automation (INRIA).

# References

Akeret, J., Refregier, A., Amara, A., et al. (2015). Approximate bayesian computation for forward modeling in cosmology. *Journal of Cosmology and Astroparticle Physics*, 2015(08), 043. https://doi.org/10.1088/1475-7516/2015/08/043

Bakker, E., Pacejka, H., & Lidner, L. (1989). A new tire model with an application in vehicle dynamics studies. *SAE Transactions*, 98, 101–113. https://doi.org/10.4271/890087

Bousmalis, K., Irpan, A., Wohlhart, P., et al. (2018). Using simulation and domain adaptation to improve efficiency of deep robotic grasping. *2018 IEEE International Conference on Robotics and Automation (ICRA)*, 4243–4250. https://doi.org/10.1109/ICRA.2018.8460875

Bretó, C., Ionides, E. L., & King, A. A. (2006). Inference for nonlinear dynamical systems. *Proceedings of the National Academy of Sciences*, 103(49), 18438–18443. https://doi.org/10.1073/pnas.0603181103

Brynjarsdóttir, J., & O'Hagan, A. (2014). Learning about physical parameters: The importance of Model Discrepancy. *Inverse Problems*, 30(11), 114007. https://doi.org/10.1088/0266-5611/30/11/114007

Chen, Z., & Xiu, D. (2021). On generalized residual network for deep learning of unknown dynamical systems. *Journal of Computational Physics*, 438, 110362. https://doi.org/10.1016/j.jcp.2021.110362

Cranmer, M. D., Greydanus, S., Hoyer, S., et al. (2020). Lagrangian neural networks [Presented at ICLR Workshop 2020. Preprint at https://arxiv.org/abs/2003.04630]. arXiv preprint.

Deb, K., Hakanen, J., & Sindhya, K. (2016). Multi-objective optimization [https://link.springer.com/chapter/10.1007/0-387-28356-0_10]. In *Decision Sciences* (pp. 161–200). CRC Press.
Djeumou, F., Goubault, E., Neary, C., et al. (2021). Neural networks with physics-informed architectures and constraints for dynamical systems modeling. CoRR, abs/2109.06407. https://arxiv.org/abs/2109.06407

Dzamba, M., Greydanus, S., & Yosinski, J. (2019). Hamiltonian neural networks [https://arxiv.org/abs/1906.01563]. Advances in Neural Information Processing Systems, 32.

Ghahramani, Z., & Roweis, S. (1998). Learning nonlinear dynamical systems using an EM algorithm [https://proceedings.neurips.cc/paper_files/paper/1998/file/0ebcc77dc72360d0eb8e9504c78d38bd-Paper.pdf]. Advances in Neural Information Processing Systems, 11.

He, K., Zhang, X., Ren, S., et al. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778. https://doi.org/10.1109/CVPR.2016.90

Hu, W., Zhang, T., Deng, X., et al. (2021). Digital twin: A state-of-the-art review of its enabling technologies, applications and challenges. Journal of Intelligent Manufacturing and Special Equipment, 2(1), 1-34. https://doi.org/10.1108/jimse-12-2020-010

Jiménez-Guarneros, M., Gómez-Gil, P., Fonseca-Delgado, R., et al. (2017). Long-term prediction of a sine function using a lstm neural network. In Nature-Inspired Design of Hybrid Intelligent Systems (pp. 159-173). Springer International Publishing. https://doi.org/10.1007/978-3-319-47054-2_10

Karniadakis, G. E., Kevrekidis, I. G., Lu, L., & el al. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422-440. https://doi.org/10.1038/s42254-021-00314-5

Karniadakis, G. E., Perdikaris, P., & Raissi, M. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686-707. https://doi.org/10.1016/j.jcp.2018.10.045

Letard, A., Camp, O., Gutowski, N., et al. (2024). Bandit algorithms: A comprehensive review and their dynamic selection from a portfolio for multicriteria top-k recommendation. Expert Systems with Applications, 123151. https://doi.org/10.1016/j.eswa.2024.123151

Lionti, F., Gutowski, N., Aubin, S., et al. (2024). Bayesian approach for parameter estimation in vehicle lateral dynamics. Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17-19, 2024, Proceedings, 249-259. https://doi.org/10.1007/978-3-031-62700-2_22

Ljung, L. (1999). System identification: Theory for the user. Prentice Hall PTR.

Maclnnis, D. D., Cliff, W. E., & Ising, K. W. (1997). A comparison of moment of inertia estimation techniques for vehicle dynamics simulation. SAE Transactions, 106, 1557-1575. https://doi.org/10.4271/970951

Marin, J.-M., Pudlo, P., Robert, & al. (2012). Approximate bayesian computational methods. Statistics and Computing, 22(6), 1167-1180. https://doi.org/10.1007/s11222-011-9288-2

Mohajerin, N., & Waslander, S. L. (2019). Multistep prediction of dynamic systems with recurrent neural networks. IEEE Transactions on Neural Networks and
*Learning Systems*, 30(11), 3370–3383. https://doi.org/10.1109/TNNLS.2019.2891257

Pacejka, H. B. (2006). *Tyre and Vehicle Dynamics*. Butterworth-Heinemann. https://doi.org/10.1016/C2010-0-68548-8

Pinkus, A. (1999). Approximation theory of the mlp model in neural networks. *Acta Numerica*, 8, 143–195. https://doi.org/10.1017/S0962492900002919

Salehinejad, H., Baarbe, J., Sankar, S., et al. (2018). Recent advances in recurrent neural networks [Preprint at https://arxiv.org/abs/1801.01078]. *arXiv preprint*.

Stortelder, W. J. H. (1998). *Parameter estimation in nonlinear dynamic systems*. Centrum voor Wiskunde en Informatica. https://doi.org/10.1016/S1474-6670(17)63039-8

Toni, T., Welch, D., Strelkowa, N., et al. (2009). Approximate bayesian computation scheme for parameter inference and model selection in dynamical systems. *Journal of the Royal Society Interface*, 6(31), 187–202. https://doi.org/10.1098/rsif.2008.0172

Vapnik, V. (1999). An overview of statistical learning theory. *IEEE Transactions on Neural Networks*, 10(5), 988–999. https://doi.org/10.1109/72.788640

W., W. (2017). A proposal on machine learning via dynamical systems. *Communica-tions in Mathematics and Statistics*, 5(1), 1–11. https://doi.org/10.1007/s40304-017-0103-z

Weiss, K., Taghi, M., & Wang, D. (2016). A survey of transfer learning. *Journal of Big Data*, 3(1), 9. https://doi.org/10.1186/s40537-016-0043-6