# PINN-Based Parameters And Input Joint Estimation Of Nonlinear Systems With Non-Gaussian Noise

Mohamed Boukaf, Zehor Belkhatir, Mohamed Chadli, Taous-meriem Laleg-Kirati

► To cite this version:

Mohamed Boukaf, Zehor Belkhatir, Mohamed Chadli, Taous-meriem Laleg-Kirati. PINN-Based Pa- rameters And Input Joint Estimation Of Nonlinear Systems With Non-Gaussian Noise. ACC 2025 - American Control Conference, IEEE, Jul 2025, Denver (Colorado), United States. hal-04945073v2

HAL Id: hal-04945073

https://hal.science/hal-04945073v2

Submitted on 20 Feb 2025 (v2), last revised 7 Jan 2026 (v3)

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# PINN-Based Parameters And Input Joint Estimation Of Nonlinear Systems With Non-Gaussian Noise

Boukaf Mohamed<fnref n="1" />, Zehor Belkhatir², Mohamed Chadli ³ and Taous Meriem Laleg Kirati<fnref n="1" />

**Abstract**— This work addresses the joint estimation of inputs and parameters of nonlinear Ordinary Differential Equations (ODE) under non-Gaussian noisy measurements. Many real-world systems are subject to non-Gaussian and non-zero mean noise, which deviates from the ideal and well-used Gaussian distribution assumption. This proposed framework integrates the Physics-Informed Neural Network (PINN) framework with Energy-Based Models (EBM) to tackle the estimation problem in the presence of non-Gaussian noise. Traditional PINNs struggle with non-zero mean noise due to the conflict between data loss and ODE loss, leading to biased estimations. By introducing an EBM to model the noise distribution and incorporating this into the training of PINNs, the method achieves a consistent estimation process. The EBM is trained to capture the probability density of the residuals between the noisy measurements and PINN predictions, and the combined loss from the PINN and EBM is minimized. This approach allows for accurate modeling of the noise distribution, leading to more reliable parameters and input estimations. The proposed method is validated through an academic experiment involving nonlinear input/output ODE, demonstrating its effectiveness in accurately estimating input signals and parameters under various noise conditions.

## I. INTRODUCTION

Joint estimation of parameters and inputs in nonlinear Ordinary Differential Equations (ODEs) is crucial for accurately modeling dynamic systems across fields like biology, physics, and engineering [1]. This involves simultaneously determining system parameters and external unknown inputs, often from noisy or incomplete data. A key limitation of existing methods that solve this estimation problem is the oversimplified assumptions of linearity and Gaussian noise, leading to inaccuracies in real-world applications. Recent methods, such as Bayesian inference, machine learning, and variational approaches, aim to improve accuracy and efficiency. Bayesian methods incorporate prior knowledge and manage uncertainty by treating parameters and inputs as random variables with probability distributions. Techniques like Markov Chain Monte Carlo (MCMC) or Sequential Monte Carlo (SMC) estimate posterior distributions. In [2], [3], a Bayesian approach for parameter estimation in systems biology is proposed, showing its effectiveness with noisy, scarce data. [4] improved computational efficiency for complex models using adaptive MCMC. [5] used variational

inference for joint parameter and input estimation in non-linear ODEs, balancing accuracy and computational speed, which is essential for large datasets or real-time applications where MCMC is too slow. Optimization-based methods solve the joint estimation by minimizing a loss function, which measures the discrepancy between observed data and model predictions. Gradient-based optimization is commonly used in this context. [6], [7] proposed an optimization framework with sensitivity analysis to estimate parameters and time-varying inputs, especially for strongly nonlinear systems where traditional methods struggle. Machine learning has introduced new possibilities for joint estimation in non-linear ODEs. Neural networks, especially recurrent neural networks (RNNs) and physics-informed neural networks (PINNs), can learn parameters and system dynamics from data. [8] introduced neural ODEs (NODEs), where a neural network approximates the ODE solution, learning underlying parameters and inputs. PINNs, as developed by [?], embed ODE structures into neural networks, ensuring that learned models adhere to physical laws. Hybrid methods, like [9], combine variational inference with deep learning to improve parameter and input estimation, blending machine learning's flexibility with classical techniques. Despite recent advances, challenges remain. Non-identifiability issues arise when multiple parameter sets produce similar model outputs, making it difficult to determine accurate parameters. Computational complexity is another hurdle; iterative methods like Bayesian inference and Monte Carlo techniques are computationally intensive, particularly for high-dimensional systems, making them unsuitable for online applications. Moreover, these methods require large, high-quality datasets, leading to poor estimates when data is sparse or noisy. Sensitivity to initial conditions and model uncertainties can also result in local convergence, limiting their robustness for complex, real-world problems. These challenges underline the need for more efficient and flexible approaches for joint estimation in nonlinear ODEs. In this paper, we propose a novel approach for the joint estimation of parameters and input of nonlinear ODE based on PINN and EBM, which is robust in the presence of non-Gaussian noise. This paper is structured as follows: section 2 discusses the materials and methods used, section 3 introduces the proposed framework for joint input and parameter estimation under non-Gaussian noise, section 4 covers the experiments conducted, and section 5 concludes with future perspectives.

<footnote n="1">¹National Institute for Research in Digital Science and Technology (INRIA), Paris Saclay, France
²School of Electronics and Computer Science, University of Southampton, UK
³IBISC Lab, University Paris-Saclay Evry, France
mohamed.boukaf@inria.fr, Z.Belkhatir@soton.ac.uk, taous-meriem.laleg@inria.fr, mchadli20@gmail.com</footnote>
## II. PRELIMINARIES

### A. Physics Informed Neural Network (PINN)

Physics-Informed Neural Networks (PINNs) are a class of neural networks designed to solve problems governed by PDEs by incorporating known physical laws directly into the network's architecture and training process. The key idea is to enforce physical constraints by embedding them within the neural network's loss function, thereby reducing the reliance on large datasets and improving the model's generalization [?], [10]. Mathematically, the loss function $\mathcal{L}$ in PINNs can be expressed as $\mathcal{L} = \mathcal{L}_{\text{data}} + \mathcal{L}_{\text{physics}}$ where $\mathcal{L}_{\text{data}}$ represents the error between the neural network's prediction and the observed data, and $\mathcal{L}_{\text{physics}}$ is the term that enforces the satisfaction of the physical laws (e.g., the PDE residuals). Unsupervised learning with PINNs is also possible when data is unavailable; in such cases, the loss function is derived entirely from the governing physical laws, as discussed in [11].

### B. Energy-Based-Models

Energy-Based Models (EBMs) are a powerful method for learning probability densities from data [12]. They are commonly used for image generation and modeling [?], [?] and have been effectively applied to regression tasks. EBMs maintain the flexibility of a neural network through the following parameterization:

$$p(y|x, \hat{E}_{\theta}) = \frac{e^{-\beta \hat{E}_{\theta}}(x, y)}{Z(x, \hat{E}_{\theta})}, \quad Z(x, \hat{E}_{\theta}) = \int e^{-\beta \hat{E}_{\theta}(x, \tilde{y})} d\tilde{y}, \quad (1)$$

where $\hat{E}_{\theta}(x, y) = \hat{E}_{\theta}(x, y|\theta_{\text{EBM}})$ is the scalar output of the neural network with weights $\theta_{\text{EBM}}$, and $\beta$ is usually equal to 1. This expressive capacity, however, comes with a more complex training procedure, as the partition function $Z(x, \hat{E}_{\theta})$ is usually analytically intractable. For high-dimensional $y$, numerical evaluation is costly or infeasible, often requiring Monte Carlo methods to approximate the integral. In this report, we opt to directly minimize the negative log-likelihood (NLL), which can be expressed as:

$$\begin{aligned} \text{NLL}(\{x, y\}, \hat{E}_{\theta}) &= -\log \left( \prod_{i=1}^{N} p(y_i|x_i, \hat{E}_{\theta}) \right) \\ &= \sum_{i=1}^{N} \log Z(x_i, \hat{E}_{\theta}) + \beta \hat{E}_{\theta}(x_i, y_i). \end{aligned} \quad (2)$$

In our scenario, accurately evaluating the partition function remains feasible using numerical integration since the noise distribution is one-dimensional.

## III. MATERIALS AND METHODS

### A. Problem formulation

In this paper, We consider the following form for input/output nonlinear ordinary differential equation:

$$\mathcal{F}(\theta_a)y(t) = \mathcal{G}(\theta_b)u(t), \quad (3)$$

where $y(t)$ and $u(t) \in \mathbb{R}$ are the output and input of the nonlinear ODE, respectively. The parametric forms of the differential operators $\mathcal{F}(\theta_a)$ and $\mathcal{G}(\theta_b)$ are given. According to Eq. 3, $\theta = \{\theta_a, \theta_b\}$ are the parameters to be estimated as well as the input $u(t)$. We will refer to input estimates as $\hat{u}$, output reconstruction as $\hat{y}$, and parameters estimates as $\hat{\theta}$. One important constraint would be the possibility of analytical solving of the previous equation for $u(t)$.

**Remark 1:** For the following, we consider a sampled ODE, with a uniform sampling time $\Delta t$; thus, we consider output measurements $\{y_i^*, i = 1, ..., N\}$, contaminated with homogeneous measurement noise (static proprieties over time) $\epsilon$, where $y^* = y + \epsilon$ and $y_i^* = y^*(t_i)$, $\epsilon$ being the noise and $N$ the number of samples.

The objective of this study is to develop an unsupervised PINN approach for joint estimation of parameters and input of a nonlinear ODE. Given a nonlinear ODE, in the previous form as described in Eq. 3, and $N$ noisy output measurements, contaminated with non-Gaussian noisy, we aim to simultaneously estimate the input, $\hat{u}(t)$ and the parameters $\theta = \{\theta_a, \theta_b\}$ derived from the model. To achieve this, we will introduce a hybrid approach based on PINNs and EBMs.

### B. PINN with non-Gaussian noisy measurements

To understand why non-Gaussian noise (and especially non-zero mean noise) poses a challenge in the standard PINN framework, consider the classical PINN loss function:

$$L_{\text{tot}} = \frac{1}{N} \sum_{i=1}^{N} (\hat{y}(t_i) - y_i^*)^2 + \frac{\omega}{N'} \sum_{i=1}^{N'} f(t_i)^2, \quad (4)$$

where $f(t_i)$ refers to the ODE residuals (see Eq. 3), and $\omega$ is a weighting parameter ($\omega > 0$).

A significant difficulty in training PINNs, even with Gaussian noise, is that the data loss and ODE loss counteract each other. The data loss term encourages overfitting to each data point, while the ODE loss term drives the solution towards satisfying the ODE (the physics prior).

**Proposition 1:** Consider the previous input/output nonlinear ordinary differential equation, with a noisy output $y^*$, where the noise is a non-zero mean noise (i.e., $y = y^* + \epsilon$ with $\mu_{\epsilon} \neq 0$), let us consider the following loss function for the PINN adapted to input and parameters estimation is:

$$L_{\text{tot}} = \frac{1}{N} \sum_{i=1}^{N} (\hat{y}(t_i) + \theta_{\text{off}} - y_i^*)^2 + \frac{\omega}{N'} \sum_{i=1}^{N'} f(t_i)^2, \quad (5)$$

where $\theta_{\text{off}} = \mu_{\epsilon}, \mu$ being the mean, $N$ is the number of output samples and $N'$ the number of dynamical model samples. Thus, the previous loss function leads to correct input and parameters estimates by aligning data and physics loss as follows:

$$\begin{aligned} \lim_{N \to \infty} L_{\text{data}} &= \mathbb{E}[(\hat{y} + \theta_{\text{off}} - y - \epsilon)^2] \\ &= \mathbb{E}[(\hat{y} + \theta_{\text{off}} - y - \mu_{\epsilon})^2] + \text{Var}(\epsilon) \end{aligned} \quad (6)$$

where the optimal values $\hat{y} = y$ and $\theta_{\text{off}} = \mu_{\epsilon}$ align with the ODE loss.
First, consider a zero-mean noise distribution ($\mathbb{E}(\epsilon) = \mu_{\epsilon} = 0$) with an infinite amount of data generated according to Eq. 3. Assuming the PINN has enough capacity to accurately represent the solution, both loss terms would be minimal for $\hat{y}(t) = y(t)$: in the limit, the data loss would become:

$$ 
\begin{aligned}
\lim_{N \to \infty} L_{\text{data}} &= \mathbb{E}[(\hat{y} - y^*)] = \mathbb{E}[(\hat{y} - y - \epsilon)^2] \\
&= \mathbb{E}[(\hat{y} - y)^2] + \mathbb{E}[\epsilon^2] - 2 \times \mathbb{E}[(\hat{y} - y)] \times \mathbb{E}(\epsilon) \\
&= \mathbb{E}[(\hat{y} - y)^2] + \text{Var}(\epsilon),
\end{aligned}
\quad (7) $$

leading to $\hat{y} = y$, since $\mathbb{E}(\epsilon) = \mu_{\epsilon} = 0$ and $\mathbb{E}(\epsilon^2) = \mathbb{E}[(\epsilon - \mu_{\epsilon})^2] = \text{Var}(\epsilon)$. Consequently, the ODE loss would also be minimal (specifically zero) for $\hat{y} = y$, due to Eq. 7. However, with non-zero mean noise ($\mathbb{E}(\epsilon) = \mu_{\epsilon} \neq 0$), the mismatch between the two loss terms is not merely a finite data issue that could be addressed with adequate regularization; it has deeper implications. In the infinite data limit, we have

$$ 
\begin{aligned}
\lim_{N \to \infty} L_{\text{data}} &= \mathbb{E}[(\hat{y} - y^*)] = \mathbb{E}[(\hat{y} - y - \epsilon)^2] \\
&= \mathbb{E}[(\hat{y} - y - \mu_{\epsilon} + \mu_{\epsilon} - \epsilon)^2] \\
&= \mathbb{E}[(\hat{y} - y - \mu_{\epsilon})^2] + \mathbb{E}[(\mu_{\epsilon} - \epsilon)^2] \\
&\quad + 2 \times \mathbb{E}(\hat{y} - y - \mu_{\epsilon}) \times \mathbb{E}(\mu_{\epsilon} - \epsilon) \\
&= \mathbb{E}[(\hat{y} - y - \mu_{\epsilon})^2] + \text{Var}(\epsilon),
\end{aligned}
\quad (8) $$

where the minimizer of the data loss becomes $\hat{y} = y + \mu_{\epsilon}$, with $\mu_{\epsilon}$ denoting the mean of the noise distribution. For the ODE loss, however, the minimizer remains $\hat{y} = y$ (assuming the correct parameters $\{\theta_a, \theta_b\}$ of $\mathcal{F}$ and $\mathcal{G}$ are known). As a result, the two loss terms now counteract each other, leading the optimization process to find a compromise between them, even if the parameters $\{\theta_a, \theta_b\}$ of the ODE are known precisely. If the parameters of $\mathcal{F}$ and $\mathcal{G}$ are unknown, the optimization process will tend to converge to parameter values that yield a lower least-squares loss rather than the correct ones. One method to ensure consistency between the two losses in the infinite data limit is by introducing an offset parameter, $\theta_{off}$, to the PINN prediction $\hat{y}$ in the data loss term. This parameter $\theta_{off}$ is designed to capture the bias in the noise term. The data loss, in the limit as $N \to \infty$, is expressed as:

$$ 
\begin{aligned}
\lim_{N \to \infty} L_{\text{data}} &= \mathbb{E}[(\hat{y} + \theta_{\text{off}} - y - \epsilon)^2] \\
&= \mathbb{E}[(\hat{y} + \theta_{\text{off}} - y - \mu_{\epsilon})^2] + \text{Var}(\epsilon)
\end{aligned}
\quad (9) $$

where the optimal values $\hat{y} = y$ and $\theta_{off} = \mu_{\epsilon}$ align with the ODE loss. Thus, theoretically, the optimization process could converge to an accurate solution. However, the maximum likelihood estimator is asymptotically optimal only when the correct likelihood is used [13]. Therefore, this data loss is not the best choice, except in the case of Gaussian noise with a zero mean noise. Moreover, with finite data, outliers can disproportionately influence the least-squares loss. Consequently, explicitly considering the non-Gaussian nature of the noise could enhance both the optimization speed and the final learning outcome. In the following section, we illustrate that EBMs can be used for this purpose.

### C. Proposed EB-PINN for parameters and input joint estimation

As introduced in the previous sections, the classical PINN approach fails when dealing with non-zero mean noise. To overcome this challenge, we propose a novel framework for joint estimation of input and parameters of nonlinear ODE in the previous form, described in Eq. 3. Having $N$ noisy measurements, $\{y_i^*, i = 1, \dots, N\}$, where $y^* = y + \epsilon$, $\epsilon$ being a non-zero mean noise, from the previous nonlinear ODE described in Eq. 3, we aim to jointly estimate the unknown inputs $u$, parameters $\{\theta_a, \theta_b\}$ and noise distribution. We propose a framework for the joint estimation of parameters and input of nonlinear ODE with noise distribution. The Figure 1 describes the data pipeline. The EB-PINN (Energy-Based Physics-Informed Neural Network) training algorithm involves iteratively updating both the PINN and the Energy-Based Model (EBM) parameters to optimize their predictions. After pre-processing our inputs (i.e., model output measurements), depending on the measurements, it begins by passing the input signals (i.e., the noisy measurements $y^*$) through the parallel PINN to generate predictions for parameters $\hat{\theta}$ and system input $\hat{u}$. Differential operators $\mathcal{F}(\theta_a)$ and $\mathcal{G}(\theta_b)$ are applied to these predictions to compute the ODE residuals, which contribute to the ODE loss ($L_{ODE}$). Simultaneously, the EBM is trained on the noise estimates derived from the residuals between the noisy input signals $y^*$ and the predicted signals $\hat{y}$. This process computes the likelihood of the residuals of estimated output and noisy measurements, forming the data loss ($L_{data}$) and allowing us to estimate the noise distribution. At each iteration, the total loss is computed by combining the ODE and data loss, and the gradients are used to update the parameters of both the PINN and EBM, resulting in the following total loss:

$$ L_{\text{tot}} = L_{\text{EBM}}^{\text{data}}(\{y_d^* - \hat{y}_d\}_{mb}, \hat{E}_\theta|\theta_{\text{PINN}}, \theta_{\text{EBM}}) + \omega L_{\text{ODE}}(\mathcal{F}, \mathcal{G}, \hat{u}, \{y\}|\theta_{\text{PINN}}, \hat{\theta}), \quad (11) $$

where

$$ L_{\text{EBM}}^{\text{data}}(\cdot, \hat{E}_{\theta}) = \frac{1}{N_d'} N \text{LL}(\cdot, \hat{E}_{\theta}) = \log Z(\hat{E}_{\theta}) + \frac{1}{N_d'} \sum_{i=1}^{N_d'} \hat{E}_{\theta}(\cdot). \quad (12) $$

Note that $\hat{E}_{\theta}$ and hence the NLL is independent of $t$ because we consider homogeneous noise. Since the actual noise magnitude at each data point is unknown, we use the residuals $(y_d^* - \hat{y}_d)$ between the current PINN prediction and the measurements as our best guess. These noise estimates serve as training data for the EBM. Essentially, we use an unconditional EBM to model the PDF of the residuals. Both models are trained jointly with the loss in Eq. 11 until convergence. The estimation of noise distribution allows us to compute the estimated offset, which contributes to the loss function of the PINN model, leading to exact minimization and better estimates of parameters and inputs, which leads to better estimates of the offset in an iterative but simultaneous procedure. The algorithm runs for a specified number of iterations, continually refining the models' predictions through
<figure><img src="image_2.png" /><figcaption>Fig. 1: The proposed parallel EB-PINN (for Energy Based PINN) method.</figcaption></figure>

backpropagation. Since training both PINN and EBM in parallel is challenging, it is advantageous to start with the standard PINN loss to get a solution close to the data, then switch to the EBM loss after a waiting time (or a fixed number of iterations) to fine-tune the solution. Before using the EBM loss, initialize the EBM by training it with $\theta_{off} = 0$, for the first iterations on the current noise estimates ($y_d^* - \hat{y}_d$). Training continues for $N_{tot}$ iterations. Initializing the EBM later in the training process is beneficial for two reasons: firstly, the least-squares loss provides a good initial guess, so training the EBM on residuals from the pre-trained network gives a reasonable likelihood form. Secondly, it gives a better idea of the residuals range between the PINN prediction and data, allowing better normalization of EBM inputs for more efficient training. Algorithm 1 summarizes the training procedure.

**Algorithm 1:** Training the EB-PINN

**Input:** $N_{tot}$, $N_{EBM}$; PINN $\hat{u}(\cdot|\theta_{PINN})$; EBM $\hat{E}_{\theta}(\cdot|\theta_{EBM})$; input signals $y^*$; differential operators $\mathcal{F}(\theta_a)$, $\mathcal{G}(\theta_b)$; weighting factor $\omega$

**Output:** optimized $\theta_{PINN}$, $\theta_{EBM}$, $\{\theta, \hat{u}\}$

$i \leftarrow 0$;

**while** $i \le N_{tot}$ **do**

Initialize EBM for $N_{EBM}$ iterations;

Calculate data loss $L_{\text{data}}^{\text{EBM}}(\{y_d - \hat{y}_d\}_{m_b}, \hat{E}_{\theta})$ from (7);

if $i \ge N_{EBM}$ **then**

$\omega' \leftarrow \omega$;

Compute $\theta_{\text{off}}$;

else

$\omega' \leftarrow 1$;

$\theta_{\text{off}} \leftarrow 0$;

Compute ODE loss $L_{\text{ODE}}(\mathcal{F}, \mathcal{G}, \hat{u}, \{y\}, \hat{\theta})$;

Compute total loss $\mathcal{L}_{\text{tot}} = \mathcal{L}_{\text{data}} + \omega' \mathcal{L}_{\text{ODE}}$;

Calculate gradient $\nabla_{\theta} \mathcal{L}_{\text{tot}}$ and update

$\theta = \{\theta_{\text{PINN}}, \theta_{\text{EBM}}, \theta_{\text{off}}\}$;

$i \leftarrow i + 1$;

IV. RESULTS AND DISCUSSION

To validate the previous proposed EB-PINN architecture, an academic experiment is conducted, illustrating a dynamical

system under various noisy environments. An in-depth study of the results will allow us to understand the behavior of the trained models under various circumstances. A comparative study with a classical PINN approach is conducted to highlight the advantage of using EBMs for non-Gaussian noises.

A. System

The Duffing oscillator is a well-known example of a non-linear dynamic system. It is frequently used in physics and engineering to study the behavior of oscillatory systems with nonlinear restoring forces. The system exhibits a rich variety of dynamics, including periodic, quasi-periodic, and chaotic behavior, making it an important model for understanding nonlinear phenomena [14], [15]. The equation of motion for the Duffing oscillator is given by:

$$\ddot{x} + 2\beta\dot{x} + \alpha x + \gamma x^3 = F \cos(\omega t), \quad (13)$$

where $x$ is the displacement of the oscillator, $\beta$ is the damping coefficient, $\alpha$ is the linear stiffness coefficient, $\gamma$ is the nonlinear stiffness coefficient, $F$ is the amplitude of the external driving force, $\omega$ is the frequency of the external driving force, and $t$ is time. This equation characterizes the nonlinear behavior of the system, where the term $\gamma x^3$ introduces the nonlinearity [?]. In the context of the Duffing oscillator, the input $u(t)$ is the external driving force $F \cos(\omega t)$, and the output $y(t)$ is the displacement $x(t)$, resulting in the following nonlinear input/output ordinary order equation:

$$\ddot{y} + 2\beta\dot{y} + \alpha y + \gamma y^3 = u(t). \quad (14)$$

Let us set the parameters values as follows: $\beta = 0.1$, $\alpha = 1$, $\gamma = 5$, $F = 5$. The motivation behind this first experiment is the fact that the parameters are identifiable [?], which allows us to have strong confidence in the parameter estimates assessment. To generate our data, we used various values of frequency, varying around the resonance frequency $\omega_0 = 1$, i.e., $0.5 < \omega < 2$ with a step of $\Delta\omega = 2 \times 10^{-4}$, allowing us to generate a dataset of 7500 input signals for which we derive the corresponding outputs using Eq. 13. In this example, we used a nonlinear system where the inputs, outputs, and parameters are known. Since the data is generated numerically, we will manually introduce various types of noise to the outputs to simulate noisy
<table><caption>TABLE I: Results for experiment 1 (Duffing oscillator), comparing the performance of EB-PINN and classical PINN, using three different metrics. The results are the average obtained for the test data.</caption><thead><tr><th rowspan="2">Noise</th><th rowspan="2">Model</th><th colspan="3">Metric</th></tr><tr><th>|Δθ| (%)</th><th>RE (%)</th><th>f<sup>2</sup></th></tr></thead><tbody><tr><td rowspan="2">G<sub>0</sub></td><td>PINN</td><td>1.3</td><td>1.63</td><td>1.9</td></tr><tr><td>EB-PINN</td><td><strong>1.24</strong></td><td><strong>1.5</strong></td><td><strong>1.85</strong></td></tr><tr><td rowspan="2">U</td><td>PINN</td><td>5.6</td><td>9.92</td><td>12.9</td></tr><tr><td>EB-PINN</td><td><strong>1.59</strong></td><td><strong>1.8</strong></td><td><strong>2.3</strong></td></tr><tr><td rowspan="2">GM</td><td>PINN</td><td>7.8</td><td>11.6</td><td>15.69</td></tr><tr><td>EB-PINN</td><td><strong>1.6</strong></td><td><strong>1.9</strong></td><td><strong>2.36</strong></td></tr></tbody></table>

measurements. The first noise to be introduced is a zero-mean Gaussian noise $G_0(\epsilon) = N(\epsilon|0, 5^2)$, we will also introduce a Gaussian mixture $GM$ and uniform noise $U$, both are non-Gaussian noise with non-zero mean. The Gaussian mixture noise will take the following form: $GM(\epsilon) = [N(\epsilon|-10, 2.5^2) + N(\epsilon|-2, 5^2) + N(\epsilon|8, 3.5^2)]$, resulting in a noise with a mean of $\mu[GM(\epsilon)] = -4$, and $U(\epsilon) = U(0, 20)$, resulting in a noise with a mean of $\mu[U(\epsilon)] = 10$. To assess our results, we will introduce three metrics. Firstly, $|\Delta\theta|$, the mean absolute error between the estimated parameters $\hat{\theta}$, and the true parameters, where $n$ is the number of parameters to estimate. The second metric is the relative error $RE(\%)$ between the estimated input and the ground truth input, and finally, the sum of squared residuals (SSR): $f^2$ of the nonlinear ODE. After training the EB-PINN using the strategy outlined in Algorithm 1, the results of the trained models are summarized in Table 1. To emphasize the importance of accurately estimating the noise distribution, we compare the performance of the proposed EB-PINN with that of the classical PINN.

## B. Results

The results are presented for the trained models with the optimal architecture found and for the test data. First, we can observe that the EB-PINN performs better than the PINN in all configurations, with similar results when dealing with zero-mean Gaussian noise, which is consistent with the theoretical findings in Section III-B. When considering the Gaussian mixture, we can see that the errors in parameters and input estimation are significant for the standard PINN, which confirms the bias introduced by the non-zero mean noise. On the other hand, the EB-PINN shows comparable results to the Gaussian noise scenario, demonstrating that the Energy-Based Model effectively learns the noise distribution and the offset, leading to improved estimations. The same conclusions can be drawn for the uniform noise. Figure 2 illustrates the estimated and ground truth noise distributions for the Gaussian mixture noise and the uniform noise. The noise estimates produced by the trained EB-PINN model are highly accurate and closely align with the ground truth, enabling an excellent estimate of the noise mean $\theta_{off}$. This, in turn, facilitates the correction of the offset in the total loss function (Eq. 11), thereby improving the estimations of inputs and parameters in the presence of non-zero mean noise.

<figure><img src="image_3.png" /><figcaption>Fig. 2: Estimated noise distributions and the ground truth noise distributions associated to the previous FWK3 model.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Models performance and input estimation errors based on the weighting factor $\omega$.</figcaption></figure>

## C. Discussion

One important hyperparameter to be discussed here is the weighting parameter $\omega$. Adjusting the weighting parameter in the loss function 11 of a PINN significantly influences the model's focus between fitting empirical data and adhering to physical laws [?], [11]. To study the impact of the weighting parameter, we will vary $\omega$ from 0.1 to 10 while fixing the architecture and the other parameters to be the optimal ones. The results are shown in the Figure 3. Results in Fig 3 show that increasing this parameter emphasizes the importance of satisfying physical equations, potentially improving generalization and adherence to theoretical constraints but possibly at the cost of reduced data accuracy and potential over-constraining. On the other hand, decreasing the weighting parameter enhances the fit to observed data, which can lead to better local accuracy but risks overfitting and reduced compliance with physical laws, compromising the model's generalization. Therefore, finding an optimal balance through careful tuning is essential for achieving a model that accurately fits the data while respecting the underlying physical principles. Another way to assess the performance of the model is to monitor $\theta_{off}$ and compare it to the mean of the noise. This monitoring would allow us to assess the performance of the energy-based model solely; the results are shown in Figure 4 for the Gaussian mixture noise.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Offset monitoring as an index of the number of Epochs. The first 100 epochs where the estimated offset is equal to 0 correspond to the waiting time before the training of the Energy-Based Model.</figcaption></figure>

**Remark 2:** Note that for the first 100 epochs, the estimated mean, i.e., $\theta_{off}$, is equal to zero, which corresponds to the transition period, where the energy-based model is not in the training procedure (the waiting time). We notice an exponential convergence in both cases, with non-zero static error. For the Gaussian mixture noise, $\lim_{N \to \infty} = -3.867$, leading to a relative error of $RE(\theta_{off}) = 3.325\%$, and allowing thus, a good estimation of the noise mean and enhancing the training of the PINN.

## V. CONCLUSION

This work addresses the problem of nonlinear ODE estimations with non-zero mean noise in measurements. We first demonstrated the limitations of classical PINNs when dealing with non-zero mean noise and proposed an enhanced loss function that accounts for the characteristics of non-Gaussian noise. This enables the model to optimize both data and physics loss through the introduction of an offset parameter. We introduced a novel framework that allows accurate estimations in noisy environments. It consists of a PINN model combined with an EB model, with a suitable training strategy that ensures the simultaneous convergence of both models. To validate the proposed approach, we conducted a detailed study on the Duffing oscillator. The Duffing oscillator was chosen due to the identifiable properties of its parameters, which provide confidence in the accuracy of the estimates. A comparative analysis with the classical PINN approach highlighted the weaknesses of traditional methods when faced with unknown measurement noise. The proposed enhanced model, EB-PINN, outperformed the classical PINN in terms of both input and parameter estimation under noisy conditions. A detailed analysis of the results emphasized the importance of the weighting parameter $\omega$, which balances data loss and physics loss. Additionally, we observed that the estimated noise offset parameter $\theta_{off}$ converged close to its true value, albeit with a small static error. Future work will focus on investigating the causes of this offset and refining the model to eliminate this residual error. Through this study,

we have shown that the proposed framework, particularly the EB-PINN, is robust in handling noisy environments, effectively estimating both parameters and inputs, while classical PINN approaches struggle with unknown noise distributions. The framework's flexibility in adjusting $\omega$ and accurately estimating the noise offset $\theta_{off}$ enhances its applicability in real-world scenarios involving nonlinear dynamics.

## REFERENCES

[1] M. Boukaf, A. Akhrouf, M. Chakir, Z. Belkhatir, and T. M. Laleg-Kirati, "B-spline-based modulating function method for arterial blood flow's estimations," *IFAC-PapersOnLine*, vol. 58, no. 24, pp. 573-578, 2024.

[2] A. Raue, C. Kreutz, T. Maiwald, J. Bachmann, M. Schilling, U. Klingmüller, and J. Timmer, "Joining forces of bayesian and frequentist methodology: A study for inference in the presence of non-identifiability," *Journal of the Royal Statistical Society: Series B (Statistical Methodology)*, vol. 75, no. 1, pp. 3-32, 2013.

[3] Y. Ding, J. Gao, and M. Magdon-Ismail, "Efficient parameter inference in networked dynamical systems via steady states: A surrogate objective function approach integrating mean-field and nonlinear least squares," *Physical Review E*, vol. 109, no. 3, p. 034301, 2024.

[4] G. I. Valderrama-Bahamóndez and H. Fröhlich, "Mcmc techniques for parameter estimation of ode-based models in systems biology," *Frontiers in Applied Mathematics and Statistics*, vol. 5, p. 55, 2019.

[5] N. Malem-Shinitski, C. Ojeda, and M. Opper, "Variational bayesian inference for nonlinear hawkes process with gaussian process self-effects," *Entropy*, vol. 24, no. 3, p. 356, 2022.

[6] H. U. Voss, J. Timmer, and J. Kurths, "Nonlinear dynamical system identification from uncertain and indirect measurements," *International Journal of Bifurcation and Chaos*, vol. 14, no. 6, pp. 1905-1933, 2004.

[7] Y. Marani, I. N'Doye, and T. M. Laleg-Kirati, "Non-asymptotic neural network-based state and disturbance estimation for a class of nonlinear systems using modulating functions," in *Proceedings of the 2023 American Control Conference (ACC)*, 2023.

[8] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, "Neural ordinary differential equations," in *Advances in Neural Information Processing Systems (NeurIPS)*, vol. 31, 2018, pp. 6571-6583.

[9] C. Huang, J. Liu, and W. Fan, "Hybrid deep learning framework for parameter estimation in nonlinear ode systems," *IEEE Transactions on Neural Networks and Learning Systems*, vol. 31, no. 11, pp. 4590-4601, 2020.

[10] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, "Physics-informed machine learning," *Nature Reviews Physics*, vol. 3, no. 6, pp. 422-440, 2021.

[11] F. V. Difonzo, L. Lopez, and S. F. Pellegrino, "Physics-informed neural networks for an inverse problem in peridynamic models," *Engineering with Computers*, 2024.

[12] Y. LeCun, S. Chopra, R. Hadsell, M. A. Ranzato, and F. J. Huang, "A tutorial on energy-based learning," *The Courant Institute of Mathematical Sciences*, 2006.

[13] Y. Mao and F. Ding, "Parameter estimation for nonlinear systems by using the data filtering and the multi-innovation identification theory," *International Journal of Computer Mathematics*, vol. 93, no. 11, pp. 1869-1885, 2016.

[14] S. H. Strogatz, *Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering*. Westview Press, 1994.

[15] J. Guckenheimer and P. Holmes, *Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields*. Springer, 2013.