# Robust AI for embedded virtual sensor

Laurent Desmet, Maxime Rosano, Olivier Flebus

► To cite this version:

Laurent Desmet, Maxime Rosano, Olivier Flebus. Robust AI for embedded virtual sensor. 13th European Congress of Embedded Real Time Systems (ERTS), Feb 2026, Toulouse, France. 10.82331/ERTS.2026.10. hal-05513270

HAL Id: hal-05513270

https://hal.science/hal-05513270v1

Submitted on 19 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Robust AI for Embedded Virtual Sensors

Laurent Desmet¹, Maxime Rosano², and Olivier Flebus³

¹,²,³Schaeffler France S.A.S., 35 Boulevard des Capucines, 75002 Paris, France
laurent.desmet@vitesco.com; maxime.rosano@vitesco.com; olivier.flebus@vitesco.com

## ABSTRACT

Artificial Intelligence (AI) enables cost-effective replacement of hardware sensors with virtual ones, enabling the estimation of system states—such as electric machine (e-machine) temperatures—directly from data. This approach can significantly reduce both cost and system complexity, benefits that are particularly crucial for automotive and industrial applications. However, reliably deploying AI-driven virtual sensors in high-stakes environments brings important challenges, such as ensuring robustness to uncertainties originating from noisy data or previously unseen situations—especially in contexts where safety and compliance are essential. This paper presents and systematically compares robust AI methods specifically designed to address these sources of uncertainty. We detail strategies to quantify and manage prediction confidence, applying formal mathematical guarantees to ensure that outputs remain within acceptable bounds under input variations, and developing interpretable symbolic models to enhance transparency and validation. Our findings illustrate that the integration of these complementary techniques substantially enhances both the trustworthiness and regulatory readiness of AI-based virtual sensors, a consideration that is increasingly pertinent as new regulations like the EU AI Act shape requirements for embedded AI systems in safety-critical environments

**Keywords:** Artificial Intelligence, Machine Learning, Real-time systems, Virtual Sensors, Robust AI, AI trustworthiness, Uncertainty Quantification, Gradient Regularization, Formal Guarantees, Symbolic Regression

## 1 INTRODUCTION

By processing data from existing sensors, vehicle dynamics, or environmental inputs, virtual sensors replicate the functionality of physical sensors, thereby reducing costs, weight, and complexity. In the automotive industry, virtual sensors are increasingly common due to the push for cost-efficient designs, lightweight vehicles, and advanced driver-assistance systems (ADAS). They enable numerous functions such as real-time monitoring, predictive maintenance, integration of autonomous driving features, performance optimization, and hardware minimization to improve efficiency and scalability.

Numerous literature papers have already addressed embedded AI systems [1] [2]. Nevertheless, with the evolution of legal regulations, trustworthy AI has become critical whenever an embedded system relies on AI models. This notion

encompasses many related concepts such as explainability, transparency, and robustness [3].

This paper focuses on robustness and guarantees over noisy inputs, in the context of a virtual temperature sensor in an e-machine. While many papers have tackled adversarial attack scenarios, it often relies on pre/post processing and rely on context of use [4]. In this paper, we focus on readily deployable machine learning methods that offer varying levels of robustness to noisy inputs and do not depend on specific contexts or pre/post-processing.

Our contributions are as follows:

* We evaluate uncertainty quantification (UQ) with Bayesian neural networks to identify low-confidence predictions and enable switching to simpler models when confidence is low.

* We study sensitivity control through gradient regularization and Lipschitz neural networks to obtain smoother predictions and explicit bounds.

* We compare interpretable symbolic regression approaches (KAN and PySR) against a simple linear baseline to assess trade-offs between readability, accuracy, and analyzable behavior.

* We adopt a lightweight, common neural-network setup and a consistent evaluation protocol (MSE, smoothness, and formal output-bound ranges via *Decomon*) to ensure comparability across methods.

We briefly describe the use case and dataset, present the three method families and their training/inference setup, and report experimental results with quantitative metrics, uncertainty analyses, and verified bounds. We conclude with key trade-offs for embedded deployment, discuss limitations, and outline future work.

## 2 VIRTUAL SENSORS

Temperature estimation on e-machines is typically performed using hardware sensors, but they bring some inherent problems: calibration issues, cost, and potential breakage. Leveraging information from other sensors through a data-driven model to infer the metric of interest might provide a competitive advantage.

Twelve PT100 stator temperature sensors were installed on the e-machine test rig (figure 1), collecting data over 360 hours across 238 measured combinations of driving cycles and cooling strategies. Testing was conducted for six weeks on a physical test rig the Base+ DISCO e-motor, which is a Permanent Magnet Synchronous Motor (PMSM). The campaign covered 16 distinct driving cycles, including

<footnote>Desmet et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</footnote>
standardized WLTP Class 3 and company ‘Schaeffler CUP’
cycles.

<figure><img src="image_2.png" /><figcaption>Figure 1: Temperature sensors position in e-machine</figcaption></figure>

Our objective is to model all twelve temperature sensors. Nevertheless, to keep scope manageable and ensure comparable method evaluation, we report results for a single representative sensor (sensor 1), and we defer full multi-sensor modeling to future work. The name of the target will be referred as “XG1”.

<table><caption>Table 1: Inputs feature descriptions</caption><thead><tr><td>Variable Name</td><td>Description</td></tr></thead><tbody><tr><td>Dyno_T_ist_Sensor</td><td>Actual torque sensor value in Nm</td></tr><tr><td>Dyno_n_ist_counter</td><td>Actual engine speed counter value in rpm</td></tr><tr><td>teff</td><td>Effective current in A</td></tr><tr><td>Oel_Temperatur_Zulauf</td><td>Oil inlet temperature in °C</td></tr><tr><td>Regloplas_Volumenstrom_ist</td><td>Regloplas volume flow rate in (actual) L/min</td></tr></tbody></table>

Although the test rig provided numerous measurements from hardware sensors, we performed feature selection to retain a restricted set of inputs for the virtual-sensor models. The independent variables used for temperature prediction are described in Table 1.

## 3 COMPARED APPROACHES

This section presents the three method families evaluated in this study—Uncertainty Quantification (UQ), Formal Guarantees and sensitivity control, and Symbolic Regression. Although Scientific Machine Learning, particularly Physics Informed Neural Networks (PINNs), is a strong candidate for our problem—improving robustness by embedding governing physics (e.g., PDE residuals) into the training objective—we do not include it here. Indeed, for our e-machine use case, a consistent set of governing equations is not available, so we defer SML/PINNs to future work.

### 3.1 Uncertainty Quantification

One way to improve robustness is to enhance the capacity to identify areas where uncertainty is high. The idea is to dynamically switch between a complex machine learning model and a simpler model (such as linear regression), if available. This approach allows the system to dynamically

switch from the complex model to the simpler model when the confidence interval (CI) becomes too broad, indicating high uncertainty and potential unreliability in the complex model’s predictions. Conversely, the machine learning model is employed when the confidence interval remains narrow, ensuring high accuracy in specific or simpler scenarios.

We can identify two main difficulties here. First, the concept of “too broad” is context dependent. The acceptable margin of prediction error that allows the system to function as intended must be defined based on human expertise. In our example, the tolerance was set at ±2.5°C. Therefore, if the confidence interval (CI) at 95% exceeds 5°C, the prediction can be rejected in favor of a safer model. The second challenge is to evaluate whether the uncertainty quantification (UQ) performs as expected: ideally, the UQ should increase when the prediction becomes more difficult for the model.

In the field of UQ, there are two main sources of uncertainty. The first arises from a lack of data—that is, when predictions are made in regions of the input space where data is scarce. This type of uncertainty is known as epistemic uncertainty. The second source occurs when the input region is inherently difficult to predict due to high variability, for example, when small changes in the input space lead to large variations in the output. This is referred to as aleatoric uncertainty [5].

A Bayesian Neural Network (BNN) is a type of neural network that incorporates Bayesian inference to quantify uncertainty in its predictions. By placing probability distributions over the network weights, BNNs naturally capture epistemic uncertainty. To additionally capture aleatoric uncertainty, the network must use a heteroscedastic likelihood—for instance, by predicting both a mean and variance (outputting distribution parameters rather than a single scalar). When both modeling choices are combined, BNNs can estimate both types of uncertainty, making them particularly valuable for applications requiring robust uncertainty quantification

Since the true posterior distribution over network weights is intractable to compute exactly, we used variational inference with Flipout layers from TensorFlow Probability, optimizing the evidence lower bound (ELBO) via stochastic gradient descent during training [6].

<figure><img src="image_3.png" /><figcaption>Figure 2: Deterministic vs Stochastic Network
(TDS: Why you should use Bayesian Neural Network? -- Yeung Wong)</figcaption></figure>

Once the model is trained, the Monte Carlo method is employed to generate probabilistic predictions by sampling from the posterior distribution of the model parameters. A common question is how many samples should be drawn from this distribution. The answer depends on how stable the results are, and this must be calibrated empirically.
## 3.2 Formal Guarantees and sensitivity control

This section addresses robustness to input perturbations by controlling model sensitivity through two complementary approaches. Gradient regularization penalizes the input-output Jacobian to achieve locally smoother predictions, while Lipschitz neural networks enforce global bounds on the maximum rate of change, providing worst-case deviation guarantees under bounded noise.

### 3.2.1 Gradient Regularization

We sought methods that leverage domain expectations—specifically, that e-machine temperature varies smoothly with respect to inputs—without requiring explicit physical knowledge. Gradient regularization achieves this by penalizing the magnitude of the prediction network's derivatives with respect to its inputs, discouraging abrupt changes in model predictions. This enhances smooth outputs, reducing erratic or unstable predictions that could arise from noisy data [7].

The loss function then became:

$$L_{\theta} = \alpha * \|y - f_{\theta}(x)\|^2 + \beta * \|\nabla f_{\theta}(x)\|^2$$

Where $x$ are the inputs, $y$ are the true outputs and $f_{\theta}$ is the neural network parametrized with the weights and biases $\theta$. $\nabla f_{\theta}(x)$ represents the gradient of the neural network with respect to the inputs. Finally, $\alpha$ and $\beta$ are the respecting weight in the loss function to balance the smoothing effect and the raw performance. It is worth noting that the extra loss; while enhancing smoothness (soft constraint), it does not guarantee formal protection against abrupt change in the surface response of the neural network model.

This algorithm can be seen as a special case of Sobolev regularization where the gradient of the true value function is assumed to be zero.

### 3.2.2 Lipschitz Neural Networks

The first widely recognized paper to introduce the concept of adversarial attacks on machine learning models was “Explaining and Harnessing Adversarial Examples” by Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, published in 2014. This paper formally demonstrated how small, carefully crafted perturbations to input data—imperceptible to humans—can cause neural networks to produce incorrect predictions. It also introduced the Fast Gradient Sign Method (FGSM), a practical and efficient technique for generating such adversarial examples, which has since sparked extensive research into vulnerabilities and defenses in deep learning, as seen in illustration 1.

To guard against this kind of vulnerability, formal methods have been developed. They have ensured that predictions remain within safe limits, avoid catastrophic errors, and behave reliably even in edge cases or adversarial scenarios. While previous approaches have not provided bulletproof guarantees, the method discussed in this section offers mathematical assurances of model behavior, making such algorithms more likely to comply with strict safety regulations.

<figure><img src="image_4.png" /><figcaption>Figure 3: How imperceptible noise can lead to catastrophic deviation in prediction</figcaption></figure>

One solution relies on fulfilling Lipschitz continuity—a mathematical property that has ensured the network’s output changes are proportionally bounded relative to changes in its inputs. Thus, a Lipschitz Neural Network (LNN) is a neural network designed to satisfy this property. Specifically, a neural network is Lipschitz continuous if there exists a constant $K \in R^+$ such as for any two points in input space $(x_1, x_2)$:

$$||f_{\theta}(x_1) - f_{\theta}(x_2)|| \le K ||x_1 - x_2||$$

Where $f$ is the neural network parametrized by the weight $\theta$. $K$ quantifies the maximum rate of change of the output with respect to the input perturbation. A smaller $K$ indicates a more stable network, as it implies that small input perturbations (e.g., noise or adversarial attacks) result in proportionally small output changes. By using Spectral Normalization, one can enforce a network to be $K$ Lipschitz with norm 2 definition. For this algorithm we used DEEL library [8].

While Lipschitz constraints defend against adversarial attacks, our primary motivation is operational robustness. Temperature prediction inputs are subject to sensor noise and measurement errors in real-world conditions. Although we have not yet fully characterized this noise, domain experts estimate perturbations up to ±10%. Lipschitz continuity ensures bounded output deviations under such perturbations, preventing erratic predictions that could compromise thermal control.

## 3.3 Symbolic Regression

Symbolic regression is suitable for robust AI because it tends to produce compact and interpretable formulas that generalize well, resist attacks, perform effectively with small datasets, and can be formally verified. In an era where AI safety, alignment, and trustworthiness have become critical, symbolic regression has emerged as one of the most promising paths toward truly robust and reliable artificial intelligence [9].

### 3.3.1 Kolmogorov-Arnold-Network

We heavily used the work described in this paper [10]. The Kolmogorov-Arnold Network (KAN) is a neural network architecture inspired by the Kolmogorov-Arnold representation theorem, which states that any multivariate continuous function can be represented as a superposition of continuous univariate functions. In symbolic regression, KAN leverages this theorem by decomposing a complex target function into a sum of compositions of simpler functions, each depending on a single variable. The network typically consists
of two layers: the first layer applying learned univariate functions to the input variables, and the second layer combining these transformed inputs to approximate the target function. By structuring the network in this way, KAN efficiently captures nonlinear relationships while maintaining interpretability through its additive univariate-function decomposition.

Thus, the training procedure consists of fitting parametrized spline functions through gradient descent. Once trained, these splines have been fitted with well-known functions such as exponentials, squares, cosines, and so on. The overall process leads to an analytical formula that, while often less performant in terms of regression accuracy, has been completely interpretable and readable by humans.

<figure><img src="image_5.png" /><figcaption>Figure 4: Using KAN symbolic function to fit data</figcaption></figure>

### 3.3.2 PySR

PySR (Python Symbolic Regression) is a software package designed by Miles Cramer [11] for efficient and scalable symbolic regression, aiming to discover mathematical expressions that best fit given data. It uses a combination of evolutionary algorithms and modern optimization techniques to explore the space of symbolic expressions composed of mathematical operators and functions. PySR represents candidate models as expression trees and evolves them using mutation and crossover operations inspired by genetic programming. A key feature of PySR is its ability to leverage just-in-time compilation with Julia's modeling language, enabling fast evaluation and optimization of expressions. Additionally, PySR supports multi-objective optimization, balancing model accuracy with complexity to find simple yet accurate symbolic formulas.

## 4 EXPERIMENTAL SETUP

### 4.1 Data Processing

The data processing depends on the type of neural network architecture we intend to implement. The frequency of 1000 Hz at which the data was collected might not be necessary for training the neural network. Therefore, we downsampled to reduce the sampling rate to 1 Hz, ensuring one data point per second. A classical architecture for this kind of data would be the Long Short-Term Memory (LSTM) architecture, as it

leverages internal memory to increase prediction capability. Nevertheless, we observed that a simple multilayer perceptron (MLP) has already provided solid predictions. The reason why such a simple model fit well the problem, is because one of the features, *Oel_Temperatur_Zulauf*, has a very high correlation value (~80%), resulting into a “quite easy” problem to predict. Since we sought the lightest model to embed, we have decided to consider only the MLP architecture.

As a result, we did not consider the temporal nature of the data, and the dimensionality of the input is a simple vector of size 5, corresponding to the 5 features described in Table 1, while the output has been a single scalar.

For each model, we consistently used the same data for training and testing, meaning the same preprocessing and splitting methods.

Regarding the splitting, we had two choices:

1. Splitting at the time scale level

2. Splitting at the cycle level

We decided to proceed with option 2, as it better reflects our use case—developing a model using known cycles without knowing in advance which cycle the user will perform. To prevent similar cycles from appearing in both the training and testing sets, we performed clustering and assigned entire clusters to either the training set or the testing set. To achieve this, we leveraged the well-known Dynamic Time Warping (DTW) algorithm to compute the distances between all target profiles in the dataset. Once the distance matrix was obtained, we performed hierarchical clustering (dendrogram) to create distinct clusters.

Overall, we had 90 cycles in the training set and 30 cycles in the testing set. Some samples can be seen in figure 5.

<figure><img src="image_6.png" /><figcaption>Figure 5: Example of temperature target for training cycles (top 3) and testing cycles (bottom 3)</figcaption></figure>
Despite this splitting, we observed that the distribution in the input and output space has remained relatively similar between the training and testing sets (figure 6).

<figure><img src="image_7.png" /><figcaption>Figure 6: Distribution of inputs and outputs for training and testing set</figcaption></figure>

Before training, we applied a standard scaler to both inputs (fitted on training set and applied on testing set) and outputs (applied to the full dataset). This preprocessing step proved especially beneficial for the Lipschitz neural network.

## 4.2 Performance metrics

To provide a fair comparison, and stick with a light model, between all our machine learning models that have relied on neural network structures, we have kept the same architecture for each experiment (2 layers of 128 hidden units each with ReLU activation functions, and a single hidden unit in the output layer with a linear activation function). The loss function to minimize was the Mean Squared Error (MSE), and we used the same optimizer throughout: Adam (learning rate at 0.001). Also, an earlystopping callback has been used to stop the training when no progress was made.

To assess performance, we considered the following metrics in each of our experiments:

1. The raw performance measured by MSE.

2. The smoothness of the predicted signal, computed as the mean absolute temporal derivative (quantify fluctuations).

3. The robustness of the algorithm by computing the “Formal Bounds Spread” as the average width of the verified output interval computed by *Decomon* under inputs perturbations.

We computed each metric for every cycle and averaged the results over all cycles in the testing set, so that each cycle weighs the same.

For the final metric, we employed *Decomon*, an open-source tool available on GitHub [12], to compute formal output bounds. *Decomon* uses forward linear relaxation algorithms to derive provable upper and lower bounds on neural network predictions under input perturbations. Specifically, it represents these bounds as affine functions of the perturbed variables. Given a neural network and a bounded perturbation region (a "box" around nominal inputs), *Decomon* guarantees

that all predictions for inputs within that region will lie within the computed bounds. Tighter bounds indicate greater robustness—the model's output varies less under perturbation. We computed these bounds for each data point in the cycle, then averaged them across the cycle to obtain the final robustness metric.

This final metric has been particularly important for addressing scenarios where input reliability is uncertain. Our primary concern is that some features used by the regression algorithm may be prone to noise, potentially misrepresenting the true state of the system. This motivates our focus on quantifying how much the network's output might deviate when inputs are perturbed by noise. In our case, and after discussion with subject matter experts, we set this perturbation level up to 10%. This means we assume that each feature can deviate up to ±10% from its original value under operational conditions.

All methods have been compared with respect to a simple Vanilla Neural Network (VNN), which has served as the baseline model.

Also, despite the scaling in the preprocessing step, all performance graphs are computed and displayed on the original data scale by denormalizing the predictions after inference.

# 5 EXPERIMENTAL RESULTS

This section presents the experimental results obtained on the test set across all evaluated models. We report quantitative metrics, verified bounds, and uncertainty analyses to characterize accuracy, smoothness, and robustness. We begin with an overview of the results before detailing the performance of each model.

## 5.1 Overview

Our quantitative results, summarized in Table 2, highlight several observations:

<table><caption>Table 2: Summary of all results</caption><thead><tr><th>Name of algorithm</th><th>Family</th><th>MSE</th><th>Mean derivative</th><th>Formal Bounds Spread</th><th>Relative Computing Time Training / Inference</th><th>Robustness guarantees</th></tr></thead><tbody><tr><td>VNN</td><td>Baseline</td><td>2.02</td><td>0.21</td><td>4.3</td><td>1.0 / 1.0</td><td>No asset</td></tr><tr><td>BNN</td><td>Uncertainty Quantification</td><td>2.35</td><td>0.204</td><td>NA</td><td>50 / 20</td><td>Warning capacity</td></tr><tr><td>Gradient regularization</td><td>Sensitivity Control</td><td>2.15</td><td>0.17</td><td>3.2</td><td>2.0 / 1.0</td><td>Soft Constraints</td></tr><tr><td>LNN</td><td>Formal Guarantees</td><td>2.45</td><td>0.14</td><td>2.7</td><td>3.5 / 1.3</td><td>Hard Constraints</td></tr><tr><td>Linear Regression</td><td>Symbolic Regression</td><td>2.88</td><td>0.13</td><td>NA</td><td>0.01 / 0.01</td><td>White box</td></tr><tr><td>PYSR</td><td>Symbolic Regression</td><td>2.72</td><td>0.12</td><td>NA</td><td>500 / 0.01</td><td>White box</td></tr><tr><td>KAN</td><td>Symbolic Regression</td><td>6.23</td><td>0.15</td><td>NA</td><td>15 / 1.5</td><td>White box</td></tr></tbody></table>

* All algorithms except KAN performed competitively in regression accuracy, largely due to the impact of the correlated feature *Oel_Temperatur_Zulauf* (correlation ≈ 0.8), while the remaining features explained residual variability. This explains also why the Linear Regression performs well compared to neural networks algorithms.

* Among neural network-based approaches, the unconstrained Vanilla Neural Network (VNN) achieved the lowest mean squared error, reflecting its flexibility.
However, this came at the cost of higher mean derivatives and wider formal bounds, illustrating less smoothness and greater sensitivity.

* Both Gradient regularization and Lipschitz networks enhance smoothness constraints effectively, resulting in tighter bounds and more controlled deviation under perturbations, as confirmed by *Decomon's* verified bounds (illustrated by Figure 7). Between the two, it remained challenging to distinguish which yields consistently tighter bounds.

* Symbolic regression methods showed two different outcomes. While the PYSR package demonstrated smooth, and comprehensive formulation, achieving performance close to linear regression, the KAN approach notably underperformed for unexplained reasons and warrants further investigation.

<figure><img src="image_8.png" /><figcaption>Figure 7: Bounds found by Decomon. VNN (green), gradient regularization (orange), and Lipschitz (blue)</figcaption></figure>

## 5.2 Bayesian Neural Network

With 50 samples and a normality assumption, we use the average as the final prediction and report a 95% uncertainty interval around it—roughly plus or minus two standard deviations. Finally, based on expert recommendation, we set a threshold of 5 degrees in terms of uncertainty. The results are shown in figure 8.

<figure><img src="image_9.png" /><figcaption>Figure 8: Evolution of absolute error with uncertainty</figcaption></figure>

This approach allowed us to identify challenging prediction scenarios, which mainly fall into two categories: either the data was out of distribution (epistemic uncertainty), or the

local input space was inherently difficult to predict (aleatoric uncertainty). In both cases, the uncertainty increased, indicating that it may be safer to switch to a “safer model

It is worth mentioning that other algorithms can also provide uncertainty quantification. We chose to focus on this approach because it employs a neural network architecture capable of handling complex problems while offering uncertainty estimates. However, it would be interesting to compare its performance with Gaussian Processes or to apply Conformal Prediction methods over the vanilla neural network (VNN), among other options data scientists might explore.

A final note regarding computational resources: the memory requirements were approximately twice as high as those of a standard neural network with the same architecture. Additionally, multiple inferences are needed simultaneously, which can be cumbersome. Nevertheless, recent research offers promising directions for implementing Bayesian Neural Networks (BNNs) efficiently in embedded systems.[13]

In our study, this situation applied to approximately 15% of the test set. The performance of the BNN was **2.35** by considering only accepted predictions.

Training this kind of model takes more time, and proved more challenging than with classical VNNs, as we observed instances of overconfident networks. A straightforward way to evaluate whether the UQ models are working as intended is to compute the correlation between prediction error and uncertainty. A high positive correlation indicates that the network is effectively aware of when it is likely to perform poorly. In our case, the value was **0.82**, which is quite satisfying. Unfortunately, we could not run this algorithm using *Decomon*, as the model is not compatible with the tool's operational requirements

## 5.3 Gradient regularization

During the training process we did a hyperparameter tuning on the weight $\alpha$ and $\beta$ to find the best combination of balance between, raw performance, and smoothing effect. When $(\alpha, \beta) = (1, 0)$, it is equivalent to the VNN. At the opposite, $(\alpha, \beta) = (0, 1)$ will provide a constant as output.

Finally, we found that $(\alpha, \beta) = (0.9, 0.1)$ was the best overall combination considering the 3 metrics. The result in terms of MSE is **2.15** with this set of hyperparameters and a spread computed by *Decomon* of **3.2**.

Some results can be seen in figure 9.

<figure><img src="image_10.png" /></figure>
<caption>Figure 9: Gradient regularization prediction versus VNN predictions for $(\alpha, \beta) = (0.9, 0.1)$</caption>

In terms of embedded integration, it is worth noting that this algorithm adds no overhead since it uses the exact same architecture as the VNN; the only difference is that the weights have been adjusted to minimize the new loss. Additionally, we did not rely on any specialized libraries to implement the extra loss term; instead, we used Keras and TensorFlow and wrote a custom training loop, which is relatively straightforward.

## 5.4 Lipschitz Neural Network

The main challenge for this kind of algorithm is to determine an appropriate value for $K$. This will depend on the dimensionality of the problem, the scale of the input and output, and how non-linear the problem is. To address this, we estimated the gradient of the response surface by analyzing the rate of change in the available data. Specifically, we computed the following for several points in the dataset (or the entire dataset if it is sufficiently small):

1. Identified 10 closest neighbors in the input space

2. Calculated the deviation in the input space (Euclidian norm) $\Delta X$

3. Calculated the deviation in the output space (Euclidian norm) $\Delta y$

4. Computed the ratio $\Delta y/\Delta X$

5. Retained the maximum ratio for each neighbor

6. Displayed histogram

This analysis helped us select an appropriate value for $K$. For instance, if the histogram contains points with very high gradient values, it means there are big variations in the surface response. As a result, it is often necessary to increase $K$ accordingly.

During the training, we noted that constraining the network led to reduced regression capabilities compared to a non-constrained network. This represents a trade-off that must be managed between raw performance and robustness guarantees. A small Lipschitz constant $K$ resulted in an over-constrained network with mediocre performance but higher robustness capabilities. Conversely, a too large $K$ may not prevent from excessive variation, limiting the regularization required to meet strict requirements.

In our case (figure 10), we can only recommend normalizing both the input and output spaces. This resulted in a small value of $K$, making a 2-Lipschitz Neural Network (LNN) suitable.

<figure><img src="image_11.png" /><figcaption>Figure 10: Histogram of maximal gradient</figcaption></figure>

To demonstrate the robustness of the LNN, we tested it against the worst-case noise (rather than random noise). On figure 11, we present a case where we added increasingly more FGSM noise on input of the testing set and compared it with original prediction (no noise added). We observed that the maximum deviation was more limited in the LNN compared to the vanilla neural network (VNN).

<figure><img src="image_12.png" /><figcaption>Figure 11: Behaviour of vanilla network (gray) vs LNN ($K=2$) (green) with Fast Signed Gradient Noise strength</figcaption></figure>

On figure 12, we show some predictions of the LNN when no noise was added.

<figure><img src="image_13.png" /><figcaption>Figure 12: Lipschitz prediction versus VNN predictions</figcaption></figure>

Our 2-LNN demonstrated reduced regression capabilities compared to the standard VNN, but it still achieved acceptable performance with an MSE of 2.45 and a spread, computed by Decomon, of only 2.7. Although the training time was relatively longer than the VNN (roughly 3.5 longer), the 2-LNN benefits from a user-friendly, plug-and-play library with excellent documentation. As for Gradient regularization training, it is worth noting that integrating the model into embedded systems incurs minimal overhead since the architecture remains the same—only the weights differ. During inference, it operates just like a vanilla neural network.

## 5.5 Kolmogorov-Arnold-Network

Unfortunately for our specific use case, the fitting performance with known function led to significant decrease in terms of predictions capability. The equation found was:

$$y = 0.1x_{3}^{2} - 0.02e^{0.54x_{3}} + 0.74x_{3} - 0.23$$

We are currently still investigating why such a poor performance. It is worth noting that KAN pruned all inputs expect the fourth input (Oel_Temperatur_Zulauf), which
made sense, as this feature alone carried roughly 80% of the information to predict our target. In terms of data volume needed, it is important to point out that providing only half of the data on the training set yielded the same results. The training time was also significantly longer than the VNN training.

## 5.6 PySR

At the difference of the KAN package, PYSR found a very simple and reasonable equation:

$$y = 0.1x_3 + 0.02\cos(x_4)$$

<figure><img src="image_14.png" /><figcaption>Figure 13: Display tree for equation</figcaption></figure>

The package balanced loss performance and equation simplicity. Each function and operator was assigned a “penalizing weight” that decreases the overall score. This approach enhances simple, sufficiently effective equations over more accurate but overly complex ones. Some examples in prediction in the test set are shown in figure 14.

<figure><img src="image_15.png" /><figcaption>Figure 14: Prediction of the symbolic regression versus linear regression</figcaption></figure>

It is worth noting that the fourth feature (Oel_Temperatur_Zulauf) was selected in its linear form, while the last feature (regloplas_Volulenstrom_ist) was first transformed using a cosine function.

Although we have no clear justification for why this transformation makes sense, it highlights a common drawback of symbolic regression: it may produce equations that lack physical interpretability, which undermines the original intent of using symbolic regression. While the resulting equation is human-readable and functions as a white box model, it remains difficult to trust if domain experts do not have confidence in its validity. Nevertheless, the equation provides the robustness guarantees we aimed for, as it can be fully

characterized, allowing us to analytically assess the range of possible values under input perturbations. In our case, the model achieved an MSE of 2.72.

# 6 CONCLUSION

This study explored four distinct modeling paradigms, each offering unique trade-offs between interpretability, predictive accuracy, and formal guarantees.

Symbolic regression methods provided complete interpretability as white-box models, requiring significantly fewer data points to converge despite longer training times. However, they exhibited lower predictive performance compared to neural network approaches. Gradient regularization training penalized abrupt changes in model predictions by constraining input-output derivatives, enhancing local smoothness and robustness to noisy data without providing formal worst-case guarantees. Lipschitz Neural Networks (LNN) offered the only approach in this study with provable bounds on model behavior, making them suitable for safety-critical applications. Finally, Bayesian Neural Networks (BNN) enabled uncertainty quantification across all methods, providing confidence measures to guide decision-making under distributional shifts or noisy data—facilitating dynamic switching between complex and simpler models based on prediction confidence

In summary, model choice depends on application priorities:

* For strict formal guarantees, LNN provides the only approach in this study with provable bounds.

* For enhanced local smoothness and reduced sensitivity to noise—without strict formal guarantees—gradient regularization provides an effective middle ground.

* For interpretability and complete analytical characterization, symbolic regression remains the preferred route.

* Meanwhile, uncertainty quantification complements all methods by offering confidence measures to guide decision-making, especially under distributional shifts or noisy data.

* Regarding training computation time, we observe that all symbolic regression algorithms (except Linear Regression) take longer to converge despite requiring less data. Among the network architectures, the Bayesian Neural Network (BNN) required the most time, followed by the Lipschitz Neural Network (LNN), and then gradient regularization. Although the overall time scale is not critical for our current tasks, it could pose challenges for more complex problems.

* For the inference computation time, all methods are either quicker or take relatively the same amount of time. Sampling multiple times over the weights for the BNN architecture is the reason why this method is longer.

One final observation about symbolic regression is that, although both packages tend to take longer to train, they require significantly fewer data points to converge effectively. While we have not conducted a detailed comparison with neural network-based algorithms to ensure a fair evaluation,
it is reasonable to except that simpler models need less data to
achieve convergence than more complex ones.

# 7 FUTURE WORK

This investigation represents an initial step toward
understanding these modeling paradigms within our domain.
Areas for future work include:

* Diagnosing the reasons behind the poor performance of KAN symbolic regression to enhance its applicability.

* Evaluating model robustness and uncertainty quantification under real out-of-distribution scenarios, which are crucial for practical deployment.

* Extending experiments to more complex datasets, more challenging prediction tasks, and diverse neural architectures to assess scalability and generalizability.

* Investigating how performance evolves with dataset size.

* Exploring alternative formal methods for comparison with *Decomon*.

Pursuing these directions will provide deeper insights and
support the development of more reliable, interpretable
predictive models tailored to real-world deployment
challenges.

# 8 ACKNOWLEDGEMENT & LIBRARIES USED

The authors would like to thank the following developers of
Python libraries:

* The *Decomon* tool was provided by the outstanding work of *Mélanie Ducoffe* and her team under the Airbus library.

* The Lipchitz library was developed in the DEEL project which is actively working on Robust AI topics and the deel-lip project from *Franck Mamalet* and his team.

* The Uncertainty Quantification was done with `tensorflow-probabilty`

* The KAN github package developed by authors in [10]

* The PySR github package developed by the author [11]

* The other popular libraries in the data science landscape (Keras, scikit-learn, pandas and so on...)

Also, we would like to deeply thank Georg Goeppert, Marcel Adrian, Tobias Moroder and Jurgen Bohn for their support and their availability during this study.

# 9 REFERENCES

[1] Zhang Z, Li J. *A Review of Artificial Intelligence in Embedded Systems*. Micromachines (Basel). 2023

[2] *Embedded AI: Algorithms and Applications*, IIP Series, Volume 3, Book 12, Part 4, Chapter 1

[3] Blazevic, R., Veledar, O., and Macher, G., *Insides to Trustworthy AI-Based Embedded Systems* SAE Technical Paper 2024-01-2014, 2024, https://doi.org/10.4271/2024-01-2014

[4] Swarnava Dey, Pallab Dasgupta, Partha P Chakrabarti *SymDNN: Simple & Effective Adversarial Robustness for Embedded Systems*, IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), https://doi.org/10.4271/2024-01-2014

[5] Hüllermeier, E., & Waegeman, W. (2021). *Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods*. Machine Learning, 110(3), 457-506.

[6] Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). *Weight Uncertainty in Neural Networks*. Proceedings of the 32nd International Conference on Machine Learning (ICML). https://arxiv.org/abs/1505.05424.

[7] Varga et al., 2018. *Gradient Regularization Improves Accuracy of Discriminative Models*. https://ar5iv.labs.arxiv.org/html/1712.09936

[8] Mathieu Serrurier, Franck Mamalet, Alberto González-Sanz, Thibaut Boissin, Jean-Michel Loubes, Eustasio del Barrio: *Achieving robustness in classification using optimal transport with hinge regularization*, Arxiv, 2020

[9] Thomas Veran, Pierre-Edouard Portier, François Fouquet: *Interpretable hierarchical symbolic regression for safety-critical systems with an application to highway crash prediction*, https://doi.org/10.1016/j.engappai.2022.105534

[10] Liu, Ziming, et al. KAN: *Kolmogorov-Arnold Networks*. Massachusetts Institute of Technology, California Institute of Technology, Northeastern University, NSF Institute for Artificial Intelligence and Fundamental Interactions, 2025.

[11] M. Cranmer, "Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl," Princeton University, Princeton, NJ, and Flatiron Institute, New York, NY, 2023.

[12] Ducoffe, M., et al. Decomon: DEep neural network CertificatiOn via linear relaxatiOn Methods. GitHub repository. https://github.com/airbus/decomon

[13] D. Bonnet et al., *Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks*, Nat. Commun., vol. 14, no. 1, Art. no. 7530, Nov. 2023, doi: 10.1038/s41467-023-43317-9.