# Getting a Better Sense of Data Drift in Dynamic Systems: Sequence-Based Deep Learning for Monitoring Slowly Evolving Degradation Processes

Tarek Berghout, Mohamed Benbouzid

► To cite this version:

Tarek Berghout, Mohamed Benbouzid. Getting a Better Sense of Data Drift in Dynamic Systems: Sequence-Based Deep Learning for Monitoring Slowly Evolving Degradation Processes. ECSA 2023 The 10th International Electronic Conference on Sensors and Applications, Nov 2023, Virtual conference, Italy. pp.70, 10.3390/ecsa-10-16229. [https://hal-05339108](https://hal-05339108)

HAL Id: hal-05339108

https://hal.univ-brest.fr/hal-05339108v1

Submitted on 30 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Proceeding Paper

# Getting a Better Sense of Data Drift in Dynamic Systems: Sequence-Based Deep Learning for Monitoring Slowly Evolving Degradation Processes †

Tarek Berghout ¹,* ID and Mohamed Benbouzid ²,³ ID

¹ Laboratory of Automation and Manufacturing Engineering, University of Batna 2, Batna 05000, Algeria

² UMR CNRS 6027 IRDL, University of Brest, 29238 Brest, France; mohamed.benbouzid@univ-brest.fr

³ Logistics Engineering College, Shanghai Maritime University, Shanghai 201306, China

* Correspondence: t.berghout@univ-batna2.dz

† Presented at the 10th International Electronic Conference on Sensors and Applications (ECSA-10), 15–30 November 2023; Available online: https://ecsa-10.sciforum.net/.

**Abstract:** Deep Learning (DL) for monitoring slowly evolving degradation processes typically involves overcoming data drift, complexity, and unavailability issues resulting from dynamic and harsh conditions and the rarity of labeled failure patterns, respectively. While degradation patterns are mostly hidden in such complex data, observation-based DL is prone to producing uncertain predictions and/or overfit models during the training process. This problem is usually caused by the insignificance of certain data representations. Therefore, and particularly due to the sequential nature of data in such a degradation process, it is necessary to consider neighboring observations to judge the accuracy of a representation or improve it. In this context, instead of providing traditional observation-based learning philosophy, this paper presents data-driven sequential mapping while additionally showing that health indices can also be represented as a vector of sequential data and not as a single regressor output changing a model’s architecture. Using a dataset generated from a mathematical model mimicking bearing degradation life cycles and responding to the aforementioned three main challenges, a comparative study built on investigating observation-based and sequence-based learning paths was conducted. According to a well-defined visual and numerical evaluation criterion, a sequence-based methodology reflects a better understanding of data representations through parameter tuning, achieving better approximation and generalization. Such results support the necessity of such a learning mechanism, especially for sequential data, dealing with some sort of correlation and degrading controversy. The files required to reproduce the findings of this work have been made publicly available.

**Keywords:** bearing; deep learning; degradation; prognostics and health management; remaining useful life; sequential data; vibration

## 1. Introduction

The monitoring of the slow degradation processes of dynamic systems under real conditions based on DL is generally a problem pertaining to the construction of a regression model where a specifically reconstructed health index needs to be predicted accurately for unseen health indicators [1]. This usually poses problem such as data drift, complexity, and unavailability [1]. The concept of drift refers to massive changes in the historical data features of a specific system lifecycle (i.e., run-to-failure data) [2]. Similarly, data complexity and unavailability refer to different kinds of distortions and the rarity of failure patterns [1]. Such distortions can be the result of the presence of noise and different outliers/anomalies in data affected by environmental conditions or physical damage propagation of the system itself. A high level of rarity of failure patterns is generally due to the fact that data are usually generated from physics-based models or accelerated aging experiments and not true

<figure><img src="image_2.png" /></figure>

### check for updates

**Citation:** Berghout, T.; Benbouzid, M. Getting a Better Sense of Data Drift in Dynamic Systems: Sequence-Based Deep Learning for Monitoring Slowly Evolving Degradation Processes. *Eng. Proc.* **2023**, *58*, 70. https://doi.org/10.3390/ecsa-10-16229

Academic Editor: Stefano Mariani

Published: 15 November 2023

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
degradation phenomena for many reasons, including financial and critical safety-related issues, reducing emulation quality with respect to reality [3].

It should be mentioned that this paper focuses on the data drift problem, while a DL model must be continuously updated to account for new changes in data and be able to generalize better for unseen samples. As a result, research gaps in this paper will be revealed based upon analysis from this perspective. Basically, DL models for health monitoring are generally constructed based on an ordinary training process consisting of mapping each observation feature separately to outputs produced at each time instant. This means that when, for instance, an observation is misrepresented due to any possible data distortion, sensor malfunction, or other disturbances, the model will automatically be affected, potentially leading to bias, misprediction, overfitting, etc. [2]. In this case, it is necessary to mitigate the negative effects of such misleading information to maintain both approximation and the generalization process of a DL model.

## 1.1. *Research Gaps*

According to the brief previous analysis, gaps in research on this topic can consequently be highlighted as follows:

1. Observation-based learning does not consider correlations between time series data, which could cause a model to produce biased results if the samples are misrepresented due to the many aforementioned reasons;

2. Observation-based learning does not reflect the actual monitoring of concept drift and its detection at some point while data are subject to continuous change.

Overall, a single observation, even if driven in chunk-by-chunk form, is not expected to carry information about neighboring samples to the learning model itself. This is a significant learning problem, especially when slowly evolving degradation process monitoring is a time series analysis problem and this fact must be considered [4].

## 1.2. *Contributions*

Based on the highlighted analysis criteria of concept drift in dynamic systems for monitoring slowly evolving degradation processes, the following contributions are made in this paper:

1. **The consideration of a sequence-based learning methodology:** One of the main solutions that this paper proposes is to follow a sequence-based learning methodology for a task such as that outlined herein. In this case, a sequence of observations of a specific length will be flattened and used as an input to the DL network. It should be mentioned that this is different from sequence-to-sequence learning presenting a series of encoding–decoding patterns and processes, as proposed in [5]. Therefore, the output of the DL regressor will be a vector instead of a single health index during sequential mapping. The tuning mechanism of the DL model will make it possible to attain a sense of the data changes and to improve their representations, taking into account the loss result.

2. **The consideration of adaptive deep learning:** An additional step of adaptive deep learning is taken into account in this case by introducing a long short-term memory (LSTM) neural network. An LSTM has a strong advantage as it allows for the consideration of correlations between driven sequences of time series data. In another way, data drift, in this case, will be addressed in two main steps: (i) a preprocessing step, where data are organized in sequences instead of as observations, and (ii) a main step, where the learning algorithm itself considers adaptive learning. This will further strengthen the learning processes and allow for more-accurate adaptive learning.

3. **Using data generated from a mathematical model:** An experiment was conducted on data generated from a mathematical model mimicking the health degradation trajectories of bearings responding to the three aforementioned health-monitoring issues of slowly evolving degradation processes. Compared to traditional observation-based DL, the experiments encourage such data mapping, especially for sequential
data with similar degradation behavior. The necessary files for reproducing the findings of this work have been made online available at [6].

### 1.3. Outlines

In attempt to ensure that the contributions of this paper are clear and well-illustrated, making this study reproducible, this paper is organized as follows: Section 1 is dedicated to data description. Section 2 is devoted to the proposed methodology. Section 3 is dedicated to the results and discussion. Finally, Section 4 concludes this work.

## 2. Materials

In this work, we used a bearing degradation dataset extracted from a mathematical model describing an exponentially growing sinusoidal wave with additive noise and distortions in an attempt to mimic real-world conditions [7]. The dataset contains two subsets dedicated to outer-race and inner-race faults, where each subset contains 20 sequences with different degradation rates. The degradation rate is defined using a different number of vibration window sizes with 30, 50, 65, 80, and 100 time cycles. For each speed profile, 4 life cycles are generated, modifying the parameters influencing the fault signatures randomly by +/- 5%. Each window has 16,348 samples, with a sampling frequency of 51.2 kHz, meaning that there are 40 life cycles in total. Row data from a single life cycle from the dataset are presented in Figure 1a, clearly showing an exponential shift in the data towards failure mode.

In this work, data were subjected to preprocessing, making it easier to extract any possible degradation signs at first glance. In this context, 11 time domain features that were the same as those used in previous work [8] (see [8], Section 2.3) were extracted. Similarly, the same denoising, outlier removing, and scaling steps applied in [8] were followed to make sure the data were ready for DL model training. Accordingly, Figure 1b is an example of extracted features from the life cycle in Figure 1a, as some degradation patterns clearly can be seen in this situation. Based on data visualization in Figure 1a,b, the health index function was defined as an exponential degradation function, better reflecting degradation mechanisms than linear trends, which also can be seen in Figure 1c.

<figure><img src="image_4.png" /><figcaption>**Figure 1.** Dataset and preprocessing: (a) raw data of a single degradation life cycle; (b) prepared data for a single degradation life cycle; (c) health index of an entire life cycle.</figcaption></figure>

## 3. Methods

In this work, a long-short term memory (LSTM) network was involved in the training process, as it is recommended for data drift and complexity problems (see Section 6.2 from [1]). In this context, for observation-based learning, a single-layer LSTM with trial-error-tuned parameters of 10 neurons, an $l_2$ regularization parameter equal to 0.01, and a learning rate of 0.01 was used. The same parameters were retained for sequence-based training, while a sequence length was fixed to six observations. The only aspect that changes in this case is the input and output layers sizes, which are altered to fit changes in data mapping and sequence length. Figure 2a is an example of an ordinary deep network
commonly used in degradation process monitoring, while Figure 2b is the new network architecture reflecting sequence-based learning.

<figure><img src="image_5.png" /><figcaption>Figure 2. DL model architecture: (a) observation-based DL model; (b) sequence-based DL model.</figcaption></figure>

## 4. Results and Discussion

In this work, from each previously mentioned speed profile, we selected three life cycles for training and one file for testing. Thus, 30 degradation profiles were used for training, and 10 degradation profiles were used for testing. This includes files of both subsets for both inner- and outer-race fault scenarios. Mini-batch size, maximum number of epochs, and iterations were fixed to 10, 1000, and 3000, respectively, for both DL networks under the same tuning mechanism with an error-trial basis. Two types of metrics were used in this case to judge the accuracy of the training process: visual and numeric. Visual metrics including curve-fitting examples, loss function behavior, and some scoring function behavior, as will be illustrated by the following numerical metrics. The numerical metrics include the root mean squared error (RMSE) in Equation (1) and the score function in Equation (2), the latter of which is usually used to evaluate data-driven models for bearing degradation analysis [9]. n, y, and ŷ are the number of samples, the desired health index, and the predicted health index, respectively. The score function penalizes early and late predictions differently to satisfy certain decision-making constraints related to maintenance planning [9]. Meanwhile, the RMSE designed to study the actual distance between a prediction reflecting a real significant measurement.

$$RMSE = \frac{1}{n} \sum_{i=1}^{n} y_i - \tilde{y}_i \qquad (1)$$

$$Score = \begin{cases} e^{-\ln(0.5)\left(\frac{100(y-\tilde{y})}{5}\right)}, & \frac{100(y-\tilde{y})}{5} < 0 \\ e^{+\ln(0.5)\left(\frac{100(y-\tilde{y})}{5}\right)}, & \frac{100(y-\tilde{y})}{20} \ge 0 \end{cases} \qquad (2)$$

Figure 3a shows the loss function behavior. For observation-based learning, we can observe faster convergence and reduced loss values. However, the model soon encounters an overfitting problem showing fluctuations in loss values (e.g., iterations 1000, 1500, and 2500). Meanwhile, despite the late convergence and slightly larger values of the loss function for sequence-based learning, the DL model shows better stability, with no signs of overfitting. The curve-fitting examples of the test set for both inner-race and outer-race profiles for a speed profile of 100 time cycles in Figure 3b,c show that the sequence-based results are closer and smoother, leading to better predictions.
<figure><img src="image_6.png" /><figcaption>**Figure 3.** Obtained results: (a) loss function behavior; (b) curve fit result example for inner-race fault degradation cycles; (c) curve fit result example for outer-race fault degradation cycles.</figcaption></figure>

Figure 4 is dedicated to addressing the behavior of the suggested scoring function. The score function was designed to explain both early prediction scores (where the percentage error is > 0) and late prediction scores (with a percentage error < 0). These predictions are related to maintenance decisions. What we see in Figure 4 are observation-based prediction scores that are further dispersed compared to the sequence-based ones when approaching a value of 1. This means that the DL model with respect to the latter has a better generalization ability (prediction using the test set). This proves the necessity of sequential learning in improving data presentation, consequently improving approximation and generalization through accurate tuning.

<figure><img src="image_7.png" /><figcaption>**Figure 4.** Score function behavior.</figcaption></figure>

For numerical evaluation, the RMSE and score results are shown in Table 1. The results encourage using sequence-based mapping when dealing with such slowly evolving degradation process rather than observation-based mapping due to the clear gap between them in terms of performance. Also, the sequence-based methodology seems less computationally expensive than the observation-based one, especially when considering the computation time results in Table 1, which confirm this information.

<table><caption>**Table 1.** Final numerical evaluation results.</caption><thead><tr><td>Method</td><td>RMSE</td><td>Score</td><td>Training Time (s)</td></tr></thead><tbody><tr><td>Sequence-based LSTM</td><td>0.0243</td><td>0.8401</td><td>16.4235</td></tr><tr><td>Observation-based LSTM</td><td>0.0246</td><td>0.7989</td><td>30.9235</td></tr></tbody></table>

## 5. Conclusions

In this work, we introduced a health index assessment experiment using deep learning under slowly evolving degradation processes. We discussed the use of a sequential learning philosophy versus the traditional observation philosophy when training a DL model for
approximating a degraded function. A bearing degradation dataset of multiple fault scenarios was used in these cases while adopting LSTM learning rules for DL model reconstruction. Many visual and numerical assessment metrics were used to evaluate the performance of the investigated approaches. The results encourage adopting the sequence-based methodology as it allows for the mitigation of mispresented observations occurring as a result of harsh operating conditions. As a future perspective, further highly dynamic systems need to be studied to solve this problem, including those with deeper architectures and targeting other problems regarding data complexity and availability, not only data drift problems. In doing so, further performance details about this methodology will be revealed.

**Author Contributions:** Conceptualization, T.B.; methodology, T.B. and M.B.; T.B. and M.B.; validation, T.B. and M.B.; formal analysis, T.B. and M.B.; investigation, T.B.; resources, T.B.; data curation, T.B. and M.B.; writing—original draft preparation, T.B.; writing—review and editing, T.B. and M.B.; All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data used in this work can be downloaded at https://figshare.com/articles/dataset/Simulated_Bearing_Degradation_Data Mat/12554690 (accessed on 14 November 2023). The necessary files for reproducing the findings of this work can be downloaded at https://doi.org/10.5281/zenodo.8142676 (accessed on 14 November 2023).

**Acknowledgments:** The authors thank Koceila, A.; Mouchaweh, M.S.; Cornez, L.; and Chiementin, X for making their simulated bearing degradation dataset public and easily accessible, enabling the results presented in this work to be produced.

**Conflicts of Interest:** The authors declare no conflict of interest.

## References

1. Berghout, T.; Benbouzid, M. A Systematic Guide for Predicting Remaining Useful Life with Machine Learning. *Electronics* **2022**, *11*, 1125. [CrossRef]

2. Khamassi, I.; Sayed-Mouchaweh, M.; Hammami, M.; Ghédira, K. Discussion and Review on Evolving Data Streams and Concept Drift Adapting. *Evol. Syst.* **2018**, *9*, 1–23. [CrossRef]

3. Lei, Y.; Li, N.; Guo, L.; Li, N.; Yan, T.; Lin, J. Machinery Health Prognostics: A Systematic Review from Data Acquisition to RUL Prediction. *Mech. Syst. Signal Process.* **2018**, *104*, 799–834. [CrossRef]

4. Ding, C.; Zhao, J.; Sun, S. Concept Drift Adaptation for Time Series Anomaly Detection via Transformer. *Neural Process. Lett.* **2023**, *55*, 2081–2101. [CrossRef]

5. Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. *Adv. Neural Inf. Process. Syst.* **2014**, *4*, 3104–3112.

6. Berghout, T. *Sequence-Based Instead of Observation-Based Deep Learning: Monitoring Slowly Evolving Degradation Processes*, Version 1.0.0; Zenodo: Geneva, Switzerland, 2023. [CrossRef]

7. Koceila, A.; Mouchaweh, M.S.; Cornez, L.; Chiementin, X. *Simulated Bearing Degradation Data*, 2020. [CrossRef]

8. Berghout, T.; Benbouzid, M.; Amirat, Y. Towards Resilient and Secure Smart Grids against PMU Adversarial Attacks: A Deep Learning-Based Robust Data Engineering Approach. *Electronics* **2023**, *12*, 2554. [CrossRef]

9. Gouriveau, R.; Medjaher, K.; Ramasso, E.; Zerhouni, N. PHM-Prognostics and Health Management De La Surveillance Au Pronostic de Défaillances de Systèmes Complexes. *Tech. l'ingénieur Fonct. Strat. la Maint.* **2013**, *9*, MT 9570 2–MT 9570 16. [CrossRef]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.