Enhancing Remaining Useful Life Prediction with Ensemble Multi-Term Fourier Graph Neural Networks
Authors: Ya Song
Laurens Bliek
Yaoxin Wu
Yingqian Zhang
Venue: Accepted by TMLR ()

Abstract: Remaining useful life (RUL) prediction is crucial in predictive maintenance. Recently, deep learning forecasting methods, especially Spatio-Temporal Graph Neural Networks (ST-GNNs), have achieved remarkable performance in RUL prediction. Most existing ST-GNNs require searching for the graph structure before utilizing GNNs to learn spatial graph representation, and they necessitate a temporal model such as LSTM to leverage the temporal dependencies in a fixed lookback window. However, such an approach has several limitations. Firstly, it demands substantial computational resources to learn graph structures for the time series data. Secondly, independently learning spatial and temporal information disregards their inherent correlation, and thirdly, capturing information within a fixed lookback window ignores long-term dependencies across the entire time series. To mitigate the issues above, instead of treating the data within the lookback window as a sequence of graphs in ST-GNN methods, we regard it as a complete graph and employ a Fourier Graph Neural Network (FGN) to learn the spatiotemporal information within this graph in the frequency space. Additionally, we create training and test graphs with varying sizes of lookback windows, enabling the model to learn both short-term and long-term dependencies and provide multiple predictions for ensemble averaging. We also consider scenarios where sensor signals exhibit multiple operation conditions and design a sequence decomposition plugin to denoise input signals, aiming to enhance the performance of FGN. We evaluate the proposed model on two benchmark datasets, demonstrating its superior performance on the RUL prediction task compared to state-of-the-art approaches.

Published in Transactions on Machine Learning Research (01/2025)
Enhancing Remaining Useful Life Prediction with Ensemble Multi-Term Fourier Graph Neural Networks
Ya Song
y.song@tue.nt
Industrial Engineering and Innovation Sciences Eindhoven Universily of Technology
Laurens Blick
LblickOlue.nl
Industrial Engineering and Innovation Sciences Eindhoven University of Technology
Yaoxin Wu
y. y.wu20lue.nl
Industrial Engineering and Innovation Sciences Findhoven University of Technology
Yingqian Zhang
ygshang ml
Industrial Engineering and Innovation Sciences Eindhoven Universily of Technology
Reviewed on OpenReview: https: // openreview. net/ form?? id= t=FjcVgmmu
Abstract
Remaining useful life (RUL) prediction crucial in predictive maintenance. Recently, deep learning forecasting methods, especially Spatio-Tempora Graph Neural Networks (ST- GNNs), have achieved remarkable performance RUL prediction. Most existing ST-GNNs require searching for the graph structure before utilizing GNNs to learn spatial graph rep- resentation, and they necessitate a temporal model such as LSTM to leverage the temporal dependencies in fixed lookback window. However, such an approach has several limita- lions. Firstly, it demands substantial computational resources to learn graph structures for the time series data. Secondly, independently learning spatial and temporal informa- tion disregards their inherent correlation, and thirdly, capturing information within a fixed lookback window ignores long-term dependencies across the entire time series. To mitigate sequence
the issues above, instead of treating the data within the lookback window of graphs in ST-GNN methods, regard it as complete graph and employ Fourier Graph Neural Network (FGN) to Iearn the spatiotemporal information within this graph in the frequency space. Additionally, we create training and test graphs with varying sizes of lookback windows, enabling the model to learn both short-term and long-term dependencies and provide multiple predictions for ensemblle averaging. We also consider scenarios where sensor signals exhibit multiple operation conditions and design sequence decomposition plugin to denoise input signals, aiming to enhance the performance of FGN. We evaluate the proposed model two benchmark datasets, demonstrating its superior performance the RUL prediction task compared to state-of-the-an approaches.
1 Introduction
The widespread adoption Sabber-hhysccll Systems (CPS) and the Internet of Things (IoT) has enabled organizations to leverage predictive maintenance techniques more effectively. This has resulted in extended equipment operational lifecycles, prevented unscheduled downtime, and decreased energy consumption. Pre- dictive maintenance is proactive approach that estimates the equipment's Remaining Useful Life (RUL).

---

Published in Transactions on Machine Learning Research (01/2025)
i.e., forecasts the time point at which equipment may fail or become ineffective in future usage. It then develops appropriate maintenance plans and procedures to ensure the equipment's reliability and continu- ous operation. Among these, forecasting RUL is considered the most significant and valuable task (Zhou ct al. 2021b). RUL prodiction typically depends on historical operational data and condition monitoring information, including sensor data, operation records, maintenance histories, and so on. Common prediction approaches include those based on physical models, data-driven methods, and hybrid methods (Ferreira & Gonçalves 2022). These models examine equipment's operational status and health to prediet probable failures. With the advancements in sensing technology and data analytics, data-driven strategies, espe- cially deep learning-basee RUL prediction, are emerging significant research and application area in engineering.
When utilizing deep learning techniques for RUL prediction, we tackle the problem as a multivariate time series regression task. Generally, we apply the sliding time window approach to generate samples of time serics data. The basic idea is to segment raw time serics data by sliding a fixed-length time window, taking the data within each time window as a sample, and then using these samples for machine learning model training and prediction. Most current deep learning models for estimating RUL primarily utilize
Convolutional Neural Networks (CNNs) (Yngg et al.] 2019 Ren et al. 2020 Xu et al.) 2022). Long Short- Term Memory (LSTMs) (Shi & Chehade 2021 Wu et al.) 2021b), and Transformers (Li et al. 2022 Zhang
et al. 2022b Jiang et al., 2023). These methods have effectively captured the temporal dependencies in
time series data. However, they are limited as they do not consider the potential interactions between sensor signals, hindering prediction models' effectiveness. To address this issue, researchers have begun to employ Spatio-Temporal Graph Neural Networks (ST-GNNs) (Jin et al. 2023). This approach involves creating graph for sensor signal data at each time point, leveraging GNNs to capture spatial information in the graph, and then using sequence models such as LSTMs to encapsulate the temporal information of a sequence of graph embeddings.
2024) show ST-GNNs can outperform traditional sequence models RUL prediction lasks, the current ST GNNs models still have the following four main drawbacks: (1) Need to learn the graph structure. Unlike the natural graph structure inherent in problems like traffic flow prediction in road networks, there is no explicit graph structure among sensor signals in RUL prediction. Therefore, it often necessitates the use of graph structure learning methods, demanding substante antial computational resources. (2) Modeling spatial and lemporal information seporately. The conventional ST-GNNs separately employ GNN to capture spatial information and LSTM to capture temporal information. This technique fails to consider the possible spatiotemporal deperdeepessscccss present in sensor signals. (3) Fized and short-lerm dependency modelling. Traditional models employ a single fixed lookback window to generate samples, often resulting in a window size too small for longer time series data. Consequently, the model struggles to capture long-term dependencies across the entire time series. (4) Ignore operation condition information. Equipment may operate under various operation conditions, and analyzing historical operational records can facilitate the learning of potential degradation trends. Existing models solcly focus on modcling sensor signals, disregarding operation condition information.
Although the experimental results in KKng et al. 2022 Wang et al. 2023a
To tackle the above problems, we propose an RUL prediction model named Ensemble Multi-Term Fourier Graph Neural Networks (MT-FGNE). The characteristics and advantages of this model are as follows:
We adopt novel approach to time series processing. We longer view a sample as a sequence of graphs like ST-GNNs; instead, we consider samples complete graphs. After converting it to the frequency domain using the Discrete Fourier Transform (DFT), we utilize Fourier Graph Neural Networks (FGN) to capture the degradation trends. This approach avoids separately modeling spa- tial and temporal information, enabling the learning of potential spatiotemporal interdependencies within sensor signal data.
We propose a multi-term learning framework to address the issue of traditional models' inadequate learning of long-term dependencics. In this framework, we generate training and test graphs with variable lookback window sizes, allowing the model to learn both short-term and long-term depen- dencies while providing multiple predictions for ensemble averaging.
2

---

Published in Transactions on Machine Learning Research (01/2025)
We consider the specificity of degradation under multiple operation conditions by incorporating historical operational recording data into the modeling process. After decomposing and interpolating the original signals, we imput them into the model and average the prediction results.
We evaluate our MT-FGNE model on two widely studied benchmarks and achieve competitive performance against state-of-the-art ST-GNN methods.
2 Related Work
2.1 Deep learning models for RUL prediction
Due to their ability to handle complex nonlinear relationships and perform end-to-end learning, deep learning models have been extensively applied to RUL prediction tasks over the past decade. CNNs excel at extracting local temporal patterns from time series data, while LSTMs specialize in capturing long-term dependencies through their galing mechanisms.. The early models consisted mainly of CNNs (Yang el al.) 2019 Ren al.
2020 Xu et al. 2022) and LSTMs (Da Costa et al. 2019 Shi & Chehade 2021 Wu al. 2021b). Hybrid models combining both have also been widely utilized, leveraging their capability to extract both spatial and temporal features (Zribi et al. 2021: Ren et al. 2020). Transformer architectures, leveraging their self- attention mechanisms, can model relationships between all positions in the input sequence simultaneously, enabling more effective capture of both long-term dependencies and global patterns in time series data. Rescarchers have also applied them to RUL prodiction and made various improvements the attention mechanism (Li et al. 2022 Zhang et. al. 2022b Jiang et. al. 2023
The emergence of GNNs brought new perspectives to spatial information modeling in RUL prediction. Researchers proposed to represent sensor signals as graphs, where each time step corresponds to graph structure, enabling effective spatial feature learning through various GNN architectures. The main GNN architectures include Graph Convolutional Network (GCN) (Wang et al. 2021 2023a), Graph Attention Network (GAT) (Zhang et al. 2022b; Kong et al. 2022 and custom Message Passing Neural Networks (MPNNs) [Wang et al.] 2023b). terms of graph construction, one study generates graphs based on domain knowledge Kong et al. 2022), while another calculates adjacency matrices based on Pearson Correlation Coefficients among sensors (Wang al. 2021). Recent rescarch prefers to apply graph structure lcarning approaches. Chen & Zeng (2023) construct the graph structure by computing the cosine similarity between the embedding vectors outputted by GAT. In (Wang et al. 2023a), a dynamic graph learning module is proposed to capture the dynamic relationships between sensor data and generate multi-scale structural insights by dividing data into segments within a lookback window.
In summary, existing ST-GNN methods commonly view samples as sequences of graphs, often requiring graph construction and learning on the original data in the time domain. The FGN method employed in this paper differs significantly in treating samples complete graphs, eliminating the need to learn graph structures, and transforming the samples into the frequency domain through DFT.
2.2 Multi-term learning
The closest idea to our proposed multi-term learning is multi-scale learning. Multi-scale learning is com- monly employed in a variety of domains and tasks, including computer vision and time series data analysis. By employing multi-scale learning, CNNs can extract features from receptive fields of different sizes simul- taneously, enabling more comprehensive capture of information in images, including both local details and global structures {Cai et al. 2016). CiossViT (Chen et al., 2021a) divides input images into multiple patches of different sizes and employs multi-scale feature fusion mechanism to integrate feature representa- tions from different scales. As time series data typically contains patterns and trends at multiple time scales, adopting multi-scale learning in time series analysis helps models gain a more comprehensive understanding and capture structural information within the data (Cui et al.) 2016). A classical approach is to apply multi-scale convolution, which generates feature maps at different scales to capture information along the time axis (Chen & Shi. 2021 Chen et al.) 2021b). Chen et al. (2023) apply a multi-scale pyramid network to preserve the various temporal dependencies. The model's input remains at equal scales in the aforemen-
3

---

Published in Transactions on Machine Learning Research (01/2025)
tioned multi-scale methods. In contrast, multi-term learning approach generates imputs of varying scales by employing different. lookback window sizes. Some samples contain long-term dependency information, while others only contain short-term information. In the following sections, we will go into the details of the proposed multi-term learning framework.
3
Method
The overview of MT-FGNE shown in Figure 1 which consists of two main components and one plugin. The first component FGN, which an individual model to learn spatial and temporal dependencies. The other component is multi-term ensemble learning framework, which constructs samples at different scales to enable the model to abstract both short-term and long-term dependencies. Considering that some sensor signals generated when the equipment operates under various operation conditions, we designed time series decomposition plugin to enhance the model's performance for such inpuls.
1DPT
FNN
Preduction
K-rress
Long Sequence
Phombble
IDFT
ENN0
Prodiction
Short Sequenoe
Prvemble
IDP
FNN
Soguenoe coomppoition plugin
Multi-lem samples
FGN
Figure 1: The overall framework of the proposed MT-FGNE. MT-FGNE, a multi-term ensemble learning strategy applied construct samples different scales to enable the model to abstract both short-term and long-term dependencies, and multiple FGNs are used learn spatial and temporal dependencies within multi-term samples. Besides, sequence decomposition plugin is designed to tackle sensor signals recorded under different operation conditions.
3.1 Preliminaries and motivations
Given multiple condition monitoring time series data {x(), xi2),..,, X(M), where X()
x1
E 1 represents the i-th time series with length L, and feature dimension N,
E R represents the value of features at timestamp We convert the raw time serics into samples by applying the sliding time window approach, with the lookback window size T, each con- taining T observations at one time step as input features, and the corresponding output label y,() x x x ( TxN denotes the input features one sample at timestamps The RUL prediction task involves predicting the label y, based on the input features x("). When employing traditional sequential models to abstract the temporal information, the prediction process be formulated
(
x
by:
( 2:00 = Fo,(x! Fo, (xx "0"
(1)
where 2:(0 are the forecasts matching the ground truth Y The forecasting function is denoted Fe, parameterized by 0, When using the ST-GNNs method, we first design the graphs or apply graph structure learning approaches to transform x(() into g: at each timestep i, and then the RUT. prediction be
(s)
4

---

Published in Transactions on Machine Learning Research (01/2025)
expressed as the following formulation:
y = Fo (x = For (Br 11 81-1+2--... 8: [0
(2)
where the forecasting function is denoted as o parameterized by Og and Og, indicating ST-GNNs separately model spatial and temporal dependencies.
3.2 Fourier Graph Neural Networks
A recent study (Yi et al. 2024) introduces FGN address the oversight of potential spatiotemporal inter- dependencies that arise from modeling spatial and temporal dependencies separately in ST-GNN- FGN aims to enhance learning efficiency by learning unified spatiotemporal dependencies. FGN no longer treats input samples as a sequence of graphs; instead, it regards them as one complete graph. Therefore, Equation 2 can be rewritten as:
p1 = F=,,,,, A!9),
(3)
where x,0) [R(TNN)XI, 110 E (1)(TXN)(TXXN) is the adjacency matrix of complete graph, and Og are the paramcters of the FGN. FGN assumes complete connectivity between all sensor signals, with the adjacency matrix A predetermined as a fully connected matrix (where every element is set to 1); thus avoiding the typical challenges associated with learning graph structures. In FGN, we initially project the node features into a higher dimensional space d to obtain node embeddings x! E ((TxN)xd, and perform a Discrete Fourier Transform (DFT) to transform node embeddings into the frequency domain and get
F(X(") E cl [ +1)xd Then we conduct recursive multiplications between F(X!") and Fourier Graph Operators (FGOs) in the Fourier space and make summations. Finally, we transform the node embeddings to the time domain using the Inverse Discrete Fourier Transform (TDFT), and utilize fully connected layers to map embeddings to labels, as illustrated in Figure The detailed FGN procodure can be formulated as follows:
K FGN,, (X, (,0, (0): := F- o (F(X! Sox + bx)), So.k II S. ==0
k
(4)
where F()) and (-) stand for DFT and IDFT, respectively. S₁ E Cdxd is the FGO in the k-th layer. a is the activation function, and by E Cd are the complex-value valued bias parameters. By treating time series samples as complete graphs and performing transformations in the frequency domain, FGN effectively captures potential spatiotemporal inter-dependencies within sensor signal data, while eliminating the need for the graph structure learning phase typically required in conventional ST-GNNs.
3.3
Multi-term ensemble learning framework
3.3.1
Multi-term Training Process
When generating samples using the sliding time window method, one key parameter to set is the lookback window size, denoted as T. If ouly a single fixed window size is used, T must not exceed the length of the shortest sequence in the dataset. Otherwise, some sequences would be excluded from the prediction and processing steps. This introduces the constraint T S min.g(.........)) Li, where L; represents the length of the i-th sequence. However, this constraint often forces the choice of relatively small T, which may be insufficient to contain long-term dependencies in the data. To address this limitation, we propose multi- term ensemble learning strategy. Instead of using single lookback window, we employ multiple lookback windows to create a diverse set of training samples:
T (Tmin: Tmin + 11x D,..., + (C
(5)
where C is the number of lookback windows, and D the increment in window size. The minimum lookback window Tmin is set to be less than or equal to the shortest sequence length in the test dataset. This ensures
5

---

Published in Transactions on Machine Learning Research (01/2025)
that predictions can be made for all test sequences. Next, we gradually enlarge the window size to gather extended long-term information. By utilizing various lookback windows of differing dimensions, we can generate multi-term samples, as depicted in Vigure[] This process results in C' groups of training samples, each corresponding to a different lookback window size, incorporating temporal dependencies at multiple scales. During the process of creating training samples, it is possible to encounter situations where the length of training sequence shorter than the designated lookback window size. In such cases, we discard the entire training sequence to ensure that all training samples maintain consistent input dimensions. Once the training samples are generated, we train C' individual FGN models, where each model is trained exclusively on the sample group generated by specific lookback window size. This design allows each individual FGN to focus on capturing different inherent temporal patterns within the data, thereby enhancing the ensemble model's ability to learn from various time perspectives. By leveraging the complementary strengths of multiple FGNs, the overall performance of the ensemble model can be significantly improved. After training, we obtain a set of trained FGN models: {FGN trained: FG trained>..... FGNeeainedd Each of these models contributes uniquely to the final ensemble, making the model more robust and versatile in handling diverse time dependencies present in the data.
3.3.2 Adaptive Length-grouped Ensemble for Inference
Traditional methods commonly use data from the last time window of a test sequence as the test sample, which is then fed into a trained model to obtain a single prediction result. In contrast, our proposed framework leverages multiple lookback window sizes to generate multiple test samples from the end of each test sequence. These samples correspond to the same target, namely the RUL at the current point. Given that some shorter test sequences may not allow the generation of complete samples for larger time windows, our strategy is to group the test sequences according to their length and utilize applicable trained models for each group. Figure D shows the test set divided into two groups: long and short test sequences. For long test sequences, multiple samples can be generated using several lookback window sizes, and these samples are then input into the corresponding trained models, result ing in multiple prediction results. The extensive temporal information contained in long sequences allows them to leverage multiple models trained on diverse window sizes, facilitating the production of more robust and comprehensive predictions. Each prediction captures distinct temporal dependencies relevant to different time spans. In contrast, short sequences, which are more limited in the amount of historical data available, are processed using fewer models or, in certain cases, only single model with the minimun window size. Let the set of test sequences be denoted by (xx)x(() ...., ...... X(M).. For a specific test sequence X with length Li, the applicable set of lookback window sizes is determined as T {Th E T' Th S L.). For each window size T, E T a test sample is generated from the last T3 time steps of the sequence X²), This sample is subsequently fed into the corresponding pre-trained model, FGN())
trained to generate a prediction g,()) This results in a collection of predictions given by: Dj = to 0!9) D!
(!("),, ), where m represents the number of valid window sizes
for the sequence X(). Finally, we employ a simple averaging aggregate all the predictions, defined as: 0. = Σm 9 where g is the prediction from the 1-th model.
This adaptive grouping strategy optimizes the utilization of available data for each test sequence while still adhering to the constraints imposed by the sequence length. It allows the ensemble of models to effectively capture both short- and long-term dependencies, improving the overall accuracy and stability of the predictions. By leveraging multiple trained models and dynamically selecting the applicable models for each lest sequence, our framework adapts to the varying lengths of the test data, providing tailored predictions that are better suited to the inherent characteristics of each sequence.
3.4 Sequence decomposition under multiple operation conditions
Due to the equipment's varying operation conditions, the collected sensor signals often adhere to multiple distributions. A typical example of monitoring data with two operation conditions is illustrated in Figure? 2 Sensor values frequently fluctuate between two distinct patterns, reflecting varying statistical propertics and frequency components over time. Applying DFT directly to such mixed sequences may cause spectral aliasing, where the frequency components of each sequence interfere with one another. This results in overlapping
6

---

Published in Transactions on Machine Learning Research (01/2025)
frequency information from multiple sequences, which diminishes the clarity and interpretability of the extracted frequency characteristics. Besides, abrupt transitions in the time series due to changing operation conditions may be mistakenly interpreted by the DFT as low-frequency components. These components are not truly indicative of equipment degradation but instead represent noise that should be climinated. To miligate these issues, we employ a simple sequence decomposition method to minimiie interference from mixed signals and reduce noise. Our first step involves identifying operation conditions. If the condition monitoring data already includes records of operation conditions each time point, we can analyze them directly. Alternatively, straightforward and efficient k-means (Ahmed et al. 2020) method for clustering analysis of the raw sensor signals to determine which operation condition a given sensor signal belongs to at timestep t. Then we segment the sensor signals based on the identified operation conditions. This involves dividing the data inlo subsets, each corresponding to a specific operation condition, as shown in the following formulation.
extract information operation conditions from sensor signals. We utilize the
(
k-mcans
(x X(12)
X((A),,
(6)
where (9) represents the i-th original time series, and {x (x6,)660)) .x.,,) represent the time series resulting from the k-means clustering process, where h the number clusters obtained by the k-means algorithm. The newly generated multiple time series data are complementary each other, with many miss- ing values that require interpolation. While numerous novel interpolation methods have been proposed (Oh) et al. 2020,, given the subslantial ial number of data points requiring interpolation, we adopt straightlor- ward equal-value interpolation method, where missing values are filled using their nearby counterparts. The interpolation process can be formally expressed as:
(x(1))) x((2) ((kk) interpolation (8611, 8 {
(7)
Finally, we utilize a sliding time window approach to generate samples ( x(1.2) x((1))) for cach operation condition sequence, inputting them into FGNs. The predicted mean under different operation conditions is then the final output:
FGN
(8)
Sensor singals multiple operation conditions
Frequency distributions under multiple openation condition
Operation Condition
Operation Condition Operation Conditan
Operation Condition?
Prequetcy Dkatibutton
Value
Time
Value
Figure 2: A typical example illustrates original sensor signals and their frequency distributions under multiple operation conditions.
4 Experiment and Results
In this section, we present a comprehensive evaluation of the proposed MT-FGNE framework on two widely used benchmark datasets for RUL prediction.

---

Published in Transactions on Machine Learning Research (01/2025)
4.1 Datasets description and analysis
The Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset is a widely used public dataset in the field of RUL prediction (Xia et al. 2020). There are four subsets in this dataset. Each subset is divided into a training set and a test sct. The training sct contains multiple turbofan engine condition monitoring data from healthy operation to complete failure. The condition monitoring data in the test set ends before complete faīlure occurs. The goal is to forecast the engines' RUT. in the test set. The characteristics of each subset are detailed in Table [I] Among the four data sets F'D001-FD0O4, the engines in FD001 and FD003 operated under single operation condition, while those in FD002 and FD004 operated under six different operation conditions, increasing the prediction complexity. Additionally, engines in FD001 and FD002 only have one fault mode, which is the High-Pressune Compressor (HPC) failure, while FD003 and FD004 hold two fault modes. Table [ displays the minimum and maximum sequence lengths of the dataset, indicating significant variations in sequence lengths across different engines. Since training data captures engine operation up to the point of failure, the signal record is usually relatively long. Also, variations in initial engine states and failure processes lead to different sequence lengths for each engine. Existing models (Kong et al.) 2022 Wang et al., 2021 Chen & Zeng) 2023) commonly use a single fixed lookback window to generate samples. However, this lookback window size camnot exceed the minimum sequence length of the test engines; otherwisc, the model cannot be applied to obtain predictions for all test engines. This relatively small lookback window size is inappropriate for lest engines with extensive sensor data and may hinder the model's ability to learn long-term dependencies.
C-MAPSS:
N-CMAPSS: the N-CMAPSS datasct introduced by Arias Chao et al. (2021). it provides a more compre- hensive and realistic simulation of aircraft engine operations. This dataset represents significant advance- ment over C-MAPSS by incorporating complete flight phases (climb, cruise, and descent) and offering more complex, real-world operational scenarios, making it an excellent benchmark for assessing bustness and generalization capabilities. The N-CMAPSS dataset contains eight sets of data from 128 units, and the DS02 subset is utilized for data-driven prognostics. The units in DS02 include two failure modes: one characterized by High-Pressure Turbine (HPT) efficioncy degradation and the other involving combinod Low-Pressure Turbine (LPT) efficiency and flow issues along with HPT degradation. Six units (2, 5, 10, 16, 18, and 20) serve as the training dataset, while units 11, 14, and 15 are allocated for testing purposes. As shown in Table I the dataset contains significantly long sequences due to high-froquency signal sampling within each flight cycle, providing sufficient data for flexible window size selection in sample generation.
model's ro-
Table 1: Description of C-MAPSS and N-CMAPSS turbofan engine dalaset.
Dataset
Subsets Operation Condition Fault Mode Training units
Test units
Max length
Min length
FD001
HPC
100
100
362
31
FD002
HPC
260
259
378
21
C-MAPSS
FD003
Fan
100
100
525
38
FD004
HPC-Fan
249
248
543
19
HPT
N-CMAPSS
DS02
6
3
1074k
187k
HPT+LPT
Table 2: Comparison of multiple time windows used in the proposed framework against existing single time windows.
Dataset
Subsets
Min length Single window size
WWng et al. 2021
Window sizes in MT-FGNE
FD001
31
30
30/
FD002
21
20
20/ ///400060///111 1//0000010001000 1200 140/160/180
C-MAPSS
FD003
38
30
FD004
19
15
18 0B//4//62/44/10///2////0 177/194
N-CMAPSS
DS02
1.87k
50
100/200
8

---

Published in Transactions on Machine Learning Research (01/2025)
FD001
FD002
30
Training data Test data
50
Training data
Test data
25
Kotnel Denett
20
Kemel Densiyy
30
15-
20
10-
10-
0-
100
200
300
400
500
100
200
300
400
500
Sequence Length
Sequence Length
FD003
FD004
30
Training data
Training data
25
Test data
40
Test data
Komell Density
20
Xemsel Density
30
15-
20-
10:
10-
5-
0-
100
200
300
100
400
500
Sequence Length
Sequence Length
Figure 3: The training and test sequence length distribution of four subsets in the C-MAPSS dataset, multiple dashed lines parallel to the y-axis represent the various sizes of the lookback window we employed. These dashed lines partition the training and testing time series into multiple subgroups.
4.2 Implementation details
We maintain consistency in the data preprocessing settings in (Kong et al. 2022 Wang et al. 2021 to ensure a fair comparison with existing models. For C-MAPSS, we first normalize the 14 effective features selected from the original 24 features. Then, we apply a piecewise function to rectify the training and test labels, ensuring that they do not exceed 125 to mitigate the possibility of overestimating RUL. Next, we apply the proposed multi-term learning approach by employing multiple lookback windows to generate various samples. For the four subsets, the lookback window sizes used are shown in Table 2 Unlike existing methods that use a single window size, which cannot exceed the minimum sequence length in the subset, resulting in short samples that fail to capture long-term dependencies in the time scries, adopt set of lookback window sizes. We keep the first window size smaller than the minimun sequence length to ensure the applicability of the predictive model. Subsequently, we gradually increase the window size to generate longer samples, allowing the model to learn potential long-term dependencies. The employed lookback time windows and their comparison with the length of the time series are illustrated in Figure B We partition the training and testing sequences into subgroups using the multiple lookback windows we defined. We generate training and testing samples adaptively basod on the length of sequences within cach subgroup. Given the relatively longer sequences on the right side of each sub-figure, they could adopt all lookback windows on the left side of the sequences, while the reverse was not valid. For N-CMAPSS, the data is downsampled to rate of 0.01 Hz, and a similar approach is employed to generate training and test samples. a comprehensive and equitable comparison with the cutting-edge models cited in Wang et al. (2021). we evaluated our model on the complete degradation trajectories of individual Lest units as well as on the entire test dataset.
To guarantee
For C-MAPSS, the operation of FD002 and FD004 under six different conditions may result in a low signal- to-noise ratio when FGN is applied directly for frequency domain learning. To address this, we select six
9

---

Published in Transactions on Machine Learning Research (01/2025)
features highly correlated with the labels fs7, s11, s12, (Huang et al. 2023 and categorize the data into six distinct clusters. For N-CMAPSS, the raw signals are clustered into four groups based on four parameters of the operation condition: altitude, flight Mach number, Throttle Resolver Angle (TRA), and total temperature at the fan inlet (T2). After completing the clustering process, we apply a straightforward equal-value interpolation to address the missing values. Next, FGN is used to learn from each of the subsequences separately. We set the number of FGO layers to three, which is deep enough for the RUL prediction task. The proposed model's training parameters were optimized using the RMSprop optimizer, with the following hyperparameters: 100 epochs, learning rate of 0.001, and batch size of 256. Two metrics are employed in the evaluation, including the Root Mean Square Error (RMSE) and Score function (Kong
lt al. 2022) as described in the following equation:
Score(u, 0,) = [ M
if 6 < 0/; -1)
(9)
E T0 1) a 0 U/:
where iy and 0; represent the true and predicted RUL values, respectively. The asymmetric Score function imposes more significant penalty for overestimating RUL, as overestimating RUL leads to more severe con- soquences. Similarly to RMSE, a lower value of the Score function indicates a better prediction performance.
4.3 Comparisons with state- afrt
This section compares our method with the most advanced RUL prediction techniques available. We mainly compare our method with ST-GNNs because of their outstanding performance on this task. Table 3 and Table ! present comparisons of various approaches on the C-MAPSS and N-CMAPSS datasets, respectively. The current advanced approaches can be broadly categorized into two groups. The first group consists of sequence models, with the most popular being Transformer-bant based models. The second group comprises ST-GNNs, which excel in predictive performance by capturing spatial information, generally outperforming sequence models. Our approach differs significantly from existing methods. First, FGN conducts learning in the frequency domain space instead of the time domain. Secondly, we transform samples into a graph rather than a sequence of graphs. Finally, we adopt a multi-term learning approach to enhance the model's 's learning of long-term dependencies within samples.
We compare our method with several statc-of-tho-art models on the C-MAPSS dataset. The values reported in Table 3 are taken directly from the papers as their implementations are not publicly available. One immediate observation on the performances of these baseline models is that there is no single best model that is capable of outperforming others on all four datasets, demonstrating the difficulty of the prediction task and the diversity shown in the datasets. In comparison to these baselines, our proposed MT-FGNE demonstrates superior performance on the first three datasets, FD001, FD002, and FD003. It shows a 13.6%, 2.3%, and 5.3% improvement compared to the second-best bascline on these three datasets, respoctively. On FD004, our model is ranked second among eighteen models in terms of RMSE, with only small gaps to the best-performing models. The results indicate that even under various operation conditions and relatively low hignal-to-noiss ratios in original sensor signals, FGN can effectively Iearn dependencics after proper sequence decomposition and processing. Compared to traditional ST-GNNs, method does not require learning graph structures separately learning spatial and temporal information. Moreover, the Fourier Graph Operator is significantly less computationally expensive than the message-assing operators of GNNs. We follow STEA (Kong et al.) 2022) and create an ST-GNN. The training time of ST-GNN and FGN on FD001 is 2518s and 729s, and the trainable parameters of ST-GNN and FGN are 183,905 and 74,064, respectively, showing that FGN has a lower computational burden. The total training time of MT-FGNE on FD001 is 5616s, which reflects the computational overhead of training multiple FGN models in the ensemble. Additionally, for FD002 and FD004, which require sequence decomposition, the processing time increases approximately six times due to the additional decomposition operations. Although this represents a notable increase in computational requirements compared to the base FGN, we believe this trade-off is justified by the significant improvements in prediction accuracy and data utilization rates demonstrated in our experimental results.
The experimental results shown in Table demonstrate the superior performance of our proposed MT-FGNE compared to the state-of-the-art methods in different. units of the N-CMAPSS dataset. Across different units,
10

---

Published in Transactions on Machine Learning Research (01/2025)
our method consistently achieves lower Score metrics compared to baseline models, with significant reductions of up to 40% in Score values compared to transformer-based methods such as Transformer and Informer: Although LOGO (Wang et al. 2023b) achieves slightly better RMSE values in some cases, MT-FGNE demonstrates superior performance in terms of Score, which a more comprchensive evaluation criterion for the N-CMAPSS dataset. Incorporating the N-CMAPSS dataset into our evaluation provides thorough assessment of the model's generalization capabilities realistic scttings. Overall, our method consistently delivers outstanding performance across different scenarios, demonstrating its ability to provide significant contributions to RUL prediction.
Table 3: Compare the RMSE and Score values of MT-FGNE with other advanced sequence models and ST-GNN methods for the C-MAPSS dataset (bold: best; underline: runner-up).
FD001
FD002
FD003
FD004
Average
Models
RMSE
Score
RMSE Score
RMSE
Score
RMSE
Score
RMSE
Score
DA-Transformer
Liu et al,
2022
12.25
198
17.08
1575
13.39
290
19.86
1741
15.65
951.00
BIGRU-TSAM
Zhang et al.
2022a
12.56
213
18.94
2264
12.45
233
20.47
3610
16.11
1580.00
MSIDSN
Zhao et al.
2023
11.74
206
18.26
2047
12.04
196
22.48
2911
16.13
1340.00
EAPN Zhang et al.
2023
12.11
245
15.68
1127
12.52
267
18.12
2051
14.61
922.50
Crossformer
Wang et al.
2023b
12.11
216
14.16
837
12.32
260
14.81
956
13.35
567.25
HAGCN
Li et al. 2021
11.93
222
15.05
1144
11.53
240
15.74
1219
13.56
706.25
STGCN (Wang al.
2021
14.55
402
14.58
943
13.06
14.60
1065
14.20
701.00
STEA Kong et al. 2022)
11.35
194
19.17
2493
11.64
225
21.41
2760
15.89
1418.00
DAST
Zhang el al.
2022b
11.43
203
15.25
925
11.32
155
18.36
1491
14.09
693.50
GGCN
Wang ct al.
2022)
11.82
187
17.24
1494
12.21
243
17.36
1372
14.66
824.50
ConvGAT
Chen & Zeng
2023
11.34
197
14.12
772
10.97
235
15.51
1231
12.99
608.75
CDSG
Wang et al.
2023a
11.26
188
18.13
1740
12.03
218
19.73
2332
15.29
1119.50
DCFA
Gao et al.
2023)
11.74
190
16.81
1076
10.71
198
17.77
1571
14.26
758.75
LOGO
WWng et al. 2023b
12.13
226
13.34
832
12.18
261
14.29
944
13.04
565.75
NSD-TGTN
(Gao et al. 2024)
12.13
226
15.87
1477
12.01
220
16.64
1493
14.16
854.00
DVGTformer
WWang et al. 2024
11.33
180
14.28
797
11.89
255
15.50
1108
13.25
585.00
THGNN Wen et al.
2024
13.15
285
13.84
806
12.61
255
14.65
1166
13.56
628.00
MT-FGNE
9.73
152
13.23
694
10.14
178
14.40
958
11.88
495.50
Table 4: Compare the RMSE and Score values of MT-FGNE with other advanced sequence models and ST-GNN methods for the N-CMAPSS dataset (bold: best; underline: runner-up).
Unit11
Unit14
Unit15
All
Models
RMSE
Score
RMSE
Score
RMSE
Score
RMSE.
Score
Transformer (Mo et al.
2021
5.86
7725
7.69
2397
5.34
3391
6.54
17075
Informer (Zhou et al. 202 a
6.28
8019
7.67
2437
5.03
3195
6.24
16066
Autoformer [Wu et al.)
202 La
10.73
23987
13.72
9533
11.29
15383
11.48
50322
Crossformer [Zhang & Yan
2023
6.89
11816
8.58
2855
7.39
5733
6.87
16704
DAG (Li al.] 2019
9.00
21282
9.37
4211
7.30
7970
8.82
36780
STGCN (Wang et. al.
2021
8.96
32080
11.96
17665
8.39
20169
9.31
37710
HAGCN Li ct al. 2021
6.39
9956
7.03
2009
7.48
9376
6.67
17918
MAGCN CChe et al.]
2023
8.33
13835
10.52
4566
5.11
3623
7.37
20821
LOGO (Wang et al.
2023b
5.73
7509
6.72
1940
4.54
3017
6.07
15127
MT-FGNE
6.16
4711
8.12
1347
5.81
2388
6.32
8447
11

---

Published in Transactions on Machine Learning Research (01/2025)
4.4 Ablation studies
To demonstrate the effectiveness of the proposed framework, we conducted an ablation study comparing MT- FGNE with its variants on the C-MAPSS dataset. We primarily evaluated two components of MT-FGNE: the multi-term ensemble learning (MTE) framework, and the sequence decomposition (SD) plugin.
4.4.1 Multi-term ensemble ablations
The first variant is the single FGN model. Like existing methods, individual FGN adopts smaller, fixed time windows to generate samples at a certain scale, resulting in unsatisfactory predictive performance, As shown in Table 5] compared to individual FGN, our MT-FGNE reduced 18.0% prediction error on the FD001 dataset. Similar effects are observed across all four datasets, demoustrating the high applicability of the proposed framework. As analyzed earlier, the multi-term ensemble learning framework primarily relies generating longer samples to enhance the model's ability to learn long-teru tengtermman dependencies. The prerequisite for performance improvement is that the adopted model possesses strong sequence modeling capabilities. For instance, MT-FGNE w/o SD applied multi-term learning, yet its performance on FD002 did not show significant enhancement. is attributed to the scnsitivity of the adopted model to noise in sensor signals, leading poor learning capabilities. Merely increasing the time window does not contribute to resolving the noise issue in this situation.
The proposed multi-term ensemble learning framework is not limited to specific individual model for learn- ing and theoretically can be applied to all existing models to further enhance performance. We employed a simple sequence modeling tool, CNN, as the individual model. Following the adoption of multi-term learning, its predictive performance saw significant improvement. Particularly noteworthy is its performance on the FD002 and FD004 datasets, where the prediction error surpassed that of MT-FGNE and even outperformed the current state-of-the-art model by 16.2%. This result suggests that employing smaller, fixed time window settings hinders the model's ability to capture long-range dependencies in the data, thereby limiting the performance of most existing models. Figure | illustrates the performance of individual FGN models and ensemble results the last four test subsets of the FD001 dataset. In subset 3, the test sequence length is insufficient to apply all FGN models, while in subset 6, the sequence length exceeds the maximum time window, set at 180, thus allowing the application of all trained FGN models for prediction. FGN performs better in the majority of cases (24 out of 34, or 70%) when a larger time window is applied. A larger time window for generating samples enables the FGN to capture more extensive long-range dependencies. However, extending the time window also results in fewer generated samples, thereby impacting the modcl's training. Consequently, in some cases the prediction performance fails to improve might even decline. In most cases, the ensemble predictions further enhance accuracy and significantly reduce the variance of prediction errors, indicating improved stability of the predictive models.
Table 5: Ablation study on the C-MAPSS dataset.
FD001
FD002
FD003
FD004
Variants
FGN
MTE
SD
RMSE
Score
RMSE
Score
RMSE Score
RMSE
Score
CNN
X
X
X
12.71
241
15.74
1504
11.47
185
17.81
1809
MT-CNNE w/o SD
X
J
X
11.83
537
13.76
853
MT-CNNE w/o MTE
X
X
17.21
1901
19.67
3124
MT-CNNE
X
V
11.76
248
14.11
851
10.77
240
15.54
1147
FGN MT-FGNE w/o SD
X
X
11.87
194
20.87
2221
12.64
208
27.22
4777
/
X
19.21
1563
23.38
2781
MT-PGNE w/o MTE
X
16.39
1411
18.37
2168
MT-FGNE
/
9.73
152
13.23
694
10.14
178
14.40
958
MTE: Multi- Term Ensemble, SD: Sequence Decomposition.
12

---

Published in Transactions on Machine Learning Research (01/2025)
Stblset
Subset
Sulset
14
14
14
12
10-
6-
6-
6-
6
TW 10
TW 60
.9)
TW 120
TW 150
TW 150
finemblle
TW: 10
Tw 60
TW .0
TW 120
TW 150
Fw. 180
finerndle
TW: 10
TW: 60
TW. .00
TW 120
TW 150
TW 180
Finemble
TW 110
TW: 60
og
TW 120
TW 130
TW 150
Fimemble
Figure 4: The individual FGN models' performance and ensemble results on the last four test subsets in
FD001.
The proposed framework can also enhance the utilization of test data. We define the test data utilization rate the ratio of the length of the time steps used (specifically, the final window for prediction) to the entire length of the test time series. The original FGN models exhibit relatively low utilization rales ranging from 9.03% to 22.91%, our MT-FGNE framework significantly increases these rates across all four benchmark datasets, achieving utilization rates between 80.65% and 88.32%. This marked improvement demonstrates our model's capability to effectively leverage a larger portion the test sequences. A straightforward hypothesis suggests that similar results might be achieved by using a single model with a larger lookback time window. However, our empirical results point a different outcome. We conducted comparative experiments datasets FD001 and FD003 to empirically validate this hypothesis. As illustrated in Figure 5 simply increasing the time window size in the single FGN model leads to consistent degradation in performance. This performance deterioration can be attributed to the necessary padding operations required for shorter sequences when using larger window sizes. The padding introduces artificial data points that do not represent actual sensor measurements or degradation patterns, thereby corrupting the model's ability to capture genuine temporal dependencies. In contrast, our proposed MT-FGNE, effectively captures temporal dependencies without compromising prediction accuracy through artificial data padding.
ID001
FD003
25.0
Single RGN
Single PGN
MEPGNE
25
MI.
22.5
20.0
RMSE
RMSE
20
17.5
13.0
15
60
90
120
150
30
120
180
210
Time Window Size
Time Window Size
Figure 5: Performance comparison between Single FGN with larger time window sizes and MT-FGNE on FD001 and FD003 datasets.
13

---

Published in Transactions on Machine Learning Research (01/202))
Proquency Speckiu for Condition
Froquency Coadition
Frequescy Spectrun Condition
10
10"
Nigniiak
Nagntal
16.1
107
10
10
0.4
6.0
6.2
044
6.4
60
0.2
0.
0.4
0.2
0.0
0.2
0.4
Preguency
Proquency Spectrum Conditiin
Troquency Spectrum Conditiio
Trequency Spectrum Conditiion
Orianal sgm
Orignul agai
Onignal
16'
10'
Magnaak
10.
10
t0
1017
10"
10"
10"
Froqucay
Figure 6: FD002.
The frequency spectrum comparison of raw sensor signals and signals after decomposition in
Silbouette Szore: RMSE: 13.79
Score: 0.92
Silhouctte Score: 0.73
RMSF: 13:23
RMSE: 14.81
0.00 PCA3
0.00 PCAS
-0 10
-0.10
-0.00
0.6
0.6
0.6
-10
-1.5
-0.3
0.0
0.2
-1.0 -0.5
-0.2
0.0
0.2
-1.0 -0.5
0.2
0.0
0.2
PCA2
PCAJ
1.0
PCAL
1.0
-0.4
PCAL
1.0
-0.4
a) Test Data with- 4 Clusters
b) Test Data with Clustors
e) Test Data with Clusters
Figure 7: Comparison of clustering quality and prediction performance under different numbers of clusters (k=4, 6, 8) on the FD002 dataset.
4.4.2 Sequence decomposition ablations
The FD002 and FD004 datasets were generated under six different operation conditions. Our study primarily focuses on these two datasets to validate the effectiveness of sequence decomposition. As shown Table although the individual FGN w/o SD method exhibits decent predictive performance on FD001 and FD003, its predictive errors on FD002 and FD004 are quite significant. By comparing FGN with MT-FGNE w/o SD, can find that the prediction performance of MT-FGNE w/o SD FD002 is not significantly improved even with the MTE strategy. It indicates that even when the samples contain more information long-term dependencies, the FGN model fails to learn from them effectively, diminishing the value of MTE. In contrast to FGN, the MT-FGNE w/o MTE model incorporates the SD plugin, resulting in a substantial improvement in predictive performance on FD002 and FD004. Figure iillustrates the frequency spectrum of an FD002 sample's 's original signal and the signal after denoising with SD. We observe a more uniform energy distribution in the original signal with frequent random fluctuations in frequency and no clear frequency range with high-amplitude signals. After denoising with SD, there significant increase in
14

---

Published in Transactions on Machine Learning Research (01/2025)
amplitude in the oow-frqquency region of the frequency spectrum, indicating enhancement in the signal- to-noise ratio. Finally, a significant enhancement in predictive performance is observed when combining SD for signal denoising, followed by the MTE module. This phenomenon underscores the necessity of applying SD for denoising signals, particularly for datasets with multiple operation conditions.
In real-world applications, the exact number of operation conditions often unknown, posing a challenge for clustering- based SD methods. To address this, we utilized the Silhouette Score, a reputable clustering quality metric, to evaluate the SD quality and to detect potential unseen operation conditions. We conducted experiments the FD002 dataset using different numbers of clusters, and examined Silhouette Score and the corresponding prediction performance. As shown in Figure 7 there are distinct and well-separated clusters for each condition, indicating that the k-means decompasition provides & clear segmentation of the operation conditions. Besides, we observed a strong correlation between clustering quality and model accuracy. Specifically, the configuration with six clusters achieved the highest clustering quality (Silhouctte Score = 0.92) and the best prediction accuracy (RMSE = 13.23). Alternative configurations such as four clusters (Silhouette Score 0.88, RMSE = 13.79) and eight. clusters (Silhouette Score 0.80, RMSE 14.81) resulted in lower clustering quality and suboptimal predictive performance. These findings confirm that the SD approach can effectively capture underlying operation conditions, even when their exact number is unknown. Furthermore, by monitoring the Silhouette Score prior to training, we can detect potential unseen operational conditions through observed decreases in clustering quality, providing an early indication that the model configuration may require adjustment. This systematic approach offers a practical and robust solution for real-world scenarios, where operation conditions are often complex and unspecified.
5 Conclusion
This paper investigates the RUL prediction problem. Unlike existing deep learning models with fixed-size input samples, we consider the diversity in the time series lengths and propose a model capable of lcarning multi-term dependencies. We adaptively employ multiple lookback windows based on the time series length to generate multiple samples with the same label. Subsequently, these samples are transformed into complete graphs, and FGN models are employed to learn both spatial and temporal dependencies across multiple terms. The predictions from FGN models are then integrated to obtain the final prediction. This approach enhances the model's capability to learn long-range temporal dependencies in long sequences and further improves prediction performance by integrating predictions from different terms. For condition monitoring data generated under various operation conditions, cluster the data based on operation conditions and then interpolate missing values, learn spectral features through FGN after DFT on data from different operation conditions, and finally average the prediction results. Extensive experimental results demonstrate that our proposed MT-FGNE model achieves state-of-the-art predictive performance on two turbofan engine datasets.
References
Mohiuddin Ahmed, Raihan Seraj, and Syed Mohammed Shamsul Islam. The k-means algorithm: A com- prehensive survey and performance evaluation. Electronics, 988):1295, 2020.
Manuel Arias Chao, Chetan Kulkarni, Kai Goebel, and Olga Fink. Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics. Data, 6(1):5, 2021.
Zhaowei Cai, Quanfu Fan, Rogerio S Feris, and Nuno Vasconcelos. A unified multi-scale deep convolutional neural network for fast object detection. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdamm The Netherlands, October 1 11-14, 2016, Proceedings, Part IV 14, pp. 354-370. Springer, 2016.
Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 357-366, 366, 2021a.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Ling Chen, Donghui Chen, Zongjiang Shang, Binqing Wu, Cen Zheng, Bo Wen, and Wei Zhang. Multi-scale adaptive graph neural network for multivariate time series forecasting. IEEE Transaclions on Knowledge and Data Engineering, 2023.
Wei Chen and Ke Shi. Multi-scale attention convolutional neural network for time series classification. Newral Networks, 136::26-140, 2021.
Xiao Chen and Ming Zeng. Convolution-grape attention network with sensor embeddings for remaining useful life prediction of turbofan engines. IEEE Sensors Journal, 2023.
Xiaohan Chen, Beike Zhang, and Dong Gao. Bearing fault diagnosis base on multi-scale enn and 1stm model. Journal of Intelligent Manufacturing 32(4):971199) -877, 2021b.
Zhicheng Cui, Wenlin Chen, and Yixin Chen. Multi-scale convolutional neural networks for time series classification. arviv preprint or Xiv: 1603. 06995, 2016.
Paulo Roberto De Oliveira Da Costa, Alp Akcay, Yinggian Zhang, and Uzay Kaymak. Attention and long short-term memory network for remaining useful lifetime predictions of turbofan engine degradation. International journal of prognostics and health manegement, 10:034, 2019.
Carlos Ferreira and Gil Gonçalves. Remaining useful life prediction and challenges: A literature review on the use of machine learning methods. Journal of Monufacturing Systems, 63:550 562, 2022.
Hui Gao, Yibin Li, Ying Zhao, and Yan Song. Dual channel feature-attentioo--ass approach for rul prediction considering the spatiotemporal difference of multisensor data. IEEE Sensors Journal, 2023.
Zhan Gao, Weixiong Jiang, Jun Wu, Tianjiao Dai, and Haiping Zhu. Nonlinear slow-varying dynamies- assisted temporal graph transformer network for remaining useful life prediction. Reliability Engineering & System Safety, 248:110162, 2024.
Lin Huang, Xin Pan, Yajie Liu, and Li Gong. An unsupervised machine learning approach for monitoring data fusion and health indicator construction. Sensors, (16):72333, 2023.
Li Jiang, Tianao Zhang, Wei Lei, Kejia Zhuang, and Yibing Li. A new convolutional dual-channel transformer network with time window concatenation for remaining useful life prediction of rolling bearings. Advanced Engineering Informalies, 56:101966, 2023.
Guangyin Jin, Yuxuan Liang, Yuchen Fang, Zezhi Shao, Jincai Huang, Junbo Zhang, and Yu Zheng. Spatio- temporal graph neural networks for predictive learning in urban computing: A survey, IEEE Transactions on Knowledge and Data Engineering, 2023.
Ziqian Kong, Xiaohang Jin, Zhengguo Xu, and Bin Zhang. Spatio-temporal fusion attention: A novel approach for remaining useful life prediction based on graph neural network. IEEE Transactions on Instrumentation and Measurement, 71:1-12, 2022.
Jialin Li, Xueyi Li, and David He. A directed acyelic graph network combined with ean and Istm for remaining useful life prediction. IEEE Access, 7:75464-75475, 2019.
Tianfu Li, Zhibin Zhao, Chuang Sun, Ruqiang Yan, and Xuefeng Chen. Hierarchical attention graph convo- lutional network to fuse multi-sensor signals for remaining useful life prediction. Reliability Engineering & System Safety, 215:107878, 2021.
Xinyao Li, Jingjing Li, Lin Zuo, Lei Zhu, and Heng Tao Shen. Domain adaptive remaining useful life prediction with transformer. IEEE Transactions on Instrumentation and Measurement, 71:1-13, 2022.
Lu Liu, Xiao Song, and Zhetao Zhou. Aircraft engine remaining useful life estimation via double attention- based data-driven architecture. Reliability Engineering & System Safety, 221:108330, 2022.
Yu Mo, Qianhui Wu, Xiu Li, and Biqing Huang. Remaining useful life estimation via transformer encoder enhanced by gated convolutional unit. Journal of Intelligent Manufacturing, 32(7):1997-25) 2021.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Cheolhwan Oh, Seungmin Han, and Jongpil Jeong. Time-scries data augmentation based on interpolation. Procedia Compuler Science, 175:64-71. 71, 2020.
Lei Ren, Jiabao Dong, Xiaokang Wang, Zihao Meng, Ti Zhao, and M Jamal Deen. A data-driven auto- cnn-lstm prediction model for lithium-ion battery remaining useful life. IEEE Transactions on Industrial Informalics, 17(5):3478 3487, 2020.
Zunya Shi and Abdallah Chehade. A dual-Istm framework combining change point detection and remaining useful life prediction. Reliability Engineering & System Safety, 205:107257, 2021.
Hongfei Wang, Zhuo Zhang, Xiang Li, Xinyang Deng, and Wen Jiang. Comprehensive dynamic structure graph neural network for aero-engine remaining useful life prediction. IEEE Transactions on Instrumen- tation and Measurement, 2023a.
Lei Wang, Hongrui Cao, Hao Xu, and Haichen Liu. A gated graph convolutional network with multi-sensor signals for remaining useful life prediction. Knowledge- Based Systems, 252:109340,, 2022.
Lei Wang, Hongrui Cao, Zhisheng Ye, Hao Xu, and Jiaxiang Yan. Dvgtformer: A dual-view graph trans- former to fuse multi-sensor signals for remaining useful life prediction. Mechanical Systems and Signal Processing, 207:110935, 2024.
Mengni Wang, Yuanxiang Li, Yuxuan Zhang, and Lei Jia. Spatio-temporal graph convolutional neural network for remaining useful life estimation of aircraft engines. Aerospace Systems, 4(1):29 36, 2021.
Yucheng Wang, Min Wu, Ruibing Jin, Xiaoli Li, Lihua Xie, and Zhenghua Chen. kocal-global correlation fusion-based graph neural network for remaining useful life prodiction. IEEE Transactions Neural Networks and Learning Systems, 2023b.
Zhihao Wen, Yuan Fang, Pengcheng Wei, Fayao Liu, Zhenghua Chen, and Min Wu. Temporal and het- erogeneous graph neural network for remaining useful life prediction. arXiv preprint arXiv:2405.0 04333, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34: 22419-22430 2021a.
Ji- Yan Wu, Min Wu, Zhenghua Chen, Xiao-Li Li, and Ruqiang Yan. Degradation-aware remaining useful life prediction with lstm autoencoder. IEEE Transactions on Instrumentation and Measurement, 70:1-10, 2021b.
Tangbin Xia, Ya Song, Yu Zheng, Ershun Pan, and Lifeng Xi. An ensemble framework based on convolutional bi-directional Istm with multiple time windows for remaining useful life estimation. Computers in Industry, 115:103182, 2020.
Xingwei Xu, Xiang Li, Weiwei Ming, and Ming Chen. A novel multi-scale cmn and attention mechanism method with multi-sensor signal for remaining useful life prediction. Computers & Industrial Engineering, 169:108204, 2022.
Boyuan Yang, Ruonan Liu, and Enrico Zio. Remaining useful life prediction based on a double-convolutionaal neural network architecture. IEEE Transactions on Industrial Electronics, 66(12):9521-953) 2019.
Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu, Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. Advances in Neural Information Processing Systems, 36, 2024.
Jiusi Zhang, Yuchen Jiang, Shimeng Wu, Xiang Li, Hao Luo, and Shen Yin. Prediction of remaining useful life based on bidirectional gated recurrent unit with temporal self-attention mechanism. Reliability Engineering & System Safely, 221:108297, 2022a.
17

---

Published in Transactions on Machine Learning Research (01/2025)
Xiong Zhang, Yunfci Guo, Hong Shangguan, Ranran Li, Xiaojia Wu, and Anhong Wang. Predicting re- maining useful life of a machine based on embedded attention parallel networks. Mechanical Systems and Signal Processing, 192:110221, 2023.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for mul- tivariate time series forecasting. In The eleventh international conference learning representations, 2023.
Zhizheng Zhang, Wen Song, and Qiqiang Li. Dual-aspect self-attention based on transformer for remaining useful life prediction. IEEE Transactions on Instrumenlation ond Measurement, 71:1-11, 2022b.
Ke Zhao, Zhen Jia, Feng Jia, and Haidong Shao. Multi-scale integrated deep self-allention network for predicting remaining useful life aero-engine. Engineering Applications of Artificial Intelligence, 120: 105860, 2023.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, In- former: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021a.
Yexu Zhou, Michael Hefenbrock, Yiran Huang, Till Riedel, and Michael Beigl. Automatic remaining useful life estimation framework with embedded convolutional lstan as the backbone. In Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, Port IV, pp. 461-477. Springer, 2021b.
Brahim Zraibi, Chafik Okar, Hicham Chaoui, and Mohamed Mansouri. Remaining useful life assessment for lithium-ion batteries using cnn-lstm hybrid method. IEEE Transactions on Vehicular Technology. 70 (5):4252-443) 4261, 2021.
18


Reviews:
[{"review_id": "iNQxihCM67", "reviewer": "Reviewer_9m7r", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n•The paper introduces a novel use of Fourier transforms in graph neural networks to improve spatiotemporal modeling for RUL prediction.\n•The multi-term ensemble approach is interesting and may provide better generalization across varying operational conditions.\n•Empirical results show statistically significant performance improvements over baseline models, particularly in common benchmark datasets like C-MAPSS.\nWeaknesses:\n•The paper does not sufficiently compare the proposed model to existing integrated spatiotemporal models, such as attention mechanisms or dual-branch networks.\n•The choice of Fourier transforms is underexplained, and there is little discussion on the potential loss of local temporal details, which could affect RUL predictions.\n•The datasets used for evaluation may not fully reflect real-world operational variability, limiting the generalizability of the results.\n•Some sections of the paper, especially the methodological descriptions, are difficult to follow due to complex language and notation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces an Ensemble Multi-Term Fourier Graph Neural Network (MT-FGNE) model aimed at improving Remaining Useful Life (RUL) prediction by leveraging Fourier transforms to capture spatiotemporal dependencies. The proposed approach offers a novel solution to model complex interactions between spatial and temporal data while adapting to varying operational conditions. The authors claim that their method addresses key shortcomings in existing models and demonstrates superior performance on standard RUL prediction datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tComparison to Existing Models:\no\tProvide a more comprehensive comparison with existing spatiotemporal models, such as spatial-temporal attention mechanisms or condition-aware GNNs, and clarify how the proposed method improves upon them.\n2.\tAblation Studies:\no\tConduct ablation studies to evaluate the individual contributions of Fourier transforms and the multi-term ensemble strategy, helping readers understand which components drive performance improvements.\n3.\tHandling of New Operational Conditions:\no\tOffer a clearer strategy for handling unseen operational conditions during inference, potentially drawing on existing work in transfer learning or condition-specific embeddings.\n4.\tBroader Dataset Evaluation:\no\tIn addition to the C-MAPSS dataset, evaluate the model on more recent and realistic datasets, such as the dataset provided by Chao et al. (Chao, Manuel Arias, Chetan Kulkarni, Kai Goebel, and Olga Fink. \"Aircraft Engine Run-to-Failure Dataset under Real Flight Conditions for Prognostics and Diagnostics.\" Data 6, no. 1 (2021): 5. https://doi.org/10.3390/data6010005) to ensure robustness in practical applications.", "broader_impact_concerns": "No significant ethical concerns are apparent in the paper’s current scope. However, it would be beneficial to add a broader impact section discussing the potential real-world implications of improved RUL prediction models in industries where predictive maintenance is critical, such as aviation and manufacturing. Specifically, attention should be given to ensuring that such models do not inadvertently lead to over-reliance on predictive systems, which could introduce new risks if the models are imperfect or biased.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pCljK0dr7B", "reviewer": "Reviewer_iwPa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The writing of this paper is clear and the design of the method is easy to follow. The experiments also show some good performance compared with some SOTA approaches.\n\nWeaknesses:\n\n* The overall technical depth of the proposed method is limited. (a) Since the Fourier Graph Neural Network has already been proposed and this paper directly uses it to model the samples, the contribution in this paper is unclear. (b) The multi-term ensemble learning framework simply uses predictions from multiple lookback windows for the ensemble. This is more like a trick of data pre-processing, rather than a technical contribution in machine learning. It also makes the comparison between MT-FGNE and other baseline methods unfair, as these baselines receive less input information. (c) The motivation for decomposing operation conditions is not clear enough. Is it because that different conditions have different amplitudes? Can this be solved by data normalization? The designed method is a simple utilization of K-means. Why can we use K-means to separate these conditions?\n\n* Some other statements and claims are not fully supported. (a) It is unknown why the proposed method can solve the issue of ‘need to learn the graph structure’. (b) It does not make sense to claim that existing methods ‘struggle to capture long-term dependencies’ because ‘window sizes are too small’. Can they simply use a larger window size to solve the problem?\n\n* Some designs in the method, including treating data as a complete graph, using multi-term ensemble, and performing K-means to decompose different operation conditions may all introduce additional computational costs, which are not fully discussed in this paper. Besides, does this method need multiple terms of data in inference? Will this limit its practical application?\n\n* The experimental evaluation is also a bit limited. MT-FGNE is only evaluated on one benchmark, and its size is small as shown in Table 1.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Ensemble Multi-Term Fourier Graph Neural Networks (MT-FGNE) for remaining useful life (RUL) prediction. MT-FGNE treats data as complete graphs and employs a Fourier Graph Neural Network (FGN) to model the graph in the frequency space. It also creates training and test graphs with varying sizes of lookback windows and uses multiple predictions for the ensemble. Additionally, MT-FGNE designs sequence decomposition for multiple operation conditions. The proposed method is evaluated on the C-MAPSS benchmark and outperforms some state-of-the-art approaches.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the Strengths And Weaknesses part. This submission needs improvements by adding more significant technical contributions, more clarifications on some statements, and experiments on more large-scale datasets.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DbYpt2IE3L", "reviewer": "Reviewer_ALmo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This paper is overall well written. The ideas are easy to follow.\n2. The proposed multi-term ensemble learning framework is reasonable for capturing both long-term and short-term dependencies and boosting the model's performance.\n3. The sequence decomposition method is a natural way to tackle the real-world scenarios where the sensor values switch between several distinct distributions. The data analysis in $\\underline{\\text{Section 3.4}}$ proves the existence of such problems.\n\nWeaknesses:\n\n1. In $\\underline{\\text{Section 4.4.1}}$, the results show a simple CNN model accompanied by multi-term learning and sequence decomposition perform better than the proposed method on dataset FD002 and FD004. While this shows that the multi-term framework and sequence decomposition approach is effective, the FGN model may be suboptimal for specific tasks. I suggest that ablations with regard to the network backbone should be conducted.\n2. This paper utilizes k-means to determine which operation condition a signal belongs to. This may be somehow ad-hoc, as  k-means is effective to classify signals with different amplitudes into different clusters, as can be seen in $\\underline{\\text{Figure 2}}$. Is this always  the case? Maybe this paper can provide more analysis on how different operation conditions influence the final signals.\n3. In $\\underline{\\text{Section 4.4.1}}$, the authors state that \"FGN models trained with samples generated from larger time windows demonstrate superior predictive performance\". However, if I’m not mistaken, the results in $\\underline{\\text{Figure 4}}$ seem to show that models trained with smaller windows can also show superior performance, for example, in Subset 3, \"TW_60\" shows better results than \"TW_90\", which is a little confusing for me.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a model name Ensemble Multi-Term Fourier Graph Neural Networks for Remaining Useful Life prediction. The proposed approach contains several major components: a Fourier Graph Neural Network to model the spatial-temporal dependencies within sensor data; a multi-term ensemble learning framework to learn both long-term and short-term temporal dependencies; a sequence decomposition method to tackle the varying operation conditions. The proposed method is evaluated on a widely-studied RUL prediction dataset and shows superior performance compared to several traditional and spatial-temporal GNN models. The motivations for each of these components are clear, and ablation studies are conducted to demonstrate the necessities of these components.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My main concerns are shown in the strengths and weaknesses section. Overall I think this is a paper of good quality. However, if the authors can provide more ablations on the choice of the network backbone, illustrate how different operation conditions influence the sensor signals and explain the results in $\\underline{\\text{Figure 4}}$ more clearly, I can be more confirmed towards acceptance of this paper.", "broader_impact_concerns": "This paper does not present significant ethical implications or potential negative societal impacts that makes a Broader Impact Statement necessary.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a practical approach towards remaining useful life prediction. The approach consists of a Fourier graph neural network, a multi-term ensemble learning framework, and a sequence decomposition method. Empirical results validate the effectiveness of the approach.

While all reviewers recommended (weak) acceptance, the writing and format of the paper needs improvement. For example, Fig 1 is not readable as the modules are not expanded, and the upright room is all wasted; Misuses in citations format (please tell between \cite, \citeyear, and \citeauthor); just to name a few. Please carefully polish the paper and submit again.
Decision: Accept (poster)