LSTM-BASED-AUTO-BI-LSTM for Remaining Useful Life (RUL) Prediction: the first round of test results
Authors: Carlos Ferreira
Gil Gonçalves
Venue: Submitted to ICLR 2023 (2023)
Keywords: Remaining Useful Life
Predictive Maintenance
Machine Learning
Deep Learning
Autoencoder

Abstract: The Remaining Useful Life (RUL) is one of the most critical indicators to detect a component’s failure before it effectively occurs. It can be predicted by historical data or direct data extraction by adopting model-based, data-driven, or hybrid methodologies. Data-driven methods have mainly used Machine Learning (ML) approaches, despite several studies still pointing out different challenges in this sense. For instance, traditional ML methods cannot extract features directly from time series depending, in some cases, on the prior knowledge of the system. In this context, this work proposes a DL-based approach called LSTM-based-AUTO-Bi-LSTM. It ensembles an LSTM-based autoencoder to automatically perform feature engineering (instead of manually) with Bidirectional Long Short-Term Memory (Bi-LSTM) to predict RUL. We have tested the model using the Turbofan Engine Degradation Simulation Dataset (FD001), an open dataset. It was generated from the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) from the Prognostics Center of Excellence (PcoE), from the National Aeronautics and Space Administration (NASA). The objective is to release the first round of analytical results and statistical visualisations of the model application, which will guide us in future improvements.

Under review as a conference paper at ICLR 2023
LSTM-BBASDDAUTTO-BBBLSMM FOR REMAINING USEFUL LIFE (RUL) PREDICTION: THE FIRST ROUND OF TEST RESULTS
Anonymous authors Paper under double-bling review
ABSTRACT
The Remaining Useful Life (RUL) is one of the most critical indicators to detect a component's failure before it effectively occurs. It can be predicted by histori- cal data or direct data extraction by adopting model-based, data-driven, or hybrid methodologies. Data-driven methods have mainly used Machine Learning (ML) approaches. despite several studies still pointing out different challenges in this sense. For instance, traditional MI methods cannot extract features directly from time series depending, in some cases, on the prior knowledge of the system. In this context, this work proposes DL-based approach called LSSTM-based-AUTO-BB LSTM. It ensembles an LSTM-based autoencoder to automatically perform fea- engineering (instead of manually) with Bidirectional Long Short-Term Mem- ory (Bi-LSTM) to predict RUL. We have tested the model using the Turbofan En- gine Degradation Simulation Dataset (FD001), an open dataset. was generated from the Commercial Modular Jero-Propulsion System Simulation (C-MAPSS) from the Prognostics Center of Excellence (PcoE), from the National Aeronautics and Space Administration (NASA). The objective is release the first round of analytical results and statistical visualisations of the model application, which will guide in future improvements.
INTRODUCTION
Cyber-Physical Systems (CPS), Internet of Things (IoT), Internet of Services (IoS), and Data Analyt- ies have built Industry 4.0, which has improved manufacturing efficiency and helped industries face challenges such as economic, social, and environmental (Ruiz-Sarmient et al., 2020). Condition- Based Maintenance (CBM) performs machines and components' maintenance routines based on their needs, and Prognostics and Health Management (PHM) monitors components' wear evolution using indicators. PHM is proactive way of implementing CBM by predicting the Remaining Use- ful Life (RUL), one of the most critical indicators to detect a component s failure before it effectively occurs /Wang et al. 2021
Huang al. 2019 Wu et al. 2017 Kan et al., 2015)).
RUL can be predicted by historical data or direct data extraction by adopting model-based, data- driven, or hybrid methodologies. Model-based methods are challenging, expensive, and time- consuming to develop in complex equipment due to the need for prior system knowledge. Data- driven methods have mainly used Machine Learning (ML) approaches. They less complex and expensive, more applicable and provide suitable trade-off between complexity, cost, precision, and applicability (Cheng et al.) [022]] Mrugalskal 2019 Li et al. 2019 Yang et 2016). although they require large amounts of historical data for development (Liewald et al. 2022
Meanwhile, despite the increased use of MI. to predict RUL,, several studies have still pointed out different challenges in this sense (Huang et al.] (019)). For example, most ML methods' accuracy in predicting RUL largely depends on the feature extraction quality, and their performance is affected in the case of very complex systems with multiple components, multiple states, and considerable amount of parameters (Zhao et al. 2021 Chen et al. 2019). Moreover, the literature has also reported that most of these models do not consider operation conditions; the machines operate in different states, even on the same shop floor. It significantly impacts the degradation behaviour and raw sensor signals that may be non-stationary, nonlinear, and mixed with much noise (Liu et al.

---

Under review as a conference paper at ICLR 2023
2020a). Finally, traditional ML methods cannot extract features directly from time series depending on the complex intermediate transformation and, in some cases, depending on the prior knowledge of the system (Cabrera et al. 2020).
To overcome several challenges and improve the accuracy of RUL prediction, there has been a prominent use of Deep Learning (DL) Methods, especially Recurrent Neural Networks (RNN) and
Long Short-Term Memory (LSTTM, besides other variations (Zhu et al.) 2019 Li et al. 2020 Liu al. 2020b). They have emerged and achieved outstanding results in different areas due to their strong capacity to map the relationship between degradation paths and measured data. Also, these methods can learn feature representation automatically, such that it is not necessary to design fea- tures manually, eliminating the need for previous knowledge of the system (Zhu et al. (211). Fi- nally, DI. methods have high capacity to deal with many complex data (Kong et al.] 2019) [17]. Nonetheless, the literature reports some drawbacks, such as the data deficit issue, especially con- sidering the varying operation conditions and the degradation mode of the components in practical industrial applications (Liu et al.
2020a).
In this context, Ferreira & Gonçalves (2022). aamong other results, have mapped 14 challenges in using ML methods for RUL prediction and pointed out some approaches used in the literature to overcome these challenges. From this collection of approaches. it was possible to propose an architecture called SSSTM-based-AUTO -----AUTO---LLS Bi-LSM,, which ensembles Autoencoder (Unsu- pervised/Reconstructtv Learning Technique) with the DI. method Bidirectional Long Short-Term Memory (Bi-LSTM). The autoencoder aims to perform feature engineering automatically (instead of manually). The Bi-LSTM aims to predict the RUL based on the outputs of the autoencoder. This type of ensembling is, at least, very few applied in the RUL prediction process. To test our model, we have explored the turbofan engine problem through the dataset gathered from PCoE/NASA. Therefore, this work aims to release the first round of analytical results and statistical visualisations of the model application.
The remaining of this work is as follows. Section 2 describes the problem and the used dataset, and Section 3 introduces the experimental context, and Section presents the results and compares them with the literature. Finally, Section 6 concludes this work by giving some directions for future works.
architecture. Section 4 describes the
THE PROBLEM AND DATASET
2.1 THE PROBLEM
PHM has been essential topic in the industry for predicting the state of assets to avoid downtime and failures (NASA) 2022). In the aircraft industry, attempted maintenance is critical to ensure operation safety (heng et al. 2018), besides increasing economic efficiency (Deng et al. 2019). According to the International Air Transport Association (IATA), maintenance costs of the major aviation companies reached $15.57 billion between 2012 and 2016, which represented growth of 3% (Kraus & Feuerriegel 2019). Turbofan engines, specifically, are responsible for about 30% of the failures in an aircraft, and in great-proportion accidents, these systems have been the root cause in 40% of the cases. Besides. propulsion device maintenance costs share about 40% of the full aircraft maintenance costs (Tang al. (002)). The main components of turbofan engine include the fan, low-pressure compressor (IPI.), high-pressure compressor (HPC), combustor, high-pressure turbine (HPT), and low-pressure turbine (L.PT), and nozzle.
2.2
THE DATASET
The dataset was gathered from the Prognostics Center of Excellence PCoF, from the National Aeronautics and Acrospace Administration (NASA). In this sense, the information provided in this subsection was retrieved from that source NASA (2022) and [Saxena et al. (2008).
Engine degradation simulation was carried out using Commercial Modular Aero- Proo-Propulsio System Simulation (C-MAPSS). Four different datasets (FD001, FD002, FD003, and FD004) were simu- lated under various operational conditions. They comprised a range of values for three operating conditions Altitude, from to 42K ft., Mach Number, from 0 to 0.84, and Throttle Resolver An- gle (TRA), from 20 to 100 and fault modes High-Pressure Compressor Degradation or/and Fan
2

---

Under review as a conference paper at ICLR 2023
Degradation combinations. Records of several sensor channels to characterise fault evolution. The objective of these datasets is to predict the RUL of each engine in the test dataset. RUL can be defined as the equivalent number of flights remaining for the engine after the last data point in the test dataset.
The datasets consist of multiple multivariate time series. Each data set is further divided into training FD001, train FD002, train_ FD003, and train_FDoOBe and test subsets (test_DD00) FDs02, test_ FDO03, and test_ FD001). Each time series is from a different engine, i.e., the data can be considered from a fleet of engines of the same type. Each engine starts with different de- grees of initial wear and manufacturing variation, unknown to the user. This wear and variation are considered normal, i.c., it is not considered fault condition.
Three operational settings substantially affect engine performance were also included in the datasets: Altitude, Mach Number, and TRA. The data is contaminated with sensor noise. The engine usually operates at the start of each time series, developing fault at some point. In the training dataset, the fault grows in magnitude until system failure. The time series sometimes ends before system failure the test dataset. also provides a vector of true Remaining Useful Life (RUL) values for the test data (RUL_FDO01, RD022 RUL_FD003, and RUL_FD001). The train and test datasets are presented through 26 columns of numbers (Unit/Engine Number, Time (Cycles), Operational Setting ..... Operational Setting 3, Sensor Measurement I, ... Sensor Measurement 21), comprehending a different variable. Each row is a snapshot of data taken during a single operational cycle. Table A presents summarised description of the datasets, and Table A2 describes the sensor measurement variables (columns to 26).
3 THE LSSM-BASHD-AUTO--B-SSS
The proposed architecture consists of ensembling two methods. First, we used an LSTM-based Au- toencoder perform automatic feature engineering (instead of manually) through the raw dataset. Then we applied Bi-LSTM initialised through the autoencoder outputs to predict the RUL (predic- tion model). Both methods are explained in detail, and an overview of the entire architecture is presented in the following subsections.
3.1 LSTM-BASED AUTOENCODER
The autoencoders are unsupervised NN structures (Ren et al. 2021], completely symmetrical (Ren et al. 2018), with one input layer, hidden layer, and one output layer (Xia et al. 2019). The input layer and first half of the hidden layer build the encoder. The second half of the hidden layer and the output layer build the decoder. The fundamental objective is reconstructing original data by minimising the error between the network output data and the original data and initialising a deep NN (Xia et al.] 2019). The number of nodes in each hidden layer is less than the number of nodes in the input layer and the output layer, creating a type of bottleneck (Chen et al. 2020). These structures have improved the optimal model determination through the random network initialisation (Xia et al. 2019) and have also demonstrated great ability for feature extraction (Chen et al. 2020). The generic mathematical autoencoder construction is shown in equations (1) and (2).
y oW b) x' = o W b)
(2)
In equation (1), y represents the features learned by the encoder (code), represents the input vector, W is the weight matrix between the input layer and the hidden layer, b is the bias, and is the activation function. equation (2), represents the vector constructed through the features learned from the hidden layer, and W" is the weight matrix between the input layer and the hidden layer. The remaining parameters are the same in equation (1).
In this work, we have set up autoencoder by applying Long Short-Term Memory (LSTM) model in the encoder and decoder layers (LSTM-basee Masedd Autoencoder). The main idea behind the LSTM to capture the dependence of the current state on the previous state, which means in the forward direction (Zhao et al., 2017). The general LSTM cell (neuron) structure can be divided into three parts gates. First, the input gate decides when the model can receive new information state. Next,
3

---

Under review as a conference paper at ICLR 2023
(a) General LSTM structure
(b) LSTM-Based Autoencoder structure
Figure 1: General LSTM and LSTM--BasedAAutoencoderr structure.
the forget gate decides when the model can forget the previous information state. Finally, the output gate decides which information to output. These gates determined by the input at the current instant and the output at the previous instant through sigmoid and hyperbolic tangent activation functions (Liu et al. (2021)) Figure presents the general LSTM structure and the LSTM-Basee Autoencoder structures.
3.2 THE BI-LSTM
The Bi-LSTM derives from the LSTM, adapted to learning the most representative features hidden in the Condition Monitoring (CM) data (Huang et al. 2019). With Bi-LSTM, the main idea is to capture past and future information by processing the CM data forward and backwards through two separate hidden layers. Equations sets (3) and (4) describe the separated hidden layers function at step t. The symbols + and + denote, respectively. the forward and backward process (Huang al. 2019). Deeper detailed considerations regarding Bi-LSTM can be found in Zhao et al. (2017) and Zhang et al. (2018a). Figure shows the general Bi-LSTM structure, and Figure shows an overview of the proposed architecture.
7 f(Z h LSTM)
=
Z tanh (W (W R. Tr - T.))
{ o T 12 h b b))
7 a (W b
(3)
d. = 2.W.2t+H, 6.)
J. غ. @ 7
h. tanh (7))
h. I(F Tr 5 (ITT)
E tanh @Wn - RR 1-1 + +T))
< oW, R, h + b.)
T a aW, R 1-1 [,))
(4)
5 a oi R. b.)
R.
C & @ T,t T f.
h. = tanh (t, 5
In the equations sets (3) and (4), 3 BLSTM and 5 LSTM are the parameters set of the forward and backward processes, shared by all the time steps and learndd during model training. W k, Wr
4

---

Under review as a conference paper at ICLR 2023
R lxp are imput weights (related to Zi, () of the forward and backward process, respectively. Rs, R E L are recurrent weights (related to i-1 process, respectively. kR, Tx E R2 are bias weights of the forward and backward process, respec- tively. Finally, a (logistics sigmoid) and tanh (hyperbolic tangent) are pointwise nonlinear activa- tion functions, denotes pointwise multiplication of two vectors, L denotes de dimensionality of the hidden neurons and k E f,,,,f.0.
+ to e+1) of the forward and backward
Iepetid
LST Betword
Figure Bi-LSTM structure.
Figure 3: SSTM-based-AUTO--BI
4 EXPERIMENTAL CONTEXT
4.1 NORMALISATION
The normalisation was performed by applying the z-score method, as described in Che et al. (2019); Wang et al. (2019: : Tiang et (2020). The mean of the normalised values is and the standard deviation of the normalised values is 1. The normalised values represent the number of standard
deviations that the original value from the mean and are calculated by the equation (5).
I m4)
2,
(5)
The 2; is the normalised value; x; is the raw data in the sensor i; μ4 is the average value of the ith sensor, and a; is the standard deviation of the ith sensor.
5

---

Under review as a conference paper at ICLR 2023
4.2 PIECE- WISE FUNCTION
In real applications, machine component degrades less at the beginning of life. On the other hand, degradation increases as is close to end of life (Zheng et al. 2017). means that the com- ponent degradation is early unclear, and the RUL of similar sensor data may vary sensible (Saxena al. a 2008). For instance, Figure shows ten selected sensors from engine 65 the training dataset. As we can perceive, for all chosen sensors, it is only possible to obtain some trend, approximately from cycle 68, A Piece-Wise linear RUL target function was assumed. The maximum threshold for RUL was 125 to deal with this condition and better model the RUL behaviour throughout time, such as in several literature examples (see, among others, Mrugalska (2019), Liu et al. (2020b); [Saxena al. (2008); Jiang al. (020)). This is crucial concern since if large RUL is assumed, there can be significant fluctuation of the predicted RUL in the early stage. Otherwise, if small RUL
considered, the predicted RUL can be confined to small range (Saxena et al. 2008
1.30
4.70
50
501
111
125
153
Figure 4: Selected sensors of engine 65 from the FD001 training dataset.
4.3 PERFORMANCE EVALUATION CRITERIA
To evaluate the proposed method's performance, we have adopted the Root Mean Square Error (RMSE) between the predicted RUL and the ground truth RUL, which is standard metric reported in the literature (see, among other, Li et al. (219); Zheng et al. (2018): Falcon et al. (2020); Zhao al. (2019)). allowed us to compare our results with the literature state-of-the-ee mathematically. The RMSE gives equal weights for early and late predictions and is defined by equation (6).
N RMSE I
RUL, R
In equation (6), RMSE is the computed error; N is the total number of data samples, and RÚ L, and RUL, respectively the predicted RUL and the ground truth RUL concerning the ith data point.
4.4 HOMM-BASED-AUTO-BEEE
SETUP
The proposed architecture was tested in two ways. First, an utterly aleatory setup was performed regarding the hyperparameters. This setup was considered the initial setup baseline setup. Second, the literature was surveyed more in-depth, and some hyperparameters were changed accordingly. Specifically. regarding the number of epochs in the LSTM-based Autoencoder, have performed specific runs to determine the best number. This setup was considered the testing setup. The initial setup and testing setup fully detailed follows.

---

Under review as a conference paper at ICLR 2023
4.4.1 INITIAL SETUP (BASELINE SETUP)
LaaM- Autoencoder * LSTM-based layers (encoder and decoder) with 24 neurons each (number of features in the dataset). Optimizer = ADAM and Loss Function Binary Crossentropy. Validation Split = 10% (standard throughout the literature). Epochs 32. Batch-size 50 (about 0.25% of the training dataset).
= 31 (minimum
engine running length in the test dataset).
Bi-LSTM = Single layer with Activation Function = Hyperbolic Tangent. Number of Neurons 100. Selected features (automatic selection) from the LSTM-based Autoencoder = 12. Dense = 30, with Activation Function = ReLu. Output Layer with Activation Func- tion Linear. Optimizer RMSProp and Loss Function MSE. Epochs = 32. Batch-size 200 (about 1% of the training dataset). Time- Window = 31 (minimum engine running
=
length in the test dataset).
4.4.2 TESTING SETUP
LSTM-based Autoencoder LSTM-based layers (encoder and decoder) with 24 neurons each (number of features in the dataset). Optimizer ADAM and Loss Function = MSE (most common used). Validation Split = 10% (standard throughout the literature). Epochs 50 (see next section). Batch-size 50 (about 0.25% of the training dataset). Time-Window =3 (minimum engine running length in the test dataset).
Bi-LSTM = Single layer with Activation Function Hyperbolic Tangent. Number of Neurons in the set {24, 48, 72, 96, 120, 144, 168, 192, 216, 240). Selected features (automatic selection) from the LSTM-based Autoencoder in the set {6. 12, 18, 24). Dense = 30, with Activation Function = ReLu (most common used). Output Layer with Activation Function Linear (most common used). Optimizer ADAM and Loss Function = MSE (most common used). Epochs = 32. Batch-size = 200 (about 1% of the training dataset). = 31 (minimum engine running length in the test dataset).
In this work, four different setups were tested for the LSTM-based Autoencoder, and 41 different configurations (one for baseline and 40 for testing setup) were tested for the LSTMM-ased-AUU -UUTO- Bi-LSTM. The results and comparison with state of the art in literature are presented the next Section. To reduce the influence of random factors, the reported results in the next Section are the average of ten independent runs. Finally, all the experiments were performed using Colaboratory from Google Research (Colab).
RESULTS
First, we have analysed the effective gain after each epoch in the LSTM-based Autoencoder, consid- ering the loss (training and validation) and accuracy (training and validation). The objective was to determine the best value for this hyperparameter. The analysis also considered the processing time for 32, 64, 96 and 128 epochs. Regardless of the number of epochs setup, we could observe that, from epoch 32, the expressive gain in loss and accuracy decreases fast. However, the processing time of the LSTeM-based Autoencoder increases as quickly as we increase the number of epochs (329.54, 628.79, 928.83 and 1169.35 seconds, respectively). Thus, considering this trade-off (pro- cessing time against effective gain) and aiming to avoid possible discrepancies (due to the stochastic nature of the method) the best value considered was 50 epochs. Figure 5 shows the loss (a) and accuracy (b) for the LSTM-based autoencoder.
Second, we have analysed the behaviour of the Bi-LSTM by fixing the LSSTM-basd Autoencoder setup and varying the configuration of the number of neurons and selected features. The remaining hyperparameters were as the testing setup. The objective was to compare the results with our baseline and the literature's state of the art. To do this, we ran the DaT---aaed--UTO--R Bi-L 400 times (about 106 running hours) through 40 different configurations. The results are graphically presented as follows. Figure 6 shows the RMSE evolution throughout the tests performed. The three best averages are marked in the graph, with two occurring using 24 features and one using only six features. It is also possible to perceive that the lines for 12, 18 and 24 features have smoother path than the line for six features. Also, the final trend appears to decrease (faster with 24 features) in those lines while it seems increase with six features.
7

---

Under review as a conference paper at ICLR 2023
Tain
Val
0.30
0.25
Loss
0.15
0.10
Epochs
(a) L.STM-basse Autoencoder Loss
(b) LSTM-based Autoencoder Accuracy
Figure 5: LSSTM-Based-AAtteenedee Loss and Accuracy
16:30
15:0
15:40
1520
1452
1495
1460
Number Nearona
fetore
Roteres
Rutern
Festanc
Figure 6: RMSE evolution throughout the tests (average of ten runs).
Still, regarding the behaviour of the Bi-LSTM, we have plotted the boxplots of the tests performed considering 6 (a), 12 (b), 18 (c) and 24 (d) features. Figure shows these results, and it is possible to visualise that the more stable distribution relies on 18 features, which also contain the lowest number of outliers.
17.50
10.00
22.50
Figure 7: Distributions of the tests performed considering 6, 12, 18 and 24 features.
Third, we have captured the total average processing time of the LSTM-based-aeeee AEL--AUTOO--SSSSS Figure 8 shows the evolution of this time. The minimum time obtained was 471.38 seconds (using 24 features in the Bi-LSTM), and the maximum was 1768.08 seconds (using 18 features in the Bi- LSTM).. Also, it is possible observe more stable trend when using 24 features in the Bi-L.TM..
8

---

Under review as a conference paper at ICLR 2023
%
144
192
216
Ninber of (8-5 (B-5TM))
foorge, SHiSTI
Tent, Avenge BASTRU
HenTon, F-55TP
Tomt BASTTU
Figure 8: LSTM-baced-UUTO--B---ST total average processing time.
Finally, we compared results with the state-of-the-art literature. Figure 9 highlights the RMSE average of the baseline setup (17.38 in red) and the best RMSE average of the testing setup (14.67 in blue) with the state-of-the-art literature. Considering the testing setup result, it possible to that it is well positioned in terms of the studied dataset. Indeed, from the architectures designed for automatic feature extraction (dark grey), only one (Bi-LSTM + CNN) outperforms the result in terms of RMSE. The remaining architectures that outperform the proposed one must perform some feature (or sensor) selection. This implies previous knowledge regarding the system and makes challenging apply them directly. A list of the methods and their acronyms is presented in Table/A3 Additional characteristics of the state-of-the-art methods are shown in Table A4 The symbol N.A. means that data/information did not apply to the method is not available in the text to the best of our evaluation.
11.55
1614
K112
6.M
11%
16:44
Figure 9: State-of-the-art Results (RMSE- Dataset FD001).
CONCLUSIONS
architecture was shown in these initial tests be within the state-of-the-ar literature for subdataset FD001, considering the results demonstrated. Especially when compared with methods that performed automatic feature extraction, our architecture proved far superior to the others, except for the work in Remadaa et al. (2020). Although Colab may have presented some instability and influenced the capture of processing times, it is estimated that the obtained times are reasonable regarding the number of instances used.
The
First, we want to vary the number of neurons in the encoder and decoder layers of the LSTM- based autoencoder. Then, we want to test different hyperparameters from those tested in this work, namely varying the number of epochs used and the number layers used the predictive model (Bi-LSTM). Next, intend to perform more tests using the subdatasets FD002, FD003 and FD004, which are more complex in terms operational conditions and fault modes. Further, we intend to evaluate these future results using the Score Function introduced in Saxena al. (2088 jointly with RMSE. Finally, we will test this architecture with categorical datasets generated through embedded monitoring systems (system log files).

---

Under review as a conference paper at ICLR 2023
REFERENCES
Gurkan Aydemir and Burak Acar. Anomaly monitoring improves remaining useful life estimation of industrial machinery. Journal of Manufacturing Systems, 56(June):463466) 2020. doi: 10.
j0s6/..jmsy.22200066
Sourajit Behera and Rajiv Misra. Generative adversarial networks based remaining useful life es- timation for НоТ. Computers and Electrical Engineering, 92(June 2020);107195, 2021. doi:
10.1016/j co0.:eoo...mpeeeeeg2222.......
Diego Cabrera, Adriana Guamán, Shaohui Zhang, Mariela Cerrada, René Vinicio Sánchez, Juan Ce- vallos, Jianyu Long, and Chuan Li. Bayesian approach and time series dimensionality reduction to LSTM-based
for fault diagnosis of reciprocating compressor. Neurocomputingg 380:51 66, 2020. doi: wtt.1//...euuomm200 2019. 11.006.
Changchang Che, Huawei Wang, Qiang Fu, and Xiaomei Ni. Combining multiple deep learning algorithms for prognostic and health management of aircraft. Aerospace Science and Technology, 94:105423, 2019. doi:
Chong Chen, Ying Liu, Shixuan Wang. Xianfang Sun, Carla Di Cairano-Gilfeddees Scott Titmus, and Aris A. Syntetos. Predictive maintenance using cox proportional hazard deep learning. Ad- vanced Engineering Informatics, 44(February): 101054 2020. doi: 10. 10.16.//j.aii.20 aei.222.. 10100.0
Jinglong Chen, Hongjie Jing, Yuanhong Chang, and Qian Liu. Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process. Reliability Engineering and System Safety, (85(Janaar):3322338 2019. doi: 10.1016/j res0////.ess...... res1. 2019.01.006
Yiwei Cheng, Kui Hu, Jun Wu, Haiping Zhu, and Xinyu Shao. A convolutional neural network based degradation indicator construction and health prognosis using bidirectional long short-term memory network for rolling bearings. Advanced Engineering Informatics, 48(June 2020): (000)):01244: 147, 2021. doi: 10. (0.1016/j.ei.2222 202................... 1011247.
Kunyuan Deng, Xiaoyong Zhang, Yijun Cheng, Zhiyong Zheng, Fu Jiang, Weirong Liu, and Jun Peng. A Remaining Useful Life Prediction Method with Automatic Feature Extraction for Air- craft Engines. In 2019 18th IEEE International Conference On Trust, Security And Privacy In Computing And Communications// Engineering (TrustCom//Bel ConnCooom//BDaaE) volume 2019-January, pp. 686-692. IEEE, aug 2019. doi: 10. DW000TTTSSCCm//BBDautt.200000
IEEE International Conference On Big Data Science And
Alex Falcon, Giovanni D' Agostino, Giuseppe Serra, Giorgio Brajnik, and Carlo Tasso. A neural turing machine-based approach to remaining useful life estimation. Proceedings of the Annual Conference of the Prognostics and Health Management Society, PHM, 2020-June:1-8, 2020. doi: 10. N0.11099/CPHM44022.220 20200.9222222011121121 918703..
Carlos Ferreira and Gil Gonçalves. Remaining Useful Life prediction and challenges: A literature review on the use of Machine Learning Methods. Journal of Manufacturing Systems, 63:550-562, apr 2022. doi: 10. 1iy.016/..myy2222255.000
Narendhar Gugulothu, Vishnu TV, Pankaj Malhotra, Lovekesh Vig, Puncet Agarwal, and Gautam Shroff. Predicting Remaining Useful Life using Time Series Embeddings based on Recurrent Neural Networks. Journal of Learning Disabilities, 12(6):408-44 sep 2017. doi: 10.48550/ 17090.1709
Chang Woo Hong, Kwangsuk Lee, Min-Seung Ko, Jae-Kyeong Kim, Kyungwon Oh, and Kyeon Hur. Multivariate Time Series Forecasting for Remaining Useful Life of Turbofan Engine Using Deep-Stacked Neural Network and Correlation Analysis. In 2020 IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 63-70. IEEE, feb 2020. doi: 10. 10.1109/iiCoomp4861 8.2000000-98.
Cheng Geng Huang, Hong Zhong Huang, and Yan Feng Li. A Bidirectional LSTM Prognostics Method Under Multiple Operational Conditions. IEEE Transactions on Industrial Electronics, 66 (11):8792-8802 2019. doi:
2019.2891463
10

---

Under review as a conference paper at ICLR 2023
Yijie Jiang, Yi Lyu, Yonghua Wang, and Pin Wan. Fusion network combined with bidirectional LSTM network and multiscale CNN for useful life estimation LSTM network and multiscale CNN for useful life estimation. 12th International Conference Advanced Computational In- telligence, ICACI 2020, pp. 620-627, 2020. doi: 10.1 109/CAC149188 85 2020.9177774.
Man Shan Kan, Andy C.C. Tan, and Joseph Mathew. A review on prognostic techniques for non- stationary and non-linear rotating systems. Mechanical Systems and Signal Processing, 62-63: 1-20, oct 2015.
doi: 10. 10.1016/j.ymoe 20016f.ymssp.2015..222
Zhengmin Kong, Yande Cui, Zhou Xia, and He Lv. Convolution and long short-term memory hybrid deep neural networks for remaining useful life prognostics. Applied Sciences (Switzerland), 9(19), 2019. doi: 10.3390/app9 1944ppppp1111.61
Mathias Kraus and Stefan Feuerriegel. Forecasting remaining useful life: approach via variational Bayesian inferences. Decision Support Systems, (25(Apri):113310) 2019. doi: (0.1016/j.dss.2111 11100111100.....1111111111111000
Interpretable deep learning
Jialin Li, Xueyi Li, and David He. A Directed Acyclic Graph Network Combined With CNN and LSTM for Remaining Useful Life Prediction. IEEE Access, 7:75464-75475, 2019. doi:
Xiang Li, Xiaodong Jia, Yinglu Wang. Shaojie Yang, Haodong Zhao, and Jay Lee. Industrial Remaining Useful Life Prediction by Partial Observation Using Deep Learning With Super- vised Attention. IEEE/ASME Transactions on Mechatronics, 25(5)22211-2251, oct 2020. doi: 10. 10.11099MMEC...2202209999
Mathias Liewald, Thomas Bergs, Peter Groche, Bernd Arno Behrens, David Briesenick, Martina Müller, Philipp Niemietz, Christian Kubik, and Felix Müller. Perspectives on data-driven models and its potentials in metal forming and blanking technologies. Production Engineering, 2022.
doi: 10.1007/s1 17000002--1000200115----0
Chenyu Liu, Alexandre Mauricio, Junyu Qi, Dandan Peng, and Konstantinos Gryllias. Domain adaptation digital twin for rolling element bearing prognostics. Proceedings of the Annual Con- ference of the Prognostics and Health Management Society, PHM, 12(1):1-10, 2020a. doi: 10.36001/phmmcof222200 129..1294.
Chongdang Liu, Linxuan Zhang, Jiahe Niu, Rong Yao, and Cheng Wu. Intelligent prognostics of machining tools based on adaptive variational mode decomposition and deep learning method with attention mechanism. Neurocomputing, 417:239 254, 2020b. doi: 10. 00.1016j..eucom.. 2020. 06.116.
Hui Liu, Zhenyu Liu, Weiqiang Jia, and Xianke Lin. Remaining Useful Life Prediction Using Novel Feature-Attention- Based End-to-End Approach. IEEE Transactions on Industrial Infor- matics, 17(2):1197-1207, 2021a. doi: 10.1 10.1109/T1.222222283777.
Kailong Liu, Yunlong Shang, Quan Ouyang, and Widanalage Dhammika Widanage. A BriveDriven Approach with Uncertainty Quantification for Predicting Future Capacities and Remaining Useful Life of Lithium-ion Battery. IEEE Transactions Industrial Electronics, 68(4):3170-3180 b021b. doi: 10. 2..109//1E.2022.297333
Yuanjun Liu and Xingang Wang. Deep Attention: A Self-Attention based Neural Network for Remaining Useful Lifetime Predictions. 2021 7th International Conference on Mechatronics and Robotics Engineering, ICMRE 2021, pp. 98-105, 2021. 9384841.
doi: 10.1 11199//CRE51111..22000
Beata Mrugalska. Remaining Useful Life as Prognostic Approach: A Review. Human Systems Engineering and Design, pp. 689-695. Springer International Publishing, 2019. doi: 10. 1007/
978-3-030-02053-8.0
NASA. National Aeronautics and Aerospace Administration - Prognostics Cen- ter of Excellence PCoE. 2022. URL https::/ httts://www.naaa..ovo
prognostics- center-eeeeeeee -eteeer----eeeeeleeee wieneeeeeleeeeeeee data- -tt.eeeeddttt-ttteepptttttrrr visited on
2022-09-01.

---

Under review as a conference paper at ICLR 2023
Ikram Remadna, Sadek Labib Tenrissa, Ryad Zemouri, Soheyb Ayad, and Noureddine Zerhouni. Leveraging the Power of the Combination of CNN and Bi-Directional LSTM Networks for Air- craft Engine RUL Estimation. In 2020 Prognostics and Health Management Conference (PHM- Besançon), pp- 116-121. IEEE, may 2020. doi: 10.11 (e///HM--Baanccoeee 06.2020.00025.
Lei Ren, Li Zhao, Sheng Hong, Shiqiang Zhao, Hao Wang, and Lin Zhang, Remaining Useful Life Prediction for Lithium-lon Battery: A Deep Learning Approach. IEEE Access, 6:50587-50598,
2018. doi: 00.110/ACCESS22288228558
Lei Ren, Jiabao Dong, Xiaokang Wang. Zihao Meng. Li Zhao, and M. Jamal Deen. A Data-Drive CNILLCCN--STMM Prediction Model for Lithium-Ion Battery Remaining Useful Life. IEEE Transactions Industrial Informatics, 17(5):3478-344 2021. doi: 10.
Jose Raul Ruiz-Sarmiento, Javier Monroy, Francisco Angel Moreno, Cipriano Galindo, Jose Maria Bonelo, and Javier Gonzalez-Jimenes A predictive model for the maintenance of industrial ma- chinery in the context of industry 4.0. Engineering Applications of Artificial Intelligence, 87 (January 20199:003289, 2020. doi:
2019 103289.
Abhinay Saxena, Kai Goebel, Don Simon, and Neil Eklund. Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 International Conference on Prognostics and Health Management, volume 41, pp. 1-9. IEEE, oct 2008. doi: 10. 10000000...0000.771111
Ran Tang, Gang Fang, Guangjia Liu, and Heng Wang. Propulsion life prediction based on support vector machine. IOP Conference Series: Earth and Environmental Science, 687(1), 2021. doi:
Vishnu TV, Diksha, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. Data-driven Prognostics with Predictive Uncertainty Estimation using Ensemble of Deep Ordinal Regression Models. ternational Journal of Prognostics and Health Managemen//ppecial Issue on PHM Applications of Deep Learning Emerging Analytics, 10(4), 2019, doi: v1014.2612.
Biao Wang, Yaguo Lei, Naipeng Li, and Tao Yan. Deep separable convolutional network for maining useful life prediction of machinery. Mechanical Systems and Signal Processing, 134: 106330, 2019.
doi: 10.1016/j. (0.1016/j.ymsss.222 wmtss.ymsspp1p11m 2019........1.00. 106330.
Jiujian Wang, Guilin Wen, Shaopu Yang. and Yongqiang Liu. Remaining Useful Life Estimation in Prognostics Using Deep Bidirectional LSTM Neural Network. In 2018 Prognostics and System Health Management Conference (PHM- SrMM-hhngggnn), pp. 1037-1042. IEEE, oct 2018. doi: 10. 1109/PHM- CaMMM--Cnngggig2200 2018.0018 144.
Ruihan Wang, Hui Chen, and Cong Guan. A Bayesian inference-based approach for performance prognosties towards uncertainty quantification and its applications on the marine diesel engine. ISA Transactions, 118:159-173, dec 2021. doi: 10.1016/j isatra.202....00
Dazhong Wu, Connor Jennings, Janis Terpenny, Robert X. Gao, and Soundar Kumara. A Com- parative Study on Machine Learning Algorithms for Smart Manufacturing: Tool Wear Prediction Using Random Forests. Journal of Manufacturing Science and Engineering, 139(7):1-9) jul 2017. doi: 10. 11 11/51114000 101.5003350.
Min Xia, Teng Li, Tongxin Shu, Jiafu Wan, Clarence W. De Silva, and Zhongren Wang. A Two-Stage Approach for the Remaining Useful Life Prediction of Bearings Using Deep Neu- ral Networks. IEEE Transactions on Industrial Informatics, 15(6):3703-371,, 2019. doi: 10. 10.1109/T1..018.1 28668677.
Zhe Yang, Piero Baraldi, and Enrico Zio. comparison between extreme learning machine and artificial neural network for remaining useful life prediction. In 2016 Prognostics and System Health Management Conference CChenddu,, number 201506280015, pp. 1-7. IEEE, oct 2016. doi: 10.1109/HHM.2016.78899999
Wennian Yu, II Yong Kim, and Chris Mechefske. Remaining useful life estimation using bidi- rectional recurrent neural network based autoencoder scheme. Mechanical Systems and Signal Processing, 129:764-788 780,,,0000000 aug 2019. doi: 10. 1100166.ymssp.2019..555
12

---

Under review as a conference paper at ICLR 2023
Wennian Yu, II Yong Kim, and Chris Mechefske. An improved similarity- based prognostic algo- rithm for RUL estimation using an RNN autoencoder scheme. Reliability Engineering System Safery, 999(Februrr):1009995 jul 2020. doi: 2...1666..ess.222....o. 106926.
Ansi Zhang, Honglei Wang. Shaobo Li, Yuxin Cui, Zhonghao Liu, Guanci Yang, and Jianjun Hu. Transfer learning with deep recurrent neural networks for remaining useful life estimation. Ap- plied Sciences (Switzerland), 8(12). 2018a. doi: 11.3390/app8122444
Jianjing Zhang. Peng Wang. Ruqiang Yan, and Robert X. Gao. Long short-term memory for machine remaining life prediction. Journal of Manufacturing Systems, 48(June):7--66, jul 2018b. doi:
imsy 0.00.111.
Huimin Zhao, Haodong Liu, Yang Jin, Xiangjun Dang, and Wu Deng. Feature Extraction for Data-
Driven Remaining Useful Life Prediction of Rolling Bearings. IEEE Transactions Instrumen- tation and Measurement, 70, 2021. doi: 10.1 10.1109/1M..22213335555
Rui Zhao, Ruqiang Yan, Jinjiang Wang, and Kezhi Mao. Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks. Sensors, 17(2):273, jan 2017. doi: 10.3390/ $17020273.
Sen Zhao, Yong Zhang, Shang Wang, Beitong Zhou, and Cheng Cheng. A recurrent neural network approach for remaining useful life prediction utilizing a novel trend features construction method. Measurement: Journal of the International Measurement Confederation, 146:279-225 288, 2019. doi: 10. ..a../....masuurmmet..22... 10....9.00004....
Caifeng Zheng, Weirong Liu, Bin Chen, Dianzhu Gao, Yijun Cheng, Yingze Yang, Xiaoyong Zhang, Shuo Li. Zhiwu Huang, and Jun Peng. A Data-driven Approach for Remaining Useful Life Pre- diction of Aircraft Engines. IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2018-Novembers 189, 2018. doi: 10. 10.1109/TCC.2118... 8569915
Shuai Zheng, Kosta Ristovski, Ahmed Farahat, and Chetan Gupta. Long Short-Term Memory Net- work for Remaining Useful Life estimation. 2017 IEEE International Conference Prognostics and Health Management, ICPHM 2017, pp. 88-95, 2017. doi: 10.1 00.100//CCHM22201779999
Jun Zhu, Nan Chen, and Weiwen Peng. Estimation of Bearing Remaining Useful Life Based on Multiscale Convolutional Neural Network. IEEE Transactions on Industrial Electronics, 66(4): 3208-3216, 2019, doi: 1991101112.2118222200
A APPENDIX
Table A1: C-MAPSS Dataset Description
Description
C-MAPSS Dataset
FD001
FD002
FD003
FD004
Training Engines Number
100
260
100
248
Testing Engines Number
100
259
100
248
Operational Conditions
6
6
Fault Modes
1
2
Max. Life Spam (Cycles)
362
378
525
546
Min. Life Spam (Cycles)
128
128
145
128
Train Instances (rows)
20.631
53.759
24.720
61.249
Test instances (rows)
13.096
33.991
16.569
41.214
13

---

Under review as a conference paper at ICLR 2023
Table A2: Sensor measurement variables description
Variable (Sensor Measurement)
Description
Unit
T2
The total temperature the fan inlet The total temperature at the LPC outlet The total temperature at the HPC outlet The total temperature at the LPT outlet
R
T24
R
T30
R
T50
R
P2
Pressure at fan inlet
psia
PI5
Total pressure in bypass-duct
psia
P30
Total pressure at the HPC outlet
psia
Nf
Physical fan speed
Rpm
Nc
Physical core speed
rpm
epr
Engine pressure ratio (P50/P2)
Ps30
Static pressure at HPC outlet pps/psi
psia
phi
Ratio of fuel flow to Ps30
NRf
Corrected fan speed
pm
NRc
Corrected core speed Bypass Ratio
rpm
BPR
farB
Burner fuel-air ratio
htBleed
Bleed Enthalpy
Nf dmd
Demanded fan speed
rpm
PCNfR_dmd
Demanded corrected fan speed
rpm
W31
HPT coolant bleed
Ibm/s
W32
LPT coolant bleed
lbm/s
Table A3: Methods described the literature and their acronyms
Method
Acronym
Anomaly Triggered Long Short-Term Memory
AT-LSTM
Bidirectional Gated Recurrent Units
BGRU
Bidirectional Long Short-Term Memory
Bi-LSTM
Deep- Bidirectional Long Short-Term Memory
Deep-Bi-LSTM
Conditional Generative Adversarial Network
CGAN
Convolutional Neural Networks
CNN
Deep & Attention Network
DAN
Deep Gated Recurrent Unit Network
DGRU
Deep Long Short- Memory
Deep LSTM
Deep Separable Convolutional Network
DSCN
Direct Acyelic Graphic Network
DAG
D-Long Short-Temn Memory
DLSTM
Extreme Learning Machine
ELM
Feature Attention Mechanism BRGU CNN
AGCNN Gated Recurrent Unit
GRU
Long Short-Term Memory
LSTM
Multiscale Convolutional Neural Networks
MSCNN
Neural Turing Machine
NTM
Ordinal Regression Censored Estimation Long Short-Term Memory
ORCE-LSTM
Polynomial Regression
PR
Stacked Sparse Autoencoder
SSA
Time Window
TW
14

---

Under review as a conference paper at ICLR 2023
Table A4: Additional characteristics of the state-of-the-ard methods
Method
Learinng Rate
Epoccs
Batch
Size
Max RUL
Features Selected
Calculation
RMSE
Ref.
PRALSTMFONN
N.A.
N.A.
N.A.
[Kone et
2019
TW+ELM
N.A.
N.A.
N.A.
125
14
N.A.
Zivae a
2013
SSA+GRU
N.A.
N.A.
N.A.
125
Auto.
Aver. in
Deng al.
20191
Selection
10
DSCN
N.A.
N.A.
N.A.
130
14
Aver. in
Wang
2019
BILSTM-MSCNN
0.00001
300
64
125
N.A.
N.A.
Jiang et al.
2020
LSTM+NTM
0.005
130
N.A.
Falcon er 2020
DLSTM
N.A.
N.A.
N.A.
130
13
N.A.
Ziao
2019
Auto.
AT-LSTM
N.A.
N.A.
N.A.
125
Selection
N.A.
Aydemir Acar
2020
0.001/
ORCE-LSTM
Max2000
32
130
N.A.
N.A.
TV al.
2019
DeepLSTM
N.A.
N.A.
N.A.
130
Selection
Auto.
N.A.
[Zheng 2017
DAN
0.001
500
256
125
N.A.
N.A.
Waing
CGAN+DGRU
0.0001/
100/000
54/512
130
N.A.
Aver. in
Behera & Misra
2021
0.001
tests
Deep-Bi-LSTM
N.A.
Max300
N.A.
125
14
N.A.
[aan et al
2018
0.01/
Bi-LSTM
500/140
30
130
N.A.
Zhang et al.
2018b
0.015
Bi-LSTM-CNN
0.0001
Max2000
N.A.
N.A.
Auto.
N.A.
Remadna et al.
2020
Selection
DAGHLSTM-CNN
0.005
40
200
125
N.A.
[fre 2019
Embed-LR
N.A.
N.A.
N.A.
120
N.A.
N.A.
Gueulothu 201
ID-CNN + LSTM Bi-LSTM
0.0001
200
200
N.A.
14
N.A.
MHooe et 2020
AGCNN
N.A.
N.A.
N.A.
Bi-RNN
0.005 0.05 0.02
N.A.
N.A.
135
14
N.A.
Yu 20.0
Bi-NNN
N.A.
N.A.
N.A.
Yu al
2019
15


Reviews:
[{"review_id": "z2fdqkRpZu", "reviewer": "Reviewer_fVJc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method combining autoencoders and Bi-LSTM layers to predict the remaining useful life of physical equipments. The author argues that besides promising approaches, traditional ML methods cannot extract sufficient information from the equipment data and they need manual specialist inputs to obtain consistent results. The idea is to use the proposed approach to extract automatic features from the equipment time series data and predict the remaining useful life information. The method was evaluated in the Turbofan Engine Degradation Simulation Dataset and the preliminary results are presented. ", "summary_of_the_review": "The proposed paper presents a promising approach with interesting results. The analysis is direct, but I was expecting more insights. The authors also should discuss the proposed approach's novelty and limitations. It seems to be an ongoing project yet.", "strength_and_weaknesses": "+ The proposed approach is interesting and presents promising results. \n+ The idea of using autoencoders and LSTM layers to model time series is not new and the authors should emphasize the novelty of the proposed approach.\n- I miss some qualitative discussions. The authors explain this is the first round of test results, but I was expecting more insights and conclusive analysis.\n- The authors did not discuss the limitation of the proposed method. Therefore, it will be meaningful to discuss the gap between the experiments in the current version of the paper and the real-world applications.\n- It is an interesting paper, but it seems to be an ongoing project yet.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "awzn90efAs", "reviewer": "Reviewer_Lf7p", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work utilizes (1) a Long Short-Term Memory (LSTM) autoencoder to learn unsupervised latent representations of multivariate time series data, and (2) a supervised Bidirectional LSTM (Bi-LSTM) that is initialized with the representations generated in step (1) and tasked to predict a continuous-valued output. The resulting architecture, referred to as LSTM-based-AUTO-Bi-LSTM, has been applied to the task of Remaining Useful Life (RUL) prediction based on engine degradation data from four datasets simulated under various operational conditions; and its effectiveness has been assessed and compared against several neural architecture-based baselines.", "summary_of_the_review": "This work addresses the problem of RUL prediction, which is rather interesting and impactful considering that RUL is one of the most critical indicators of component failure. Nevertheless, in my view, the contributions of this work cannot be considered novel from a methodological perspective. Moreover, certain experiments appear to be missing from the paper (although the datasets concerning those experiments are discussed by the authors), and the effectiveness of the proposed architecture is not well supported empirically; among other weak points (outlined in the “Weaknesses” part of this review). That being said, the weak points of this paper seem to outweigh its strengths. Therefore, I am not convinced that this work is a good fit for ICLR. Nevertheless, I am looking forward to the authors’ response and I would be willing to adjust my score in case I have misunderstood or misinterpreted certain aspects of the work.", "strength_and_weaknesses": "Strengths:\n\n* The problem addressed in this paper is a problem of considerable importance as RUL is one of the most critical indicators of a component’s failure before the failure occurs. In a real-world setting, having reliable and accurate RUL estimates can substantially reduce maintenance costs and enforce proactive equipment replacement.\n\n* To bypass manual feature engineering from the multivariate sensor datasets, the authors opt for well established neural network variants for sequence encoding that allow for automatic learning of latent features. This design choice suits their RUL task well as opposed to handcrafting features manually or using conventional machine learning methods to extract features directly from multivariate time series.\n\n* The effectiveness of the proposed architecture has been assessed on engine degradation datasets simulated under various operational conditions, each consisting of multivariate time series representing records of several sensor channels that may characterize fault evolution.\n\n* The authors provide a comprehensive and quite informative overview of LSTM autoencoders and supervised Bi-LSTM networks in Sections 3.1 and 3.2, respectively.\n\n-------------------------------------------------------------------------------------------------\n\nWeaknesses:\n\n* In a couple of instances throughout the paper, the authors state that RUL prediction can be performed based on historical data or direct data extraction by adopting “model-based, data-driven, or hybrid methodologies”. Nevertheless, apart from data-driven methods, no model-based or hybrid methods have been included among the baselines. Namely, all of the considered baselines (summarized in Table A3 and Table A4) are indeed data-driven methods. Thus, I believe that this work would have benefited from a comparison of LSTM-based-AUTO-Bi-LSTM with several representative methods from the classes of model-based and hybrid methods, in addition to the already considered data-driven methods.\n\n* The authors discuss that for the purpose of overcoming several challenges and improving RUL prediction accuracy, there has been “a prominent use of Deep Learning (DL) Methods, especially Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM), besides other variations (Zhu et al., 2019; Li et al., 2020; Liu et al., 2020b)”. If that is the case, I am wondering why the proposed architecture has not been compared to some recently published RNN/LSTM-based variants specifically designed for RUL prediction, some of which are listed as follows:\n\n  * Hong, C. W., Lee, K., Ko, M. S., Kim, J. K., Oh, K., & Hur, K. (2020, February). Multivariate time series forecasting for remaining useful life of turbofan engine using deep-stacked neural network and correlation analysis. In 2020 IEEE International Conference on Big Data and Smart Computing (BigComp) (pp. 63-70). IEEE.\n\n  * Liu, H., Liu, Z., Jia, W., & Lin, X. (2020). Remaining useful life prediction using a novel feature-attention-based end-to-end approach. IEEE Transactions on Industrial Informatics, 17(2), 1197-1207.\n\n  * Yu, W., Kim, I. Y., & Mechefske, C. (2020). An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme. Reliability Engineering & System Safety, 199, 106926.\n\n  * Yu, W., Kim, I. Y., & Mechefske, C. (2019). Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme. Mechanical Systems and Signal Processing, 129, 764-780.\n\n  * Park, D., Kim, S., An, Y., & Jung, J. Y. (2018). LiReD: A light-weight real-time fault detection system for edge computing using LSTM recurrent neural networks. Sensors, 18(7), 2110.\n\n  * Gugulothu, N., Tv, V., Malhotra, P., Vig, L., Agarwal, P., & Shroff, G. (2017). Predicting remaining useful life using time series embeddings based on recurrent neural networks. arXiv preprint arXiv:1709.01073.\n\n  Although the proposed architecture is directly based on (Bi-)LSTMs, it has neither been compared to any of the above listed methods nor has any of their respective publications been cited and discussed as a part of the related work.\n\n* The results from the comparison of LSTM-based-AUTO-Bi-LSTM with the state-of-the-art methods are included and discussed only in the case of the FD001 dataset. For some reason, the results obtained on the remaining three engine degradation datasets have been omitted from the paper as well as from the supplementary material / appendix. Therefore, I would encourage the authors to include the results obtained on these datasets and discuss them accordingly in the main paper, if possible. Alternatively, I would suggest that those results are included in the supplementary material / appendix.\n\n* Figure 9 suggests that there seem to be several baselines that outperform LSTM-based-AUTO-Bi-LSTM in terms of RMSE. For instance, among the automatic feature learning methods, Bi-LSTM+CNN outperforms the proposed architecture by a significant margin. Unless I have somehow misinterpreted the findings from Figure 9 (in which case I would strongly suggest that the authors correct me), I would encourage the authors to elaborate on this observation of mine in their response; since, if such an observation holds, that brings into question the effectiveness of the proposed architecture.\n\n* I consider that assessing RUL prediction performance simply by measuring RMSE is not sufficient. In addition to RMSE, I would recommend that the authors consider evaluation metrics that capture the accuracy of the methods’ predictions relative to the variation within the time series. Such metrics include: Mean Relative Absolute Error (MRAE) which is sensitive to outliers, Relative RMSE, and coefficient of determination.\n\n-------------------------------------------------------------------------------------------------\n\nOther (minor) weaknesses:\n\n* Hyperparameter choices and experimental settings such as the number of training epochs, batch size, optimizers, loss functions, and so forth, are specific to the training process and are already mentioned in Section 4.4.1, thus I see no reason for those details to be duplicated in Section 4.4.2.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L8XxXG6KdCZ", "reviewer": "Reviewer_5eu9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "* The authors propose a method for predicting the Remaining Useful Life (RUL) of components. They combine an LSTM-based autoencoder to extract features and a Bi-LSTM network to predict RUL. They validate the approach in one dataset and make an extensive test of hyperparameters and execution setups. In the end, they compare the method with baselines from the literature.", "summary_of_the_review": "* The paper clearly propose a different approach and uses good images to help explaining it. Unfortunately, many aspects lack justification, and the experimental setups are bad detailed (with possible mistakes). They do not introduce nothing entirely new, and when the methodology is compared with other work it underperforms.", "strength_and_weaknesses": "* Strength:\n    * Images are nicely made and help explaining the method and how both parts are combined.\n    * The authors perform hyperparameter search to help finding a good configuration overall.\n\n* Weaknesses:\n    * The method's name is more complex and confusing than it should be.\n    * I actually did not understand the general idea of the proposed approach. The rationale behind deep learning models is to learn/extract features automatically, so why not using only a single bi-LSTM to learn how to extract them? There isnt enough support for the autoencoder in the beginning, especially with only 24 features.\n    * The initial setup is not clear in defining losses: how is the Binary Cross-entropy used as the autoencoder loss? I think that needs further explanations, since it is not trivial. Also, if the authors claim that MSE is more commonly used, why not use it? \n    * In the Bi-LSTM, RMSProp is an optimization approach, not a loss function. Which loss function was used to train it?\n    * The method is compared to 16 others, but 10 of them are better than the proposed approach, and the best one is a method that also uses deep learning for feature extraction. No argument is provided to support why the proposed approach is better or has any advantage over the baselines.  \n    * Figure 5 does not help explaining anything. Processing time and epochs represent the same information (time), so it is not needed to demonstrate both. Also, too much information is shown, I would've selected only train or val gain, not both.\n    * It would be good to include the input and output in Figure 1(b).\n    * There is no need to explain all four datasets if only one of them is actually used.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject