# Optimization of Design Parameters in LSTM Model for Predictive Maintenance
## 3 Optimization of Design Parameters for an LSTM-Based Predictive Maintenance Model  
### Problem Definition and Optimization Procedure  
We aim for an optimal design of an LSTM-based predictive maintenance model by following the procedure suggested by [PERSON] et al. [3]. Firstly, we formulate an HI to describe the state of machinery. Specifically, the HI includes representative features such as mean or standard deviation calculated using raw data collected from the machinery. Then, the HS is defined to discriminate a normal stage (healthy stage) and a faulty stage (unhealthy stage) after fault occurrence in machinery. It also describes degradation states of performance after a fault occurrence. In addition, we assume that the faulty stage includes six degradation states, and deterioration of the machinery occurs following a linear model, depicted in Figure 2, obtained from a pre-test and literature survey. Then, we generate two LSTM models for prediction and monitoring, which have the following roles. The purpose of the prediction model is to detect fault occurrence, and we train it by using input sequences, including HI values of a certain time step. Both normal and faulty sequences with labels for the normal and faulty class, as 0 and 1, are inserted. If the output probability exceeds 0.5, the last time point of the input sequence is regarded as the fault occurrence time. Meanwhile, the monitoring model is used to classify the state of the input sequence according to the pre-defined degradation states of performance in Figure 2. The model calculates the probability that the machinery is in each defined degradation state. Then, it classifies the input sequence as the state with the highest probability, which means the machinery is in a corresponding degradation state at the last time point of the sequence. The structure of those LSTM models is determined by adjusting hyperparameters of LSTM models such as time steps, the number of LSTM layers and hidden neurons in each LSTM layer. However, since hyperparameter tuning can affect the performance of LSTM, it should be done very carefully. Specifically, short time steps are not appropriate for time series data that have a sequential relationship, and long time steps may cause prediction value to depend significantly on past data. The small number of LSTM layers cannot trace complex and latent data patterns even though there are plenty of neurons in the layer. On the other hand, too-deep LSTM layers may cause overfitting and slow convergence. The number of neurons in hidden layers plays a similar role to that of the LSTM layer. Based on these remarks, significant design parameters to be optimized in LSTM models are (i) feature values used for HIs and (ii) hyperparameters of LSTM. Therefore, we propose an efficient method for selecting a good set of features used for HIs and an efficient GA to efficiently explore the solution space of hyperparameters for LSTM. The optimization procedure for developing an efficient LSTM-based predictive maintenance model using these suggested methods can be represented as in Figure 3. At first, we import raw data and calculate feature values using them. This pre-processing of raw data and information of raw data is illustrated in Section 4. Based on calculated features, we perform feature  
Figure 2: Used degradation model for HS. selection for identifying features relevant to states of machinery and establishment of an effective HI. We devise a two-phase feature selection scheme including correlation analysis and a stepwise comparison explained in Section 3.2. Then, hyperparameters of LSTM are explored by using the GA. Specifically, we design the components of the GA, including a representation scheme and genetic operators, for describing the hyperparameter tuning problem. The detailed design of the GA is described in Section 3.3. As a result, a compact and concise HI representing the state of machinery well can be obtained, which can be applied to get an optimal hyperparameter configuration of LSTM.