# Prediction of Key Parameters of Wheelset Based on LSTM Neural Network
## 2 LSTM Neural Network
### The Basic Theory of LSTM (Long Short-Term Memory) Network  
#### 2.2.1 LSTM Neural Network  
The LSTM network is called long short-term memory recurrent neural network [13]. The LSTM neural network also has a chain structure similar to a regular RNN, but it differs in terms of its improved internal structure [14]. Compared to RNN, the LSTM neural network has an additional input and output. These two additional parts are the input and output of LSTM's memory and forgetting mechanism, which is commonly represented by \(C\) to denote the cell state [15]. The emergence of the cell state effectively avoids the problems of gradient vanishing and exploding that often occur in an RNN model. The internal structure of the LSTM neural network is shown in Figure 3.  
The internal structure of an LSTM neuron consists of four parts: input gate, output gate, forget gate, and memory unit. The forget gate is represented by _sigmoid_ layer, while the other two _tanh_ layers correspond to the input and output of the memory unit [16]. The LSTM neural network has the ability to remove or add information to the _cell_ state, which is controlled and regulated by the _gate_ structure. The _gate_ structure is composed of a _sigmoid_ neural network layer and a pointwise multiplication operation and can selectively pass information. The output of the _sigmoid_ layer ranges from 0 to 1 to describe how much content each component allows to pass through. A value of 0 indicates prohibiting content from passing, while a value of 1 indicates allowing everything to pass. An LSTM neural network structure has three such gate structures to protect and control the state of the unit [17]. The functions of the three gates are as follows:  
(1) Forget gate: It is used to determine the information discarded from the node state. This determination is made by the _sigmoid_ layer, which decides whether to pass or partially pass based on the previous time step's output. Every number output of the _cell_ state \(c_{t-1}\) between 0 and 1 is determined by inputting \(h_{t-1}\) and \(x_{t}\). The calculation is given in Equation (1):  
\[f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}) \tag{1}\]  
(2) Input gate: It determines the new information stored in the _cell_ state and is responsible for the input at the current sequence position. In the first step, the _sigmoid_ layer called the input gate layer decides which values to update. At the same time, the _tanh_ layer creates a new candidate value vector \(\widetilde{c_{t}}\) and adds it to the _cell_ state. Then these two components are combined to update the _cell_ state. The calculation is shown in Equations (2) and (3):  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{2}\]  
Figure 3: The internal structure of LSTM neural network.  
\[\widetilde{c}_{t}=tanh(W_{c}\cdot[h_{t-1},x_{t}]+b_{c}) \tag{3}\]  
Next is the update of the old _cell_ state \(c_{t-1}\) to the new _cell_ state \(c_{t}\). This is done by multiplication between the old cell state \(c_{t-1}\) and \(f_{t}\) to forget the information decided in the previous step. Then, the new candidate value \(c_{t}\) is added to scale to varying degrees to update the new state value \(c_{t}\). The calculation is given in Equation (4):  
\[c_{t}=f_{t}*c_{t-1}+i_{t}*\widetilde{c}_{t} \tag{4}\]  
(3) Output gate: It determines the output layer. First, determine the output _cell_ state by the _sigmoid_ layer. Then, the _cell_ state is scaled between \(-1\) and \(1\) by the _tanh_ layer. The scaled cell state is pairwise-multiplied with the initial output obtained from the _sigmoid_ layer, resulting in the model's output, which contains the desired information. The calculation is shown in Equations (5) and (6):  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{5}\]  
\[h_{t}=o_{t}*tanh(c_{t}) \tag{6}\]