# Prediction of Process Quality Performance Using Statistical Analysis and Long Short-Term Memory
## 3 Materials and Methods
### Prediction of PQP Using LSTM
#### 3.3.2 Long Short-Term Memory  
LSTM is a powerful type of artificial recurrent neural network (RNN) [33] designed to recognize patterns in sequence data. RNNs can remember input data, which enables this model to predict the next sequence. However, RNNs are only capable of dealing with short-term dependencies [34], whereas LSTM is capable of learning long-term dependencies. An LSTM is composed of a cell, input gate, output gate, and forget gate. The cell handles remembering values over arbitrary time intervals. The input gate determines whether or not to let new input in and the forget gate determines whether or not to delete information because it is not important or to let that information impact the output at the current time step in the output gate. The LSTM gates are analog, which enables them to do back propagation in the form of a sigmoid; their values range from 0 to 1. This architecture is \begin{table}
\begin{tabular}{c c c} \hline \hline
**Classification** & **Outcomes** & **Data Type** \\ \hline \multirow{4}{*}{MctA} & Workpiece cycle time (WCt) & Long \\  & Process cycle time (PrCt) & Long \\  & Process interval cycle time (PlnCt) & Long \\  & Production cycle time (PCIt) & Long \\ \hline PTAD & Abnormal process status value & Float \\ \hline \multirow{4}{*}{SPCA} & Mean value of a production process condition X & Float \\  & Upper control limit of X & Float \\  & Lower control limit of X & Float \\  & Standard deviation R of X & Float \\  & Upper control limit of R & Float \\  & Lower control limit of R & Float \\ \hline \multirow{2}{*}{PCCA} & Process capability Cp & Float \\  & Process capability index Cpk & Float \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Outcomes of composite statistical analysis. designed to overcome the limitations of RNN by improving the ability to remember and forget historical data with a long-term period. The cell gates in LSTM work on the signals they receive. They are similar to the nodes of a neural network. They can block or pass information. They have their own sets of weight, which they can use to filter, and their weight is the modulating input and hidden states, which are adjusted by the learning process of the recurrent network. The LSTM cells learn when open information should enter, leave, or be deleted by adjusting weights of gradient descent and back propagation error. The LSTM can remove or add information to the cell state using structures called gates. The gates are composed of a sigmoid function and a pointwise multiplication operation. The sigmoid function outputs a result from 0 to 1. Let \(\sigma\) be a sigmoid activation function. To calculate the gate activations, a logistic sigmoid function of the input and weights will be used. The weights are distributed by the LSTM unit via each sequence. The cell and hidden states are under the control of the input and output gates. The states of block input, input, output, cell, hidden, and forget of each sequence are denoted by \(z\), \(i\), \(o\), \(c\), \(h\), and \(f\). A set of weights is denoted as W, and connects between the input and each state in the LSTM unit. The output state manages the information propagated in the previous time step. A new LSTM architecture is introduced in [35] with forget gates. A forget gate is able to restart the LSTM memory cell after it finishes learning one sequence and before starting a new sequence. The LSTM unit equations can be represented by the following [36]:  
\[z_{t}=\text{tanh}(W^{\text{z}}x_{t}+R^{\text{z}}h_{t-1}+b^{z}) \tag{16}\]  
\[i_{t}=\sigma\Big{(}W^{\text{z}}x_{t}+R^{\text{z}}h_{t-1}+b^{\text{z}}\Big{)} \tag{17}\]  
\[f_{t}=\sigma\Big{(}W^{\text{z}}x_{t}+R^{\text{f}}h_{t-1}+b^{\text{f}}\Big{)} \tag{18}\]  
\[o_{t}=\sigma(W^{o}x_{t}+R^{o}h_{t-1}+b^{o}) \tag{19}\]  
\[c_{t}=z_{t}\odot i_{t}+c_{t-1}\odot f_{t} \tag{20}\]  
\[h_{t}=\text{tanh}(c_{t})\odot o_{t} \tag{21}\]  
where \(x_{t}\) is the current input time step.