# Statistical Process Control with Intelligence Based on the Deep Learning Model
## Methodology
### Long Short-Term Memory Network Model  
In recent years, deep learning has made remarkable achievements in various fields. The deep learning algorithm has advantages that the traditional shallow machine learning algorithm does not have, such as complex data preprocessing and feature engineering are no longer needed. The raw data can be used as the input of the model. The deep-seated of neural network and the special design of network structure make it can learn the potential deeper knowledge from the raw data and be competent for more complex tasks. The features it learns are the most suitable for this classification task and fares better than the features designed by human experts. LSTM is a typical deep learning model and a variant of RNN. The basic idea is still to take the previous output of the network as the next input, as shown in Equation (7). Where \(\mathbf{x}_{t}\) is the \(t^{\text{th}}\) input of the network, \(\mathbf{h}\) is the output of the network, and \(H\) is the nonlinear transformation function. \[\mathbf{h}_{t}=H(\mathbf{x}_{t},\mathbf{h}_{t-1}) \tag{7}\]  
Compared with traditional RNN, LSTM can capture long-distance dependency. It is different from the standard RNN in structure. LSTM adds some gate structures for each cell, which allows information to pass selectively. It can be understood as a mechanism of feature learning selection and update [45]. The cell structure of LSTM is shown in Figure 4.  
The forget gate (the red part in Figure 4) is used to decide what information will be discarded from the cell state \(\mathbf{C}_{t-1}\). A forget gate is mathematically represented by [45]:  
\[\mathbf{f}_{t}=sigmoid\Big{(}\mathbf{W}_{f^{\prime}}[\mathbf{x}_{t},\mathbf{h}_{t-1}]+\mathbf{b} _{f}\Big{)} \tag{8}\]  
The input gate (the blue part in Figure 4) is used to determine what new information \(\mathbf{C}^{\prime}_{t}\) will be stored in the cell state. A input gate is mathematically represented by [45]:  
\[\mathbf{i}_{t}=sigmoid\big{(}\mathbf{W}_{i^{\prime}}[\mathbf{x}_{t},\mathbf{h}_{t-1}]+\mathbf{b} _{i}\big{)} \tag{9}\]  
\[\mathbf{C}^{\prime}_{t}=tanh\big{(}\mathbf{W}_{c}[\mathbf{x}_{t},\mathbf{h}_{t-1}]+\mathbf{b}_{c} \big{)} \tag{10}\]  
Then the cell state is updated, and the decisions made in the previous steps are implemented to get a new cell state \(\mathbf{C}_{t}\). The mathematical representation of the state update is as follows:  
\[\mathbf{C}_{t}=\mathbf{f}_{t}\star\mathbf{C}_{t-1}+\mathbf{i}_{t}\star\mathbf{C}^{\prime}_{t} \tag{11}\]  
Figure 4: Typical LSTM cell structure. The output gate (the green part in Figure 4) determines the final output \(\mathbf{h}_{t}\) according to the updated cell state. An output gate is mathematically represented by [45]:  
\[\mathbf{o}_{t}=sigmoid(\mathbf{W}_{o}\cdot[\mathbf{x}_{t},\mathbf{h}_{t-1}]+\mathbf{b}_{o}) \tag{12}\]  
\[\mathbf{h}_{t}=\mathbf{o}_{t}*tanh(\mathbf{C}_{t}) \tag{13}\]  
where \(\mathbf{W}_{f}\), \(\mathbf{W}_{i}\), \(\mathbf{W}_{c}\) and \(\mathbf{W}_{o}\) represent the weight of forget gate, input gate, current cell and output gate respectively. \(\mathbf{b}_{f}\), \(\mathbf{b}_{i}\), \(\mathbf{b}_{c}\) and \(\mathbf{b}_{o}\) represent the bias. The final output \(\mathbf{h}_{T}\) of the last cell of LSTM is the feature extracted adaptively from the raw data (where \(T\) refers to the last cell). The final recognition can be achieved by inputting this feature into the traditional ANN. The last layer is the output layer, whose activation function is Softmax, and the number of neurons is \(N\), representing the number of types that want to identify patterns. Finally, the stochastic gradient descent (SGD) algorithm is used to optimize the weight (\(\mathbf{W}_{f}\), \(\mathbf{W}_{i}\), \(\mathbf{W}_{c}\) and \(\mathbf{W}_{o}\)) and bias (\(\mathbf{b}_{f}\), \(\mathbf{b}_{i}\), \(\mathbf{b}_{c}\) and \(\mathbf{b}_{o}\)) in LSTM. Commonly used SGD methods include SGD with momentum (SGDM), root mean square propagation (RMSProp), and adaptive moment estimation (Adam) [50].