# Multistage Quality Prediction Using Neural Networks in Discrete Manufacturing Systems
## 2 Materials and Methods
### LSTM Prediction Model  
Traditional neural networks are typically fully connected from the input layer to the hidden layers and then to the output layer. However, they lack intra-layer connections between neurons. This limitation can result in substantial deviations when processing sequential data [28]. In contrast, recurrent neural networks (RNN) can store and utilize information from previous computations to calculate current outputs. This is accomplished by establishing connections between neurons in the hidden layers. The input of the hidden layer consists of the output from the input layer and the output from the hidden layer in the previous time step. One common training method for processing sequential data in an RNN is the back-propagation through time (BPTT) algorithm. This algorithm locates optimal points along the negative gradient direction of parameter optimization until convergence is achieved. However, during optimization, it is necessary to retrace information for all the previous time steps to compute the partial derivative of the parameters at a given moment. The overall partial derivative function is then calculated as the sum of all time steps. The inclusion of activation functions can result in multiplicative interactions between their derivatives, which can lead to issues such as \"gradient vanishing\" or \"gradient explosion\" [29]. LSTM is an improved recurrent neural network that is used to extract temporal information and characteristics during processing. A structural diagram of the LSTM network is shown in Figure 3. In comparison to RNN, LSTM incorporates memory cells with a forgetting gate method, allowing for selective memory and the filtering out of noise information, thereby reducing the memory burden. As in an RNN, LSTM uses a backpropagation method to update a model; LSTM can fit faster than RNN, but the number of parameters is large, so LTSM needs a lot of data as support. \[i_{t}=\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+b_{i}) \tag{4}\]  
\[f_{t}=\sigma(W_{xi}x_{t}+W_{hf}h_{t-1}+b_{f}) \tag{5}\]  
Figure 3: LSTM model network. \[o_{t}=\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+b_{o}) \tag{6}\]  
\[c_{t}=c_{t-1}\odot f_{t}+g_{t}\odot i_{t} \tag{7}\]  
\[h_{t}=o_{t}\odot m_{t} \tag{8}\]  
In this study, we propose an improved MV\(-\)LSTM prediction model for analyzing the production characteristics of discrete manufacturing. Compared to traditional LSTM models, our model expands the input layers to multiple dimensions. Figure 4 illustrates a comparison between the input and output variables per unit time of LSTM. By learning from historical data, our model maps manufacturing parameters and quality characteristics to improve the accuracy of prediction and identify quality divergence points. To achieve this, we adjusted the number of neuron cells, LSTM layers, and dense layers. Similar to the RNN, the LSTM network also undergoes a process of data back-propagation, where the error value propagates along the time series and spreads between layers. After obtaining the updated gradient of the horizontal and vertical weights and bias terms through the hidden layers' structures, we can calculate the updated value of each weight and bias term using the same method as the RNN network. It is important to set the value of the learning rate \(\alpha\) to control the gradient of the updated error and the speed of error decline. In our training model, we introduced three evaluation metrics to assess predictive ability, which are defined in the following text. The root mean square error (RMSE), also known as the standard error, is calculated by taking the square root of the mean square error. The rationale behind introducing the RMSE is similar to that of introducing the standard deviation. This is because the magnitude of the RMSE may not visually reflect the degree of dispersion in the data. Hence, taking the square root of the RMSE helps to obtain a more meaningful measure of the RMSE.