# Time-Series Prediction of Failures in an Industrial Assembly Line Using Artificial Learning
## 3 Materials and Methods
### Modelling Using the LSTM Network  
LSTM is a model with an RNN structure. This network consists of layers. The main layers are an input layer, LSTM layer(s), and an output layer. The input layer processes the data received from external sources. The LSTM layers hold hidden units, the core components responsible for the model's learning capability. These units calculate data sequentially and learn temporal dependencies. The output layer transforms the information processed by LSTM into a final output format (e.g., classification, regression, or sequence prediction). The recurrent structure of LSTM specifically provides the ability to learn long-term dependencies in time-series data. The general architecture of an LSTM network is shown in Figure 5.  
A hidden unit includes three gates, the input gate, forget gate, and output gate. The gates determine which information the model retains and which it discards. Each gate employs sigmoid activation functions that are regulated by weights and biases. This is shown in Figure 6, where \(i,f,g\), and \(o\), respectively, represent the input gate, forget gate, cell candidate, and output gate. \(h\) denotes the hidden state. \(W\in R^{dxe}\), \(R\in R^{dxe}\), and \(b\in R^{e}\) represent the input weights, recurrent weights, and biases, respectively. \(t\) denotes the time step, \(\{x\}\) is the set of input variables, \(\{h_{t}\}\) is the set of outputs (hidden states), and \(c\) denotes the cell state. In LSTM networks, learning occurs through the update of weights and biases. The process of updating is illustrated by the signal flow within a unit cell in the diagram. At time step \(t\), the values of \(c_{t}\) and \(\{h_{t}\}\) are calculated using their previous values \(c_{t-1}\) and \(h_{t-1}\). The network state is updated at each step by adding or removing information, and this process is governed by the gates. At time step \(t\), the cell state \(c_{t}\) is computed using Equation (7):  
\[c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot g_{t} \tag{7}\]  
The symbol \(\odot\) represents the Hadamard product. The hidden state \(h_{t}\) is then calculated using Equation (8):  
\[h_{t}=o_{t}\odot\tanh(c_{t}) \tag{8}\] \begin{table}
\begin{tabular}{l l} \hline
**Parameter** & **Value** \\ \hline Training algorithm & Levenberg–Marquardt \\ Delay (\(d\)) & 3, feedback of \(\mathit{Configuration}_{t-1}\), \(\mathit{Configuration}_{t-2}\), \(\mathit{Configuration}_{t-3}\) \\ Inputs & \(\mathit{Configuration}_{t-1}\), \(\mathit{Configuration}_{t-2}\), \(\mathit{Configuration}_{t-3}\) \\ Output & \(\mathit{Configuration}_{t}\) \\ Output threshold & 0.99 \\ Learning rate & 0.1 \\ Momentum & 0.1 \\ Learning threshold & 0.0001 \\ Batch size & 80\% (modelling *), 20\% (forecasting); all data normalised \\ \hline \end{tabular} (*) Modelling includes 75\% training, 15\% testing, and 15\% validation.  
\end{table} 
Table 1: _Cont._  
Figure 5: Layer topology of the LSTM network. In this equation, _tanh_ refers to the hyperbolic tangent activation function. The terms in the above equations are computed using Equation (9):  
\[i_{t}=\sigma(W_{t}x_{t}+R_{t}h_{t-1}+b_{i})f_{t}=\sigma\Big{(}W_{f}x_{t}+R_{f}h_{ t-1}+b_{f}\Big{)}o_{t}=\sigma(W_{o}x_{t}+R_{o}h_{t-1}+b_{o})g_{t}=tanh\big{(}W_{g}x_{t}+R_{g}h_ {t-1}+b_{g}\big{)} \tag{9}\]  
where \(\sigma(x,h,b)\) denotes the sigmoid activation function in the gates. Each gate has weights and biases optimised during training. Weights capture the relationships between the input data and the hidden state, while the biases ensure the correct functioning of each gate. The cell state of the LSTM enhances its ability to learn long-term dependencies, and the structure also helps to mitigate issues such as vanishing gradients. The hyper-parameters of the LSTM and GRU networks used in this study are presented in Table 2, and their optimal values are generally determined through experimental methods and cross-validation. The input layer is defined in accordance with the characteristics of the dataset.