# Using Hybrid LSTM Neural Networks to Detect Anomalies in the Fiber Tube Manufacturing Process
## 2 Materials and Methods
### Hybrid LSTM Network Model for Anomaly Detection  
To process the data stream recorded during the manufacturing of fiber optic cables, which are subjected to \(K\)--means clustering in three channels simultaneously, a dedicated model of a hybrid LSTM network has been proposed. Its task is to analyze the states of the production line in each of the channels, see Figure 6. The integrating module of the network is an aggregating layer along with a set of auxiliary layers, whose task is to detect anomalies based on the recommendations of the three predictors operating in each of the channels. Figure 5: Production RLM line data feautures with bad sessions indicator \(p\) component. The hierarchical, binary cluster tree (**a**) and 3D view of clusters first three principal components (**b**) respectively. Figure 6: The diagram of the hybrid LSTM network designed for the classification of production process states. Considering the three-channel architecture of the LSTM network, the following single cell model was used, see Figure 7.  
Assuming that the input data of a single cell at time step \(t\) is the input vector \(x_{t}\), the mathematical model for a fixed time step enabling the update of the memory state \(c_{t}\) is as follows:  
\[f_{t}^{Ch}=\sigma\Big{(}u_{f}h_{t-1}+w_{f}x_{t}+b_{f}\Big{)} \tag{9}\]  
\[i_{t}^{Ch}=\sigma(u_{i}h_{t-1}+w_{i}x_{t}+b_{i}) \tag{10}\]  
\[\widehat{c}_{t}^{Ch}=tanh(u_{c}h_{t-1}+w_{c}x_{t}+b_{c}) \tag{11}\]  
\[o_{t}^{Ch}=\sigma(u_{o}h_{t-1}+w_{o}x_{t}+b_{o}) \tag{12}\]  
\[h_{t}^{Ch}=tanh(o_{t}\odot c_{t}) \tag{13}\]  
\[c_{t}^{Ch}=f_{t}\odot c_{t-1}+i_{t}\odot\widehat{c_{t}} \tag{14}\]  
and the generalized matrix is written in the form:  
Figure 5:0  
where: \(\sigma\)--is the sigmoid activation function, \(h_{t-1}\)--previous hidden state and \(tanh\) is the hyperbolic tangent activation function and \(W^{Ch}\) denotes the given channel weights set. Figure 8 represents the structure of a single LSTM layer operating in each of the channels. Figure 5:1 LSTM Layer Diagram. Figure 5:2 LSTM cell from the given channel layer. Assuming the clustering mechanism described by Equation (7), we obtain the relationship for the expected values of the signature at time \(t\) of the industrial process on the RLM line:  
Figure 5:3  
As in the supervised training model the goal of the LSTM model is to learn a function \(f:R^{S_{N}\times W_{S}}\rightarrow\left\{1,2,\dots,C_{k}^{Ch_{j}}\right\}\) such that the output \(\hat{y}^{Ch_{j}}(t)\) predicted by the model is as close as possible to the true label \(y^{Ch_{j}}(t)\). This we achieved by minimizing the loss function \(L\left(W^{Ch}\right)\):  
Figure 5:4  
where \(W^{Ch}\) represents the weight set collection sought and \(N\) denotes the number of samples in the learning set. Individual channels of \(LSTM^{Ch_{(8,16,32)}}\) have the following numerical details, see Tables 4 and 5. For the chosen LSTM network model, a training set was constructed with parameters summarized in Figure 5:5 They represent the time interval from which individual measurement sessions originate. In total, recorded coordinates of the production process and extracted classes corresponding to individual diagnostic states and the actual production state were used. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Name** & **Type** & **Activations** & **Learnables** \\ \hline Seq 1 & Sequence input & 232 & – & \\ \hline \multirow{2}{*}{Seq 1} & \multirow{2}{*}{LSTM} & \multirow{2}{*}{512} & Input Weights & 2048 \(\times\) 232 \\  & & & Recurrent Weights & 2048 \(\times\) 512 \\  & & & Bias & 2048 \(\times\) 1 \\ \hline \multirow{2}{*}{Seq 3} & \multirow{2}{*}{Full Connected} & \multirow{2}{*}{8/16/32} & Weights & 8/16/32 \(\times\) 512 \\  & & & Bias & 8/16/32 \(\times\) 32 \\ \hline \multirow{2}{*}{Seq 4} & \multirow{2}{*}{Softmax} & \multirow{2}{*}{8/16/32} & – & \\ \cline{1-1}