# Using Hybrid LSTM Neural Networks to Detect Anomalies in the Fiber Tube Manufacturing Process
## 2 Materials and Methods
### Preprocessing of Input Data for Training Set
As we can observe a binary clustering tree is as a kind of dendrogram, where each node represents a cluster, and each leaf node represents an individual observation. The height of the nodes in the tree corresponds to the distance (or dissimilarity) at which consecutive clusters are merged. A spatial visualization of the location of the leading features of the clusters is provided in Figure 4b. As can be seen, there is a classic \"energy leakage effect\" causing the extracted classes of process states to interpenetrate and individual clusters to differ to a very small degree. Therefore, a multi-resolution \(K\)--means Clustering mechanism was adopted in later part of this work to cluster the input data at different resolutions of the space of feature channels \(Ch_{j}\in\left\{8,16,32\right\}\), for \(j=1,2,3\). So, having \begin{table}
\begin{tabular}{c c} \hline
**Column Indexes** & **Name** \\ \hline \{[ 4]\} & {’BAZ1\_iSpeed’} \\ \{[30]\} & {’KAP1\_iTens17\_20’} \\ \{[31]\} & {’KAP1\_iTens21\_24’} \\ \{[32]\} & {’KAP1\_iTens58\_’} \\ \{[57]\} & {’POF1\_iDancMode\_N’} \\ \{[58]\} & {’POF1\_iFlCCodeTrav’} \\ \{[59]\} & {’POF1\_iLoad’} \\ \{[60]\} & {’POF1\_iRealDian’} \\ \{[61]\} & {’POF1\_iSpeed’} \\ \{[63]\} & {’OSD2\_ValDel’} \\ \{[49]\} & {’MES\_MeterCnt’} \\ \{[56]\} & {’POF1\_iDancerPos’} \\ \{[50]\} & {’MES\_MeterCnt2’} \\ \{[52]\} & {’MES\_OkaltRe’} \\ \{[40]\} & {’MES\_Dwarm’} \\ \hline \end{tabular}
\end{table} 
Table 3: Corresponding process coordinates names. an RLM line dataset with \(n=256\) observations for each stepping window, where each observation is represented as a \(d\)-dimensional vector we have:  
\[\textbf{x}_{i}^{Ch_{j}}=\left(x_{i1}^{Ch_{j}},x_{i2}^{Ch_{j}},\ldots,x_{id}^{Ch_{ j}}\right) \tag{5}\]  
Then we performed \(K\)--means clustering in each of the channels to partition the observations into channel independent clusters:  
\[K^{Ch_{j}}\in\left\{C_{1}^{Ch_{j}},C_{2}^{Ch_{j}},\ldots,C_{K}^{Ch_{j}}\right\} \tag{6}\]  
such that sum of squared distances between observations and their corresponding cluster centroids is minimized. Given \(\boldsymbol{\mu}_{1},\boldsymbol{\mu}_{2},\ldots,\boldsymbol{\mu}_{k}\), as initial cluster centroids we assign each observation \(\textbf{x}_{i}^{Ch_{j}}\) to the cluster:  
\[C_{k}^{Ch_{j}}=\left\{\textbf{x}_{i}^{Ch_{j}}\mid\arg\min_{k}\parallel \textbf{x}_{i}^{Ch_{j}}-\boldsymbol{\mu}_{k}\parallel^{2}\right\},k=1,\ldots,Ch_{j} \tag{7}\]  
where \(\parallel\textbf{x}_{i}^{Ch_{j}}-\boldsymbol{\mu}_{k}\parallel\) is the Euclidean distance between observation \(\textbf{x}_{i}^{Ch_{j}}\) and centroid \(\boldsymbol{\mu}_{k}\). The update of the centroid for each cluster in a given channel \(Ch_{j}\), is calculated as a mean of the observations assigned to that cluster:  
\[\boldsymbol{\mu}_{k}=\frac{1}{|C_{k}^{Ch_{j}}|}\sum_{\textbf{x}_{i}^{Ch_{j}} \in\textbf{c}_{k}^{Ch_{j}}}\textbf{x}_{i} \tag{8}\]  
where \(|C_{k}^{Ch_{j}}|\) is the number of observations in cluster \(C_{k}^{Ch_{j}}\). This way of preprocessing the input data was intended to maximize the disparity of the process states in order to facilitate the task of building a training set for the LSTM network. One of the key difficulties that arises in anomaly detection tasks is that technological limitations do not allow precise labeling of training set classes. Therefore, an analysis of individual measurement sessions was carried out in cooperation with RLM line technologists. As a result of it, it was indicated after which sessions the quality procedure checks showed any defects in the final product. In this way, it was possible to estimate the index of potential defect for each measurement session: \(p=1\) for normal production, \(p=0\) for the interval of the production process in which any defects were detected during the post manufacturing quality control in a given measurement session. A dendrogram and a spatial visualization of the most relevant features were constructed again for such a completed input data set, see Figure 5.  
Figure 4: Production RLM line data feautures.