# Control Chart Concurrent Pattern Classification Using Multi-Label Convolutional Neural Networks
## 3 Experimental Analysis
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Metric\(\backslash\)Algorithm** & **RF** & **SVC** & **CNN** \\ \hline Accuracy & 0.8019 (0.0023) * & 0.9008 & 0.9256 (0.0003) \\ Recall & 0.8752 (0.0019) & 0.9508 & 0.9563 (0.0005) \\ Precision & 0.9228 (0.0019) & 0.9521 & 0.9588 (0.0006) \\ \(F_{1}\) & 0.8910 (0.0019) & 0.9491 & 0.9568 (0.0004) \\ \(F_{2}\) & 0.8805 (0.0019) & 0.9496 & 0.9563 (0.0005) \\ Hamming loss & 0.0383 (0.0005) & 0.0228 & 0.0198 (0.0001) \\ Jaccard\_score & 0.8355 (0.0019) & 0.9027 & 0.9150 (0.0005) \\ \hline \hline \end{tabular}  
* Numbers in parentheses are standard deviations.  
\end{table} 
Table 2: Classification performance in experiment 1. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Layer** & **Type** & **Filter Size** & **\#Filters** & **Stride** & **Padding** \\ \hline
1 & Convolution/ReLU & \(1\times 5\) & 32 & 1 & same \\
2 & Max pooling & \(1\times 2\) & \(1\) & 2 & - \\
3 & Convolution/ReLU & \(1\times 3\) & 128 & 1 & same \\
4 & Max pooling & \(1\times 2\) & 1 & 2 & - \\
5 & Dropout (30\%) & - & - & - & - \\
6 & Convolution/ReLU & \(1\times 3\) & 256 & 1 & same \\
7 & Max pooling & \(1\times 2\) & 1 & 2 & - \\
8 & Dropout (30\%) & - & - & - & - \\
9 & Fully connected (128) & - & - & - & - \\
10 & Dropout (30\%) & - & - & - & - \\
11 & Sigmoid (6) & - & - & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Convolutional neural network architecture for experiment 2.  
multi-label CNN outperformed the feature-based SVC in classifying the concurrent pattern which is composed of cycle, upward trend and downward shift. It seems that multi-label CNN has the advantage of classifying complex concurrent patterns. The advantages of using a multi-label classification model instead of a multi-class model for concurrent CCPR can be illustrated with the following example. The original paper [51] only considered a concurrent pattern of cycles and upward trends. Therefore, the concurrent pattern of cycles and downward trends can be considered an unknown pattern. Restricted by the predefined pattern classes of the multi-class model, in the best case, this concurrent pattern can be classified as a class label including the features of either a cycle or a downward trend. In the worst-case scenario, it can be classified as a normal pattern or as some other pattern. Ten additional concurrent patterns that featured a cycle and downward trend were simulated. Figure 9 depicts these 10 concurrent patterns. The results of the multi-label CNN are presented in Figure 10. According to the coding scheme of the original paper, the desired output should be (0,1,0,1,0,0). As evident, only one misclassified pattern was present. Pattern six was misclassified as a single cyclic pattern. As can be seen in Figure 10, only the neuron corresponding to the cycle was fired. That is, the output value was greater than the default threshold value of 0.5. As expected, the traditional classifiers misclassified all 10 patterns. For RF, three patterns were classified as a single downward trend and seven patterns were classified as concurrent patterns involving a cycle and a downward shift. For SVC, all 10 patterns were misclassified as concurrent patterns involving a cycle and a downward shift. These results indicate that the multi-class classification models can, at best, select a class label that most resembles the unseen input pattern as the output. This example demonstrates the benefits of using multi-label CNN in control chart concurrent pattern recognition.