# Control Chart Concurrent Pattern Classification Using Multi-Label Convolutional Neural Networks
## 3 Experimental Analysis  
In this study, a series of experiments were conducted to illustrate the benefits of the proposed method. The performance of the proposed method was compared with various traditional classification algorithms. In the first experiment, the data of non-random patterns were generated based on the pattern parameter settings of [PERSON] et al. [24]. They set the analysis window size to 32, and explored 16 patterns, including 7 single patterns and 9 concurrent patterns (Figure 5). In the training set, there were 5000 samples for each pattern, and in the test dataset, there were 1250 samples for each pattern. In the present study, Keras [44] was used to build a multi-label convolutional neural network. To identify an optimized 1D CNN architecture, various structural and learning parameters were needed, such as the number of filters in each convolution layer, the filter size, the stride, and the number of neurons in the fully connected layers. The input data were a 1D vector containing 32 observations, and the output layer contained 7 neurons corresponding to 7 single pattern types. Since the amount of data for training was limited, it was not feasible to use deeper architectures without causing significant overfitting. The proposed 1D CNN had two convolution layers. The first convolution layer had 16 filters of size \(1\times 5\), and the second convolution layer had 32 filters of size \(1\times 3\). A large filter size was used in the first convolutional layer because it was assumed that the first layer should have a higher-level view of the control chart data. The convolution stride in the network was fixed to 1. The padding type was set to \"same\" (zero padding) to avoid the loss of spatial dimensions. The size of the max pooling window was \(1\times 2\) with a  
Figure 4: Example of subset accuracy. Figure 5: Single and concurrent patterns. stride of 2. Dropout was added to each convolution layer to reduce model complexity and prevent overfitting. The proposed end-to-end 1D CNN included one fully connected layer. The fully connected layer had 32 neurons that were chosen after trial and error. Figure 6 illustrates the effect of the number of neurons on accuracy. Dropout was also added to the fully connected layer. The ReLU activation function (\(f(x)=\max(x,0)\)) was used for all layers, except for the output layer where a sigmoid activation function was used. The CNN architecture is summarized in Table 1.  
The Adam optimization algorithm and cross-entropy loss function were used to train the 1D CNN with a batch size of 16. The 1D CNN was trained up to 100 epochs with a learning rate of 0.001. To avoid overfitting, an early stopping procedure was employed; specifically, the loss value of the validation set was monitored in each epoch and the training procedure was halted if training yielded no improvements within the previous 10 epochs. This study selected RF [49] and SVC [47] as the competing methods, both of which can support multi-label classification problems. In this study, a one-vs-the-rest (OVR) multi-label classifier was established based on SVC. OVR is also called one-vs-all, which is to build a binary classifier for each class. The above-mentioned classifiers were built in Python with scikit-learn v0.24.1 [48]. For RF, the maximum depth of each tree was not restricted. The number of features used for each decision split was set to the square root of the total feature number. The number of trees (n_estimators) was identified by evaluating the results for a range of values. Figure 7 illustrates the performance of RF when the number of individual decision trees were varied. From Figure 7, it follows that increasing the number of trees beyond 1450 would result in minor improvement of accuracy. Therefore, 1450 trees were used to reduce the computation time.