## 4 Anomaly Detection Model Training
### Convolutional Neural Network (CNN)  
A CNN is the network architecture that learns directly from data, eliminating the need for manual feature extraction. CNNs are particularly useful for finding patterns in images to recognize objects, faces, and scenes. They can also be quite effective for classifying non-image data such as audio, time series, and signal data [64; 65; 66; 67; 68; 69; 70; 71; 72; 73]. Figure 9 shows an example of image classification using a CNN [65]. As shown in the figure, a convolution operation is repeatedly performed to extract features, and extracted features are entered into the fully connected network to classify the images. Figure 8: LSTM-based anomaly detection model training architecture. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Model**} & **The Number of** & **The Number of** \\  & **LSTM Layers** & **Hidden Units per** \\  & & **LSTM Layer** \\ \hline LSTM-m1 & 1 & 200 \\ LSTM-m2 & 1 & 400 \\ LSTM-m3 & 3 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: LSTM layer and hidden unit settings for the LSTM-based anomaly detection models. We trained the engine clutch engagement/disengagement anomaly detection model using a CNN by configuring the input/output structure as shown in Figure 10. As shown in the figure, the architectures with one and three convolution layers were constructed. In [74], the authors configured the channel for time-series data to enter the data into the convolution layer to classify time-series data. Similarly, we configured the channel for each target signal to enter the data into the first convolution layer. The output of the network is the normal/anomaly per pattern, as in the aforementioned MLP and LSTM. For input signals of the network, we matched the length of data per pattern in the same way as the MLP and LSTM, using Equation (1), and then entered the data into the network. The composition of layers for each architecture in Figure 10 is as follows. First, Table 6 shows the hyperparameter setting of the convolutional layer for each architecture. In the table, [w, h] of the filter size means [height, width (time axis)] of the filter, and [a, b] of the stride means [vertical step size, horizontal step size]. Because the input data is one-dimensional, the height of the filter size and vertical step size of the stride is always 1. We made the input size and output size of the layer the same by using zero padding. In the batch normalization layer, z-score normalization is conducted on input data for each channel. In the ReLU layer, the activation function shown in Figure 5c is applied to the input data. The max pooling layer performs downsampling by outputting a maximum value for a specific region (pooling region) of input data. The sizes of the pooling regions of the two max pooling layers of the 3-convolutional layer architecture were set to (1, 3) and (1, 4), respectively, and the stride values were also set to (1, 3) and (1, 4), respectively. Here, (w, h) of the pooling regions size means height, width (time axis)] of pooling region. The set values for each layer above were selected according to a general method or through trial and error. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{4}{c}{**3-Convolution Layer Architecture**} \\ \hline Hyperparameter & 1 st convolutional layer & 2 nd convolutional layer & 3 rd convolutional layer \\ \hline Filter size & (1, 50) & (1, 50) & (1, 50) \\ The number of filters & 64 & 128 & 256 \\ Stride & (1, 1) & (1, 1) & (1, 1) \\ \hline \hline \multicolumn{4}{c}{**1-Convolution Layer Architecture**} \\ \hline Hyperparameter & 1 st convolutional layer & - & - \\ \hline Filter size & (1, 50) & - & - \\ The number of filters & 64 & - & - \\ Stride & (1, 1) & - & - \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Convolutional layer hyperparameter setting. Figure 9: Image classification example using a CNN.