## 4 Artificial Neural Networks
### Architecture of the CNN  
In this study, a CNN designed with Keras based on TensorFlow (Google \"open-source software for deep learning\") was used, which consists of an input layer, three convolutional layers, three pooling layers, a fully connected layer, and an output layer. The number of nodes in the input layer is 2500, which is the same as the number of sampling points in the signals. Table 3 outlines the parameters used in the CNN.  
The filter size of the first convolutional layer was set to 1 \(\times\) 16 to achieve good performance even in conditions with excess noise in the signals [13]; to prevent overfitting during training, a dropout layer was added after each convolutional layer. In addition, to reduce the computational load and prevent overfitting, a pooling layer was added after the dropout layer, and the kernel size (1 \(\times\) 2) and stride (1 \(\times\) 2) were set. The filter sizes of the second and third convolutional layers were reduced to 1 \(\times\) 8 each, and a dropout and pooling layers were added, respectively. The dropout parameter was set to 0.5. The number of nodes of the fully connected layer was set to 300, and the number of nodes that provided good performance was selected through multiple trials.  
For the output layers, since there are three types of signals, namely signals without defects and defect signals from depths 1 mm and 2 mm below the surface, the number was set to 3. For the activation function used in the convolution layer, the _ReLU_ function was used, which showed good performance for deep learning [13; 27; 28; 29; 30]; the sigmoid function  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Layer Type** &
\begin{tabular}{c} **Kernel Size/** \\ **Stride** \\ \end{tabular} & **Feature Maps** & **Output Size** \\ \hline
1 & Conv1 & \(1\times 16/1\times 8\) & 128 & 313 \(\times\) 128 \\ \hline
2 & Dropout 1 & 0.5 & - & - \\ \hline
3 & Max Pool 1 & \(1\times 2/1\times 2\) & - & 156 \(\times\) 128 \\ \hline
4 & Conv2 & \(1\times 8/1\times 2\) & 32 & 78 \(\times\) 32 \\ \hline
5 & Dropout 2 & 0.5 & & - \\ \hline
6 & Max Pool 2 & \(1\times 2/1\times 2\) & & 39 \(\times\) 32 \\ \hline
7 & Conv 3 & \(1\times 8/1\times 2\) & 8 & 20 \(\times\) 8 \\ \hline
8 & Dropout 3 & 0.5 & & - \\ \hline
9 & Max Pool 3 & \(1\times 2/1\times 2\) & & 10 \(\times\) 8 \\ \hline
10 & Fully Connected & 300 & & - \\ \hline
11 & Dropout & 0.5 & & - \\ \hline
12 &
\begin{tabular}{c} Softmax with \\ Cross Entropy \\ \end{tabular} & 3 & 3 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Parameters of the CNN.  
Figure 7: Architecture of the convolutional neural network.  
was not used because the output value of the function converges to 0 when there are many layers [13; 25]. The _ReLU_ function is expressed as Equation (1) below.  
\[Relu=\left\{\begin{array}{l}(x<0)f(x)=0\\ (x\geq 0)f(x)=x\end{array}\right. \tag{1}\]  
In addition, sparse categorical cross-entropy was used as the loss function and is expressed as Equation (2).  
\[-\sum_{i=1}^{D}y_{i}\log(p_{i}) \tag{2}\]  
where \(D\) is the number of data, \(y\) is a vector of the true labels, and \(p\) is the vector of predicted labels. Sparse categorical cross-entropy provides results in integer form and is widely used for multiclass classifications. For the output layer activation function, softmax is used and is expressed as Equation (3) below:  
\[\sigma(y_{i})=\frac{e^{y_{i}}}{\sum_{k=1}^{K}e^{y_{k}}} \tag{3}\]  
where \(y\) is the input vector to the output layer, and \(i\) indexes the output nodes, \(i=1,2,\ldots,\)\(k\). In this study, the number of classes is 3, so \(k=3\).