## 4 Artificial Neural Networks
### Residual Neural Network Architecture  
The ResNet used in this study was designed in Keras, based on Tensorflow (Google open-source software for deep learning). The algorithm included an input layer, nineteen convolutional layers, and three stages of ResNet blocks. Each stage features the same architecture, but a different number of feature maps. The detailed architecture of the stage with feature map FM is described in Table 5. The final ResNet architecture used when these unit stages were combined is shown in Table 6. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Adopted ResNet Stage**} \\ \hline  & **Layer Type** & \begin{tabular}{c} **Kernel** \\ **Size/Stride** \\ \end{tabular} & \
\begin{tabular}{c} **Feature** \\ **Maps** \\ \end{tabular} &
\begin{tabular}{c} **Activation** \\ **Function** \\ \end{tabular} & **Description** \\ \hline
1 & Input layer & - & - & - & x \\ \hline
2 & Conv 1 & \((1\times 3)/(1\times 1)\) & FM & Relu & y = Conv 1 (x) \\ \hline
3 & Dropout & 0.5 & - & - & y = Dropout(y) \\ \hline
4 & Conv 2 & \((1\times 3)/(1\times 1)\) & FM & Relu & y = Conv 2 (y) \\ \hline
5 & Dropout & 0.5 & - & - & y = Dropout (y) \\ \hline
6 & Shortcut & - & - & - & x = x \\ \hline
7 & Conv s & \((1\times 1)/(1\times 1)\) & FM & Relu & x = Conv s (x) \\ \hline
8 & Add shortcut & - & - & Relu & x = x + y \\ \hline
9 & Conv 3 & \((1\times 3)/(1\times 1)\) & FM & Relu & y = Conv 3 (x) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Detailed architecture of unit stage of ResNet. The number of nodes in the input layer was maintained at the same level as the sampling point of the input signal, which was 2048. The first convolutional layer contained 64 kernels of size \(1\times 32\), which remained large because of their performance for ultrasonic and vibration signals. [10; 11; 28]. The pooling layer reduces the number of layers but does not affect performance [22]. After the data have passed through stage 3, there is one fully connected (dense) layer and an output layer. The number of nodes in the fully connected layer was chosen as the best performance over multiple trials. The number of nodes in the output layer was kept equal to the number of flaws for classification. The activation function used in the layers was rectified linear unit (_Relu_), described by Equation (2)  
\[Relu=\left\{ \begin{array}{l}(x<0)\ f(x)=0\\ (x\geq 0)\ f(x)=x\end{array} \right. \tag{2}\] \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Adopted ResNet Architecture**} \\ \hline  & **Layer Type** & **Kernel Size/Stride** & **Feature Maps** & **Output Size** \\ \hline
1 & Input Layer & - & - & 2048 \\ \hline
2 & Conv 1 & \((32)/(1\times 8)\) & 64 & \((256\times 64)\) \\ \hline
3 & Dropout & 0.5 & - & \((256\times 64)\) \\ \hline
4 & Activation & Relu & - & \((256\times 64)\) \\ \hline
5 & Max Pool & \((3)/(1\times 2)\) & - & \((128\times 64)\) \\ \hline
6 & Stage 1 & - & 64 & \((128\times 64)\) \\ \hline
7 & Max Pool & \((3)/(1\times 2)\) & - & \((64\times 64)\) \\ \hline
8 & Stage 2 & - & 128 & \((64\times 128)\) \\ \hline
9 & Max Pool & \((3)/(1\times 2)\) & - & \((32\times 128)\) \\ \hline
10 & Stage 3 & - & 256 & \((32\times 256)\) \\ \hline
11 & Dense Layer & 300 & - & 300 \\ \hline
12 & Dropout & 0.5 & - & - \\ \hline
13 & Output Layer & 5 & - & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Architecture of ResNet.