# Depth Evaluation of Tiny Defects on or near Surface Based on Convolutional Neural Network
## 3 Results and Discussion  
### Neural Network Architecture  
The CNN (convolutional neural network) is one of the most extensively employed deep learning neural networks in signal analysis, with remarkable achievements in image processing and speech recognition [23; 24]. It predominantly comprises three components: the input layer, feature extraction layer, and fully connected layer, as depicted in Figure 7. The input layer is utilized for inputting the training and testing data. The feature extraction layer constitutes the core of the CNN, encompassing both the convolutional and pooling layers, which collaboratively capture pivotal features from the data and learn potential patterns from the constructed datasets. The convolutional layer filters the input data through a specified number of convolutional kernels, generating complete feature maps. This process simplifies the input data and extends them in depth. To introduce non-linearity into the model and address the issue of insufficient expressive power in linear models, enhancing the neural network's ability to linearly separate complex features, it is common practice to incorporate activation functions following the convolution operation [25]. Commonly used activation functions include the sigmoid function, the tanh function, and the rectified linear unit (ReLU) function. The sigmoid function is primarily used in binary classification neural network structures. Tanh suffers from issues like gradient vanishing and power operations. On the other hand, the ReLU function only requires a simple check on whether the input is greater than zero, leading to faster convergence and addressing the problem of gradient vanishing. Therefore, in this study, the ReLU function is selected as the activation function. The representation of the mth feature map output from the \(N\)th convolutional layer can be expressed as follows:  
\[x_{m}^{N}=ReLU\left(\sum_{i=1}^{M}\left(x_{i}^{N-1}\times k_{im}^{N-1}\right)+ b_{m}^{N}\right) \tag{3}\]  
where \(k_{im}^{N-1}\) is the kernel of the Nth filter, \(b_{m}^{N}\) is the corresponding bias matrix, and \(M\) signifies the number of feature maps inputted from the (\(N-1\))-th layer. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Type** & **0.1 mm** & **0.2 mm** & **0.3 mm** & **0.5 mm** & **No Defect** \\ \hline Training set & 140 & 140 & 140 & 140 & 140 \\ \hline Test set & 40 & 40 & 40 & 40 & 40 \\ \hline Validation set & 20 & 20 & 20 & 20 & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Dataset partitioning. Figure 7: The typical architecture of a CNN. Directly using the feature map output from the convolutional layers without any processing can lead to increased computational complexity and a higher risk of overfitting during the model's learning process. Therefore, to simplify the computational process, pooling layers are typically placed after each convolutional layer. These pooling layers down-sample the input feature maps, replacing feature information from neighboring regions with single-point data. Depending on the pooling function used, pooling can be divided into max pooling and average pooling. Average pooling computes the average of the values within the pooling range, while max pooling selects the maximum value within that range. At the end of the CNN network, there are fully connected layers responsible for transforming the output feature maps from the previous pooling layer into feature vectors. For non-linear problems, multiple fully connected layers are often used. In a study that involves multi-classification, the output layer after the fully connected layers uses the SoftMax function to output classification labels. The complete structure of the 2D CNN for processing ultrasonic signals is depicted in Figure 8. The input data consist of wavelet time-frequency images that have undergone crucial feature extraction and normalization. The parameters for the convolutional layers, pooling layers, and fully connected layers were optimized in accordance with the dataset size and image characteristics.