## 3 Classification with Machine Learning and Results
### Results  
The extracted data from the simulations have been exploited to conclude whether the proposed classification models can detect the variation of the MUT thickness. The procedure started with the training classification models with eight classes of all thicknesses of simulated specimens. The models obtained after the training were then tested to verify their accuracy. The 80 simulated structures with 101 frequency points in each one resulted in 8080 data samples in which each sample consists of five of the aforementioned input features and the corresponding thicknesses. The dataset was then split into three sets by selecting 70% of the total acquisitions for the training phase, 20% to validate the trained model, and the distinct remaining 10% for the testing set. The validation was conducted by putting 20% of the acquisitions to the trained models and comparing the attained classifications with the corresponding simulated MUT thicknesses to frequently evaluate the model. The distinct testing set was then taken into account for the final evaluation after the models had been completely trained and validated. The same amount of data in the training set, validation set, and testing set were used in the training phase, validation, and testing phase for both DNN and SVM classification models in this paper. The flowchart of the investigational platform is shown in Figure 7. First, the scattering parameters and dielectric constants of MUTs were acquired as described in Section 2. Subsequently, two predictive models were trained using the input features from the previous procedure. The evaluation of training and prediction results were then used to compare the models. Figure 6: The overall DNN structure. The hidden layers comprise multiple fully connected, ReLU and Drop-out layers. The output layer comprises the fully connected layer followed by a soft-max. In terms of the aforementioned SVM training above and based on the 10-fold cross validation results, the grid search was able to find the optimal pair value of both parameters (C and \(\gamma\)) of four kernels. The overall summary of various kernels is listed in Table 2. It can clearly be seen that the best SVM model values using the RBF kernel with the optimal C and \(\gamma\) after grid search are 224.81 and 0.26986, respectively. The reason for this is that SVM adopting RBF kernel had the highest accuracy compared to its competitors while the linear kernel provided the lowest classification ability. However, it should be noted that the search interval is also a problem. Due to the large search region, it will waste many computational resources. For example, the polynomial kernel required a relatively longer computational time than the others when employing 70% of datasets with a duration of around 3 h while the other kernels needed approximately 2 h to finish searching. However, too small a search region may lead to an unsatisfactory outcome. Therefore, one must correctly analyze the search region before conducting the optimization simulations. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Kernel Functions**} & \multicolumn{3}{c}{**Optimal Pair Value**} & \multirow{2}{*}{**Accuracy (\%)**} & \multirow{2}{*}{**10-Fold Loss**} \\ \cline{2-2} \cline{4-5}  & \(\mathbf{C}\) & \(\gamma\) & & \\ \hline Linear & 1.7079 & 0.03335 & 81.3 & 0.18907 \\ Polynomial & 0.20641 & 0.39028 & 99.1 & 0.0016732 \\ Sigmoid & 181.019 & 0.31079 & 98.6 & 0.0019681 \\ RBF & 224.81 & 0.26986 & 99.7 & 0.0011154 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Optimal pair value of four SVM kernels. Figure 7: The investigational platform. The results of the SVM classification when adopting the RBF kernel are illustrated in Figure 8a with different percentages of data being trained and tested. For each data partition, the algorithm was independently executed 20 times using the training/testing data, which were randomly selected for each time.