# SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection
## 4 Experiments and Results
### Network Training and Performance Metrics  
The four CNN architectures mentioned in Section 3.3 are trained for the roller surface classification task using three different strategies. Two of these strategies refer to training the CNN from scratch, which means starting from a random configuration of weights. The first from-scratch strategy uses a dataset augmented by a method similar to that in the GoogLeNet paper, while the second strategy uses the dataset augmented successively by the LD and SSAD methods. The third strategy is based on transfer learning from pretrained networks, which are respectively obtained from the corresponding papers cited above. Here, all of the network layers are fine-tuned, including convolution layers and fully connected layers. From this perspective, the approach is called a deep transfer strategy. The third strategy uses a dataset identical to that of the second one. We call the second and third networks an SDD-CNN because of the small data-driven augmentation method that they use. Note that these 12 training configurations (4 CNN architectures \(\times\) 3 strategies) use the same hyperparameter values (_momentum_ 0.9, _weight decay_ 0.0005, _learning rate_ 0.005, and _batch size_ 32). The number of network layers and parameters for each CNN model is displayed in Table 4. SqueezeNet benefits from its special Fire module, and the parameter size is the smallest of the four models. Inception v3 and ResNet18 have a similar number of parameters, and VGG16 is more than five times larger than Inception v3. Figure 7 shows a comparison of the time and number of iterations required to achieve convergence in the training of different network models. Obviously, in both strategies that train from scratch, SDD-CNN requires less training time and iteration times than the original CNN. This is because, for the roller defect image, the dataset of SDD-CNN preserves the defect features more completely, which enables the model to extract the target features more precisely and to reach the convergence more quickly. Specifically, the convergence time of the SDD-CNN can be reduced by at least 20% over \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model Name.** & **Training Type** & **Number of Layers** & **Number of Parameters** \\ \hline SqueezeNet v1.1 & From scratch & & \\ SDD-SqueezeNet v1.1 & From scratch & 18 & 728,139 \\ SDD-SqueezeNet v1.1 & Deep transfer & & \\ \hline Inception v3 & From scratch & & \\ SDD-Inception v3 & From scratch & 18 & 24,734,048 \\ SDD-Inception v3 & Deep transfer & & \\ \hline VGG16 & From scratch & & \\ SDD-VGG16 & From scratch & 16 & 134,305,611 \\ SDD-VGG16 & Deep transfer & & \\ \hline ResNet18 & From scratch & & \\ SDD-ResNet18 & From scratch & 18 & 11,196,107 \\ SDD-ResNet18 & Deep transfer & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Training strategies and basic scales for four CNN models. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Type** & **EFQ/EFC/EFI/EFSc/EFS/EFSF** & **CQ/CC/CI/CSo/CSt** \\ \hline Training set & 1440 & 960 \\ Validation set & 480 & 320 \\ Test set & 480 & 320 \\ Total number & 2400 & 1600 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Distribution of 11 categories of roller datasets. the original CNN, and in the case of ResNet18, this number is up to 42%. With regard to the number of iterations, the SDD-CNN can also be reduced by at least 20%; for ResNet18, the figure is 36%. The SDD-CNN from deep transfer learning benefits from its pretraining model, and the convergence time and iteration times are less than those of the general SDD-CNN. Conversely, from the perspective of architecture, Inception v3 has the shortest convergence time no matter which training strategy is adopted. Considering that this is not a small number of parameters, this convergence rate is quite good and should be attributed to its efficient architecture design. VGG16 is the slowest of the four architectures in terms of convergence speed because it has a huge weight to optimize.