## 3 The Hierarchical Steel Surface Detection Methodology
### Defect Image Classification  
For defect image classification, the combination of CNN and ensemble algorithm is carried out. In this study, three convolutional models, VGG16, VGG19 and ResNet50, are experimented with, and out of them, VGG16 outperforms the other two models. In Table 1, it is clearly depicted that VGG16 accuracy is better than the other two models. The VGG16  
Figure 1: The overall architecture of the proposed approach. convolutional neural network is used as a feature extractor model where it extracts the features of defects. The architecture is a simple and uniform convolutional network. It is a recognized and preferred choice for feature extraction. It is the pre-trained model on the imagenet dataset. The input to this network has the dimension of 256 \(\times\) 256 \(\times\) 3. The VGG16 has a total of five convolutional blocks. The first two convolutional blocks provide two convolution filters, followed by max-pooling and the remaining three convolutional blocks contain three convolution filters, followed by max-pooling. The final output of this feature extractor is in the shape of 8 \(\times\) 8 \(\times\) 512. These extracted features go through the ensemble model, the XGBoost classifier [46], for the classification of the defects. XGBoost (extreme gradient boosting) is a ML algorithm based on the decision tree. XGBoost is a scalable boosting tree that optimizes gradient boosting. It is capable of handling missing data and overfitting issues, through a parallel process. The system is optimized through parallelization. It handles sparse data and performs out-of-core computation with large datasets. In steel surface defect detection where time is the critical factor, XGBoost can handle this factor, as it is computationally fast and requires fewer resources for computation. This algorithm is the most effective ML algorithm among most of the existing ones for classification. The XGBoost's main feature is the gradient descent algorithm. The results of this ensemble algorithm are promising. The XGBoost subsists of classification and regression modules. In this study, we need the classifier module. The reason behind using this algorithm is that it involves less computational resources and requires less time for classification, compared with other ensemble algorithms. It uses the depth-first approach and avoids overfitting through regularization. It is capable of handling missing data and has an in-built cross-validation feature for determining the model's effectiveness and reduces the bias and variance. After the classification of the defects, it goes to the detection phase consisting of two models parallelly aligned. The first one is the object detection model, RetinaNet, and the other is the image segmentation model UNet. The five defects (crazing, inclusion, pitted surface, patches, rolled-in scale) pass through the RetinaNet model, and the scratch defect goes through the UNet model. Object Detection for the Pitted Surface, Inclusion, Patches, Rolled-In Scale and Crazing Defects Detection  
To provide an algorithm for real-time applications, the one-stage detectors, RetinaNet [47], as the defect detection for pitted surface, inclusion, patches, rolled-in scale and crazing is employed. The RetinaNet model, compared with other state-of-the-art detectors, has a simple architecture with good speed and acceptable accuracy. The RetinaNet architecture comprises the following three sections: first, the backbone network; second, the classification subnet and third, the regression subnet. The backbone network section comprises of a bottom-up pathway and a top-down pathway. The input image is passed through the bottom-up pathway, consisting of ResNet50 as a feature extractor. This extractor generates the multi-scale feature maps in bottom-up hierarchical form. Each feature maps belongs to the last layer of the convolution stage. In the bottom-up path, the semantic values increases and the spatial resolution decrease.