## 5 Strip Steel Surface Defect Recognition Deep Neural Network Based on Transfer Learning
### Transfer Learning Deep Neural Network Based on VGG19  
The deep learning network requires a lot of high-quality annotated image data, but strip steel defects emerge randomly on the industrial production line. It takes a long time for strip steel production lines to collect enough defective samples, especially new strip steel production lines. To solve this problem, this paper chooses the VGG19 network [27] as the basic backbone and transfers its pre-trained parameters on ImageNet to our mixed image dataset. ImageNet dataset is mainly the data of life scenes (such as cats, bicycles, people), which has a big gap with the strip defect dataset in this paper. However, ImageNet dataset has a huge amount of data, and the VGG19 network model will not be overfitted on this dataset. Therefore, this paper fine-tune the VGG19 network pre-trained on ImageNet dataset to complete strip defect detection. With the help of transfer learning, our network achieved strip steel defect-recognition successfully.  
A network trained with a transfer learning paradigm was designed to achieve automatic defect recognition. In the mixed dataset, we randomly selected 80% of the dataset to train the network classifier, and the remaining 20% of the dataset was used to test the network performance. The network's input images were pre-processed by our improved gray-scale projection algorithm and ROI image augmentation algorithm. This is a strip steel defect dataset with full annotation and class balance. Model prediction accuracy and error are closely related. The mixed image dataset was pre-processed to create a new class balanced dataset; accuracy and categorical cross-entropy loss function are ideal indicators for evaluating the performance of various algorithms. In order to reflect the performance of the models, Formulas (3) and (4) were proposed.  
\[\text{Accuracy}=\frac{\text{Number of Correctly Classified Images}}{\text{Total Number of Input Images}}\times 100\% \tag{3}\]  
\[\text{Loss}(\text{P},\text{ Q})\ =\ -\sum_{i=1}^{N}\text{P}(i)\log\text{Q}(i) \tag{4}\]  
where P (_i_) is the probability calculated after softmax activation, Q (_i_) is the prediction output of the model, and \(N\) is the number of categories.  
The VGG19 network was designed as the basic backbone of our new network. The first 15 layers of the network were non-trainable frozen layers, while the four layers of the bottom network were used to conduct fine-tuning on the training dataset in the paper. Additionally, the loss function softmax cross-entropy classification was connected. Finally, matching with output classes, the strip steel defect-recognition transfer learning network based on VGG19 was construction. The initial learning rate was set as lr = 10\({}^{-6}\), theattenuation rate was set as decay = \(10^{-6}\), the attenuation momentum moment at 0.9, and the training step was 500 steps. The model was designed using Python 3.6 programming language, with Keras and TensorFlow libraries. The experimental equipment in this paper was a desktop personal computer with a I5-8500 processor, NVIDIA GTX 1050 graphics card, 32G RAM and hard disk capacity of 1T.  
Figure 12 shows the accuracy and loss error of the network. The network has a poor convergence performance on the testing dataset. Checking the loss of error curve of the training process carefully, it was found that the verified loss of the dataset was quite turbulent in the early stage of training. In the medium term, there as occasional increases in loss. Although the loss converges after training, the unstable loss curve obviously has some unreasonable network design and network structure, needing further improvement.