# Heated Metal Mark Attribute Recognition Based on Compressed CNNs Model
## 5 Experimental Evaluation
### Recognition Accuracy Evaluation  
Recognition accuracy was used to evaluate recognition performance on different attribute. As shown in Equation (9), \(R_{i}\) is the recognition accuracy for attribute \(a_{i}\), \(N_{i}^{all}\) means the number of all testing samples containing \(a_{i}\), and \(N_{i}^{correct}\) denotes the number of correctly recognized attribute \(a_{i}\). We divided the dataset into six subgroups with attribute values equally distributed. Five randomly chosen subgroups (1500 image samples) were used for training and the remaining subgroup (300 image samples) was used for testing. The results were obtained by averaging the five independent tests.  
\[R_{i}=\frac{N_{i}^{correct}}{N_{i}^{all}} \tag{9}\]  
_MobileNet_, \(SluffleNet\) and \(SqueezeNet\) were used as backbone compressed CNNs models for evaluation. For model input, sample image size was set as 224 \(\times\) 224 \(\times\) 3 pixels. Epoch was set as 50 and batch size was set as 32. Adam was used as preferred optimization method. Learning rate was set with initial value of 0.001 and momentum was set as 0.9. Dropout was set as 0.2.  
The results of average recognition accuracy are shown in Table 3. Structure of CNNs models and data augment are listed in the first and second columns, respectively. For data augment, commonly used transformations including random cropping, vertical and horizontal flipping, perturbation of  
Figure 2: Demonstration of multi-label model training.  
brightness, saturation, hue and contrast were adopted. When the model was trained with data augment, 40% of training image in each batch was augmented, otherwise the probability was 10%. For \(a_{1}\), \(ShuffleNet\) with data augment obtained the best performance, with value of 0.803. For \(a_{2}\), \(SqueezeNet\) with data augment obtained best performance, with value of 0.837. For \(a_{3}\), \(SqueezeNet\) with data augment obtained best performance, with value of 0.825. For \(a_{4}\), \(ShuffleNet\) with data augment obtained best performance, with value of 0.812. For \(a_{5}\), \(MobileNet\) with data augment obtained best performance, with value of 0.883. For \(a_{6}\), \(MobileNet\) and \(ShuffleNet\) with data augment obtained best performance, with value of 0.817. For \(a_{7}\), \(ShuffleNet\) with data augment obtained best performance, with value of 0.894. For the overall performances, \(ShuffleNet\) model ranked first.  
We found that models training with data augment obtained better performance than those without data augment. There was about 2% accuracy improvement. It can be concluded that data augment is an effective way to train better CNN models, especially for large scale CNNs with huge parameters and lacking of sufficient training data.  
Figure 3 demonstrates the misclassified sample images. Each row corresponds to an attribute. The red texts represent ground truth label, while yellow texts represent the predicted results. GAlvanized steel and cold rolled steel normally have different corrosion degrees at the experimental condition. The misclassified sample images of \(a_{1}\) showed similar corrosion degree. For heating temperature, higher temperatures will lead to more corrosion and rougher texture. The misclassified sample images of \(a_{3}\) came from adjacent temperature. These situations can be seen as general causes of \(a_{4}\), \(a_{6}\) and \(a_{7}\). For \(a_{2}\) and \(a_{5}\), the reason for misclassification is hard to describe even for the field professional.  
Commonly used large scale datasets are mainly natural scene, animals, etc. These are easy to distinguish by humans, and the differences are easy to explain visually. The heated metal mark image we studied is a special kind of objects, the origin of its mark being caused by complex physical and chemical reactions. Moreover, the benchmark dataset we generated inevitably contains noise, which may influence the model performance. We need further research to explore the internal principle with the help of other professionals.