# Using Artificial Neural Network Models to Assess Hurricane Damage through Transfer Learning
## 4 Results and Discussion
### Damage Classification  
After the completion of the transfer learning model training using the base models of ResNet and MobileNet on our in-house flood damage dataset, the newly trained models are tested with images they did not see during the training stage. Among the test dataset, the number of _floods_ images is 92, and the number of _non-floods_ images is 107. It is noted that the _floods_ and _non-floods_ images are slightly imbalance. The same set of _floods_ images and _non-floods_ images was tested by the newly trained models using ResNet and MobileNet. In this study, the _floods_ class was the positive class, and the _non-floods_ class was the negative class, resulting in the following four possible predictions:  
* TN / True negative: an image was _non-floods_ and predicted as _non-floods_;
* TP / True positive: an image was _floods_ and predicted as _floods_;
* FN / False negative: an image was _floods_ and predicted as _non-floods_;
* FP / False positive: an image was _non-floods_ and predicted as _floods_. Figure 4: The training metrics for damage detection using classification loss, localization loss, regularization loss, and total loss up to the twenty-thousandth (20 k) epoch. Figure 5 shows the confusion matrix for both ResNet and MobileNet. Each row in a confusion matrix represents a true label (i.e., an actual class), while each column in a confusion matrix represents a predicted label. In this study, the first row and the first column have the label of _non-floods_, while the second row and the second column have the label of _floods_. The diagonal elements of the confusion matrix represent that the predicted label is equal to the true label, while off-diagonal elements are those that are mislabeled by the classifier. The values in each of the elements are normalized by the total number of images for each class. It is expected that a perfect classifier would have only true positives (lower right) and true negatives (top left). Figure 5 shows both ResNet and MobileNet are able to classify _non-floods_ and _floods_ images with the larger percentages along the diagonal elements. An accurate prediction means that a _non-floods_ image was predicted as _non-floods_, and a _floods_ image was predicted as _floods_. The prediction accuracy of ResNet is about 76%, and the accuracy of MobileNet is about 87%. Both classifiers show that true positive predictions (85% by ResNet and 89% by MobileNet) are higher than true negative predictions (66% by ResNet and 85% by MobileNet). A further comparison shows that the transfer learning model using MobileNet outperforms the one trained using ResNet for both true positive and true negative predictions. Specifically, the true positive prediction percentage of MobileNet is about 89%, while the true positive prediction percentage of ResNet is about 85%. Similarly, the true negative prediction percentage of MobileNet is about 85%, while the true negative prediction percentage of ResNet is about 66%. Model classification performance is further evaluated by using precision, recall, and F1 score, as defined in Equations (3)-(5), respectively. The result is summarized in Table 3. The precision metric measures the accuracy of the true positive predictions (i.e., _floods_ label) by dividing the true positive predictions over the sum of true positive and false positive predictions. The precision values for ResNet and MobileNet are 0.75 and 0.87, respectively. This indicates that MobileNet shows a higher accuracy (about 12%) than ResNet in terms of _floods_ image classification. The second metric recall measures the true positive rate (i.e., the ratio of _floods_ images that are correctly detected by the classifiers. The result shows that the recall of MobileNet is about 4% higher than that of ResNet. The last metric examined in this study is the F1 score, which is the harmonic mean of precision and recall, as defined  
Figure 5: The confusion matrix for flood damage classification using (**a**) ResNet and (**b**) MobileNet.