# Machine Learning-Based Highway Truck Commodity Classification Using Logo Data
## 4 Experiments
### Experimental Results
#### 4.2.3 Integrated Logo Model  
We evaluated the developed logo models using only the text-based approach, only the image-based approach and the integrated approach. It is worth noting that certain logo classes (such as 'davis') are'relatively unclear' and difficult for the image-based approach to recognize, as they are likely to be cluttered with scene context. In many cases, it may be worth considering them as texts rather than as image-based logos. This can be effectively addressed using the proposed text-based approach. For example, the 'SouthernAG' logo class can be handled by the text-based solution with high recall and precision, although it achieves poor performance with the image-based approach. Image-based Logo Detection and RecognitionWe evaluated the image-based logo solution using the mAP metric. We achieved an mAP of 66.2%. Given the relatively small number of samples for each logo in our datasets, these results are promising. We expect additional improvements as additional annotated images are added to the dataset. Text-based Logo Detection and RecognitionThe text-based solution has a high precision on matching texts to company names. In general, it achieved higher precision compared to the image-based solution. The recall is slightly worse, which can be partly attributed to the fact that the text detection tends to predict tighter bounding boxes around text regions. The logo region is usually larger than the text regions, as it usually consists of both text and figure regions. Because the ALD dataset is annotated as logos, bounding boxes predicted by our text solution are expected to be smaller than the ground truth annotations, which can worsen the recall. Ablation Study on Integration ApproachesTo evaluate the two integration approaches we presented in Section 3, we apportioned the ALD dataset into training and testing set with a 70-30 split. The evaluation result of the two approaches on the testing set of ALD is shown in Table 5. On the apportioned testing set, approach 1 (text-focused approach) yields better overall performance in terms of mAP and Acc, while approach 2 gives higher precision. The results may vary on different datasets. We choose to use approach 1. The subsequent experiments were conducted on the whole ALD dataset (as opposed to the apportioned testing set) because approach 1 does not require additional training. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Top-1 Accuracy** & **Top-3 Accuracy** & **Top-5 Accuracy** \\ \hline Deep Metric & 90.0 & 96.3 & 97.1 \\ Bag of Words & 88.3 & 95.8 & 97.3 \\ Fused & 95.3 & 97.9 & 98.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Evaluations on logo recognition models with different feature representations. _Integrated Logo Model_. The end-to-end logo detection and recognition performance were measured on the ALD dataset. Besides the aforementioned detection metrics (Rec, Prec, and mAP), we also evaluated the classification accuracy (Acc), where a single predicted class is obtained by a majority voting of all bounding boxes' labels detected in one frame. The experimental result is showed in Table 6. We obtained an mAP of 81.2%, surpassing both the text-based and image-based approach significantly. Given that there were 26 logo classes, these results are promising. As discussed earlier, additional annotation, in particular for the difficult classes, should further improve the overall accuracy. As can be found in Table 6, the text-based approach performs well in the easy and medium categories. It fails to detect logo classes such as 'OD', 'Opies', and 'E', where 'E' and 'OD' logos are designed with artistic fonts and figures. The 'Opies' logo usually appears on the body of the tank truck, where the compartment is made of reflective materials. The lighting reflection causes the text-based approach to fail to detect 'Opies'. These studies can help traffic agencies customize their specific tasks by choosing a particular solution considering the characteristics of the data.