## 4 Experimental Evaluation
### Evaluation Indicators and Inference Models  
To properly evaluate the proposed inference model, we used the following three evaluation indicators (EIs): object detection accuracy, classification accuracy, and inference time. Each performance indicator used the metrics addressed in Table 6. \begin{table}
\begin{tabular}{c c} \hline \hline
**Indicator** & **Measuring Method** \\ \hline Object Detection Accuracy & Accuracy according to Equation (1) \\ Classification Accuracy & Precision, recall, and F1 score according to Equations (2)–(4) \\ Inference Time & Frames per second (FPS) when using the real embedded device \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Method of estimating each evaluation indicator (EI). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Class** & **Number of Classes** & **Ratio** \\ \hline Safe & 41,039 & 52\% \\ Caution & 21,506 & 27\% \\ Danger & 16,878 & 21\% \\ \hline Total & 79,423 & 100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Each class distribution. Figure 10: Sample images and labels. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Datasets** & **Number of Images** & **Ratio** \\ \hline Training Set & 11,378 & 70\% \\ Validation Set & 2867 & 15\% \\ Test Set & 2867 & 15\% \\ \hline Total & 19,112 & 100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Dataset distribution. The accuracy of the object detection is the ratio of correctly classified traffic flow samples to the total number of samples, which is represented by the following equation:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN}. \tag{1}\]  
Here, we list four common variables. The binary confusion matrix is shown in Table 7. The coordinate axes represent two properties of the label: 'True' and 'False'. If both the actual and predicted labels are 'True', then we call this case true positive (\(TP\)). If both the actual and predicted labels are 'False', then we call it true negative (\(TN\)). False negative (\(FN\)) is when the actual label is 'True' and the predicted label is 'False'. False positive (\(FP\)) is the opposite. Next, to evaluate the classification accuracy, we estimated the precision, recall, and F1 score. Precision is a measure of the ratio of positive, correctly predicted classes to the total number of positive classification predictions, depicted as in the following equation:  
\[Precision=\frac{TP}{TP+FP}. \tag{2}\]  
Recall measures the ratio of the actual positive, correctly predicted traffic classes as in the following equation:  
\[Recall=\frac{TP}{TP+FN}. \tag{3}\]  
The F1 _score_ measures the average of precision and recall as in the following equation:  
\[F1\,score=\frac{2\times Precision\times Recall}{Precision+Recall}. \tag{4}\]  
Finally, to evaluate the inference time as the time cost of the inference model, we measured the frames per second (FPS) through the real embedded device (NVIDIA Jetson AGX Xavier(Nvidia. 2021. jetpack (version 4.6.4). Ubuntu. Nvidia)) used as the edge device for the proposed real-time shipping container accident inference system. To implement the inference model in order to classify the alignment status of shipping containers and then infer a container accident, we used YOLOv3, YOLOv4, and YOLOv7 as the learning algorithms. The total inference models consisted of six models: YOLOv3 with 416 pixels, YOLOv3 with 640 pixels, YOLOv4 with 416 pixels, YOLOv7 with 416 pixels, and YOLOv7 with 640 pixels. The hyperparameters used to train each model are listed in Table 8. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Actual Class**} & \multicolumn{2}{c}{**Predicted Class**} \\ \cline{2-3}  & **Safe** & **Non-Safe** \\ \hline Safe & TP & FN \\ Non Safe & FP & TN \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Confusion matrix in the example of the safe class.