## 3 Experiment and Analysis
### Experiment on OD-MF  
In the process of training the object detection model, we chose YOLOv5s as a baseline. First, the YOLOv5s model was trained. Additionally, the weights pre-trained on ImageNet were used to initialize the model parameters. Then, fine-tuning was performed on the parcel dataset made by ourselves. The laboratory data came from the data collected by the KinectV1 and KinectV2 cameras. The training set had 7500 images, the validation set had 2500 images and the test set had 2500 images. The hyperparameters required for model training are shown in Table 1. During the training process, the diversity of data was increased by horizontal and vertical inversion, rotation and other enhancement methods. During training, the batch size is set to 16, and the popular SGD optimizer with momentum was set to 0.9. The weight decay of 0.0005 was used to optimize our model for back propagation.  
In order to verify the superiority of the OD-MF proposed in this paper, the improvements in the network by adding depth map estimation and self-attention module are compared. The experimental results are shown in Table 2 below.  
It can be seen in Table 2 that compared with the original YOLOv5s model, the accuracy of parcel detection can be improved by 6.53% with multi-modal information (MMI). On the other hand, adding the self-attention module alone improved the accuracy by only 1.2%, and the recall was improved by about 4%. This proves that the method proposed  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameters** & **Value** \\ \hline batch size & 16 \\ base learning rate & 0.01 \\ learning rate decay & 9000, 14,000 \\ rate of learning rate change & 0.1 \\ maximum iterations & 16,000 \\ Momentum & 0.9 \\ weight decay & 0.0005 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The object detection network’s hyperparameter setting.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Precision (\%)** & **Recall (\%)** & **FPS** \\ \hline YOLOv5s (Baseline) & 91.75 & 92.33 & 16.13 \\ \(+\) MMI & 98.28 (+6.53) & 95.56 (+3.23) & 13.89 \\ \(+\) Self-Attention & 93.23 (+1.48) & 96.05 (+3.72) & 15.15 \\ \(+\) MMI and & 98.57 (+6.82) & 96.62 (+4.29) & 13.51 \\ Self-Attention & & & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of the precision, recall and FPS of different network structures on our parcel dataset.  
Figure 8: The robot sorting experimental platform mainly includes a UR-5 6-DOF robot, a RGB-D camera and a sucker actuator.  
in this paper can more easily supplement and integrate explicit knowledge and implicit knowledge. At last, OD-MF with MMI and the self-attention module improved accuracy by 6.82%. That is, the network significantly improved the detection accuracy of parcels. The reason is that OD-MF can better learn the interaction information in the image by using the depth image as the estimated supervision signal and the self-attention module. Through these experiments, it can be concluded that OD-MF can effectively solve the problem of detection of stacked parcels, and improve precision and recall without losing speed.  
On the other hand, it was also found in the experiment that the gradient distribution of the obtained depth map is biased and cannot be directly used as the global gradient feature of the image. The relative positions of the camera and the target shifted during the process of collecting data. To solve this problem, we used normalization and calculated the maximum gradient (MAG) in the process of acquiring global gradient characteristics of the depth map. The influences of the two improvements on the experimental results were proved by experiments. As shown in Table 3, the OD-MF using normalization and MAG could effectively improve the recall rate of objects.