# Cross-Modality Interaction-Based Traffic Accident Classification
## 5 Discussion  
### Cross-Modality Interaction  
We designated QD-DETR(Video) as the model that uses video and text features as inputs, and QD-DETR(Video+Audio) as the model that takes video, audio, and text features as inputs. In Tables 5 and 8, the QD-DETR(Video+Audio) outperforms QD-DETR(Video) across all evaluation metrics. Notably, QD-DETR(Video+Audio) CM shows a 7.23% higher performance in [EMAIL_ADDRESS], a 6.38% increase in MR-Accuracy, and a 3.88% improvement in HL-Min-Fair-mAP compared to QD-DETR(Video) CM. In Figure 10, the accident segments of the video contain sounds such as horns, crashes, and human screams. Therefore, such audio features from the accident segments, when combined with video and text features, enhance the performance of traffic accident segment prediction. Moreover, it has been demonstrated that training QD-DETR with video and audio features from the existing QVHIGHTLIGHTS dataset [12] results in better performance than training with only video features [14]. In Table 8, QD-DETR CS outperforms UMT CS with a 22.76% higher [EMAIL_ADDRESS], a 6.12% increase in MR-Accuracy, and a 3.73% improvement in HL-Min-Fair-mAP. QD-DETR CR exceeds UMT CR by 25.13% in [EMAIL_ADDRESS], by 6.65% in MR-Accuracy, and by 5.58% in HL-Min-Fair-mAP. Furthermore, QD-DETR CM surpasses UMT CM with a 25.17% higher [EMAIL_ADDRESS], a 4.52% increase in MR-Accuracy, and a 7.33% improvement in HL-Min-Fair-mAP. In Table 9, QD-DETR demonstrates superior performance in detecting traffic accidents with the query (\"Cars collide\") across all evaluation metrics when compared to UMT. QD-DETR CR boasts a 75.97% [EMAIL_ADDRESS], QD-DETR CS shows a 79.62% in MR-Accuracy, and a 77.01% in HL-Min-Fair-mAP. In Table 10, QD-DETR CS exhibits superior performance in moment retrieval metrics for the detection of \"Accident\" traffic accidents when compared to UMT CS. Specifically, QD-DETR CS shows an improvement of 24.4% in [EMAIL_ADDRESS], 7.02% in MR-Accuracy, and 4.77% in HL-Min-Fair-mAP over UMT CS. In video highlight detection, multi-modal learning, which efficiently fuses data with different characteristics, is crucial. In common, UMT [13] and QD-DETR [14] have a cross-attention structure where video and audio features are defined as query and text features as key and value. The two models differ in how they process video and audio features. Figure 11 shows the predictions from UMT and QD-DETR. Instead of combining video and audio features produced by self-attention within UMT's bottleneck transformer, \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Query** & \multicolumn{5}{c}{**“Accident”**} \\ \hline \multirow{2}{*}{**Metric**} & \multicolumn{3}{c}{**UMT**} & \multicolumn{3}{c}{**QD-DETR**} \\  & \(CS\) & \(CR\) & \(CM\) & \(CS\) & \(CR\) & \(CM\) \\ \hline [EMAIL_ADDRESS] & 42.86\% & 35.71\% & 42.86\% & 67.26\% & 59.52\% & 51.19\% \\ MR-Accuracy & 68.42\% & 64.91\% & 68.42\% & 75.44\% & 68.42\% & 59.65\% \\ HL-Min-Fair-mAP & 36.90\% & 36.55\% & 32.38\% & 41.67\% & 42.14\% & 44.52\% \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparing UMT and QD-DETR using video, audio, and text data (query: “Accident”, CCHD). \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Query** & \multicolumn{5}{c}{**“Cars Collide”**} \\ \hline \multirow{2}{*}{**Metric**} & \multicolumn{3}{c}{**UMT**} & \multicolumn{3}{c}{**QD-DETR**} \\  & \(CS\) & \(CR\) & \(CM\) & \(CS\) & \(CR\) & \(CM\) \\ \hline [EMAIL_ADDRESS] & 53.09\% & 50.62\% & 48.15\% & 75.57\% & 75.97\% & 75.41\% \\ MR-Accuracy & 73.98\% & 72.10\% & 72.10\% & 79.62\% & 79.31\% & 78.68\% \\ HL-Min-Fair-mAP & 73.46\% & 68.22\% & 69.94\% & 77.01\% & 73.69\% & 76.42\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparing UMT and QD-DETR using video, audio, and text data (query: “Cars collide”, CCHD). QD-DETR's simple concatenation approach to processing video and audio features shows superior performance. Encoding video and audio features allows the module that simply concatenates video and audio to automatically activate audio information [29].