## 2 Materials and Methods
### Fusion Rule and Meta-Classifier Evaluation  
The first step for implementing late fusion is to load the prediction.csv files from both our visual and audio prediction models. Each file consists of a list of predictions that were gathered during the testing phase of both models. Below, in Tables 2 and 3, a sample from each file is presented in Tables 2 and 3. The two prediction data frame files were loaded into memory. Merging was not based on common filenames (as we would have performed in the case of two datasets from the same source with a high correlation degree between them) but was based on the true value, the predicted label in our case. Each row was iterated based on the visual data frame and correlated with a corresponding row that exists in the audio data frame in the same order [44; 45; 46]. Traditional multimodal analysis methods are unable to solve uncertainty estimation problems among different modalities effectively. In order to avoid label mismatches or conflicts in probability uncertainties, we employed a stable and tested methodology [47] to counter these issues. First, we defined a function that will calculate the uncertainty of the predictions between the two models. This is measured based on the differentiality between the two highest probabilities, and when the two probabilities are close to each other, an indication flag of high uncertainty is being raised, indicating that the model is not confident. Next, we developed a hybrid uncertainty calibration that adjusts the final predictions of the two models based on the confidence of each model (visual and audio models). In the case of one model being more confident than the other, that model has more influence in the final decision. If the certainty of the models is equal, a weighted average based on their confidence levels is created, and this defines the final prediction. Subsequently, noise functions were applied, and two types of noise were introduced to the probabilities in order to simulate real-world conditions. The Gaussian noise was randomly added to the visual probabilities with normal distribution, and salt-and-pepper noise was added to the audio probabilities, where values of 0 and 1 were randomly set to simulate sudden data errors. As the next step, we refined the true labels into a new column named **refined_true_label**. The function responsible for this functionality adjusts the final label (aggressive or non-aggressive). The way this is achieved is by finding if either model has a probability greater than 0.7 to decide the label directly, but if the probabilities are low, it averages the probabilities to make a new decision. This multimodal methodology enables each data modality to leverage its strengths while mitigating uncertainty through rigorous confidence thresholding and the avoidance of low-reliability data points. This can apply in real-world scenarios, when a single modality might dominate in obscure cases, such as when an individual uses verbal aggression with a conflicting body stance. Merged results consist of the filenames, true labels, predicted labels, and average probabilities, as shown in Table 4. \begin{table}
\begin{tabular}{c c c c} \hline
**Filename** & **True_Label** & **Predicted_Label** & **Probabilities** \\ \hline image\_00721.jpg & aggressive & 1 & [0.23, 0.77] \\ \hline image\_00467.jpg & aggressive & 1 & [0.28, 0.72] \\ \hline image\_00556.jpg & aggressive & 0 & [0.67, 0.33] \\ \hline image\_00680.jpg & aggressive & 1 & [0.04, 0.96] \\ \hline \(\dots\) & \(\dots\) & \(\dots\) & \(\dots\) \\ \hline \end{tabular}
\end{table} 
Table 2: Sample from visual_detection_results.csv file after visual model testing.