## 4 Discussion
* **Strengths of Early Fusion:** \(\odot\) **Higher precision**: Early Fusion exhibits a lower false positive rate, which is particularly beneficial for applications where reducing unnecessary alerts is a priority; \(\odot\) **Direct integration of multimodal features**: Unlike Late Fusion, which combines predictions, Early Fusion merges raw feature data, allowing for richer feature-level interactions between audio and visual cues; \(\odot\) **Simpler training process**: A single classifier is trained on the fused dataset, eliminating the need for separate models and fusion steps, reducing training complexity; \(\odot\) **Potentially stronger feature representation**: Combining data at the feature level enables the model to learn complex correlations between visual and audio signals that might not be captured when training models independently (**S1.7**). * **Limitations of Early Fusion:** \(\odot\) **Lower recall**: The model is more prone to missing instances of aggression due to the direct integration of features, which may not always effectively capture subtle behavioral patterns; \(\odot\) **High-dimensional multimodal feature space**: The increased complexity of the dataset, resulting from concatenating visual and audio features, may lead to overfitting if not managed properly (**S1.4, S1.5, S1.7**); \(\odot\) **More sensitive to missing data**: Since the model relies on both modalities being present for training and inference, any missing or corrupted features could degrade classification performance significantly; \(\odot\) **Potential difficulty in feature alignment**: Even though concatenation techniques were used to ensure proper label correspondence, the lack of direct correlation between audio and visual features could introduce noise into the training process, affecting overall performance. **Fusion Strategy Implications**  
As mentioned above, the choice between Early and Late Fusion is challenging. Each model has its own strengths and weaknesses, which must be taken into consideration when choosing the preferred strategy for a specific application. The Late Fusion approach is preferable in applications where high recall is crucial for real-world clinical scenarios, with human supervision, where an imminent episode of aggressive behavior must be identified in time to reduce risk. Early Fusion is preferable for applications where priority is given to precision, such as in automated systems without human supervision, where there is a need to minimize false cases. Additionally, the Late Fusion method provides flexibility considering the update of the model since audio and visual features are trained separately. This can improve the maintenance and scalability of the model, since it is not dependent on all modalities at a given time; therefore, retraining is not required to make additional improvements to one modality. This provides modularity to the Late Fusion approach, which is beneficial for frequently updated systems that need to adapt to new data. **Model Robustness**  
Model robustness is another critical factor during the evaluation of the model fusion approach, especially in cases of missing modalities or handling variations in data, fast and efficiently in real-world deployment challenges. Based on the results discussed earlier, Late Fusion demonstrates superior performance thanks to its independent decision-making process, which allows for compensation in missing and corrupted data in modalities. This highlights its resilience in real-world conditions where data are often inconsistent or come with high noise. Early Fusion implementation, on the other hand, can effectively integrate multimodal features from different cues and is more sensitive to data inconsistencies between audio and visual data. Feature concatenation at an early stage compensates for data discrepancies, as well as noise in the data, and can propagate through the model. This reduces overall classification performance and also reduces real-world adaptability due to its dependence on balanced and synchronized multimodal data.