# Acute Respiratory Distress Identification via Multi-Modality Using Deep Learning
## 4 Experimental Analysis
### Ablation Study
#### 4.4.4 Late Fusion  
Due to the shortcomings of our initial approach, we adopt a late fusion strategy and evaluate three different schemes: feature concatenation (FCAT), score averaging (SA),  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Fusion**} & \multicolumn{4}{c}{**Accuracy**} & \multicolumn{4}{c}{**Precision**} & \multicolumn{4}{c}{**Recall**} & \multicolumn{4}{c}{**\(F_{1}\) Score**} \\ \cline{2-13}  & **Min** & **Avg** & **Max** & **Min** & **Avg** & **Max** & **Min** & **Avg** & **Max** & **Min** & **Avg** & **Max** \\ \hline
**Baseline** & 0.783 & 0.822 & 0.870 & 0.818 & 0.872 & 0.95 & 0.72 & 0.777 & 0.833 & 0.783 & 0.821 & 0.864 \\ \hline
**CC** & 0.696 & 0.765 & 0.848 & 0.667 & 0.762 & 0.870 & 0.792 & 0.818 & 0.833 & 0.741 & 0.788 & 0.851 \\ \hline
**FCAT** & 0.804 & 0.830 & 0.848 & 0.800 & 0.821 & 0.840 & 0.833 & 0.868 & 0.917 & 0.816 & 0.843 & 0.863 \\ \hline
**SA** & 0.804 & 0.830 & 0.848 & 0.759 & 0.808 & 0.846 & 0.875 & 0.893 & 0.917 & 0.830 & 0.847 & 0.863 \\ \hline
**FCAT-F** & 0.804 & 0.852 & 0.913 & 0.808 & 0.867 & 0.917 & 0.760 & 0.852 & 0.917 & 0.809 & 0.858 & 0.917 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance comparison of different feature fusion techniques across five folds (CC— channels concatenation, FCAT—feature concatenation, SA—score averaging and FCAT-F, feature concatenation with freezing base-model). Performance metrics, including accuracy, precision, recall, and \(F_{1}\) score, are presented as the minimum (min), average (avg), and maximum (max) scores across the five folds.  
Figure 4: Block diagram illustrating the various types of multi-modality fusion schemes: (**a**) early fusion, where input modalities are combined at the input level; (**b**) score averaging, where individual modality predictions are averaged; (**c**) late fusion, where features are combined after independent processing of each modality w/o base-model freezing.  
and feature concatenation with frozen base models (FCAT-F). In the FCAT approach, we employ a two-stream network to independently extract features from both RGB and depth modalities. These features are then concatenated at a later stage and used for classification. For the SA approach, we utilize two identical models to predict ARD condition scores for each modality, similar to the previous approach. These scores are aggregated and used for the final detection. Both FCAT and SA models are trained end-to-end. In contrast to the previous approaches, the FCAT-F method involves first training the models independently on each modality. Once trained, these models are used as feature extractors by removing their classification layers. The features from both modalities are then concatenated and passed to a single-layer neural network. During training, the weights of the base models, which were trained on the hospital dataset, are frozen, allowing only the fully connected single-layer neural network to be trained.  
The experimental results of the late fusion schemes are presented in the last three rows of Table 2. The third row presents the results of the FCAT technique, which achieved an average accuracy of 0.830, a precision of 0.821, a recall of 0.868, and an \(F_{1}\) score of 0.843, which is better than the RGB model. The fourth row shows the results of the SA technique, which achieved an average accuracy of 0.830, a precision of 0.808, and a recall of 0.893, The results of the FCAT-F technique are shown in the fifth row, demonstrating superior performance with an average accuracy of 0.852, a precision of 0.867, a recall of 0.852, and an \(F_{1}\) score of 0.858.