# Pathological Gait Classification Using Early and Late Fusion of Foot Pressure and Skeleton Data
## 6 Results and Discussions
### Performance of Multi-Modal Using Late Fusion without and with Different Weights  
As discussed in [40], various strategies have been proposed and modified for sensor fusion. Here, the previously trained single-modals, ST-GCN and MViTv2_small, which correspond to the foot pressure and skeleton data, respectively, were combined to evaluate the classification performance of our proposed multi-modal late fusion method, both with \begin{table}
\begin{tabular}{c c} \hline \hline
**Subjects** & **Accuracy** \\ \hline Subject 1 & 100\% \\ \hline Subject 2 & 100\% \\ \hline Subject 3 & 100\% \\ \hline Subject 4 & 100\% \\ \hline Subject 5 & 100\% \\ \hline Subject 6 & 99.17\% \\ \hline Subject 7 & 100\% \\ \hline Subject 8 & 100\% \\ \hline Subject 9 & 99.17\% \\ \hline Subject 10 & 100\% \\ \hline Subject 12 & 100\% \\ \hline Average & 99.86\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Subject-wise validation accuracies of multi-modal using early fusion. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Subjects** & **ST-GCN (Last Frames)** & **ST-GCN** & **ST-GCN (Last 100 Frames)** & **ST-GCN (Min No. of Common Frames)** & **ST-GCN (All Frames)** \\ \hline Subject 1 & 100\% & 99.50\% & 100\% & 100\% \\ \hline Subject 2 & 100\% & 93.67\% & 100\% & 100\% & 100\% \\ \hline Subject 3 & 100\% & 100\% & 100\% & 100\% & 100\% \\ \hline Subject 4 & 100\% & 97.67 & 100\% & 100\% & 100\% \\ \hline Subject 5 & 100\% & 95.83\% & 100\% & 100\% & 100\% \\ \hline Subject 6 & 89.50\% & 79.34\% & 90.50\% & 91.17\% & 87.29\% \\ \hline Subject 7 & 100\% & 97.00\% & 100\% & 100\% & 100\% \\ \hline Subject 8 & 100\% & 97.17\% & 100\% & 100\% & 100\% \\ \hline Subject 9 & 84.67\% & 91.67\% & 96.33\% & 97.33\% & 96.88\% \\ \hline Subject 10 & 100\% & 98.33\% & 100\% & 100\% & 100\% \\ \hline Subject 11 & 100\% & 80.83\% & 100\% & 100\% & 100\% \\ \hline Subject 12 & 100\% & 94.17\% & 100\% & 100\% & 100\% \\ \hline Average & 97.85\% & 93.76\% & 98.90\% & 99.04\% & 98.68\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Subject-wise validation accuracies using different combinations for ST-GCN. and without applying different weights. The results were computed using the following procedures:  
* A single vector of the same size by selecting the maximum values from the two input vectors (element-wise) was created. * Two input vectors of the same size (element-wise) were averaged. * Two input vectors of the same size (element-wise) were multiplied. The validation accuracies for our multi-modal method using late fusion with and without different weights, and using the three methods described above, are presented in Table 5 and Figure 8. The accuracy using the maximum without weights is 96.95%, whereas the average accuracy with weights is 96.81%. When the outputs of two models were multiplied without weights, the accuracy was 96.67%. In the late fusion method with weight, the outputs with different weights were combined. A weight of 0.9 was assigned to the skeleton dataset due to its high accuracy and 0.1 to the foot pressure dataset due to its low accuracy. In the case of the maximum, an accuracy of 99.10% was achieved, whereas in the case of averaging, a 99.17% accuracy with different weights was achieved. Similarly, for multiplication, an accuracy of 96.67% was achieved. Our proposed multi-modal using late average fusion with different weights shows the highest performance among multi-modal late fusion methods. A comparison of the suggested models with the baseline is presented in Table 6. [PERSON] et al. [16] introduced single-modals using foot pressure and skeleton datasets, achieving accuracies of 68.82% and 93.40%, respectively. In contrast, our proposed single-modals achieved higher accuracies, reaching 78.2% for the foot pressure dataset and an impressive 99.04% for the skeleton dataset. In the realm of multi-modal approaches, [PERSON] et al. reported an accuracy of 97.60%, whereas our proposed multi-modal utilizing early fusion attained an accuracy of 99.86%.