# Pathological Gait Classification Using Early and Late Fusion of Foot Pressure and Skeleton Data
## 7 Conclusions and Future Directions  
In this study, transformer-based single-modals were introduced as the first step to categorize one normal and five abnormal gaits (antalgic, lunch, steppage, stiff-legged, and Trendelenburg), following the application of 30-degree rotation augmentations to foot pressure data. Subsequently, another single-modal approach utilizing the skeleton dataset was presented, referred to as ST-GCN. Next, a multi-modal method employing early fusion was proposed, achieved by concatenating the features from both modalities. In the final step, a multi-modal approach employing late fusion, with and without different weights, was introduced to amalgamate the outputs from both modalities. Our early fusion multi-modal approach exhibited superior performance in comparison to late fusion. Our proposed models underwent a thorough comparison with the baseline, and the results undeniably demonstrate that our work surpassed both single-modal and multi-modal approaches. The findings of this study hold the potential to enhance existing gait analysis programs through the implementation of multimodality, which captures the most important information from all the modalities, thereby offering doctors and physicians more precise results in gait categorization.  
In the future, for further study, collaboration with orthopedic, otolaryngology, and rehabilitation medical facilities to collect our datasets will be useful. By utilizing actual patient datasets, the suitability of the proposed hybrid model for real-world applications will be validated.  
There are several other potential applications of gait classification. First, it can be used in gait monitoring for abnormal gait detection, the recognition of human activities, fall detection, and sports performance. Second, it can be used as gait-based biometrics with applications in person identification, authentication, and re-identification, as well as gender and race recognition. Third, it can be used as a smart gait device and in environments ranging from smart socks, shoes, and other wearables to smart homes and smart retail stores that incorporate continuous monitoring and control systems.  
Conceptualization, C.-S.L.; methodology and software, H.S. and N.-H.K.; data analysis, H.S., N.-H.K. and M.T.N.; writing--original draft preparation, M.T.N.; writing--review and editing, M.T.N. and C.-S.L.; visualization, H.S. All authors have read and agreed to the published version of the manuscript.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Baseline** & **Accuracy (\%)** \\ \hline [PERSON] et al. [16] (Foot pressure data) & 68.82\% \\ \hline Our (Foot pressure data) & 78.2\% \\ \hline [PERSON] et al. [16] (Skeleton data) & 93.40\% \\ \hline Our (Skeleton data) & 99.04\% \\ \hline [PERSON] et al. [16] (Multi-modal) & 97.60\% \\ \hline Our (Multi-modal using early fusion) & 99.86\% \\ \hline Our (Multi-modal using late fusion without weights) & 96.95\% \\ \hline Our (Multi-modal using late fusion with different weights) & 99.17\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparison and performance of the proposed model with the baseline.  
**Funding:** This research was funded by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (2021R1A6A1A03040177).  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** Publicly available datasets were analyzed in this study. This data can be found here: [[https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints](https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints)]([https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints](https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints)).  
**Conflicts of Interest:** The authors declare no conflict of interest.