# Ensemble Learning of Multiple Deep CNNs Using Accuracy-Based Weighted Voting for ASL Recognition
## 2 Materials and Methods
### Proposed Model
#### 2.2.2 Classification and the Proposed AWV Algorithm
Each dataset has 27 non-motion gesture images and two motion gesture images (J and Z) and has different background complexity. Relative to motion and non-motion gestures, the ARS-MA model prioritizes non-motion character with a more significant proportion. ARS-MA has accuracies of 98.98% and 96.88% for non-motion and motion gesture images, respectively. This result means that the accuracy for non-motion images is around 2% higher than for motion images, and that a model for motion image gestures is necessary for better performance. Table 10 compares the proposed method to other methods for sign language recognition with motions or gestures in recent papers. The proposed ARS-MA model achieved better recognition accuracy on both datasets. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Types** & **Authors** & **Works** & **Accuracy** \\ \hline \multirow{6}{*}{Sign Language Recognition} & [PERSON] et al., 2022 [31] & Ensembles for Isolated Sign Language Recognition & 73.84\% \\ \cline{2-3}  & [PERSON] et al., 2021 [23] & Self-mutual distillation learning & 80\% \\ \cline{2-3}  & [PERSON] et al., 2022 [24] & Inflated 3D ConvNet with BLSTM & 89.74\% \\ \cline{2-3}  & [PERSON] et al., 2020 [18] & Sign Language Translation with STMC-Transformer & 96.32\% \\ \cline{2-3}  & [PERSON] et al., 2022 [19] & Full transformer network with masking future & 96.17\% \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of proposed work and previous works. Figure 6: The performance scores for three types of multiple deep CNNs in the ARS-MA model: (a) AUC and (b) Accuracy. ## 4 Conclusions

ASL letters are used as an auxiliary language for exceptional cases, such as names, book titles, and letter correction. Building a reliable ASL letter recognition model is essential to improving communication as a tool for hearing-impaired people. In this paper, the ARS-MA model was proposed which consists of data preprocessing, feature extraction, and classification. Five M-CNN models with different depths and feature capture methods were designed to combine feature extraction and a novel AWV algorithm using an accuracy-based weighted voting proposed to increase accuracy in the final classification step, where two parameters, \(\alpha\) and \(\lambda\), in AWV were introduced to improve the accuracy and consumption time. The best performance was achieved when \(\alpha\) = 2 and \(\lambda\) = 5. Three new datasets, ND1, ND2, and ND3, for accuracy-based weighted voting, were created by the results of the multiple deep CNNs. The two datasets used in this paper have two types of images: non-motion gestures and motion gestures (J and Z). The ARS-MA model obtained 98.83% and 98.79% accuracies for the ASL Alphabet and ASLA datasets. In addition, it has accuracies of 98.98% and 96.88% for non-motion and motion gesture images, respectively, and the accuracy for non-motion images is around 2% higher than for motion images. In the future, the research on increasing the accuracy of motion gesture images will be conducted for a real-time ASL recognition system and it will help hearing-impaired people better communicate. Conceptualization, Y.M. and K.K. ; methodology, Y.M. and K.K. ; software, Y.M. and K.K. ; validation, Y.M. and T.X. ; formal analysis, Y.M. ; investigation, Y.M. and K.K. ; resources, Y.M. and K.K. ; data curation, Y.M. and K.K. ; writing--original draft preparation, Y.M. ; writing--review and editing, Y.M., K.K., and S.H. ; visualization, Y.M. and K.K. ; supervision, Y.M. and K.K. ; project administration, Y.M. and K.K. ; and funding acquisition, Y.M. and K.K. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The generated datasets ND1, ND2, and ND3 during the study can be found in link: [[https://data.world/ying1211/asl-three-new-datasets](https://data.world/ying1211/asl-three-new-datasets)]([https://data.world/ying1211/asl-three-new-datasets](https://data.world/ying1211/asl-three-new-datasets)). The authors declare no conflict of interest.