# Lip-Reading Classification of Turkish Digits Using Ensemble Learning Architecture Based on 3 DCNN
###### Abstract  
Understanding others correctly is of great importance for maintaining effective communication. Factors such as hearing difficulties or environmental noise can disrupt this process. Lip reading offers an effective solution to these challenges. With the growing success of deep learning architectures, research on lip reading has gained momentum. The aim of this study is to create a lip reading dataset for Turkish digit recognition and to conduct predictive analyses. The dataset has divided into two subsets: the face region and the lip region. CNN, LSTM, and 3 DCNN-based models, including C3D, I3D, and 3 DCNN+BiLSTM, were used. While LSTM models are effective in processing temporal data, 3 DCNN-based models, which can process both spatial and temporal information, achieved higher accuracy in this study. Experimental results showed that the dataset containing only the lip region performed better; accuracy rates for CNN, LSTM, C3D, and I3D on the lip region were 67.12%, 75.53%, 86.32%, and 93.24%, respectively. The 3 DCNN-based models achieved higher accuracy due to their ability to process spatio-temporal data. Furthermore, an additional 1.23% improvement was achieved through ensemble learning, with the best result reaching 94.53% accuracy. Ensemble learning, by combining the strengths of different models, provided a meaningful improvement in overall performance. These results demonstrate that 3 DCNN architectures and ensemble learning methods yield high success in addressing the problem of lip reading in the Turkish language. While our study focuses on Turkish digit recognition, the proposed methods have the potential to be successful in other languages or broader lip reading applications. lip-reading; ensemble learning; 3 DCNN +
Footnote †: journal: Applied sciences  
0000-0001-5000-0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-00  
machines with the advancement of technology. It can be seen as a necessary element for the processes of communication in human computer interaction. People unconsciously tend to extract words from the movements of the lips. This approach is purely due to a visual implication. It is seen in the studies that this process, which is dependent on visual extraction, can be analyzed with image processing techniques. In the analysis phase, more and more successful results are obtained with deep learning methods [4]. When lip-reading studies using deep learning methods are examined, it is seen that the studies focus mainly on the English language [5, 6, 7, 8]. At the same time, it is seen that there are many data sets for lip reading in English in order to be able to do these studies. A review of the literature reveals that while there are numerous datasets available in English, the number of datasets in Turkish is significantly lower. This highlights the need for a dataset specifically designed for Turkish lip-reading studies. Lip-reading, or visual speech recognition, is crucial for individuals with hearing impairments and in noisy environments where auditory information is insufficient. Despite advancements in deep learning, there is a significant gap in lip-reading research for non-English languages, particularly Turkish. This study aims to address this gap by creating a novel dataset for Turkish digit recognition and utilizing various deep learning architectures as benchmarks for evaluation. In this study, first of all, we present a novel dataset specifically designed for lip-reading in Turkish, a language with unique phonetic characteristics. This dataset, focusing on digit recognition, fills a significant gap in the current literature which predominantly concentrates on English. The different structures of languages make it difficult for a model that achieves good in one language to achieve similar results when applied to another language. The unique phonetic characteristics of the Turkish language pose additional challenges, making this study particularly significant.