# Sign Language Interpreting System Using Recursive Neural Networks
## 1 Introduction  
According to the National Institute of Statistics and Geography of Mexico (INEGI) [1], there are around 1,350,802 people with hearing disabilities in Mexico. As is known, these people communicate with each other using sign language fluently; however, most people who use speech are unfamiliar with Mexican Sign Language (MSL), so communicating between them can be difficult when there is no interpreter or shared means of communication, such as a written language that both parties are proficient in. On the other hand, artificial intelligence has advanced to rival human decisions in some areas [2]; such is the case of the discipline called deep learning. Within deep learning is the Recurrent Neural Networks or RNN, and part of this technique is the Long Short-Term Memory or LSTM networks. These machine learning techniques are used to processdata sequences widely applied to research on automatic speech recognition and natural language processing [3]. For example, [PERSON]. [4] proposed a neural network focused on human action recognition from videos. The authors utilized an attention mechanism to focus on the most important features of each frame. The authors propose a new neural network architecture to capture this temporal information. The network combines two streams:  
* An LSTM stream to analyze the sequence of frames in the video. * A CLSTM (Convolutional LSTM) stream captures spatial and transient information. As a result, the method is more accurate than previously existing methods. On the other hand, [PERSON]. [5] created a model for generating video descriptions. The model uses recurrent neural networks, specifically LSTMs, to capture the transient structure of videos and generate descriptions. The model was trained using a dataset made of YouTube videos; authors claim their model outperforms previous models regarding accuracy. From a different perspective, [PERSON]. [6] proposed a method to generate Indian Sign Language (ISL) videos from sentences using sentence preprocessing and generative adversarial networks (GAN). Their method focuses on converting sentences into glosses (a way to represent signs using written language) and generating synthetic video frames for each gloss using a GAN architecture. The generated videos use animated avatars and 2D/3D graphics. Despite the difficulties related to this research field, it is essential to highlight the importance of communication since it is fundamental to connecting people and communities; this motivates our work since, for the deaf community in Mexico, MSL is more than a means of communication, it is a pillar of both their culture and their identity. Despite the progress regarding public awareness of the importance of MSL, this type of communication still faces significant challenges in achieving its adoption by our society. For this reason, this work is more than just a technological exercise; it is a concrete manifestation of the commitment to including deaf people in our society. We believe that this can be reached by empowering the deaf community with technological tools that facilitate their communication, so we expect that this work contributes to achieving a more inclusive and equitable society. The contributions of this paper are:  
* We designed four multimodal models based on spatial tracking, gesture recognition, and an RNN (Section 3). * We tuned the proposed systems and identified the most suited model for the target application (Section 5). * We tested the proposed system both in an offline and online scenario (Section 5). * We evaluated the accuracy of the system in the offline and online scenarios (Section 5). We expect our work to impact different social areas, such as education, the health care sector, transportation, finances, and daily life in general, helping deaf people interact with the rest of the people in equal circumstances.