# Dataset Transformation System for Sign Language Recognition Based on Image Classification Network
## 1 Introduction  
Sign language is a method for the deaf to communicate in society by using a variety of information, such as hand movements and facial expressions. In general, the ultimate goal of sign language recognition is to communicate with others by interpreting a deaf individual's words or sentences. Even if the sign language performs the same action, it may be a different word or sentence depending on the range of movement [1]. In recent years, there has been an increasing interest in deep learning applied to various fields, and it has contributed to technological improvement [2]. There have recently been numerous studies in the field of sign language recognition using deep learning to classify images or videos. The reason we chose sign language recognition is because it has both the characteristics of motion recognition and the characteristics of a time series language translation. Deep learning models that classify images have low complexity compared to models that classify videos [3]. The reason is that the model used for video classification uses a time series learning model based on recurrent neural networks (RNN) [4]. The RNN-based classification model is used for the classification of data with continuity, such as natural language processing (NLP) and motion recognition. Natural language processing requires general sentence data, and the video data is required for learning for motion recognition. Deep learning models require large amounts of data for high accuracy [5]. In the case of natural language processing, even if the amount of data increases, the physical capacity occupied by the data does not increase rapidly. However, in the case of motion recognition, the video data are used for learning. When the amount of video data increases, the capacity increases rapidly. There are challenges to be solved in the field of motion recognition using a relatively high complexity model and a large capacity dataset. First, large amounts of data and high complexity increase the learning time. Second, it costs a lot to manage the data. In contrast to the existing sign language recognition system, which recognizes using a time series classification model, we propose a dataset transformation system to enablesign language recognition with a low complexity image classification model. The focal point of the proposed method is the process of converting video data into images. The amount of data can be effectively reduced only when information about human motion is extracted from the video data. Through pose estimation, it is possible to extract human skeleton information from video data [6]. In order to extract the skeleton information in a consistent range, normalization was performed. In the video that contains human movement, the person does not always move at the center of the video or at a specific location. This process is necessary because we cannot limit the position of a person. Normalized data are converted to data between 0 and 1 so that the position of the person is not affected. Subsequently, the STmap (spatial-temporal map) is generated by converting these data to the RGB coordinate system. Through this process, the system can create a dataset in a new type that can be trained on the image classification model. However, the lack of a dataset remains a problem in the sign language recognition problem. So, we perform data augmentation in order to train on a small amount of data. Data augmentation is a technique widely used when training deep learning models, and it is possible to increase the amount of data to increase the learning accuracy [7]. This paper uses two data augmentation techniques: frame skipping and salt-and-pepper noise. In this paper, we propose a data transformation system that converts video data into images and augments the data to train on an image classification model. Because the image classification model is used, the learning complexity is reduced, which is expected to reduce the learning time and increase the accuracy.