## 2 Related Work  
There have been many approaches to recognize hand languages that are used to describe letters of the alphabet with a single hand. It is relatively easier than recognizing sign languages as each letter of the alphabet simply corresponds to a unique hand shape. In [17], the authors utilized depth cameras (Microsoft's Kinect) and the random forest algorithm to recognize the English alphabet with 92% recognition accuracy. A pose estimation method of the upper body represented by seven key points was proposed for recognizing the American Sign Language (ASL) [18]. We also note t an approach by [PERSON] et al. [24] to recognize the Korean hand language by analyzing latent features of hand images. In general, researchers rely on the movements and shapes of both hands to recognize sign languages. [PERSON] et al. [22] developed a real-time system based on Hidden Markov model (HMM) to recognize sentence-level ASL. They demonstrated two experimental results: they used solidly colored gloves to make tracking of hands easier in the first experiment and the second experiment was conducted without gloves. They claimed that the word accuracy of glove-based system is 99.2% but the accuracy drops to 84.7% if they do not use gloves. It should be noted that those accuracy can be reached because they exploited the grammar for reviewing the errors of the recognition. The word accuracy without grammar and gloves is 74.5%. On the other hand, there have been approaches to automatically learning signs from weakly annotated data such as TV broadcasts by using subtitles provided simultaneously with the signs [25; 26; 27]. Following this direction, [PERSON] et al. released the RWTH-PHOENIX-Weather 2012 [1] and its extended version RWTH-PHOENIX-Weather 2014 [28] that consist of weather forecasts recorded from German public TV and manually annotated using glosses and natural language sentences where time boundaries have been marked on the gloss and the sentence level. Based on the RWTH-PHOENIX-Weather corpus, [PERSON] et al. [20] presented a statistical approach performing large vocabulary continuous sign language recognition across different signers. They developed a continuous sign language recognition system that utilizes multiple information streams including the hand shape, orientation and position, the upper body pose, and face expressions such as mouthing, eyebrows and eye gaze. There have been many attempts to recognize and translate sign language using deep learning (DL). [PERSON] et al. [29] introduced and evaluated several architectures for CNNs to predict the 3D joint locations of a hand given a depth map. [PERSON] et al. [19] developed a sign language recognition system that is robust in different video backgrounds by extracting signers using boundary and prior shape information. Then, the feature vector is constructed from the segmented signer and used as input to artificial neural network. An end-to-end sequence modeling using CNN-BiLSTM architecture usually used for gesture recognition was proposed for large vocabulary sign language recognition with RWTH-PHOENIX-Weather 2014 [21]. At the same time, one of the most interesting breakthroughs in neural machine translation or even in the entire DL was introduced under the name of \"sequence-to-sequence (seq2 seq)\" [7]. The seq2 seq model relies on a common framework called an encoder-decoder model with RNN cells such as LSTMs or GRUs. The seq2 seq model proved its effectiveness in many sequence generation tasks by achieving almost human-level performance [7]. Despite its effectiveness, the seq2 seq model still has some drawbacks such as the input sequences of varying lengths being represented in fixed-size vectors and the vanishing gradient due to the long-term dependency between distant parts. [PERSON] et al. [23] formalized the sign language translation problem based on the pre-existing framework of neural machine translation with word and spatial embeddings for target sequences and sign videos, respectively.