# Sign Language Recognition Using Two-Stream Convolutional Neural Networks with Wi-Fi Signals
## 1 Introduction  
Sign language is an important way for deaf people to understand and communicate with each other. Communication barriers are often encountered between the deaf communities and people who do not know about sign language. Many researchers try to build a sign language recognition system to break these barriers [1]. Currently, sign language recognition systems are roughly divided into two categories: (i) device-based sign language recognition systems; (ii) device-free sign language recognition systems [2; 3]. Wearable sensors are widely used in device-based sign language recognition systems. In 1983, [PERSON] et al. invented a data glove for dynamic gesture recognition [4]. [PERSON] et al. used data gloves to obtain data on Malaysian sign language letters, numbers, and words [5]. [PERSON] et al. recognized gestures through the time delay neural networks (TDNNs) algorithm, and the gesture data is obtained from data gloves based on pyrolytic graphite sheets (PGS) [6]. In general, the advantages of the wearable device-based sign language recognition method are that the input data is accurate and the recognition rate is high [2; 4; 5; 6]. The disadvantages are also obvious. For example, wearable devices are often expensive and inconvenient to carry. The device-free sign language recognition systems are usually inexpensive and not limited by the wearable device [2; 7]. Several device-free sign language recognition systems use computer vision techniques with cameras. [PERSON] conducted a survey on gesture recognition, focusing on theRWTH-PHOENIX-Weather data set recorded by a stationary standard color camera [8]. For example, [PERSON] et al. adopted a convolutional neural network (CNN) with stacked temporal fusion layers and a bi-directional recurrent neural network (RNN) to extract the spatiotemporal information of sign language videos [9]. [PERSON] et al. proposed an alignment network with iterative optimization for video-based sign language recognition, including a three-dimensional (3D) ResNet and an encoder-decoder network [10]. In addition, depth information is also considered to improve the performance. [PERSON] et al. introduced a vision-based system consisting of RGB and depth descriptors to classify gestures. This method used RGB and depth images to modify the Histogram of Oriented Gradients (HOG) function to achieve higher classification accuracy [11]. [PERSON] et al. proposed a 3D CNN which extracts automatically spatial-temporal features from raw video datasets collected with Microsoft Kinect sensor [12]. [PERSON] et al. utilized an unsupervised principal component analysis network (PCANet) to extract the local features of sign language images captured from the Microsoft Kinect sensor. The extracted features were recognized by the support vector machine (SVM) [13]. It is worth noting that computer vision-based sign language recognition systems are not only susceptible to light conditions and obstacles, but also cause privacy issues [2; 7]. With the widespread deployment of wireless networks, wireless related technologies have experienced very rapid growth. Gesture recognition based on commodity Wi-Fi sensing solutions has been widely studied [2; 7]. Thus, a sign language recognition system that is non-intrusive and insensitive to lighting conditions has attracted the interest of many researchers. The gestures recognition system proposed by [PERSON] et al. used directional antenna technology to achieve fine-grained gesture recognition [14]. [PERSON] and [PERSON] used Wi-Fi signals to recognize different gestures and arm movements, called WiSign [15]. Wi-Finger, which can recognize nine-digit finger gestures from American Sign Language (ASL), was implemented on a commercial Wi-Fi infrastructure [16]. Most Wi-Fi sensing research uses Wi-Fi channel state information (CSI), which describes how the signal propagates from the transmitter to the receiver, and reveals combined effects such as scattering, fading, and power decay with distance. [PERSON] et al.