# Finger-Gesture Recognition for Visible Light Communication Systems Using Machine Learning
## 3 Gesture Recognition System with LSTM Network  
A typical HAR system comprises data acquisition, segmentation, feature extraction, and classification stages. The categorization is based on an analysis of the pattern activity sensed on each PD. Through training, the system learns to associate the sequences with each activity.  
The concept of the hand movement recognition system is shown in Figure 2. The identification activity takes place between the LED and PD. Unobstructed light from the LED is incident on the photo-diode sensor and, as an object moves in between the two, light can become blocked. The task is to associate the sequence of incident light with the particular gesture. Typically, a hand may move at about 1 m/s or 1000 mm/s. The distance between fingers is up to about 10 mm, and therefore periods of activity and inactivity will typically last for about 10 ms. To reliably capture these movements the symbol sensing slot-time should be at least 0.1 ms. The slot time depends on the underlying use of the VLC system and is a trade-off between VLC data rate requirements, prediction accuracy, and computational complexity. The signaling rate is typically easily satisfied by modern VLC systems that operate above 1 Mbit/s.  
LSTM is a type of recurrent network that learns patterns embedded in time-series data [39] and has complexity proportional to the number of time-steps. The network is applied here to predict the finger gesture on a per time-step basis. The network comprises a sequence layer for handling the series input data, an LSTM layer for computing the learning, a fully-connected layer, a softmax layer, and finally, a classification layer. The size of the fully connected layer determines how well the network can learn the dependencies but care is required to avoid problems associated with over-fitting. The LSTM block diagram is shown in Figure 3 in which \(\mathbf{x}_{t}\) represents the input data. The hidden-state and cell-states at time \(t\) are termed \(\mathbf{h}_{t}\) and \(\mathbf{c}_{t}\), respectively. The current state and the next sequence data samples will determine the output and updated cell state. The cell state is given by Equation (5)  
\[\mathbf{c}_{t}\ = \!\!f_{t}\odot\mathbf{c}_{t\!-\!1}+i_{t}\odot g_{t} \tag{5}\]  
The hidden-state is given by Equation (6)  
\[\mathbf{h}_{t}=\mathbf{o}_{t}\odot\sigma_{c}(\mathbf{c}_{t}), \tag{6}\]  
where \(\sigma_{c}\) represents the state activation function. Control gates allow data to be forgotten or remembered at each iteration.  
The forget, cell-candidate, input, and output-states at time step \(t\) are given by Equations (7)-(10) respectively:  
\[f_{t} = \!\!\sigma_{c}(W_{f}\mathbf{x}_{t}+R_{f}\mathbf{h}_{t-1}+b_{f}), \tag{7}\] \[g_{t} = \!\!\sigma_{c}(W_{g}\mathbf{x}_{t}+R_{g}\mathbf{h}_{t-1}+b_{g}),\] (8) \[i_{t} = \!\!\sigma_{c}(W_{i}\mathbf{x}_{t}+R_{i}\mathbf{h}_{t-1}+b_{i}),\] (9) \[o_{t} = \!\!\sigma_{c}(W_{o}\mathbf{x}_{t}+R_{o}\mathbf{h}_{t-1}+b_{o}), \tag{10}\]  
where \(W_{f}\), \(W_{g}\),\(W_{i}\), \(W_{o}\) represent the forget, cell-candidate, input, and output weights. \(R_{f}\),\(R_{g}\),\(R_{i}\), and \(R_{o}\) are the forget, cell-candidate, input, and output recurrent weights. \(b_{f}\), \(b_{g}\),\(b_{i}\), and \(b_{o}\) are the forget, cell-candidate, input, and output biases.  
Figure 2: Concept of finger movement recognition system based on received patterns of light on a photo-diode sensor.