# Recognition of Hand Gesture Sequences by Accelerometers and Gyroscopes
## 3 The Proposed Gesture Recognition Algorithms and Systems
### Overview  
As shown in Figure 1, the proposed gesture recognition algorithms can be separated into two parts: feedforward neural networks and Maximum A Posteriori (MAP) estimation. The feedforward networks take sensory data as input. Examples of sensory data include the data produced by accelerometers and/or gyroscopes. Based on the input data, the feedforward neural networks carry out the gesture spotting operations. Given the spotting results, the MAP estimation is then performed to obtain the final classification outcomes. \begin{table}
\begin{tabular}{l l} \hline \hline
**Symbol** & **Meaning** \\ \hline \(A\) & The path of the sensory data. Each individual index along the path is the index of the gesture \\  & having highest probability at its corresponding time step. \\ \(a_{t}\) & The index of the gesture having the largest probability at time step \(t\). It is the \(t\)-th component of \\  & \(A\), \(t=1,...,T\). \\ \(K\) & Number of different gestures in the input sensory data sequence \(X\). \\ \(M\) & Number of elements in each input sample \(x_{t}\), \(t=1,...,T\). \\ \(N\) & Length of input window \(X_{t}\). \\ \(Q\) & Number of gesture classes. \\ \(R\) & Classification results. \\ \(r_{k}\) & The \(k\)-th element of classification results \(R\), \(k=1,...,K\). \\ \(T\) & Length of the input sequence \(X\) and output sequence \(Y\). \\ \(X\) & Input sensory data sequence. \\ \(X_{t}\) & Input window with central component \(x_{t}\). \\ \(x_{t}\) & The sample at time step \(t\) of input sensory data sequence \(X\), \(t=1,...,T\). \\ \(Y\) & Gesture spotting results. \\ \(y_{t}\) & The sample at time step \(t\) of gesture spotting results \(Y\), \(t=1,...,T\). \\ \(y_{t,j}\) & The \(j\)-th element of \(y_{t}\), \(j=1,...,Q\), \(t=1,...,T\). \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A list of frequently used symbols in this study. Let \(X=\{x_{1},...,x_{T}\}\) be an input sensory data sequence for the recognition of a hand gesture sequence containing \(K\) different gestures back-to-back. Each of the \(K\) gestures belongs to one of the pre-defined \(Q\) gesture classes. Each \(x_{t}\), \(t=1,2,...,T\), is the sample of the sequence \(X\) acquired at time step \(t\), and \(T\) is the length of of the sequence. There are \(M\) elements in each sample \(x_{t}\) of the data sequence \(X\). The dimension \(M\) is dependent on the sensors for the gesture recognition. For example, when both 3-axis accelerometer and 3-axis gyroscope are used, each sample \(x_{t}\) contains six elements, and therefore \(M=6\). However, when only one of the sensors is used, \(M=3\). Let \(Y=\{y_{1},...,y_{T}\}\) be the results produced by a feedforward neural network, where \(y_{t}\) is the output of the network at time step \(t\). There are \(Q\) elements in each output \(y_{t}\). Let \(y_{t,j}\) be the \(j\)-th element of \(y_{t}\), \(t=1,...,Q\). The activation function associated with \(y_{t}\) is the softmax activation function. We then can view \(y_{t,j},j=1,...,Q\), as the probability of the occurrence of gesture class \(j\) at the time step \(t\). Figure 2 shows the operations of a feedforward network on \(X\) for producing \(Y\). Starting from \(t=1\), the feedforward network produces \(y_{t}\) based on \(X_{t}\) for each \(t\) until \(t=T\) is reached, where \(X_{t}\) is an input window with length \(N\). The central component of \(X_{t}\) is \(x_{t}\). This allows for the full exploitation of correlation among neighboring samples of \(x_{t}\) at expense of higher latency for acquiring the input window \(X_{t}\). Although causal operations are possible, only the correlation in the past neighboring samples of \(x_{t}\) is exploited. Because the full utilization of correlation among neighboring samples is important for accurate gesture spotting, causal operations are not considered. Note that, when \(t<N/2\) or \(t>T-N/2\), parts of \(X_{t}\) is outside \(X\). These parts are filled with zeros.