# Recognition of Hand Gesture Sequences by Accelerometers and Gyroscopes
## 3 The Proposed Gesture Recognition Algorithms and Systems
### Overview
After \(Y=\{y_{1},...,y_{T}\}\) is available, post-processing operations are required to produce the final classification results. With the a priori knowledge of the number of gestures \(K\) contained in the input sensory data \(X=\{x_{1},...,x_{T}\}\), the goal of the post-processing operations is to find the set of indices \(R=\{r_{1},...,r_{K}\}\) of the gestures, where \(r_{k}\) is the index of the \(k\)-th gesture appears in the sensory data \(X=\{x_{1},...,x_{T}\}\). The selection of \(K\) is dependent on the number of different gesture sequences required for an application. Under the restriction of no occurrence of repetitive gestures, there are at most \(Q\times(Q-1)\times...\times(Q-(K-1))\) different gesture sequences. Figure 1: The overview of the proposed gesture recognition system, where \(X,Y,\) and \(R\) denote the input sensory data, gesture spotting results, and classification results, respectively. Figure 2: The feedforward operations on \(X\) for producing \(Y\). Let \(a_{t}\) be the index of the gesture having the largest probability at time step \(t\). That is,  
\[a_{t}=\operatorname*{argmax}_{1\leq j\leq Q}y_{t,j}. \tag{1}\]  
We call \(A=\{a_{1},...,a_{T}\}\) the path of the sensory data, where each individual index along the path is the index of the gesture having highest probability at its corresponding time step. The computation of final classification outcome \(R\) is based on the probability model  
\[Prob(C/A)=\prod_{k=1}^{K}Prob(c_{k}/A), \tag{2}\]  
where \(C=\{c_{1},...,c_{K}\}\) is a possible classification outcome, and \(c_{k}\) is the index of the \(k\)-th gesture of \(C\). The probability \(Prob(c_{k}/A)\) is estimated by  
\[Prob(c_{k}/A)=\frac{|I_{c_{k}}|}{T}, \tag{3}\]  
where  
\[I_{c_{k}}=\{a_{t}:a_{t}=c_{k}\}, \tag{4}\]  
and the size of \(I_{c_{k}}\) is denoted by \(|I_{c_{k}}|\). The final classification result \(R\) is then the \(C\) maximizing \(Prob(C/A)\). That is,  
\[R=\operatorname*{argmax}_{C\in S}P(C/A), \tag{5}\]  
where \(S\) denotes the set of all possible classification outcomes. From (2) and (3), the search process in (5) is equivalent to the identification of gestures having top-\(K\) occurrence. The classification results \(R=\{r_{1},...,r_{K}\}\) are then obtained from these gestures according to their locations in the path \(A\). Figure 3 shows an example of mapping from the path \(A\) to the classification results \(R\) for the recognition of three gestures (i.e., \(K=3\)). In this example, based on the results of the feedforward neural network, the operations in (1) produces index values \(1\), \(3\), \(4\), \(6\), or \(7\). The locations of the corresponding intervals \(I_{1}\), \(I_{3}\), \(I_{4}\), \(I_{6}\), and \(I_{7}\) are revealed in Figure 3. The top-three largest intervals are \(I_{1}\), \(I_{3}\), and \(I_{6}\). The set of gestures having the top-three occurrences then contains Gesture \(1\), Gesture \(3\), and Gesture \(6\). Their order of occurrence along the path A is Gesture \(3\), Gesture \(6\), and Gesture \(1\). Consequently, \(r_{1}=3\), \(r_{2}=6\), and \(r_{3}=1\). Figure 3: An example of mapping from path \(A=\{a_{1},...,a_{T}\}\) to final classification outcome \(R=\{r_{1},r_{2},r_{3}\}\) for the recognition of three gestures (i.e., \(K=3\)). In this example, \(r_{1}=3\), \(r_{2}=6\), and \(r_{3}=1\). The feedforward neural network in the proposed system could be a basic CNN, or the PairNet, Residual PairNet, PairNet with Inception, or Residual PairNet with Inception. They are presented separately in the following subsections.