# Sensor-Based Hand Gesture Detection and Recognition by Key Intervals
## 3 Proposed Algorithm
### Training Operations  
To facilitate the training and inference operations of the proposed neural network model, a list of commonly used symbols is revealed in Table A1 in Appendix A. Let \(S=\{s_{1},\ldots,s_{L}\}\) be an input sensory data sequence for training. Each \(s_{t}\in S\) is the \(t\)th sample of the input sequence, \(t=1,\ldots,L\), where \(L\) is the length of the sequence. Each sample \(s_{t}\) is an \(N\)-tuple vector, where \(N\) is dependent on the sensors adopted for the gesture recognition. For example, \(N=6\) when both 3-axis accelerometer and 3-axis gyroscope are used. A sliding window operation on the input sequence \(S\) is adopted for the training of the neural network. Let \(S_{t}\) be a window of \(W\) successive samples from \(S\), where the central sample of \(S_{t}\) is \(s_{t}\). In the proposed algorithm, we slide the window \(S_{t}\) with stride size 1. For each \(S_{t}\), three outputs \(X_{t}\), \(Y_{t}\), and \(Z_{t}\) can be obtained from the three branches of the proposed neural network model. Let \(K\) be the number of gesture classes for classification. The \(X_{t}\), \(Y_{t}\), and \(Z_{t}\) are then \((K+1)\)-tuple vectors. Let \(X_{t,j}\), \(Y_{t,j}\), and \(Z_{t,j}\) be the \(j\)th element of \(X_{t}\), \(Y_{t}\), and \(Z_{t}\), respectively. The \(X_{t,j}\), \(Y_{t,j}\), and \(Z_{t,j}=1,\ldots,K\), are the PKI score, SKI score, and classification score associated with class \(j\) for the window \(S_{t}\). When \(j=K+1\), the \(X_{t,K+1}\), \(Y_{t,K+1}\), and \(Z_{t,K+1}\) are the scores associated with the background class. Furthermore, because softmax is employed in the proposed network model, we have \(X_{t,j}\geq 0\), \(Y_{t,j}\geq 0\), \(Z_{t,j}\geq 0\), \(j=1,\ldots,K+1\), and  
\[\sum_{j=1}^{K+1}X_{t,j}=1,\ \sum_{j=1}^{K+1}Y_{t,j}=1,\ \ \sum_{j=1}^{K+1}Z_{t,j}=1. \tag{1}\]  
For each window \(S_{t}\), let \(F_{t}\), \(G_{t}\) and \(H_{t}\) be the ground truth of \(X_{t}\), \(Y_{t}\), and \(Z_{t}\), respectively. Therefore, they are also \((K+1)\)-tuple vectors. Based on the training sequence \(S\), the assignments of ground truth values to \(F_{t}\), \(G_{t}\), and \(H_{t}\) for each window \(S_{t}\) are regarded as the labelling process for the training operation. Figure 2: The proposed neural network model for gesture detection and classification. The model contains a backbone and three branches. The samples from a sliding window of a sequence \(S\) serve as input data. The model then produces PKI scores, SKI scores, and classification scores corresponding to the sliding window. The proposed labelling process operates under the assumption that the start location and end location for each gesture in the training sequence \(S\) are known. A simple algorithm is then proposed to find the PKI and SKI associated with the gesture numerically. Based on the PKI and SKI associated with each gesture, the \(F_{t}\), \(G_{t}\), and \(H_{t}\) for each window \(S_{t}\) are then computed. Figure 3 shows an example of the labelling process for a training sequence \(S\) containing only a single gesture. Let \(P_{s}\) and \(P_{f}\) be the locations of the start and final samples of the gesture, respectively. Let \(u_{c}\) and \(R\) be the centroid and radius of the gesture, respectively. That is,  
\[u_{c}=\frac{P_{f}+P_{s}}{2},\ R=\frac{P_{f}-P_{s}}{2}. \tag{2}\]  
Therefore, the set of indices of the gesture, denoted by \(\mathcal{I}_{R}\), is given by  
\[\mathcal{I}_{R}=\{t:u_{c}-R\leq t\leq u_{c}+R\}. \tag{3}\]  
Let \(u_{\text{rx}}\) and \(u_{\text{sxt}}\) be the centroid of the PKI and SKI, respectively. In this study, \(u_{\text{rx}}\) and \(u_{\text{sxt}}\) were given by  
\[u_{\text{rx}}=u_{c}-0.4R,\ u_{\text{rx}}=u_{c}+0.4R. \tag{4}\]  
Both PKI and SKI have the same length, denoted by \(I\), where  
\[I=rR, \tag{5}\]  
where \(0<r<1\) is a positive constant. In this study, we termed \(r\) the key interval-length-to-gesture-radius (ITR) ratio. For the example shown in Figure 3, the ITR ratio \(r=0.3\).