# Sensor-Based Hand Gesture Detection and Recognition by Key Intervals
## 3 Proposed Algorithm
### Inference Operations  
For the inference operations, the input sequence \(S\) could contain more than one hand gesture. Furthermore, the start and end locations of the samples in each gesture are not known for the inference operations. As shown in Figure 4, the inference process first activates the detector of key intervals, which evaluates the PKI and SKI scores for the current sliding window \(S_{t}\). The scores are then compared with a threshold, denoted by \(T\), for the detection of key intervals. If a key interval is detected, a matching process for the key intervals is then initiated to determine whether an occurrence of a gesture is found or not. When a detected PKI is followed by a detected SKI, and both PKI and SKI belong to the same gesture class, we then say that a gesture is detected and classified. For example, suppose \(t_{1}\) and \(t_{2}\) are the indices where PKI and SKI are detected. Therefore, the highest PKI score in \(X_{t_{1}}\) and the highest SKI score in \(Y_{t_{2}}\) for nonbackground classes should be higher than the threshold \(T\). That is, let  
\[j^{*}=\operatorname*{argmax}_{1\leq j\leq K+1}X_{t_{1},j},\;\;j^{**}= \operatorname*{argmax}_{1\leq j\leq K+1}Y_{t_{2},j}. \tag{21}\]  
It then follows that  
\[X_{t_{1},j^{*}}>T,\;\;Y_{t_{2},j^{**}}>T, \tag{22}\]  
where \(j^{*}\
eq K+1\), \(j^{**}\
eq K+1\), and class \(K+1\) is the background class. Suppose no PKI or SKI detections occur between \(t_{1}\) and \(t_{2}\). A matched detection then occurs when \(j^{*}=j^{**}\). In this case, a gesture is detected. In addition, the gesture is classified as class \(j^{**}\). To further enhance the robustness of the proposed algorithm, additional constraints may be imposed. In this study, we required that \(t_{\text{min}}<t_{1}-t_{2}<t_{\text{max}}\), where \(t_{\text{min}}\) and \(t_{\text{max}}\) are the minimum time and maximum time between a pair of matched PKI and SKI detections, respectively. This would reduce the possibilities of false matches due to inaccurate PKI or SKI detections. As revealed in Figure 4, a postprocessing operation is employed for enforcing the constraints for the gesture detection. For each sliding window \(S_{t}\) from input sequence \(S\), the operations of key interval detection, key interval matching, and postprocessing are carried out sequentially, where the next sliding window is obtained from the current one with stride size \(\Delta\). The same operations are repeated for each sliding window until the final window of \(S\) is processed. Algorithm 1 summarizes the detailed operations for the inference process. The sets \(\mathcal{C}\) and \(\mathcal{T}\) returned by Algorithm 1 consist of the gesture class and the occurrence time of the detected gestures, respectively. An important fact in the proposed inference algorithm is that the classification scores in \(Z_{t}\) are not required for the gesture classification. We include \(Z_{t}\) only in training process, where it is adopted for the computation of \(C\) in Equation (16) for loss function \(J\) in Equation (15). We can view the incorporation of \(C\) for the loss evaluation as a regularization scheme for the detection. The backbone of the proposed neural network is then able to provide features best for both the detection and classification. Therefore, \(Z_{t}\) may not be needed. Only the KPI scores \(X_{t}\) and \(Y_{t}\) are involved in the gesture classification. Figure 4: The inference operations of the proposed neural network for an input sequence. There are three major operations for each sliding window of the input sequence: detection of key intervals, key interval matching, and postprocessing. The same operations are repeated for each sliding window with stride size \(\Delta\) until the final window of the input sequence is reached. ```
1:Input sequence \(S\). 2:A trained neural network model. 3:Threshold \(T\) for detection of PKI and SKI. 4:Stride size \(\Delta>0\). 5:\(\mathcal{T}=\emptyset\), \(\mathcal{C}=\emptyset\). 6:PKI_Detected \(\leftarrow\) false.