# AnyGesture: Arbitrary One-Handed Gestures for Augmented, Virtual, and Mixed Reality Applications
## 3 Materials and Methods
### Gesture Recognition  
#### 3.5.1 Static Gestures  
Static gestures are recognized similar to how they were recorded, e.g., current joint positions are retrieved from the tracking module, hand scaling is applied, and hand orientation is calculated. The values from each hand frame are then compared to the stored values, i.e., gestures. A threshold \(T_{Shape}\) can be used to either relax or increase the constraint to detect a gesture. To recognize a gesture, the mean difference for each joint position from the live data and the positions stored inside a gesture are calculated and then compared with the threshold (Equation (1)).  
\[|(\sum_{n=1}^{joints}CurrentPos_{n}-StoredPos_{n})|<T_{Shape} \tag{1}\]  
The same is conducted with the orientation where \(OC_{x}\), \(OC_{y}\), and \(OC_{z}\) represents the current data from the tracker, and \(OS_{x}\), \(OS_{y}\), and \(OS_{z}\) are the stored angles between HMD facing and hand directions (Equation (2)).  
\[|(OC_{x}-OC_{y}-OC_{z})-(OS_{x}-OS_{y}-OS_{z})|<T_{Orientation} \tag{2}\]  
The thresholds \(T_{Shape}\) and \(T_{Orientation}\) can be freely adjusted for each gesture individually within the framework and therefore allow various options for the design of an interactive gesture-based system. For example, a \(T_{Orientation}=360^{\circ}\) value causes a gesture to be rotation invariant, and the gesture is then detected by shape only, regardless of the hand rotation. It was found that the threshold values \(T_{Shape}=0.05\) and \(T_{Orientation}=15^{\circ}\) are a good compromise for recognition accuracy of many gestures. A simple algorithm for detecting a single static gesture among a predefined set of static gestures is described in Algorithm 2.  
```
1:New hand frame arrives
2:Get current hand frame \(H_{current}\) from hand tracker
3:for Each known gesture in the gesture storage do
4: Set \(D_{minimum}\) to Float.Maximum value
5:for Each joint on the hand do
6: Transform joint position from world space to local space
7: Adjust joint position with hand scaling factor
8: Calculate distance \(D\) between stored joint position and current position
9:if\(D>Threshold\)then
10: Discard current gesture \(G\) (hand shape is not matching)
11:endif
12: Add \(D\) to \(D_{Sum}\)
13:endfor
14:if\(D_{Sum}<D_{minimum}\)then
15: Set \(D_{minimum}\) to \(D_{Sum}\) (store the smallest distance)
16:endif
17:endfor
18:if\(G\) is not discarded AND \(D_{Sum}\) is the lowest between gestures then
19: Current gesture is detected
20:endif
```  
**Algorithm 2** Recognizing Static Gestures.  
This algorithm is suitable for recognizing the closest matching rotation invariant hand shape in a set of known gestures. In the case of similar gestures inside the gesture storage, it is a design choice whether the system should take the gesture with the closest matching shape or the closest matching orientation of the hand. The closest matching hand orientation as the final decision to decide for a single gesture out of multiple similar gestures was chosen for the framework. Ultimately, the decision as to which factor (hand shape or orientation) is decisive is rather negligible. This is because the threshold for both factors can be chosen more strictly for each individual gesture. If a gesture is designed more strictly, it will less likely have competing gestures with a similar hand shape.