# Hand Gesture Recognition on Edge Devices:
## 1 Introduction
Depth measurement can be based on either camera data alone with stereo algorithm, or enhanced by an invisible infrared pattern. This is especially relevant in case of bad lighting conditions. Alternatively, depth calculation can be done with radar or ultrasound sensors and time of flight (ToF) calculation. Both technologies require significantly less power than cameras and are therefore especially interesting for the automotive or IoT sectors. Independent of the sensor and the technology or algorithm used, some processing has to be carried out. In IoT and edge applications, processing power is scarce, as edge devices come with low processing capabilities and should use little power. Some run on battery power only. Table 1 summarizes HGR-specific challenges. The first challenge is to accurately detect one or more hands. For a vision-based system there are tools for hand segmentation. However, vision-based systems struggle in bad lighting conditions. A similar color of a hand and the background causes further problems. Adding a depth camera can improve the accuracy of vision-based systems. In case of radar, WiFi, ultrasound, 4G, 5G, and LTE, the system relies on detecting an echo in the signal. This echo is caused by the reflection of a pulse emitted by the system from the hand. However, other objects in the field of view can also reflect the emitted pulse and produce an echo. Reflections of echoes can add further noise. After correct detection of a hand in the FOV, the second challenge is to correctly track it. Different hand speeds lead to different gesture lengths in the time domain and different frequency shifts in the spectrogram after Fourier transformation. Normalizing and dynamic time warping are possible solutions for these differences. The third challenge lies in gesture classification itself. The background plays a significant role in the difficulty of hand gesture classification. If it has a very similar color to the hand, classification is more challenging. The same applies for a surrounding that contains lots of noise and other obstacles that make differentiation of the hand echo from other echoes challenging. Humans have distinct ways of carrying out a specific gesture. This leads to differences in speed and location of the hand in relation to the sensor. This results in differences in gesture signals of the same gesture among individuals. Furthermore, depending on the size of somebody's hand, the hand image or received echo signal are bigger or smaller, which causes difficulties for correct classification. \begin{table}
\begin{tabular}{l l} \hline \hline
**Challenge** & **Reason** \\ \hline \multirow{3}{*}{Detection of one or more hands} & Similarity between hand and background \\  & Different hand sizes \\  & Different location of hand at beginning and end of the same gesture \\ \hline Tracking of a hand & Different hand speeds \\ \hline \multirow{3}{*}{Classification of a hand gesture} & Variation of person: way of doing a gesture, speed, hand size, hand positioning \\  & towards the sensor \\ \cline{1-1}  & Variation of background: noise, obstacles, moving objects \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Challenges of HGR. Figure 2: Technology-agnostic processing flow.