# Single-Handed Gesture Recognition with RGB Camera for Drone Motion Control
## 3 Methodology
### Hand Gesture Vocabulary for Drone Control  
In this section, we design a set of hand gestures for the control of drone flights, utilizing an RGB camera mounted on a user's head to capture hand gestures from an egocentric view for first-person-view (FPV) flights. Our goal is to ensure user convenience and accessibility in drone control. [PERSON] et al. [8] designed a single-handed gesture set that was as close as possible to the drone's flight direction and was easy to perform and remember. In addition, they proposed a multiple-handed gesture set to reduce the rate of hand gesture misoperation and improve the adaptability of hand gesture interaction. Nevertheless, limitations exist in executing simultaneous roll, pitch, yaw, and throttle  
Figure 1: Overview of the HGR pipeline. Hand keypoints are estimated and input into two HGR models, which detect drone motion commands from hand gestures. maneuvers. [PERSON] et al. [23] developed two gesture vocabularies, finger-based and palm-based, for virtual drone navigation, evaluating their effectiveness in a user study. However, palm-based gestures require precise hand pose estimation using a 3D sensor, while finger-based gestures lack roll movements. Building upon the insights gained from the aforementioned references, we define a set of precise hand gestures tailored for drone maneuvering. Traditional drones with four rotors, also known as quadcopters, are maneuvered by a combination of four basic controls, namely throttle, roll, pitch, and yaw. Throttle controls the speed of the rotors, which in turn controls the altitude of the drone. Roll controls the rotation of the drone around its longitudinal axis, which causes it to move left or right. Pitch controls the rotation of the drone around its lateral axis, which causes it to move forward or backward. Yaw controls the rotation of the drone around its vertical axis, which causes it to rotate clockwise or counterclockwise. In particular, throttle and yaw are interrelated and can be affected by the pitch and roll and vice versa. Therefore, the movement of a drone can be described as a combination of the following four elements:  
\[Movement=Throttle\cdot\mathbb{I}_{T}+Roll\cdot\mathbb{I}_{R}+Pitch\cdot\mathbb{I}_ {P}+Yaw\cdot\mathbb{I}_{Y}, \tag{3}\]  
where \(\mathbb{I}_{T},\mathbb{I}_{R},\mathbb{I}_{P}\), and \(\mathbb{I}_{Y}\) denote the indicator functions, which represent whether throttle, roll, pitch, and yaw are active (1) or not (0), respectively. Each element has three states: increase, decrease, and neutral. Therefore, there are at least 81 (i.e., 3\({}^{4}\)) movement commands. To ensure intuitive and convenient drone maneuvering, it is crucial to align the hand gestures closely with the desired drone flight direction. Figure 2 illustrates a method of controlling drone movement using the hand gestures that we propose. We initially establish gestures to represent the stop and neutral states of the drone. The neutral state occurs when a drone is not moving and is not controlled by the user. In this state, the drone must remain stationary and wait for commands to move. The stop command tells the drone to stop moving and remain stationary. This command stops the drone from moving in any direction. In the study conducted by [PERSON] et al. [23], the participants reported experiencing a notably high level of ease and comfort when executing index-finger-related gestures. Consequently, we design the neutral state to correspond to an index-finger-only extension motion. The stop command is defined as a closed fist, which is a gesture that is commonly associated with stopping or halting an action. Next, it is necessary to design gestures for yaw, roll, pitch, and throttle. We divide the hand gestures into two categories based on the characteristics of the four components:  
Figure 2: An illustration of the manipulation of drone movement with a combination of hand gestures. When a user presents the stop gesture, no motion command is transmitted to the drone.