## 2 Proposed System
### Joint Reason and Control Module  
As shown in Figure 14, some gestures and actions are quite similar to each other and they are difficult to be identified by either the offline learning based or the online gesture recognitions. In Figure 14a, the two gestures are quite similar and in (b) the actions of hands up and that of drawing circle are almost the same. In such cases, the performance of gesture and action recognition algorithms needs to be improved so as to be applicable for controlling the UAV. In order to solve this problem, we introduced a joint cross validation over the action and gesture recognition results. In this research, we separately selected six kinds of actions and gestures to control the flight action of a UAV. As shown in Figure 15, all the gestures and actions are arranged into a circle ring where each gesture is expressed as \(g_{j}\) (j = 1\(\sim\)6) and each action is denoted as \(a_{i}\) (i = 1\(\sim\)6). In order to separate similar gestures or actions from each other, each node is arranged by the following rules as: as for an input gesture \(g^{(in)}\) and action \(a^{(in)}\) with their correct category as \(g_{j}\) and \(a_{i}\), their probability to the other nodes should be \(P(g_{j}|g^{(in)})\gg P(g_{j\pm 1}|g^{(in)})\gg P(g_{j\pm 2}|g^{(in)})...\)  
Figure 13: Description for features. and \(P(a_{i}|a^{(in)})\gg P(a_{i\pm 1}|a^{(in)})\gg P(a_{i\pm 2}|a^{(in)})\). When \(P(g_{j}|g^{(in)})\sim P(g_{j\pm 1}|g^{(in)})\) or \(P(a_{i}|a^{(in)})\sim P(a_{i\pm 1}|a^{(in)})\), it means the action-gesture recognition results are un-reliable and we will send a special command to the UAV to keep it hovering. Such arrangement could guarantee that, as for an input action (or gesture) \(a^{(in)}\) (or \(g^{(in)}\)), even if the classification for it is wrong, the obtained action/gesture category (\(i\pm 1\) or \(j\pm 1\)) could still be the nearest neighbor to the correct one. Then, as shown in Figure 16, the sparse coding for the combination of action and gesture has been set up, where the eighteen orange blocks mean the correct codebook that the UAV pilot will use to control the UAV and the other blocks will not be applied in our system. As for the input action \(a^{(in)}\) and gesture \(g^{(in)}\), let \(id_{a}\) represent their classification result that can maximize Equation (2) and \(id_{g}\) be the output from Equation (3). The corresponding combination of \(g_{id_{g}}a_{id_{a}}\) could be checked as follows:  
\[g_{m}a_{n}= \begin{cases}\quad g_{id_{g}}a_{id_{a}}\quad\quad if\quad g_{id_{g} }a_{id_{a}}\in codebook\\ \arg\max\left(P(g_{id_{g}\pm 1})P(a_{id_{a}\pm 1})\right)\quad\quad else \end{cases}  \tag{6}\]  
(if \(id_{g/a}+1>6\), set \(id_{g/a}+1=1\); if \(id_{g/a}-1=0\), set \(id_{g/a}-1=6\).) And \(g_{m}a_{n}\) represents their final ID in the codebook through cross validation process. In the case that \(g_{id_{g}}a_{id_{a}}\) lies in the orange block of Figure 16, they will be considered as the correct result and translated to the corresponding flight action in Table 2. However, when \(g_{id_{g}}a_{id_{a}}\) is out of the codebook  
Figure 14: There are some gestures and actions quite similar to each other. In (**a**), two gestures are quite similar, and in (**b**), in five successive images, the human action in four images looks almost the same as others. Figure 15: Actions and gestures confusion ring sequence: \(g_{1}\): Rock, \(g_{2}\): Vertical palm, \(g_{3}\): Vertical knife, \(g_{4}\): Paper, \(g_{5}\): Horizontal palm, \(g_{6}\): Horizontal knife. Description for actions: \(a_{1}\): Waving left hand, \(a_{2}\): Hands up, \(a_{3}\): Draw circles in front of chest, \(a_{4}\): Swing hand up and down, \(a_{5}\): Waving right hand, \(a_{6}\): Draw circles over head. (means they lie in the white blocks), it means either the gesture or the action recognition result is incorrect. The correction for such result will performed by search for its four-neighbor block and find the very one that can maximum \((P(g_{id_{g}\pm 1})P(a_{id_{a}\pm 1}))\).