# Multiple Sensors Based Hand Motion Recognition Using Adaptive Directed Acyclic Graph
## 1 Introduction  
As an extraordinarily dexterous part of the human body, the human hand achieves most of the tasks in daily life. Hand movements provide a natural and intuitive manipulation modality while performing behaviors including gestures, postures, and manipulations. With the high-speed development of modern robot technology, robots are being used to work in more and more complex surroundings like aerospace, field operations, and repeated or dangerous works [1]. This technology has a profound impact on traditional industries, but faces some challenging problems. These applications require autonomous dexterous robots that perform increasingly human-like manipulation tasks and achieve advanced manipulation tasks such as in-hand regrasping, rotation and translation. The current level of development of a sophisticated multi-fingered robot hand is still at an early stage because of its control complexity and the immature synchronous cooperation between sensor-motor systems [2]. Hence, how to intensively investigate in-hand manipulation skills of humans, and further transfer such skills into bionic multi-fingered dexterous robotic hands have been receiving considerable attention [3]. In order to study the effectiveness of the dexterous human hand manipulation, it is crucial to accurately extract its multi-modal information, such as fingertip locations, hand skeleton, contact force, speed, etc. [4]. Current hand motion capturing systems can be mainly categorised into: data glove based capturing, attached force based capturing, electromyography (EMG) based capturing, optical markers-based capturing and vision based capturing. Increasing efforts have been made in analysing the hand motions by using uni-modal sensors. [PERSON] et al. developed a data glove-based hand motion recognition system which used a probabilistic neural network trained on a cluster set generated by a clustering ensemble [5]. Some mainstream sensors, including capacitive, piezoresistive, piezoelectric and optoelectronic techniques, were designed to detect 3D contact force and physical properties of object [6]. By utilizing multichannel surface EMG, individual and combined finger movements were classified for dexterous prosthetic control, and then offline processing was used to evaluate the classification performance [7]. [PERSON] et al. presented a kinematic model based on surface marker placement and used standard calculations to calculate the movements of the wrist, hand, fingers and thumb [8]. A comprehensive review of recent Kinect-based computer vision algorithms and applications, including preprocessing, object tracking and recognition, human activity analysis, hand gesture analysis, and indoor 3D mapping is presented in [9]. In addition to the literature mentioned above, a large number of related scientific papers as well as technical demonstrations have already appeared in various robot journals and conferences. However, most of the current research focused on a range of limited behaviours or in limited scenarios. Because of the complex properties involved in the human hand motions, hand motion capture will be faced with some additional challenges, like large posture variations, different colored skin and severe occlusions of the fingers during movements. A single sensor can get a certain feature information for some simple gesture recognition, such as static gestures. However, due to its own limitations, the lack of some key features may cause movement distortion in the complex gesture recognition, such as in-hand manipulation and re-grasps. It is crucial to compensate the drawbacks of uni-modal sensors by using a combination of varied types of sensors. For instance, data gloves do not have the basic ability of the spatial location information collection, and thus vision based sensors like Kinect, Leap motion controller can be integrated into motion capture system to compensate the limitation of data glove [10]. Hence, given the difficulty of robust control based on uni-modal sensors, multimodal sensory fusion is a promising approach.