# Trigger-Based Dexterous Operation with Multimodal Sensors for Soft Robotic Hand
## 1 Introduction  
Dexterous operation is one of the important abilities for a robotic hand to interact with the external environment [1; 2; 3; 4; 5; 6]. A soft robotic hand (SRH) is able to perform more compliant operations than a traditional rigid robotic hand [7; 8]. Therefore, SRH has attracted increasing interest from researchers [9; 10; 11]. [PERSON] and [PERSON] designed a compliant and underactuated robotic hand to perform dexterous grasping [12; 13]. [PERSON] et al. [14] embedded inner skeletons into the soft fingers to increase the hardness to grasp eggs, pens, cups, etc. [PERSON] et al. [15] designed a 13 degrees of freedom (DOF) SRH for dexterous grasping and in-hand manipulation. [PERSON] et al. [16] designed a human-like robotic hand using thin soft muscles. [PERSON] et al. [17] used electromyography (EMG) to control the SRH. [PERSON] et al. [18] designed a novel underactuated multifingered SRH for prosthetics to grasp multiple objects. However, these SRHs lacked sensors and sensing ability, which limited the dexterity of operation. To address this issue, [PERSON] et al. [19] developed an optoelectronically innervated soft prosthetic hand via stretchable optical waveguides that were fabricated with two soft silicone composites with different refractive indexes. The soft prosthetic had active haptic sensing to detect shape, texture, and softness and was able to select the ripest tomato among a group. [PERSON] et al. [20] performed an autonomous selection of closing posture by using soft matter capacitive sensors. Moreover, highly sensitive soft sensors are used in anthropomorphic robotic hands [21], and highly stretchable optical sensors are used to measure pressure, strain, and curvature [22]. In spite of this, there are still few SRHs that use multimodal sensors for human-robot interaction.  
On the other hand, operation strategy is another problem to consider. [PERSON] et al. [23] realized grasp configuration optimization based on haptic exploration information. [PERSON] et al. [24] realized robust proprioceptive grasping with an SRH. [PERSON] et al. [25] performed learning dexterous manipulation for an SRH from human demonstration.  
In summary, single modal sensors have been introduced into the SRHs to constrain perceptual skills for dexterous operation. On the other hand, the above-mentioned SRHs are mostly used to grasp objects, not to interact with humans. Therefore, we aim to develop dexterous operation for an SRH with multimodal sensors in this paper. Firstly, multimodal sensors are designed and integrated into an SRH. An optical-based fiber curvature sensor (OFCS) is designed and embedded in the soft finger together with a gas pressure sensor (GPS) to realize curvature detection for the soft finger. A capacitive pressure contacts sensor (CPCS) is attached on the fingertip to detect the touch force of the fingertip, while a resistance pressure contact sensor (RPCS) is attached to the palm to detect the touch force of the palm. These four sensors form a multimodal sensor system for the SRH. Secondly, a trigger-based dexterous operation (TDO) strategy is proposed to improve the dexterity of operation. Operation experiments were conducted to validate the effectiveness and feasibility of dexterous operation.  
The main contribution of the work is to add different sensors on the SRH to realize the multimodal dexterous operation. The paper is organized as follows: Section 2 introduces multimodal sensors integrated in the SRH; these sensors are used for curvature detection and touch force detection. Section 3 describes the framework of TDO and the design of the program flow of TDO in the embedded system. Section 4 shows the experiment results, which demonstrate that the TDO is effective and feasible. Finally, Section 5 presents the conclusions of this paper.