# A Survey of Tactile-Sensing Systems and Their Applications in Biomedical Engineering

Authors: Yousef Al-Handarish; Olatunji Mumini Omisore; Tobore Igbe; Shipeng Han; Hui Li; Wenjing Du; Jinjie Zhang; Lei Wang
Journal: Advances in Materials Science and Engineering
 (2020)

---

## Abstract

Over the past few decades, tactile sensors have become an emerging field of research in both academia and industry. Recent advances have demonstrated application of tactile sensors in the area of biomedical engineering and opened up new opportunities for building multifunctional electronic skin (e-skin) which is capable of imitating the human sense-of-touch for medical purposes. Analyses have shown that current smart tactile sensing technology has the advantages of high performance, low-cost, time efficiency, and ease-of-fabrication. Tactile sensing systems have thus sufficiently matured for integration into several fields related to biomedical engineering. Furthermore, artificial intelligence has the potential for being applied in human-machine interfacing, for instance, in medical robotic manipulation, especially during minimally invasive robotic surgery, where tactile sensing is usually a problem. In this survey, we present a comprehensive review of the state of the art of tactile sensors. We focus on the technical details of transduction mechanisms such as piezoresistivity, capacitance, piezoelectricity, and triboelectric and highlight the role of novel and commonly used materials in tactile sensing. In addition, we discuss contributions that have been reported in the field of biomedical engineering, which includes its present and future applications in building multifunctional e-skins, human-machine interfaces, and minimally invasive surgical robots. Finally, some challenges and notable improvements that have been made in the technical aspects of tactile sensing systems are reported.


---

## Body

## 1. Introduction

Human skin is the largest sensory system in the human body and contains a complex array of mechanoreceptors that perceive tactile sensation. So far, four main types of mechanoreceptors have been identified [1, 2], and their role in the human sense-of-touch has been broadly explained. It has been established that these mechanoreceptors are stimulated when touched, sending signals to the central nervous system which are processed for bodily utility and feedback [3]. Once the general principles of tactile sensing were discovered, they could be applied to enable robotic systems to interact with objects in their environment in a similar manner to humans, i.e., machines can analyze objects based on their physical properties such as pressure, dynamic strain, surface texture, and shear for recognition and interaction [2]. The interest in the sense-of-touch began in the late 1970s with several studies focusing on explaining its principles. Knibestöl and Vallbo [4] published a survey in which 61 mechanoreceptor units in glabrous skin were analyzed based on their physiognomic differences in the receptive fields. Later in 1974, Clippinger et al. [5] demonstrated the first prosthetic hand based on a physical feedback principle. Consequently, over the past two decades, tactile sensing has evolved and been widely explored by many research groups, with remarkable achievements in covering not only the fingertip or hands of humanoid robots, but also their whole bodies.The stimulation of tactile systems depends on designing efficient tactile perception strategies, technological aspects of the sensory devices, and effective tactile learning methods [6]. Many tactile sensing systems have already been proposed and developed based on, for example, piezoresistive [7, 8], piezoelectric [9, 10], capacitive [11, 12], and optical [13] principles. However, a survey of the research shows that this is still an emerging field with scientific and technical challenges requiring extensive work. The focus of research has now shifted to improving existing systems by harnessing advances in signal processing and fabrication technologies as in the use of highly flexible and stretchable materials such as polydimethylsiloxane PDMS, carbon nanotubes (CNTs), bioinspired materials, and graphene [14–16]. These microfabrication technologies and integrated materials are focused towards the development of artificial skins with embedded tactile sensing, which will play important roles in the future of multifunctional e-skin, hand prosthetics, and soft robotics.Tactile sensors are especially promising for future developments in medical robotics [17]. Tactile sensors in medical surgery have already been applied in a variety of surgical robotic tools. For instance, Ohtsuka et al. developed a piezoelectric tactile sensor able to locate small, barely-visible nodules during minimally invasive thoracoscopic surgery [18]. For minimally invasive (MIS) applications, providing the surgeon with the ability to measure coordinate tool-tissue contact force during surgical procedures is key. The human sense-of-touch can thus be enhanced by intelligent tools capable of feeling tissue and organs, which process and transmit these sensations to the surgeon as if he were really sensing [19].In this review, we focus on recent developments in tactile sensors and their applications in e-skin, human-machine interaction, and MIS. The remainder of this review is organized as follows: Section2 includes the basic concepts of the human sense-of-touch; in Section 3, the transduction mechanisms used for sensing in artificial systems are described as are the new materials that have been used in the fabrication of tactile sensors; in Section 4, current and future applications of tactile sensing systems in biomedical engineering are surveyed along with recent applicability towards in minimally invasive robotic surgery; finally, a summary of the challenges and perspectives hoped for the future with tactile sensing is presented in Section 5.


## 2. Basic Concepts of Human Sense-of-Touch

Individually, mechanoreceptors appear as jelly-like materials located under the human skin [20]. Lexically, they can be said to be a network of receptors and processing centers combining to form the haptic sensory system. The latter is responsible for the perception of the information acquired from surroundings, flashing it to the central nervous system (CNS) as signals. After analyzing and processing the signals, the body then gives feedback in the form of a physical response [21].The human sense-of-touch involves different sensory subsystems that can be classified according different factors. One of the most common classification methods is by the source of neural inputs which may be cutaneous, kinesthetic, or haptic [22–25]. The cutaneous subsystem is associated with the skin and involves physical contact with stimuli. This subsystem performs the spatiotemporal perception of external stimuli via receptors such as thermoreceptors for temperature and thermal inputs and nociceptors, which respond to pain and damage. The mechanoreceptors, which are the focus of this review, play vital roles in providing the CNS with information about mechanical effects, such as vibration and contact pressure [26]. The kinesthetic subsystem acquires sensory information received through mechanoreceptors located in the muscles, joints, and tendons of the human body system. Thus, kinesthetic information enables the CNS to know about the position and movement of the body and limb segments in both cases, static and dynamic. The haptic sense is combining sensory stimulations of both the cutaneous and kinesthetic subsystems, in purpose to perform and stimuli body activities efficiently.


### 2.1. Human Mechanoreceptors

Human skin is an active sensory system which protects our bodies from injury, dehydration, radiation, and toxic substances in the external environment by tactile sensation of stimuli [27]. The skin consists of complex layers of specialized receptors, such as the epidermis, dermis, and hypodermis. The external layer of the epidermis is responsible for regulating body temperature and consists of impervious protective surfaces. The dermis layer, which is located under the epidermis, transmits nerve information from thermal, mechanical, and chemical stimuli. The third layer is hypodermis which, depending on the study, may or may not be considered a part of human skin (Figure 1). This part of the external layer consists of connective and subcutaneous tissues that separate the dermis from the muscle and bone.Figure 1
Description of sensory touch receptors in glabrous human skin (source: adapted from [28], copyright (2018), with permission from MDPI AG).Generally, tactile units or mechanoreceptors are available in all layers and all the areas of human skin. One of the most notable areas is the human hand (glabrous and hairless) where most nerve endings are embedded in the dermis at different depths. Figure1 illustrates some of the 17,000 mechanoreceptors in human skin, which can be categorized into four main types, namely, Meissner’s corpuscles, Markel’s cells, Ruffini endings, and Pacinian corpuscles [8, 29–32]. Each mechanoreceptor responds to a specific kind of mechanical stimuli and is differentiated from the other by the structure of its receptive fields and the adaption rate. The adaptation rates can be separated into two main groups: slow adapting (S) units and rapid adapting units (RA), also known as fast adapting (FA) units. The SA units show an active and continuous response to sustained deformations, whereas rapid adapting RA units respond to physical deformations.The RA and SA units are further divided into type I receptors, which have small receptive fields and are located close to the surface of the skin, and type II receptors, which are rooted in the dermis and have large receptive fields [33–35]. Merkel cells are representative of SA-I mechanoreceptors; they respond to static pressure distributions and to low-frequency stimuli. Ruffini endings are representative of SA-II mechanoreceptors; they detect sensations and perceive the stretching of skin around joints and fingernails as well as helping in the detection of finger motion and position and the perception of object shape [31]. Meissner corpuscles are representative of RA-I mechanoreceptors; they innervate the skin and deal with the low-frequency vibrations, grip control, motion detection, and tapping [36, 37]. Pacinian corpuscles are representative of RA-II mechanoreceptors, and they are responsible for high-frequency vibration detection and perception of surface textures [38].


### 2.2. Human Tactile Perception

Based on the above, integrated tactile devices equipped into prosthetics and robotics should be provided with an electronic circuit for tactile feedback, imitating the CNS mechanism by transforming the kind of inputs which can be accepted by the human sensory system. Thus, flexible tactile sensors are a promising development due to their ability to mimic the transduction mechanism of human skin. Mechanoreceptors in the human skin collect different information by neural afferents, which transmit information to the brain. There are two main pathways to transmit tactile information to the CNS, namely, the dorsal-medial lemniscal pathway and the spinothalamic pathway [39]. The spinothalamic pathway is part of the anterolateral system, and it transmits information to the thalamus about temperature and pain, while the dorsal column-medial lemniscus pathway transmits vibration, pressure, and shared position information from mechanoreceptors to the CNS [40, 41]. Johansson et al. [23] observed that both pathways provide a classification of tactile stimuli by temporal-to-spatial conversion at the level of second-order neurons, which can function as a coincidence detector. Tactile information is processed in multiple stages and undergoes a process of selection and adaptation that enables human beings to discriminate between properties of objects and be more sensitive to the particular parts of the object as they touched [42–45]. In addition, cutaneous and kinesthetic senses work together to discriminate and detect various stimulations such as force, position, and texture.


## 2.1. Human Mechanoreceptors

Human skin is an active sensory system which protects our bodies from injury, dehydration, radiation, and toxic substances in the external environment by tactile sensation of stimuli [27]. The skin consists of complex layers of specialized receptors, such as the epidermis, dermis, and hypodermis. The external layer of the epidermis is responsible for regulating body temperature and consists of impervious protective surfaces. The dermis layer, which is located under the epidermis, transmits nerve information from thermal, mechanical, and chemical stimuli. The third layer is hypodermis which, depending on the study, may or may not be considered a part of human skin (Figure 1). This part of the external layer consists of connective and subcutaneous tissues that separate the dermis from the muscle and bone.Figure 1
Description of sensory touch receptors in glabrous human skin (source: adapted from [28], copyright (2018), with permission from MDPI AG).Generally, tactile units or mechanoreceptors are available in all layers and all the areas of human skin. One of the most notable areas is the human hand (glabrous and hairless) where most nerve endings are embedded in the dermis at different depths. Figure1 illustrates some of the 17,000 mechanoreceptors in human skin, which can be categorized into four main types, namely, Meissner’s corpuscles, Markel’s cells, Ruffini endings, and Pacinian corpuscles [8, 29–32]. Each mechanoreceptor responds to a specific kind of mechanical stimuli and is differentiated from the other by the structure of its receptive fields and the adaption rate. The adaptation rates can be separated into two main groups: slow adapting (S) units and rapid adapting units (RA), also known as fast adapting (FA) units. The SA units show an active and continuous response to sustained deformations, whereas rapid adapting RA units respond to physical deformations.The RA and SA units are further divided into type I receptors, which have small receptive fields and are located close to the surface of the skin, and type II receptors, which are rooted in the dermis and have large receptive fields [33–35]. Merkel cells are representative of SA-I mechanoreceptors; they respond to static pressure distributions and to low-frequency stimuli. Ruffini endings are representative of SA-II mechanoreceptors; they detect sensations and perceive the stretching of skin around joints and fingernails as well as helping in the detection of finger motion and position and the perception of object shape [31]. Meissner corpuscles are representative of RA-I mechanoreceptors; they innervate the skin and deal with the low-frequency vibrations, grip control, motion detection, and tapping [36, 37]. Pacinian corpuscles are representative of RA-II mechanoreceptors, and they are responsible for high-frequency vibration detection and perception of surface textures [38].


## 2.2. Human Tactile Perception

Based on the above, integrated tactile devices equipped into prosthetics and robotics should be provided with an electronic circuit for tactile feedback, imitating the CNS mechanism by transforming the kind of inputs which can be accepted by the human sensory system. Thus, flexible tactile sensors are a promising development due to their ability to mimic the transduction mechanism of human skin. Mechanoreceptors in the human skin collect different information by neural afferents, which transmit information to the brain. There are two main pathways to transmit tactile information to the CNS, namely, the dorsal-medial lemniscal pathway and the spinothalamic pathway [39]. The spinothalamic pathway is part of the anterolateral system, and it transmits information to the thalamus about temperature and pain, while the dorsal column-medial lemniscus pathway transmits vibration, pressure, and shared position information from mechanoreceptors to the CNS [40, 41]. Johansson et al. [23] observed that both pathways provide a classification of tactile stimuli by temporal-to-spatial conversion at the level of second-order neurons, which can function as a coincidence detector. Tactile information is processed in multiple stages and undergoes a process of selection and adaptation that enables human beings to discriminate between properties of objects and be more sensitive to the particular parts of the object as they touched [42–45]. In addition, cutaneous and kinesthetic senses work together to discriminate and detect various stimulations such as force, position, and texture.


## 3. Tactile Transduction Techniques

The most common techniques currently being employed for tactile transduction are based on piezoresistive, capacitive, optical, piezoelectric, and triboelectric tactile sensing methods (TESM). Each of these transduction methods has some unique characteristics, as illustrated in Figure2, upon which sensing can be achieved. The principle aim of these mechanisms is to convert tactile stimuli into electrical signals. Robotic grasping, manipulation, and related activities will be based on measurement of the tactile stimuli. Below, we present only a brief review of TESM methods and their performance.Figure 2
Schematic illustrations for four typical transduction mechanisms of tactile sensors: (a), piezoresistive, (b) piezocapacitive, (c) piezoelectric, and (d) triboelectric (adapted from Ref. [46], copyright (2018), with permission from WILEY-VCH publishing group).


### 3.1. Piezoresistive Tactile Sensors

Piezoresistive tactile sensors make use of the changes in the electrical resistance of materials as forces and contact are applied to them. This is known as the piezoresistive effect; a mechanism that involves conversion of mechanical stimuli to electrical signals. The piezoresistive effect of a conductor can be described mathematically by the expressions of equations (1) and (2) where R is the sensitivity of a conductor, v is Poisson’s ratio, and ε is the strain. Equation (2) accounts for the electrical resistance, where ρ is the resistivity, L is the length, and A is the cross-sectional area of the conductor [47].(1)ΔRR=1+2νε+Δρρ,(2)R=ρLA.According to the sensing principle, piezoresistive sensors can be separated into two categories [48]: piezoresistive tactile sensors based on changing contact area at the microscale [8, 9, 49] and those based on pressure detection and volume changes in sensitive materials [50].Owning to their excellent electrical conductivity, low-cost, easy read-out circuit, and its nanoscale flexibility piezoresistive tactile sensors have received great attention and have become the most commonly used in various wearable sensing devices to detect tactile signals. This has motivated engineers as well as researchers to employ tactile sensing based on piezoresistive effects in various medical and industrial applications [51]. For microstructure fabrication of sensing elements, the most common choices include the use of nanomaterials such as soft polymer, nanoscale, and microscale conductive matrices with silicon composites as strain gauges. Tactile sensors fabricated with nanomaterials have high mechanical flexibility and chemical resistance, which enable them to be used in e-skin and a variety of robotics applications. Metallic fillers such as Ni and Ag [52, 53], carbon-based materials [54, 55], carbon black [56], and silicon [57] have been evaluated as ideal choice for piezoresistive tactile sensors [25, 26].Recently, several works have extensively studied carbon-based materials such as polymer/CNTs for use in pressure sensing [58] and strain sensing [59–61]. However, pressure sensitivity still needs to be improved to detect low pressure range and external stress. To achieve a piezoresistive tactile sensor with better sensitivity, several assembly methods have been applied in recent studies.


### 3.2. Capacitive Tactile Sensors

Capacitive tactile sensing is another common tactile sensing mechanism, which is based on assessing changes in the geometry of a capacitor by mechanical effects via changes in its capacitance. Broadly defined, capacitance is the ability of a capacitor to store electrical charge. For a parallel-plate capacitor, the capacitance can be described with by equation (3), where ε0 is the permittivity of vacuum, εr is the relative static permittivity of the dielectric layer between two plates, A is the area of overlap between the two parallel plates, and d is the distance between two parallel plates:(3)C=ε0εrAd.Changes in the overlap and distance enable effective measurement of the shear forces, strain, and pressure that are applied on the sensing structure [62, 63]. Due the possibility variations in electrode structure and dielectric layer design, capacitive tactile sensors have a wide domain for applications in different area such as e-skin, touch screens, and surgical robotics.By equation (3), the performance of tactile sensing systems can be improved by using materials of small thickness and high capacitances for the dielectric layers. Polymer materials, such as polydimethylsiloxane (PDMS) [64–66], and silicon [67], are currently popular for fabrication of capacitive tactile sensors, while highly compressible materials are being explored [62]. Materials with good electrical conductivity, low-cost, and mechanical flexibility have become great candidates for developing piezocapacitive tactile sensors.


### 3.3. Piezoelectric and Triboelectric Tactile Sensors

Piezoelectrical tactile sensors include those sensors for which external pressure causes their constituent materials to generate electrical potentials by the piezoelectric effect. The magnitude of the electrical potential is described by equation (4) where dijk is the piezoresistive coefficient of the causal materials, xjk is the external applied effect, and Di is the charge density generated in the ith direction [68]. Piezoelectric materials include materials that can be polarized and which can therefore also be used as dielectric materials whose capacitance changes when dynamic forces are applied to them. Piezoelectrical materials such as polyvinylidene fluoride (PVDF) and its copolymers are suitable choices for covered surfaces or cases where large overlaps in material are desired.(4)Di=dijkXjk.Another type of piezoelectricity sensing is based on the triboelectric mechanism. This has not yet been widely explored but it also involves the conversion of mechanical energy, as a result of external effects operating on the underlying material, into electrical signals. Materials with ambipolar transport behaviors such as graphene and organic based materials such as PDMS, PVC, PTFE, Al, are considered to be the ideal candidates for fabricating triboelectric tactile sensors [69, 70]. Figure 2(d) shows the triboelectric mechanism operating on the same material in different states. The triboelectric potential, which can be negative or positive, is induced at the interface between the two different materials and the amount of charge generated depends on the difference between the triboelectric polarities [70]. However, when pressure is applied to two materials with different triboelectric polarities and they come into contact with each other, the triboelectric effect induces opposite charges in both surfaces [71]. One advantage of triboelectric devices is that they can use surface friction which enhances the triboelectric effect due to the higher output voltage the device can produce.Recently, Wang et al. [72] integrated a self-powering system with a high-resolution pressure sensor for real-time tactile mapping and motion tracking. The tactile information sensing is done with a signal-electrode nanogenerator (TENG). Electrical responses were obtained from the charges that are transferred between the ground and electrodes as the object moves on the surface above the PDMS layer. As shown in Figure 3, the self-powered flexible signal-electrode triboelectric sensor matrix has a resolution of 5 dpi and 16 × 16 pixels besides the fact that data acquisition can be stimulated from multiple channels in real-time. These promote durability and an excellent pressure sensitivity of 0.06 kPa−1. It is important to note that the triboelectric devices can have wider range of applications, for instance, in self-powering devices, tactile sensing for touch pad, and human-machine interface [72].Figure 3
Schematic illustration of TESM. (a) Structure of a 16 × 16 TESM. (b) Schematic of the pressure mapping process. (c) Demonstration of the mapping output voltage of the sensor matrix under the pressure. (d) The output voltage increases with increasing contact area at a pressure of ≈200 kPa. (e) The output voltage simulated using the finite element method with different contact areas. (f) Photograph of a fabricated 16 × 16 TESM with good flexibility (reprinted from Ref. [72], copyright (2016), with permission from WILEY-VCH publishing group).
(a)
(b)
(c)
(d)
(e)
(f)


### 3.4. Optical Tactile Sensors

The principle of optical tactile sensors is based on employing changes in illumination (light) properties to obtain tactile information. In general, optical sensors are characterized by their light transmission, fast response, physical flexibility, and spatial resolution [73]. Optical tactile sensors have a wide range application for compliance [73], roughness sensing [73], shear, and vertical stress [74–76]. In addition, optical tactile sensors exhibit many potential applications which require the features of portability and flexibility [77, 78]. Such sensors must generally be immune to electromagnetic interfaces in order to enhance robotic manipulation during minimally invasive surgery (MIS) and related procedures such as robotic grasping and navigation [78, 79]. Yamazaki et al. [78] reported a tactile sensor that is based on heterocore fiber optics, with the design structure shown in Figure 4. The heterocore fiber optic converts the applied force into a bending curvature on a heterocore fiber optic sensor. The hemispheric tactile sensor can be used to detect surface roughness with periodic changes of less than 0.05 mm and a periodic pattern of 0.74 mm. This tactile sensor also has the capability of hardness and texture detection for discriminating touched objects, which will allow it to be utilized in intelligent robots and tactile feedback for the detection of lumps in biomedical applications [78].Figure 4
Schematic illustration showing the structure of the hemispheric tactile sensor: (a) the structure of a heterocore fiber optic sensor and (b) the structure of a hemispheric of a hemispheric heterocore fiber optic tactile sensor (adapted from Ref. [78] Copyright (2018), with permission).
(a)
(b)


## 3.1. Piezoresistive Tactile Sensors

Piezoresistive tactile sensors make use of the changes in the electrical resistance of materials as forces and contact are applied to them. This is known as the piezoresistive effect; a mechanism that involves conversion of mechanical stimuli to electrical signals. The piezoresistive effect of a conductor can be described mathematically by the expressions of equations (1) and (2) where R is the sensitivity of a conductor, v is Poisson’s ratio, and ε is the strain. Equation (2) accounts for the electrical resistance, where ρ is the resistivity, L is the length, and A is the cross-sectional area of the conductor [47].(1)ΔRR=1+2νε+Δρρ,(2)R=ρLA.According to the sensing principle, piezoresistive sensors can be separated into two categories [48]: piezoresistive tactile sensors based on changing contact area at the microscale [8, 9, 49] and those based on pressure detection and volume changes in sensitive materials [50].Owning to their excellent electrical conductivity, low-cost, easy read-out circuit, and its nanoscale flexibility piezoresistive tactile sensors have received great attention and have become the most commonly used in various wearable sensing devices to detect tactile signals. This has motivated engineers as well as researchers to employ tactile sensing based on piezoresistive effects in various medical and industrial applications [51]. For microstructure fabrication of sensing elements, the most common choices include the use of nanomaterials such as soft polymer, nanoscale, and microscale conductive matrices with silicon composites as strain gauges. Tactile sensors fabricated with nanomaterials have high mechanical flexibility and chemical resistance, which enable them to be used in e-skin and a variety of robotics applications. Metallic fillers such as Ni and Ag [52, 53], carbon-based materials [54, 55], carbon black [56], and silicon [57] have been evaluated as ideal choice for piezoresistive tactile sensors [25, 26].Recently, several works have extensively studied carbon-based materials such as polymer/CNTs for use in pressure sensing [58] and strain sensing [59–61]. However, pressure sensitivity still needs to be improved to detect low pressure range and external stress. To achieve a piezoresistive tactile sensor with better sensitivity, several assembly methods have been applied in recent studies.


## 3.2. Capacitive Tactile Sensors

Capacitive tactile sensing is another common tactile sensing mechanism, which is based on assessing changes in the geometry of a capacitor by mechanical effects via changes in its capacitance. Broadly defined, capacitance is the ability of a capacitor to store electrical charge. For a parallel-plate capacitor, the capacitance can be described with by equation (3), where ε0 is the permittivity of vacuum, εr is the relative static permittivity of the dielectric layer between two plates, A is the area of overlap between the two parallel plates, and d is the distance between two parallel plates:(3)C=ε0εrAd.Changes in the overlap and distance enable effective measurement of the shear forces, strain, and pressure that are applied on the sensing structure [62, 63]. Due the possibility variations in electrode structure and dielectric layer design, capacitive tactile sensors have a wide domain for applications in different area such as e-skin, touch screens, and surgical robotics.By equation (3), the performance of tactile sensing systems can be improved by using materials of small thickness and high capacitances for the dielectric layers. Polymer materials, such as polydimethylsiloxane (PDMS) [64–66], and silicon [67], are currently popular for fabrication of capacitive tactile sensors, while highly compressible materials are being explored [62]. Materials with good electrical conductivity, low-cost, and mechanical flexibility have become great candidates for developing piezocapacitive tactile sensors.


## 3.3. Piezoelectric and Triboelectric Tactile Sensors

Piezoelectrical tactile sensors include those sensors for which external pressure causes their constituent materials to generate electrical potentials by the piezoelectric effect. The magnitude of the electrical potential is described by equation (4) where dijk is the piezoresistive coefficient of the causal materials, xjk is the external applied effect, and Di is the charge density generated in the ith direction [68]. Piezoelectric materials include materials that can be polarized and which can therefore also be used as dielectric materials whose capacitance changes when dynamic forces are applied to them. Piezoelectrical materials such as polyvinylidene fluoride (PVDF) and its copolymers are suitable choices for covered surfaces or cases where large overlaps in material are desired.(4)Di=dijkXjk.Another type of piezoelectricity sensing is based on the triboelectric mechanism. This has not yet been widely explored but it also involves the conversion of mechanical energy, as a result of external effects operating on the underlying material, into electrical signals. Materials with ambipolar transport behaviors such as graphene and organic based materials such as PDMS, PVC, PTFE, Al, are considered to be the ideal candidates for fabricating triboelectric tactile sensors [69, 70]. Figure 2(d) shows the triboelectric mechanism operating on the same material in different states. The triboelectric potential, which can be negative or positive, is induced at the interface between the two different materials and the amount of charge generated depends on the difference between the triboelectric polarities [70]. However, when pressure is applied to two materials with different triboelectric polarities and they come into contact with each other, the triboelectric effect induces opposite charges in both surfaces [71]. One advantage of triboelectric devices is that they can use surface friction which enhances the triboelectric effect due to the higher output voltage the device can produce.Recently, Wang et al. [72] integrated a self-powering system with a high-resolution pressure sensor for real-time tactile mapping and motion tracking. The tactile information sensing is done with a signal-electrode nanogenerator (TENG). Electrical responses were obtained from the charges that are transferred between the ground and electrodes as the object moves on the surface above the PDMS layer. As shown in Figure 3, the self-powered flexible signal-electrode triboelectric sensor matrix has a resolution of 5 dpi and 16 × 16 pixels besides the fact that data acquisition can be stimulated from multiple channels in real-time. These promote durability and an excellent pressure sensitivity of 0.06 kPa−1. It is important to note that the triboelectric devices can have wider range of applications, for instance, in self-powering devices, tactile sensing for touch pad, and human-machine interface [72].Figure 3
Schematic illustration of TESM. (a) Structure of a 16 × 16 TESM. (b) Schematic of the pressure mapping process. (c) Demonstration of the mapping output voltage of the sensor matrix under the pressure. (d) The output voltage increases with increasing contact area at a pressure of ≈200 kPa. (e) The output voltage simulated using the finite element method with different contact areas. (f) Photograph of a fabricated 16 × 16 TESM with good flexibility (reprinted from Ref. [72], copyright (2016), with permission from WILEY-VCH publishing group).
(a)
(b)
(c)
(d)
(e)
(f)


## 3.4. Optical Tactile Sensors

The principle of optical tactile sensors is based on employing changes in illumination (light) properties to obtain tactile information. In general, optical sensors are characterized by their light transmission, fast response, physical flexibility, and spatial resolution [73]. Optical tactile sensors have a wide range application for compliance [73], roughness sensing [73], shear, and vertical stress [74–76]. In addition, optical tactile sensors exhibit many potential applications which require the features of portability and flexibility [77, 78]. Such sensors must generally be immune to electromagnetic interfaces in order to enhance robotic manipulation during minimally invasive surgery (MIS) and related procedures such as robotic grasping and navigation [78, 79]. Yamazaki et al. [78] reported a tactile sensor that is based on heterocore fiber optics, with the design structure shown in Figure 4. The heterocore fiber optic converts the applied force into a bending curvature on a heterocore fiber optic sensor. The hemispheric tactile sensor can be used to detect surface roughness with periodic changes of less than 0.05 mm and a periodic pattern of 0.74 mm. This tactile sensor also has the capability of hardness and texture detection for discriminating touched objects, which will allow it to be utilized in intelligent robots and tactile feedback for the detection of lumps in biomedical applications [78].Figure 4
Schematic illustration showing the structure of the hemispheric tactile sensor: (a) the structure of a heterocore fiber optic sensor and (b) the structure of a hemispheric of a hemispheric heterocore fiber optic tactile sensor (adapted from Ref. [78] Copyright (2018), with permission).
(a)
(b)


## 4. Potential Applications of Tactile Sensors

Because of the advantages of tactile sensors and their ability to mimic human skin, tactile sensors have been widely used to develop multifunctional e-skins with high sensitivity at low-cost. Many works have reported various potential applications of tactile sensors in health-care devices, human-machine interaction, and MIS [45, 80–82]. Here, we review the state of art of the potential applications of tactile sensors.


### 4.1. Multifunctional Artificial Skin

To achieve multifunctional e-skins that are capable of high perception from a variety of stimulations in static and dynamic cases, it is necessary to integrate a large number of macroscales of sensing elements based on thin flexible tactile substrates [83]. In addition, mechanosensory e-skins consisting of mechanical and stretchable sensor networks have exhibited a wide range of application in robotics [6, 84, 85], health-monitoring devices [86–89], and prosthetics [90–92].Recently, Hua et al. [93] reported a skin-inspired, highly stretchable sensor with conformable matrix networks (SCMNs), based on a structured polyimide network, as shown in Figure 5. The structure has the potential for integration into three-dimensional configurations. The SCMN illustrated successfully expands the e-skin sensing functionality in pressure, magnetic field, humidity, temperature, strain, and proximity. However, to achieve multifunctional sensing performance, the SCMN was comprised of 100 sensory nodes connected to each other by meandering wires as shown in Figure 6(a) [93]. Furthermore, when SCMN covered an artificial hand as e-skin, it exhibited superior expandability for multisensing. Figure 5(d) shows the spatial pressure achieved before and after 300% expansion of the SCMN, enabling it to identify position and pressure load and estimate the size of the loading object. The reported SCMN has various applications in human-machine interfaces, prosthetics, and health-monitoring technologies [93]. Chou et al. [94] introduced a chameleon-inspired stretchable e-skin, which is capable of interactive color changes and tactile sensing properties. Color easily can be changed and controlled through varying the applied pressure along with the applied pressure duration using ultraviolet-visible measurements.Figure 5
Schematic illustration of the skin-inspired high stretchable and conformable matrix networks. (a) Illustration showing layout of SCMNs: an integrated sensor array with eight functions; (b) photographic image indicating the fabricated polyimide network (10 × 10 array, scale bar: 5 mm); (c) schematic illustration of an SCMN as artificial skin equipped on hand (showing sensing expandability and adjustability); (d) plot of pressure mapping before and after the 300% expansion of an SCMN (Figures5(a)–5(d), reproduced from Ref. [93] with permission; copyright 2018; Nature publishing group).
(a)
(b)
(c)
(d)Figure 6
(a) Schematic illustration showing the concept of a chameleon-inspired e-skin, showing the structure of the neutral and oxidized states of electrochromic polymer, also schematic of the circuit layout (PS, pressure sensor; ECD, electrochromic device). (b) The schematic layout of interactive color-changeable e-skin (reproduced with permission from Ref. [94], copyright 2015, Nature publishing group).
(a)
(b)Chou et al. [94] introduced a chameleon-inspired stretchable e-skin, which is capable of interactive color changes and tactile sensing properties. Color easily can be changed and controlled through varying the applied pressure along with the applied pressure duration using ultraviolet-visible measurements. The idea of mimicking the sense-of-touch capacities of by human skin has motivated the use of fully stretchable organic electrochromic devices (ECDs) which can be fabricated with reduced consumption power and can be integrated with highly tunable resistive pressure sensing. As demonstrated in Figure 6, changes in pressure can be achieved by utilizing multiple and tunable colors in the flexible sensor. The proposed stretchable color changeable e-skin can maintain its skin color without any applied pressure, indicating its low power consumption [94].Another emerging aspect in the development of e-skin is the use of elastic materials for tactile sensor fabrication. PDMS has been widely studied for its virtues of biocompatibility, mechanical flexibility, and simplicity in the fabrication of microchannels (less than 100μm) [95–97]. Liquid and bioinspired materials are widely used due to their characterization, which includes high deformability and good electrical conductivity [98]. Gao et al. [99] demonstrated a microfluidic tactile diaphragm pressure sensor, characterized by embedded Galinstan (a eutectic alloy of gallium, indium, and tin) microchannels with a 70 μm width and a 70 μm height, as shown in Figure 7(a) [99]. The pressure sensor with a diaphragm configuration utilized a Wheatstone bridge circuit, which allowed it to achieve high performance with a fast time-response (90 ms), high resolution (sub-100 Pa), high sensitivity (0.0835 kPa−1), linearity, and temperature self-compensation. Such sensors have also been developed for use as in smart gloves with multipressure sensing, using three-dimensional printed hand molds as shown in (Figure 7(b)) for their design and fabrication. These results are promising for various applications, especially for hand motions such as grasping, moving, touching, squeezing, and gripping [99].Figure 7
(a) Photograph of a microfluid tactile diaphragm pressure sensor (left). Optical image of demonstrated microfluidic diaphragm sensor (middle). Schematic circuit diagram of an equivalent Wheatstone bridge (right). (b) Photograph of tactile sensing gloves embedded on devices (top left), diagram showing the real-time response recorded during the motion of thumb and index while grasping and releasing grape (bottom left). Photograph of tactile sensor worn while grasping (adapted with permission from Ref. [99] copyright 2017, John Wiley publishing group).
(a)
(b)


### 4.2. Human-Machine Interface

Recently, human-machine interaction has played a promising and effective role in wearable intelligent systems with wide applicability in robotics, mobility and bionics. In particular, the development of prosthetic hands or artificial skin based on human-like sensory systems with high stretchability and multifunctionality is important for intelligent robots and prosthetics [100, 101]. However, the achievements in prosthetic limbs cannot yet fully replace natural human limbs, because of differences between the digital signals of humans and those from artificial tactile sensors [37]. Thus, the first stage in data mining is typically feature interaction, as it provides a meaningful stimulation that can be transmitted by tactile sensing [102]. Moreover, mechanical flexibility of electronic systems is also required to develop devices based on human-machine interface.Several studies have been carried out on different applications of human-machine interaction such as manipulation, grasping, position recognition, and pressure evaluation [103, 104]. Another important direction of in the development human-machine interaction devices is methods based on deep learning. Recently, the deep learning technique was used to realize an extremely simple macroscale electronic skin without macro-, nano-, and micropatterns [105]. The deep learning network (DNN) architecture that has been used is shown in Figure 8. The reported deep learning-based method enables the use of a sample of bulky sheet (40 × 40 mm2) piezoresistive MWCNT-PDMS to play a role in the smart sensory devices (e.g., e-skin). The results show that the proposed e-skin based on deep learning obtained a 97.22% level of test accuracy for position recognition and had a reliable pressure estimation with a 3.12% RMSE and therefore approximated the capability of human skin. Furthermore, DNN-based e-skin showed high performance in pressure sensitivity and high spatial resolution (0.78 ± 0.44 mm) for position recognition [105]. The great potential of this revolutionary concept could open a new era for many fields, not only for e-skin application but also high-end applications such flexible keyboard, sign language interpreting, touch panels, and diagnosis motility [105].Figure 8
The schematic illustration of basic concept for e-skin and DNN architecture for reliable sensing. (a) A schematic elucidating the comparison between the human skin and the proposed e-skin. (b) The DNN architecture for tactile sensing (reprinted from Ref. [105] with permission, copyright 2017, Nature publishing group).


### 4.3. Minimally Invasive Surgery

The rapid developments in sensing technologies and robotics have encouraged the development of tactile sensors, especially for minimally invasive surgery (MIS). The tactile stimuli can be identified as any form of stimuli that can be transmitted by palpation. Recently, less invasive surgical robots have attracted great attention from both researcher groups and industry. Robot-assisted MIS enables surgeons to remotely control surgical tools or devices to perform surgical task that typically can be achieved by the surgeon. There are two main systems of RMIS which involves either Master-slave systems like the da Vinci Surgical System or autonomous systems (Curexo Technology Corporation, CA, USA) [106]. Surgeons depend on robotics assistance because they can provide high-resolution view which makes RMIS broadly applied in various operations such as thyroid surgeries, hysterectomies, and prostatectomies [107]. In the following, the most recent advances of tactile technologies in MIS are briefly summarized.


#### 4.3.1. Background of Tactile Systems in MIS

The first tactile feedback system reported for MIS is found in da Vinci surgical robots. The system contains an end-effector, which comprises piezoresistive force sensor and pneumatic balloon both used for creating tactile feelings, and it is driven with a semiautomated control system during robotic surgery. The tactile feedback in the system was evaluated by 16 novices and 4 experts peg transfer tasks, as reported by King et al. [108]. During the experiments, the force of effectors was measured from three blocks, but only the middle set provided tactile feedback.Control system of the da Vinci surgical robot was equipped with digital signal processor which enables the system to process the voltage signal and accordingly, based on signal conditioning electronics, determined the inflation level corresponding to the input voltage as well as for the generated an output signal to affect the inflation. The signal was relayed to the pneumatic balloons mounted at the master control to provide the thumb and index finger with feedback, as shown in Figure9. This system was the first complete tactile system known to be applied to commercial robotic surgical system; it is used to evaluate the direct tactile feedback in robotic surgery. Additionally, it has been adopted for various applications including prosthetics rehabilitation, surgical training, and robotic manipulation [108].Figure 9
(a) Modified FlexiForce sensor mounted on the top and bottom jaws of the Cadiere grasper for the da Vinci System. (b) The balloon tactile displays were mounted directly onto the da Vinci master control for thumb and index finger (adapted with permission from Ref. [108]; copyright 2009, IEEE publishing group).
(a)
(b)Recently, Kuchenbacker et al. [109] designed another acceleration feedback system named by vibrotactile feedback for the da Vinci system (Verro-touch). This design provides tactile feedback to the surgeon by measuring the vibrations caused by contact and directly reproduce them for the surgeon to feel by grasper tools. Similar to the system designed by King et al. [108], the signals are relayed to the master handles of the system. In order to detect high-frequency vibrations in the da Vinci S surgical system tools, that system included MEMS-based accelerometer, which held against the surface of the tool arms. Moreover, sensors were attached to the main controller manipulators in order not to interfere with arm or tool motion and measure acceleration signals and then transfer it to the main receiving unit (MRU). The MRU modifies, filters, and amplifies the measured acceleration signals before transferring to the vibration actuators. Accordingly, voice-coil actuators available containing commercial modules were mounted onto the da Vinci master handles in position to minimize interference and optimize signal transmission. The integrated system is shown in Figure 10 [109].Figure 10
Schematic of Verro-Touch installed on an Intuitive da Vinci S surgical system (adapted with permission from Ref. [109]; copyright 2010, Springer publishing group).Additionally, Verro-touch has made evaluation in the development of surgical system due specially in acceleration data, that can be obtained, compared, and contrasted at da Vinci needle driver (slave) and master controls. Furthermore, this evaluation included tool contact acceleration feedback via audio. That system enables measuring the texture of rough surfaces and the duration of the contact period.


#### 4.3.2. Trending Applications of Tactile Sensing in MIS

The progresses of tactile feedback systems encouraged many researchers to continue developing this field and explore new systems. Recently, McKinely et al. [110] presented a low-cost, single-use probe that is mounted in RMIS for localizing subcutaneous structure like blood vessel and tumors. It can sense the differences in probe tip force by continuous measurement of tip deflection with respect to a known spring constant by using Hall Effect sensor. In order to identify subcutaneous blood vessel phantoms, the probe used discrete measurements to generate a surface profile of unknown silicon tissue as shown in Figure 11. Additionally, sensor probe was characterized for machine tool with a computer numerical controlled tool with respect to various parameter setting such as silicone cylinders (1.58–4.75 mm), at varying subcutaneous depth (1–5 mm) with sliding speed (0.5–21 mm/s) and range of indentation depths (0–8 mm). Finally, this device can be used in surgery for localizing and searching a subcutaneous inclusion in large area surfaces where access is limited. Also, it can be mounted in RMIS devices [110].Figure 11
(a) Schematic illustration of disposable palpation mounted on da Vinci Research Kit (dVRK) needle tool. (b) Photograph shows the silicone tissue phantom with blood vessel inclusions and dermal phantom (adapted with permission from Ref. [110]; copyright 2015, IEEE publishing group).
(a)
(b)Later, Kim et al. [111] proposed a tactile device based on magneto-rheological (MR) sponge cell that is capable of representing the viscoelastic sensation for real organs. To realize tactile recognition of the proposed MR cell, a 3-axis robot was designed and manufactured. The robot enables measuring accurate tactile motion and feedback, due to the role of MR in mimicking the human fingertip with various materials. This can be important to control palpation exercises as shown in Figure 12. In addition, the presented MR sponge cell was used in testing a 3-axis robot on three different pork specimens: pork, pork rind, and pork heart, individually with various electromagnetic conditions. Their experiments showed the capability of the MR sponge to represent similar relaxation times and equilibrium forces depending on the voltage under which it was operated and the magnetic field. The MR cell sponge showed superior result; thus, it has been verified for surgical robotic applications [111].Figure 12
Schematic illustration of RMIS using MR sponge cell as a tactile sensor (adapted with permission from Ref. [111]; copyright 2016, Elsevier publishing group).Despite the significant progresses achieved in various tactile feedback systems, there are several issues that still limit the performance of tactile devices in surgical robotics systems. The lack of tactile information including visual, auditory, and temperature limits the performance of surgical system. Thus, much effort on not only the development of surgical tools but also providing tactile information to sensors other than touch is required. Schostek et al. [112] developed the artificial tactile feedback system (ATF) for laparoscopic surgery where a special software application was developed for processing data measurement from the tactile grasper to generate visual information. That system graphically provided different display modalities including pressure distribution which was used as force pictogram with the opening angle. In order to determine object size and ship, the software uses logarithm based on the opening angle of the forceps and pressure profile. However, the ATF of the grasper enables to enhance feedback information with significant positive effects on speed and accuracy. The sensor information is acquired and transmitted via Bluetooth to a remote computer. The major advantage of the ATF system is that it has relatively low-cost and easy to fabricate [112]. Having a tactile sensor during robotic surgery offers the surgeon the ability to feel perception of the surgical site and measure the magnitude of applied force to the tissue, which can accordingly help improve surgeon’s performance. Researchers have explored and developed various tactile sensors with different functions and mechanisms, which were mounted on the tip of skinny tools [113]. Although there is still great attention to continue research in this area toward high-performance applications with smart robotic systems, the ongoing research is promising with equipped flexible tactile sensing system during the surgery in the near future.


## 4.1. Multifunctional Artificial Skin

To achieve multifunctional e-skins that are capable of high perception from a variety of stimulations in static and dynamic cases, it is necessary to integrate a large number of macroscales of sensing elements based on thin flexible tactile substrates [83]. In addition, mechanosensory e-skins consisting of mechanical and stretchable sensor networks have exhibited a wide range of application in robotics [6, 84, 85], health-monitoring devices [86–89], and prosthetics [90–92].Recently, Hua et al. [93] reported a skin-inspired, highly stretchable sensor with conformable matrix networks (SCMNs), based on a structured polyimide network, as shown in Figure 5. The structure has the potential for integration into three-dimensional configurations. The SCMN illustrated successfully expands the e-skin sensing functionality in pressure, magnetic field, humidity, temperature, strain, and proximity. However, to achieve multifunctional sensing performance, the SCMN was comprised of 100 sensory nodes connected to each other by meandering wires as shown in Figure 6(a) [93]. Furthermore, when SCMN covered an artificial hand as e-skin, it exhibited superior expandability for multisensing. Figure 5(d) shows the spatial pressure achieved before and after 300% expansion of the SCMN, enabling it to identify position and pressure load and estimate the size of the loading object. The reported SCMN has various applications in human-machine interfaces, prosthetics, and health-monitoring technologies [93]. Chou et al. [94] introduced a chameleon-inspired stretchable e-skin, which is capable of interactive color changes and tactile sensing properties. Color easily can be changed and controlled through varying the applied pressure along with the applied pressure duration using ultraviolet-visible measurements.Figure 5
Schematic illustration of the skin-inspired high stretchable and conformable matrix networks. (a) Illustration showing layout of SCMNs: an integrated sensor array with eight functions; (b) photographic image indicating the fabricated polyimide network (10 × 10 array, scale bar: 5 mm); (c) schematic illustration of an SCMN as artificial skin equipped on hand (showing sensing expandability and adjustability); (d) plot of pressure mapping before and after the 300% expansion of an SCMN (Figures5(a)–5(d), reproduced from Ref. [93] with permission; copyright 2018; Nature publishing group).
(a)
(b)
(c)
(d)Figure 6
(a) Schematic illustration showing the concept of a chameleon-inspired e-skin, showing the structure of the neutral and oxidized states of electrochromic polymer, also schematic of the circuit layout (PS, pressure sensor; ECD, electrochromic device). (b) The schematic layout of interactive color-changeable e-skin (reproduced with permission from Ref. [94], copyright 2015, Nature publishing group).
(a)
(b)Chou et al. [94] introduced a chameleon-inspired stretchable e-skin, which is capable of interactive color changes and tactile sensing properties. Color easily can be changed and controlled through varying the applied pressure along with the applied pressure duration using ultraviolet-visible measurements. The idea of mimicking the sense-of-touch capacities of by human skin has motivated the use of fully stretchable organic electrochromic devices (ECDs) which can be fabricated with reduced consumption power and can be integrated with highly tunable resistive pressure sensing. As demonstrated in Figure 6, changes in pressure can be achieved by utilizing multiple and tunable colors in the flexible sensor. The proposed stretchable color changeable e-skin can maintain its skin color without any applied pressure, indicating its low power consumption [94].Another emerging aspect in the development of e-skin is the use of elastic materials for tactile sensor fabrication. PDMS has been widely studied for its virtues of biocompatibility, mechanical flexibility, and simplicity in the fabrication of microchannels (less than 100μm) [95–97]. Liquid and bioinspired materials are widely used due to their characterization, which includes high deformability and good electrical conductivity [98]. Gao et al. [99] demonstrated a microfluidic tactile diaphragm pressure sensor, characterized by embedded Galinstan (a eutectic alloy of gallium, indium, and tin) microchannels with a 70 μm width and a 70 μm height, as shown in Figure 7(a) [99]. The pressure sensor with a diaphragm configuration utilized a Wheatstone bridge circuit, which allowed it to achieve high performance with a fast time-response (90 ms), high resolution (sub-100 Pa), high sensitivity (0.0835 kPa−1), linearity, and temperature self-compensation. Such sensors have also been developed for use as in smart gloves with multipressure sensing, using three-dimensional printed hand molds as shown in (Figure 7(b)) for their design and fabrication. These results are promising for various applications, especially for hand motions such as grasping, moving, touching, squeezing, and gripping [99].Figure 7
(a) Photograph of a microfluid tactile diaphragm pressure sensor (left). Optical image of demonstrated microfluidic diaphragm sensor (middle). Schematic circuit diagram of an equivalent Wheatstone bridge (right). (b) Photograph of tactile sensing gloves embedded on devices (top left), diagram showing the real-time response recorded during the motion of thumb and index while grasping and releasing grape (bottom left). Photograph of tactile sensor worn while grasping (adapted with permission from Ref. [99] copyright 2017, John Wiley publishing group).
(a)
(b)


## 4.2. Human-Machine Interface

Recently, human-machine interaction has played a promising and effective role in wearable intelligent systems with wide applicability in robotics, mobility and bionics. In particular, the development of prosthetic hands or artificial skin based on human-like sensory systems with high stretchability and multifunctionality is important for intelligent robots and prosthetics [100, 101]. However, the achievements in prosthetic limbs cannot yet fully replace natural human limbs, because of differences between the digital signals of humans and those from artificial tactile sensors [37]. Thus, the first stage in data mining is typically feature interaction, as it provides a meaningful stimulation that can be transmitted by tactile sensing [102]. Moreover, mechanical flexibility of electronic systems is also required to develop devices based on human-machine interface.Several studies have been carried out on different applications of human-machine interaction such as manipulation, grasping, position recognition, and pressure evaluation [103, 104]. Another important direction of in the development human-machine interaction devices is methods based on deep learning. Recently, the deep learning technique was used to realize an extremely simple macroscale electronic skin without macro-, nano-, and micropatterns [105]. The deep learning network (DNN) architecture that has been used is shown in Figure 8. The reported deep learning-based method enables the use of a sample of bulky sheet (40 × 40 mm2) piezoresistive MWCNT-PDMS to play a role in the smart sensory devices (e.g., e-skin). The results show that the proposed e-skin based on deep learning obtained a 97.22% level of test accuracy for position recognition and had a reliable pressure estimation with a 3.12% RMSE and therefore approximated the capability of human skin. Furthermore, DNN-based e-skin showed high performance in pressure sensitivity and high spatial resolution (0.78 ± 0.44 mm) for position recognition [105]. The great potential of this revolutionary concept could open a new era for many fields, not only for e-skin application but also high-end applications such flexible keyboard, sign language interpreting, touch panels, and diagnosis motility [105].Figure 8
The schematic illustration of basic concept for e-skin and DNN architecture for reliable sensing. (a) A schematic elucidating the comparison between the human skin and the proposed e-skin. (b) The DNN architecture for tactile sensing (reprinted from Ref. [105] with permission, copyright 2017, Nature publishing group).


## 4.3. Minimally Invasive Surgery

The rapid developments in sensing technologies and robotics have encouraged the development of tactile sensors, especially for minimally invasive surgery (MIS). The tactile stimuli can be identified as any form of stimuli that can be transmitted by palpation. Recently, less invasive surgical robots have attracted great attention from both researcher groups and industry. Robot-assisted MIS enables surgeons to remotely control surgical tools or devices to perform surgical task that typically can be achieved by the surgeon. There are two main systems of RMIS which involves either Master-slave systems like the da Vinci Surgical System or autonomous systems (Curexo Technology Corporation, CA, USA) [106]. Surgeons depend on robotics assistance because they can provide high-resolution view which makes RMIS broadly applied in various operations such as thyroid surgeries, hysterectomies, and prostatectomies [107]. In the following, the most recent advances of tactile technologies in MIS are briefly summarized.


### 4.3.1. Background of Tactile Systems in MIS

The first tactile feedback system reported for MIS is found in da Vinci surgical robots. The system contains an end-effector, which comprises piezoresistive force sensor and pneumatic balloon both used for creating tactile feelings, and it is driven with a semiautomated control system during robotic surgery. The tactile feedback in the system was evaluated by 16 novices and 4 experts peg transfer tasks, as reported by King et al. [108]. During the experiments, the force of effectors was measured from three blocks, but only the middle set provided tactile feedback.Control system of the da Vinci surgical robot was equipped with digital signal processor which enables the system to process the voltage signal and accordingly, based on signal conditioning electronics, determined the inflation level corresponding to the input voltage as well as for the generated an output signal to affect the inflation. The signal was relayed to the pneumatic balloons mounted at the master control to provide the thumb and index finger with feedback, as shown in Figure9. This system was the first complete tactile system known to be applied to commercial robotic surgical system; it is used to evaluate the direct tactile feedback in robotic surgery. Additionally, it has been adopted for various applications including prosthetics rehabilitation, surgical training, and robotic manipulation [108].Figure 9
(a) Modified FlexiForce sensor mounted on the top and bottom jaws of the Cadiere grasper for the da Vinci System. (b) The balloon tactile displays were mounted directly onto the da Vinci master control for thumb and index finger (adapted with permission from Ref. [108]; copyright 2009, IEEE publishing group).
(a)
(b)Recently, Kuchenbacker et al. [109] designed another acceleration feedback system named by vibrotactile feedback for the da Vinci system (Verro-touch). This design provides tactile feedback to the surgeon by measuring the vibrations caused by contact and directly reproduce them for the surgeon to feel by grasper tools. Similar to the system designed by King et al. [108], the signals are relayed to the master handles of the system. In order to detect high-frequency vibrations in the da Vinci S surgical system tools, that system included MEMS-based accelerometer, which held against the surface of the tool arms. Moreover, sensors were attached to the main controller manipulators in order not to interfere with arm or tool motion and measure acceleration signals and then transfer it to the main receiving unit (MRU). The MRU modifies, filters, and amplifies the measured acceleration signals before transferring to the vibration actuators. Accordingly, voice-coil actuators available containing commercial modules were mounted onto the da Vinci master handles in position to minimize interference and optimize signal transmission. The integrated system is shown in Figure 10 [109].Figure 10
Schematic of Verro-Touch installed on an Intuitive da Vinci S surgical system (adapted with permission from Ref. [109]; copyright 2010, Springer publishing group).Additionally, Verro-touch has made evaluation in the development of surgical system due specially in acceleration data, that can be obtained, compared, and contrasted at da Vinci needle driver (slave) and master controls. Furthermore, this evaluation included tool contact acceleration feedback via audio. That system enables measuring the texture of rough surfaces and the duration of the contact period.


### 4.3.2. Trending Applications of Tactile Sensing in MIS

The progresses of tactile feedback systems encouraged many researchers to continue developing this field and explore new systems. Recently, McKinely et al. [110] presented a low-cost, single-use probe that is mounted in RMIS for localizing subcutaneous structure like blood vessel and tumors. It can sense the differences in probe tip force by continuous measurement of tip deflection with respect to a known spring constant by using Hall Effect sensor. In order to identify subcutaneous blood vessel phantoms, the probe used discrete measurements to generate a surface profile of unknown silicon tissue as shown in Figure 11. Additionally, sensor probe was characterized for machine tool with a computer numerical controlled tool with respect to various parameter setting such as silicone cylinders (1.58–4.75 mm), at varying subcutaneous depth (1–5 mm) with sliding speed (0.5–21 mm/s) and range of indentation depths (0–8 mm). Finally, this device can be used in surgery for localizing and searching a subcutaneous inclusion in large area surfaces where access is limited. Also, it can be mounted in RMIS devices [110].Figure 11
(a) Schematic illustration of disposable palpation mounted on da Vinci Research Kit (dVRK) needle tool. (b) Photograph shows the silicone tissue phantom with blood vessel inclusions and dermal phantom (adapted with permission from Ref. [110]; copyright 2015, IEEE publishing group).
(a)
(b)Later, Kim et al. [111] proposed a tactile device based on magneto-rheological (MR) sponge cell that is capable of representing the viscoelastic sensation for real organs. To realize tactile recognition of the proposed MR cell, a 3-axis robot was designed and manufactured. The robot enables measuring accurate tactile motion and feedback, due to the role of MR in mimicking the human fingertip with various materials. This can be important to control palpation exercises as shown in Figure 12. In addition, the presented MR sponge cell was used in testing a 3-axis robot on three different pork specimens: pork, pork rind, and pork heart, individually with various electromagnetic conditions. Their experiments showed the capability of the MR sponge to represent similar relaxation times and equilibrium forces depending on the voltage under which it was operated and the magnetic field. The MR cell sponge showed superior result; thus, it has been verified for surgical robotic applications [111].Figure 12
Schematic illustration of RMIS using MR sponge cell as a tactile sensor (adapted with permission from Ref. [111]; copyright 2016, Elsevier publishing group).Despite the significant progresses achieved in various tactile feedback systems, there are several issues that still limit the performance of tactile devices in surgical robotics systems. The lack of tactile information including visual, auditory, and temperature limits the performance of surgical system. Thus, much effort on not only the development of surgical tools but also providing tactile information to sensors other than touch is required. Schostek et al. [112] developed the artificial tactile feedback system (ATF) for laparoscopic surgery where a special software application was developed for processing data measurement from the tactile grasper to generate visual information. That system graphically provided different display modalities including pressure distribution which was used as force pictogram with the opening angle. In order to determine object size and ship, the software uses logarithm based on the opening angle of the forceps and pressure profile. However, the ATF of the grasper enables to enhance feedback information with significant positive effects on speed and accuracy. The sensor information is acquired and transmitted via Bluetooth to a remote computer. The major advantage of the ATF system is that it has relatively low-cost and easy to fabricate [112]. Having a tactile sensor during robotic surgery offers the surgeon the ability to feel perception of the surgical site and measure the magnitude of applied force to the tissue, which can accordingly help improve surgeon’s performance. Researchers have explored and developed various tactile sensors with different functions and mechanisms, which were mounted on the tip of skinny tools [113]. Although there is still great attention to continue research in this area toward high-performance applications with smart robotic systems, the ongoing research is promising with equipped flexible tactile sensing system during the surgery in the near future.


## 4.3.1. Background of Tactile Systems in MIS

The first tactile feedback system reported for MIS is found in da Vinci surgical robots. The system contains an end-effector, which comprises piezoresistive force sensor and pneumatic balloon both used for creating tactile feelings, and it is driven with a semiautomated control system during robotic surgery. The tactile feedback in the system was evaluated by 16 novices and 4 experts peg transfer tasks, as reported by King et al. [108]. During the experiments, the force of effectors was measured from three blocks, but only the middle set provided tactile feedback.Control system of the da Vinci surgical robot was equipped with digital signal processor which enables the system to process the voltage signal and accordingly, based on signal conditioning electronics, determined the inflation level corresponding to the input voltage as well as for the generated an output signal to affect the inflation. The signal was relayed to the pneumatic balloons mounted at the master control to provide the thumb and index finger with feedback, as shown in Figure9. This system was the first complete tactile system known to be applied to commercial robotic surgical system; it is used to evaluate the direct tactile feedback in robotic surgery. Additionally, it has been adopted for various applications including prosthetics rehabilitation, surgical training, and robotic manipulation [108].Figure 9
(a) Modified FlexiForce sensor mounted on the top and bottom jaws of the Cadiere grasper for the da Vinci System. (b) The balloon tactile displays were mounted directly onto the da Vinci master control for thumb and index finger (adapted with permission from Ref. [108]; copyright 2009, IEEE publishing group).
(a)
(b)Recently, Kuchenbacker et al. [109] designed another acceleration feedback system named by vibrotactile feedback for the da Vinci system (Verro-touch). This design provides tactile feedback to the surgeon by measuring the vibrations caused by contact and directly reproduce them for the surgeon to feel by grasper tools. Similar to the system designed by King et al. [108], the signals are relayed to the master handles of the system. In order to detect high-frequency vibrations in the da Vinci S surgical system tools, that system included MEMS-based accelerometer, which held against the surface of the tool arms. Moreover, sensors were attached to the main controller manipulators in order not to interfere with arm or tool motion and measure acceleration signals and then transfer it to the main receiving unit (MRU). The MRU modifies, filters, and amplifies the measured acceleration signals before transferring to the vibration actuators. Accordingly, voice-coil actuators available containing commercial modules were mounted onto the da Vinci master handles in position to minimize interference and optimize signal transmission. The integrated system is shown in Figure 10 [109].Figure 10
Schematic of Verro-Touch installed on an Intuitive da Vinci S surgical system (adapted with permission from Ref. [109]; copyright 2010, Springer publishing group).Additionally, Verro-touch has made evaluation in the development of surgical system due specially in acceleration data, that can be obtained, compared, and contrasted at da Vinci needle driver (slave) and master controls. Furthermore, this evaluation included tool contact acceleration feedback via audio. That system enables measuring the texture of rough surfaces and the duration of the contact period.


## 4.3.2. Trending Applications of Tactile Sensing in MIS

The progresses of tactile feedback systems encouraged many researchers to continue developing this field and explore new systems. Recently, McKinely et al. [110] presented a low-cost, single-use probe that is mounted in RMIS for localizing subcutaneous structure like blood vessel and tumors. It can sense the differences in probe tip force by continuous measurement of tip deflection with respect to a known spring constant by using Hall Effect sensor. In order to identify subcutaneous blood vessel phantoms, the probe used discrete measurements to generate a surface profile of unknown silicon tissue as shown in Figure 11. Additionally, sensor probe was characterized for machine tool with a computer numerical controlled tool with respect to various parameter setting such as silicone cylinders (1.58–4.75 mm), at varying subcutaneous depth (1–5 mm) with sliding speed (0.5–21 mm/s) and range of indentation depths (0–8 mm). Finally, this device can be used in surgery for localizing and searching a subcutaneous inclusion in large area surfaces where access is limited. Also, it can be mounted in RMIS devices [110].Figure 11
(a) Schematic illustration of disposable palpation mounted on da Vinci Research Kit (dVRK) needle tool. (b) Photograph shows the silicone tissue phantom with blood vessel inclusions and dermal phantom (adapted with permission from Ref. [110]; copyright 2015, IEEE publishing group).
(a)
(b)Later, Kim et al. [111] proposed a tactile device based on magneto-rheological (MR) sponge cell that is capable of representing the viscoelastic sensation for real organs. To realize tactile recognition of the proposed MR cell, a 3-axis robot was designed and manufactured. The robot enables measuring accurate tactile motion and feedback, due to the role of MR in mimicking the human fingertip with various materials. This can be important to control palpation exercises as shown in Figure 12. In addition, the presented MR sponge cell was used in testing a 3-axis robot on three different pork specimens: pork, pork rind, and pork heart, individually with various electromagnetic conditions. Their experiments showed the capability of the MR sponge to represent similar relaxation times and equilibrium forces depending on the voltage under which it was operated and the magnetic field. The MR cell sponge showed superior result; thus, it has been verified for surgical robotic applications [111].Figure 12
Schematic illustration of RMIS using MR sponge cell as a tactile sensor (adapted with permission from Ref. [111]; copyright 2016, Elsevier publishing group).Despite the significant progresses achieved in various tactile feedback systems, there are several issues that still limit the performance of tactile devices in surgical robotics systems. The lack of tactile information including visual, auditory, and temperature limits the performance of surgical system. Thus, much effort on not only the development of surgical tools but also providing tactile information to sensors other than touch is required. Schostek et al. [112] developed the artificial tactile feedback system (ATF) for laparoscopic surgery where a special software application was developed for processing data measurement from the tactile grasper to generate visual information. That system graphically provided different display modalities including pressure distribution which was used as force pictogram with the opening angle. In order to determine object size and ship, the software uses logarithm based on the opening angle of the forceps and pressure profile. However, the ATF of the grasper enables to enhance feedback information with significant positive effects on speed and accuracy. The sensor information is acquired and transmitted via Bluetooth to a remote computer. The major advantage of the ATF system is that it has relatively low-cost and easy to fabricate [112]. Having a tactile sensor during robotic surgery offers the surgeon the ability to feel perception of the surgical site and measure the magnitude of applied force to the tissue, which can accordingly help improve surgeon’s performance. Researchers have explored and developed various tactile sensors with different functions and mechanisms, which were mounted on the tip of skinny tools [113]. Although there is still great attention to continue research in this area toward high-performance applications with smart robotic systems, the ongoing research is promising with equipped flexible tactile sensing system during the surgery in the near future.


## 5. Conclusions

The skin has been regarded as the largest sensory system with which human surgical experts can carry out different intelligent tasks such as surgery. Thus, having a replica in intelligent machines that have been developed for robotic surgery remains a thing of focus in the world today. In this study, we reported the survey that was carried out about the application of tactile-sensing systems in some related fields of biomedical engineering. The review covered the main aspects that are essential to mimicking human’s sense-of-touch mechanism when building multifunctional electronic skin and haptic systems in medical surgical robots. These include description of the fundamental theories about human skin and the role of mechanoreceptors in tactile sensation, the design structures of mechanoreceptors with the basic requirements needed for tactile sensing, some techniques that are employed for tactile transduction, the state-of-the-art and novel materials commonly used for sensing, and potential applications of tactile sensors, specifically towards building the human’s sense-of-touch for minimally invasive robotic surgery. Finally, this survey contains some of the limitations and challenges of each technical aspect and some notable improvements that have been made in the research trends.



---

*Source: 4047937-2020-01-08.xml*