# Proximity Perception in Human-Centered Robotics: A Survey on Sensing Systems and Applications

Stefan Escaida Navarro, Stephan Mühlbacher-Karrer, Hosam Alagi, Hubert

Zangl, Keisuke Koyama, Björn Hein, Christian Duriez, Joshua R Smith

► To cite this version:

Stefan Escaida Navarro, Stephan Mühlbacher-Karrer, Hosam Alagi, Hubert Zangl, Keisuke Koyama, et al.. Proximity Perception in Human-Centered Robotics: A Survey on Sensing Systems and Applications. IEEE Transactions on Robotics, 2022, 38, pp.1599-1620. 10.1109/TRO.2021.3111786. hal-03156830v2

HAL Id: hal-03156830

https://inria.hal.science/hal-03156830v2

Submitted on 17 Aug 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Proximity Perception in Human-Centered Robotics: A Survey on Sensing Systems and Applications

Stefan Escaida Navarro*, Stephan Mühlbacher-Karrer*, Hosam Alagi*, Hubert Zangl, Keisuke Koyama, Björn Hein, Christian Duriez, and Joshua R. Smith

***Abstract***—Proximity perception is a technology that has the potential to play an essential role in the future of robotics. It can fulfill the promise of safe, robust, and autonomous systems in industry and everyday life, alongside humans, as well as in remote locations in space and underwater. In this survey paper, we cover the developments of this field from the early days up to the present, with a focus on human-centered robotics. Here, proximity sensors are typically deployed in two scenarios: first, on the exterior of manipulator arms to support safety and interaction functionality, and second, on the inside of grippers or hands to support grasping and exploration. Starting from this observation, we propose a categorization for the approaches found in the literature. To provide a basis for understanding these approaches, we devote effort to present the technologies and different measuring principles that were developed over the years, also providing a summary in form of a table. Then, we show the diversity of applications that have been presented in the literature. Finally, we give an overview of the most important trends that will shape the future of this domain.

***Index Terms***—Perception for Grasping and Manipulation; Collision Avoidance; Reactive and Sensor-Based Planning; Object Detection, Segmentation and Categorization

<figure><img src="image_2.png" /><figcaption>Fig. 1. A robot skin with proximity sensing capability can be deployed in scenarios with intense human-robot interaction and collaboration. The skin helps closing the gap between vision-based perception and tactile/force perception.</figcaption></figure>

## I. INTRODUCTION

IN the current robotics research landscape, a lot of effort is still dedicated to overcoming the challenges posed by unstructured environments. Areas, such as medicine, health care, agriculture, Industry 4.0, and exploration endeavors in space as well as underwater are awaiting to profit from the robotics technologies currently in development. Furthermore, in terms of unstructured environments or situations, one of the main challenges is to develop robotics technologies that enable a safe and reliable interaction with the human. At the same

Stefan Escaida Navarro and Christian Duriez are with Inria Lille-Nord Europe, France, e-mail: stefan.escaida-navarro@inria.fr

Stephan Mühlbacher-Karrer is with JOANNEUM RESEARCH ROBOTICS, Institute for Robotics and Mechatronics, Klagenfurt, Austria, e-mail: stephan.muehlbacher-karrer@joanneum.at

Hosam Alagi is with Karlsruhe Institute of Technology, Institute for Anthropomatics and Robotics - Intelligent Process Automation and Robotics Lab (IAR - IPR), Karlsruhe, Germany, e-mail: hosam.alagi@kit.edu

Björn Hein is with Karlsruhe Institute of Technology, IAR - IPR, Karlsruhe, Germany, e-mail: bjoern.hein@kit.edu and with University of Applied Sciences Karlsruhe, Germany, e-mail: bjoern.hein@hs-karlsruhe.de

Hubert Zangl is with Alpen-Adria-Universität Klagenfurt, Institute of Smart System Technologies, Sensors and Actuators, Klagenfurt, Austria, e-mail: hubert.zangl@au.at

Keisuke Koyama is with Osaka University, Department of Systems Innovation, Graduate School of Engineering Science, Osaka, Japan, e-mail: koyama@sys.es.osaka-u.ac.jp

Josuha R. Smith is with University of Washington, Allen School of Computer Science and Engineering, Department of Electrical Engineering, Seattle, United States of America, e-mail: jrs@cs.washington.edu

time, the functionality and intelligence provided by the robot system must justify the investment, meaning its autonomous behavior, oftentimes in environments made for humans, must contribute real value. A hallmark of robust and efficient robot behavior is that task execution does not need to be interrupted in the presence of emergent events. A technology that is capable of addressing these challenges is proximity perception. It has been developed over the years, with first, impactful applications being shown in the late 1980s and early 1990s, which were sparked by seminal developments in robot control. Proximity perception is complementary to the main robotics perceptive modalities of vision and touch. Its use is often motivated by closing the perception gap left by occlusions and blind spots as well as by dealing with pose uncertainty of the robot with respect to objects and its environment. Therefore, one of the big challenges in this domain is to find sensor designs that can coexist with the main existing modalities of vision and touch.

In human-centered robotics, the typical applications of proximity perception can be broadly divided into two categories: the first one is pertaining a sensitive skin covering the links of a robot manipulator for safety and interaction functionality, which we call *applications of type I (AT-I)*. The second one is where a robot gripper or hand is equipped with sensors to support grasping and exploration tasks, which we call *applications of type II (AT-II)*. In Fig. 1, a typical scenario of human-robot interaction and collaboration is illustrated

<footnote>* These authors contributed equally.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2. Reactive preshaping to an object based on proximity perception has three characteristic phases: (1) object detection by vision and approaching of the hand to the object, (2) detection of the object by the hand's proximity sensors, start of closed-loop control and occlusion of the object in the camera view, and (3) finalized preshaping control, where the fingers and palm of the hand are aligned with the object.</figcaption></figure>

(AT-I). As the human approaches the robot, the view of the camera monitoring the robot and its workspace will become increasingly occluded. A tactile skin covering the robot is not adequate to handle this perception gap in general. This is because detecting the human or the environment only when contact is established, implies operating the robot at very low velocities, thus undermining the purpose of installing such a system in the first place. To address scenarios like these, a sensitive skin with proximity perception capabilities has been proposed by several authors. In Fig. 2, a typical scenario for grasping supported by proximity perception shown (AT-II). Since the robotic hand can detect an object’s surface before touch is established, a *pre-touch* closed-loop control can be implemented to adjust the hand posture during this phase. This is called *reactive preshaping* and can also have a diversity of use-cases. In Fig. 2, the three typical phases of this procedure are shown. Proximity perception also has the potential to play an important role in robotic solutions that are compliant with norms and standards, such as ISO/TS 15066 for the operation of collaborative robots.

In this paper, we want to provide an up-to-date perspect-
tive on the field of proximity perception in human-centered
robotics as well as an introduction to the principles and
technologies developed. Proximity perception in areas such
as autonomous vehicles or unmanned aerial vehicles (UAVs)
usually aim at autonomous driving or flying and thus address
larger distances and speeds and avoidance of contact and
interaction with objects and humans. Recent surveys that
include discussions on proximity perception in these domains
are [1] (millimeter wave Radar), [2] (sensor fusion), both for
autonomous driving, and [3] for indoor localization of UAVs.
In human-centered robotics, proximity perception is related

to short distances between humans and robots and aiming
for improving human-robot interaction as well as safety.
Nonetheless, many ideas presented in this paper can be valid
in the automotive domain and for UAVs as well, especially
those about the sensing principles and their applicability. We
hope to give the readers a starting point to understand the
principles and applications of proximity perception that have
been developed in human-centered robotics. Furthermore, we
want to provide a perspective on what, in our opinion, are the
important trends that will shape the developments within the
next years.

The main contributions of this paper are as follows:

* We provide an introduction to the concept of proximity perception and an overview of the possible use-cases. We provide a categorization according to the application types and the complexity of the implemented behavior (Sec. II). We use this categorization throughout the paper to organize the different works found in the literature.

* We give an introduction to the working principles of the most important proximity sensor designs (capacitive, optical, radar, etc.) and give a review of the related work in the context of robotics (Sec. III).

* We cover what use-cases for proximity perception have been studied in robotics research and industrial robotics. Here, we give a detailed account of the two most important basic applications, AT-I and AT-II (Sec. IV, see also Figs. 1, 2, and 4). We start by giving a historical account and cover the basic forms of behavior possible to the more advanced, cognitive approaches.

* We provide a systematic comparison of the technologies from the field summarized in Table I

* Finally, we project the current developments into the future and finish the paper with concluding remarks (Sec. V and VI).

## II. PROXIMITY SENSOR: CHARACTERIZATION, APPLICATIONS AND SAFETY CONSIDERATIONS

### A. Characterization

Providing a concise characterization of proximity sensors is challenging. One thing common to all proximity sensors is that they detect objects without physical contact. However, this alone does not distinguish them from cameras, which is problematic, as both modalities are considered to be complementary. To address this, we propose a series of attributes that generally characterize proximity sensor designs. At the same time, not all of the attributes need to be present at once in a particular case. Thus, proximity sensors provide non-contact detection of objects *and* more often than not

* use active measurement principles, i. e. they probe the nearby environment to detect an object’s presence, item provide limited sensing range and even small detection ranges can be considered to be useful,

* are *skinlike*, i. e. they can be deployed on surfaces such as robot arm segments as well as fingers where they can form a network of sensing elements,
<figure><img src="image_4.png" /><figcaption>Fig. 3. Definition of the proximity sensing range.</figcaption></figure>

* are suitable for being highly integrated into the sensory-motor functionality of the robot, enabling reflex-like behaviors due to low latency measurements,

* are used to handle occlusions in vision systems, i. e. are complementary to vision,

* are used to supervise approaching objects which are bound to enter in contact with the robot, i. e. are complementary to tactile sensing.

In Fig. 3, we propose a definition for the sensing range of a proximity sensor. Any detection distance below 50 cm can be considered to be within the proximity range. This limit is not strict, but in human-robot interaction (HRI) and human-robot collaboration (HRC), this is an approximate distance at which visual occlusions begin to become problematic. As discussed later in Sec. II-C, this is a similar range in which monitoring of separation distance is relevant for compliance with safety standards. At larger distances, i. e. mid-range and long-range perception, other technologies (LIDAR, long-range stereo vision, etc.) can provide better performance in workspace surveillance or for providing HRI functionality. This is especially true here because the requirements on reactivity can be relaxed at larger distances. Furthermore, it is interesting to consider the contributions by anthropologist Edward T. Hall, who describes the *intimate space* of humans as part of his studies on *proxemics* [4]. The intimate space starts at a distance of typically 45 cm, which is also close to the range proposed above. Therefore, proximity sensing is easy to understand from the perspective of humans, as they can intuitively relate this perception to the “intimate space” of the robot by analogy.

Finally, a distinction can also be made for a range below 10 cm that we call *pre-touch-range*. This is the type of sensing that precedes contact interactions, for instance during grasping. Here it is especially important to have uninterrupted sensing until contact. Some sensor designs might not feature a long detection range, but the sensing capabilities provided are still useful for closed-loop control of finger and hand posture, which is executed until touch is established. A more in-depth discussion of the available proximity sensing technologies is provided in Sec. III. In Fig. 5, an example of a modern humanoid robot covered in a multi-modal skin is shown, displaying many of the characteristics discussed in this section.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Categorization of some use-cases for proximity sensors in robotics according to the possible application types (AT-I and AT-II) and behavior types (BT-I and BT-II)</figcaption></figure>

## B. Application and Behavior Types

To talk about proximity sensors in human-centered robotics as a whole, it is useful to consider first a categorization of the possible applications and desired behaviors. In the introduction, we already mentioned that a broad classification of applications into two categories is possible: the ones relating to safety and HRI/HRC and the ones relating to preshaping and grasping (Figs. 1 and 2). Beyond this, automated behaviors based on proximity sensors can be organized according to their conceptual complexity and how instantaneous their effect is on the movement of the robot. One example for a low-complexity behavior is a *safety stop*, i. e. enabling the brakes of the robot based on a sensor signal surpassing a threshold value. This behavior is closely tied to the update-rate of the sensors and the low-level robot controller. In that sense, it can be called *reactive* or *reflex-like*. Modern collaborative robots, e. g. the Franka Emika Panda [5] or the KUKA LBR iiwa [6] have control loop cycles of $t_{cl} = 1$ ms. Thus, the closer the response time of the proximity sensor is to $t_r < t_{cl}$, the better. An example of high-complexity behavior is object exploration. It involves managing an object model as well as a planner to complete this model with purposeful exploration steps, resulting in a robot behavior that is executed in several phases and over a longer time compared to the basic control loop cycle times. This behavior is also characterized by being executed at different layers, reaching, as mentioned, up to the planning and cognitive components in the robot’s architecture. Fig. 4 illustrates the categorizing of applications and behaviors we propose as well as providing some examples (not an exhaustive list). As a result, we have a broad classification of applications into two types, AT-I (left) and AT-II (right), and behaviors into two types, reactive or reflex-like behavior (low complexity) (BT-I) (bottom) and cognitive or model-based behavior (high complexity) (BT-II) (top). In general, BT-I will appear as subsystems of BT-II.

## C. Safety Considerations and Norm Compliance

From the safety perspective, a proximity sensor deployed on a collaborative robot in an industrial environment has
<figure><img src="image_6.png" /><figcaption>Fig. 5. Left: Researchers at the Technical University of Munich, Chair for Cognitive Systems, have developed a multi-modal, modular skin, which they have deployed on their robot H-1. Right: A single cell of the multi-modal sensor. (Copyright left picture. A. Eckert / TU München. Right picture ©2019 IEEE) [7]. Both reprinted with kind permission of the authors.)</figcaption></figure>

to fulfill the requirements of the ISO/TS 15066 complying with the ISO 10218 for robots and robotic devices, as well as the performance level defined in the safety of machinery ISO 13849. Proximity sensing hardware is very well suited to operate a collaborative robot in the speed and separation monitoring mode as defined in the ISO/TS 15066 to monitor the protective separation distance $S_p$ between a human and a robot's surface. According to the standard, the following equation has to be fulfilled during the operation mode:

$$S_p(t = t_0) = v_h(T_r + T_s) + v_r T_r + S_s + C_i + Z_d + Z_r, \quad (1)$$

where $v_h$ is the human speed (if not monitored, $v_h = 1.6 \text{ m/s}$) $T_r$ is the reaction time of the robot, $T_s$ is the robot stopping time, $v_r$ is the robot speed, $S_s$ is the robot stopping distance, $C_i$ is the intrusion distance, $Z_d$ is the position uncertainty of the human and $Z_r$ is the position uncertainty of the robot.

To provide an illustrative example for a state of the art collaborative robot, we look at the UR10e series. It has a control loop cycle of 500 Hz and the safety parameters can be configured $v_r = 5 \text{ m/s}$ (max end-effector speed) $T_r = 4$ ms (two control loop cycles), $T_s = 100$ ms, $S_s = 50$ mm, $Z_r = 0.05$ mm. This configuration results in separation distance of $S_p = 0.236 \text{ m}$ excluding the uncertainty of the position of the human and the intrusion distance, as this depends on the sensor parameters monitoring the area.

### III. MEASUREMENT PRINCIPLES FOR PROXIMITY SENSING

In this section, we will give an introduction to the main physical principles available to implement proximity sensing. The idea is to be able to ease the process of reviewing articles by starting with an explanation of the basics. This will also help us in the systematization in Table I. There, we use the abbreviations introduced in this section. Here, we will already do a review of some representative works in the field focusing on how the proximity sensing technology is implemented. This section is closely linked to Sec. IV, where the details of

the implemented applications are discussed. We try to cross-reference the most relevant relationships. However, favoring readability, cross-referencing is not exhaustive.

#### A. Capacitive Sensing

In this section, we provide a short introduction to capacitive sensing, its measurement techniques as well as the work done dedicated to capacitive proximity sensing in robotics. The capacitive measurement principle has been widely adopted in various other fields and has well-established applications in research and industry, [8] gives an overview on basic principles and applications. A recent survey paper reviewing capacitive sensing for human-computer interaction (HCI) is due to Grosse-Poppendahl et al. [9]. In this section, we will concentrate on the technologies related to robotics.

The capacitive proximity sensing principle uses electrically conductive elements (electrodes) to generate and measure electric fields. Objects interfere with this electric field when they approach the electrodes and the observed changes are utilized to estimate their distance as well as properties of the object, such as its material. Therefore, capacitive sensing is called electric field sensing in some literature. Essentially, the capacitance between the sensor and an object depends on the geometry of an object, its relative pose to the electrode(s), its coupling to electrical ground, and its material. The nonlinear relation between the relative pose and the material of the object to the measured signal presents a significant challenge for developing signal processing for and applications based on capacitive sensing. However, its ubiquitous use for HCI is explained by the fact that humans can be detected reliably.

Commonly, alternating electric potentials are used to generate the electrical field and displacement currents that are proportional to the capacitances are measured. Another popular approach is measuring the oscillation frequency in an oscillator-circuit based on the capacitance of interest. Typically, the alternating frequency is rather low, i. e. not much larger than 1 MHz, and thus the corresponding wavelength is long compared to the size of the electrodes such that wave-propagation effects can be neglected and the quasi-static assumption can be used.

Mainly two different modes of operations are distinguished for capacitive sensors: The first mode, called capacitive single-ended mode (C-SE), uses the influence of an object on the capacitance between sensor electrodes and distant ground (see Fig. 6, left). This mode is also called self-capacitive mode or shunt mode in literature. The second mode, called mutual-capacitance mode (C-M), sometimes also differential mode, uses the influence of an object on the capacitances between electrodes of the sensor (see Fig. 6, right). Both modes are widely used. An advantage of the single-ended mode is a typically higher capacitance and thus a higher signal to noise ratio. The mutual capacitance mode has the advantage of providing more independent measurements, as all the combinations between electrodes can be measured. Therefore, electrical capacitance tomography (ECT) (e. g. [10]) that allows obtaining images of material distributions usually utilize the latter, sometimes in combination with the single-ended mode.
<figure><img src="image_7.png" /><figcaption>Fig. 6. Capacitive single-ended mode (C-SE) (left): The electrode is driven with an electrical potential and generating an electrical field between the measurement electrode and the object. Mutual-capacitance mode (C-M) (right): the right electrode transmitter (Tx) is driven, generating an electrical field, which ends at receiver (Rx). A conductive object within the measurement range can block/shield the field lines (green) between the electrodes.</figcaption></figure>

Fig. 6 (left) shows the self-capacitance mode and illustrates the electrical field between a transmitter electrode and an object. Some electrical field lines end in the ground representing unwanted coupling to the environment, also called parasitic effects. A second layer can be used as an active guard, which has the same electrical potential as the transmitter. This is a popular method to reduce parasitic effects and to actively shape the measurement lobe of the electrode. In this mode, approaching objects in general increase the measured capacitance. Conductive objects with strong coupling to ground show a strong sensing effect. Having reasonable conductivity and body parts that offer large areas, humans behave like such objects and are thus well recognized, achieving a high coupling capacitance in the range of 100 pF. Small conductive objects and non-conductive objects show lower sensing effects and are more difficult to detect. In case that the sensors are intended to detect humans, the low sensitivity with respect to small objects can be an advantage as this implies that contamination of the electrode surface e. g. with water due to condensation or accumulation of dirt will have little impact on the sensor performance.

Fig. 6 (right) shows the mutual capacitance mode and illustrates the electrical field between a transmitter electrode, an object, and a receiver electrode. Conductive objects with high coupling to ground (such as humans) can partially shield the field and thus – in contrast to the single-ended mode – reduce the capacitance between the electrodes. If coupling to ground is low, then the opposite effect can occur and the coupling increases due to polarization within the object. This is in particular common for non-conductive objects as well as for higher frequencies of the excitation frequencies. In mutual capacitance mode, shaping of the field can be achieved using passive ground electrodes.

Using the electrostatic representation, the relation between charges Q on the electrodes and the potentials Φ on the electrodes can be described as

$$ \begin{pmatrix} Q_1 \\ \vdots \\ Q_n \end{pmatrix} = \begin{pmatrix} C_{1,1} & \cdots & -C_{N,1} \\ \vdots & \ddots & \vdots \\ -C_{1,N} & \cdots & C_{N,N} \end{pmatrix} \begin{pmatrix} \Phi_1 \\ \vdots \\ \Phi_n \end{pmatrix} = C \begin{pmatrix} \Phi_1 \\ \vdots \\ \Phi_n \end{pmatrix} \quad (2) $$

where $C_{i,j}$ represents the capacitance between electrode i and electrode j and the diagonal elements represent the capacitances between the electrode i, ground (reference potential)

and all other electrodes. The self-capacitance mode typically determines the diagonal elements, whereas the mutual capacitance mode determines off-diagonal elements. To perform the inversion and obtain the capacitances, linearly independent excitation patterns are needed to determine the full matrix C.

1) Early Capacitive Technologies in Robotics: One of the first applications of proximity (and tactile) sensor for robots goes back to 1988 presented by Yamada at el. [11]. Two links of a manipulator are equipped with mutual-capacitance sensors (C-M) and the capability of detecting conductive and insulating approaching obstacles is demonstrated. It is established that conductive objects are detected more reliably than non-conductive objects. In the 1990s, with the same motivation of avoiding obstacles, other groups worked on capacitive sensing, for instance focusing on the electrode design, like Vranish et al. in [12]. This technology was evaluated for use in collision avoidance by Wegerif et al. [13], but was dropped in favor of infrared (IR) sensing (see also Secs. III-B1 and IV-A1). However, authors like Novak and Feddema favored capacitive sensing for these kinds of approaches, developing large sensor arrays to cover greater areas on robot arms [14] [15] (see also Sec. IV-A1).

2) Capacitive Sensing for AT-I: Since the first developments, many groups in the robotics community have worked on capacitive based proximity sensing. On the one hand, capacitive sensing has been further investigated to cover robot links (AT-I, see Sec. IV-A). In [16] and [10], the authors propose a mutual-capacitance sensor (C-M) having several electrodes for detecting obstacles on robot links. In [16], the ability to detect non-conductive materials due to the mutual-capacitance sensing principle is highlighted. Collision avoidance on a mobile robot based on capacitive proximity sensing for a variety of materials was shown in [17]. Covering robot links with modular single-ended capacitive proximity (C-SE) skins for collision avoidance and HRI is proposed in [18], [19], [20], [21]. These works show the potential for these technologies, especially for HRI. However, for example in [19], it is discussed that C-SE is not suitable enough to detect insulating materials for the intended application of collision avoidance. As a solution, in [20], [21], a combination with time-of-flight (O-ToF) sensing (see Sec. III-B4) is proposed to compensate for the shortcomings of C-SE sensors. Capacitive proximity sensing has also been adopted by some robotics companies to implement safety-features, especially in HRI. Examples are BOSCH APAS [22], [23], FOGALE Robotics [24], [25] as well as MRK-Systeme [26] (see Sec. IV-D).

3) Capacitive Sensing in AT-II: While the first developments in capacitive proximity sensing in robotics were focused on AT-I, more recently, AT-II have started to gain interest. Smith et al. showed the use of multi-transmitter and receiver in robotic grippers [27], [28], [29]. The placement of transmitter and receiver electrodes allows covering diverse sensing ranges for the different aspects of reactive preshaping, i. e. short, mid, and long-range sensing for adjusting the poses of the fingers and the palm. Furthermore, they show it is possible to detect the grounding state of an object as it changes due to a human holding on to it. Göger et al. show the integration of a tactile proximity sensor into fingertips [30]. In [31], [32],
Escaida Navarro et al. show the integration of the sensor presented in [33] into a two-jaw gripper for reactive preshaping and telemanipulation with force-feedback. In [31], [32], the limitations of self-capacitance sensing with regards to material properties are on display. This is addressed to some extent in [34] with mutual-capacitance sensing and flexible spatial resolution. In [35], sensors are integrated into the fingers of a humanoid robot, which help in finding an object's fill state. These works have shown the feasibility of integrating capacitive sensors into the fingertips of robot hands. However, the reduced size of the electrodes remains a challenge. A smaller size is desirable for integration and spatial resolution but is attained at the cost of reduced electrode surface area, which limits the possible sensing range/sensitivity. A further interesting use-case for capacitive sensing is introduced by Erickson et al. in [36], [37]. Using off-the-shelf electronics (MPR121 and the Teensy-board respectively), they implement a capacitive end-effector for the PR2 that is capable of detecting human limbs for dressing and washing tasks in health-care scenarios. The mechanical robustness of capacitive sensors also makes them suitable for harsh industrial environments, like investigated in [38] for a grasper of an autonomous forestry crane.

4) *Further Aspects of Capacitive Proximity Sensing Technologies in Robotics:* As the capacitive measurement principle is suitable for implementing both tactile and proximity sensors, there have been efforts to realize both modalities in a single sensor design, e. g. [39], [33], [30], [40], [41]. This is a special case of multi-modal sensors (see Sec. III-E). Other works have made use of the material dependency and investigated material recognition using multi-exciter frequencies [42], [43], and [44]. Moreover, tomographic measurements using capacitive sensors were also studied including side effects and material dependencies [17], [45] and the potential for flexible spatial resolution was explored [41], [34], [46]. The possible applications are further discussed in Sec. IV.

In contrast to optical proximity sensors (see Sec. III-B), it is not as common for researchers to use off-the-shelf solutions for implementing capacitive proximity sensors. More often than not, capacitive sensor circuits have been developed by the robotics researchers themselves. Another difference to optical sensing is the attainable sensing rate. The typical rates reported fall in the range of 20-125 Hz (with some exceptions up to several kilohertz) for capacitive sensing, whereas recent optical sensing approaches report update rates > 1 kHz (see Sec. III-B). The difference can be explained by the fact that the effect an object has on an electric field is often quite weak, leading to low signal to noise ratios and the comparatively low-frequency carrier frequencies for the measurement circuitry. Stronger excitation signals might compensate for the low sensitivity but this is limited due to increasing costs and higher power consumption. Also, often a single sensing front-end is addressing several sensing elements in a time-multiplexed manner, further decreasing the update rate.

A somewhat unique domain in robotics, where capacitive-like sensing plays an important role, is in bio-inspired underwater robots. Here, mimicry of weakly electric fish, that is, fishes that use this sensing modality for navigation, preying,

<figure><img src="image_8.png" /><figcaption>Fig. 7. (a) The *freshwater elephantfish* native to Africa feature an electric organ and electroreceptors, creating a mutual impedance system, for sensing their surroundings, and for communication. This makes them attractive subjects for studying bio-inspired proximity sensing approaches. (b) An illustration of the electric field generated by the *freshwater elephantfish* [47]. (Sensing type: similar to C-M), images courtesy of Frédéric Boyer and Vincent Lebastard ©2013 IEEE</figcaption></figure>

and communication, is studied (see Fig. 7). The electric fish live in low-visibility and cluttered environments where the *electrosense* becomes a crucial tool. They use an electric organ to generate voltage pulses or oscillations and have voltage receptors on their skin to detect disturbances of the field, i. e. they implement a mutual impedance system in water similar to a capacitive system (C-M) in air. Examples are the research by Boyer and Lebastard et al. [48] as well as MacIver et al. [49]. Both groups have published an important number of articles on this research topic.

5) *Spatial Resolution and Sensing Range for Capacitive Sensors:* Regarding spatial resolution, capacitive sensing is highly adaptable. Reducing the electrode size is not necessarily a problem in terms of fabrication, but sensitivity becomes more challenging as the size decreases. While using cells of size $1 \times 1 \text{ mm}^2$ for tactile sensing is no problem [39], an area of $\approx 15 \times 25 \text{ mm}^2$ is needed for detecting conductive objects at a distance of about 40 mm for a sensor mounted on a finger (AT-II) in [30]. However, sensing range is also determined by the distance of the electrodes in mutual-capacitive mode [29] (AT-II) as well as the circuit design. In [25], a detection distance of about 30 cm for electrodes of size between $50 \times 50 \text{ mm}^2$ and $100 \times 100 \text{ mm}^2$ is reported for HRI (AT-I). Finally, as the sensing range increases, self-influence becomes an issue that needs to be addressed [11], [50].

## B. Optical Sensing

In this section, we describe the principles, research background, and the latest research in optical sensing technology. Optical sensing is one of the most popular and traditional forms of proximity sensing in robotics. The main principles, as shown in Fig. 8, are:

* Reflected light intensity (O-RLI),

* Time-of-flight (O-ToF),

* Triangulation (O-Tri) and

* Break-beam (O-BB).

In the cases of O-RLI, O-ToF, and O-Tri, a light emitter and a receiver are placed next to each other on the same surface. Then, the proximity of an object is measured based on the reflected light intensity, return time of reflected light, or light incident position (or angle) respectively. Especially for O-RLI, a paired set of an IR LED and a photodiode can be called
<figure><img src="image_9.png" /><figcaption>Fig. 8. Illustration of the most common optical sensing working principles: (a) reflected light intensity (O-RLI), (b) time-of-flight (O-ToF), (c) triangulation (O-Tri), and (d) break-beam (O-BB).</figcaption></figure>

*photoreflector*. Furthermore, O-RLI often uses modulated light to suppress extraneous light influences. In the case of O-BB, a light emitter and a detector are arranged on distinct surfaces to detect the interruption of the ray due to an obstacle.

With O-RLI, the proximity value depends on the reflectance of the object, which affects the measured light intensity. With O-ToF and O-Tri, the actual distance without direct dependency on the reflectance is measured. However, most instances of O-ToF and O-Tri have difficulties with specular reflections as they can occur for instance on metallic surfaces. When reflectance properties of surfaces are problematic, O-BB is an interesting alternative, as it can detect objects, even with very shiny surfaces. In robotics, the O-RLI-type has been widely used, as the sensor structure and processing are simple, easily complying with the requirements on integration. Therefore, it has been a popular choice for equipping manipulators (AT-I) and grippers (AT-II) with proximity sensors.

1) *Reflected Light Intensity-Type Sensors (O-RLI) at an Early Stage:* In 1973, Lewis et al. [51] at NASA's Jet Propulsion Laboratory (JPL) proposed a gripper with the O-RLI-type in a JPL program (see also [52]). The main goal of the program was

> "[...] to demonstrate the integration of sensory and motor functions in the autonomous performance of manipulation and locomotion tasks in response to global commands issued by an operator." [51]

O-RLI-type sensors are suitable for both application types described in Sec. II-B, because of their small sizes and fast response times. However, in the 1970s, the sizes of LEDs, detectors, lenses, and amplifier circuits still were too large. Also, CPU performance was not sufficiently high. For this reason, it was technically difficult to mount an array of multiple optical sensors on manipulator links or grippers.

In the late 1980s and early 1990s, advancement in this technology already allowed Cheung and Lumelsky to show designs for an O-RLI-type proximity skin for collision avoidance tasks [53] based on an Opto Diode Corp. OD8810 infrared emitter and an Osram SFH205 photodiode as shown in Fig. 9 (see also Sec. IV-A). This technology was adapted by Wegerif et al. in [54] for their own work on collision avoidance. In both cases, these solutions required the design of an analog front-end to drive the sensors and custom made

electronics for the skin as a whole. Modulation of light is used to handle potential cross-talk between different sections of the skin. Similar technology is featured in the work of Petryk and Buehler [55], [56], who equipped a two-jaw gripper with distributed sensing.

2) *Reflected Light Intensity-Type Sensors (O-RLI) Using Photoreflectors and Custom Electronics:* In the 2000s, many companies released surface-mounted photoreflectors and small microcontroller/amplifier circuits. As a result, researchers were able to develop an array of multiple sensors suitable for ATs I and II more easily. An interesting example is due to Tar et al. [57], who show the realization of an 8×8 matrix of sensors capable of imaging approaching objects using the TCRT1000 photoreflector. Hsiao et al. [58] developed an O-RLI-type sensor for the finger of a Barrett Hand as shown in Fig. 10. The sensor was constructed using four photoreflectors and an amplifier circuit/microcontroller, embedded in each finger segment. In [59], Mittendorfer and Cheng first showed their multi-modal and modular sensor design that uses the Sharp GP2S60, having a footprint of 3 × 4 mm², as a proximity sensor (see Fig. 5 as well as Sec. III-E). A special case of O-RLI-type sensors can be implemented using optical fibers.

<figure><img src="image_10.png" /><figcaption>Fig. 9. Using flexible circuit boards, Lumelsky and Cheung show the realization of a proximity skin covering whole manipulator arm as discussed for instance in [53]. Sensor type: O-RLI (Photo courtesy of www.edcheung.com)</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>Fig. 10. a) Barrett hand equipped with finger-size optical proximity sensors b) Design criteria leading to a final concept of sensor integration in the fingertips of the Barrett hand. [58] Sensor type: O-RLI (Figures reprinted with kind permission of the authors ©2009 IEEE)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 11. (a) High-speed proximity sensor [65], [66], [67]. (b) High-speed, high-precision proximity sensor [70], [71]. Both sensors detect tilt angles and distance of an object's surface. The measurement time of both sensors is less than 1 ms. The distance resolution of the sensor (b) is less than 51 µm. Sensor type: O-RLI (©2013 and 2019 IEEE)</figcaption></figure>

Since the fibers are easy to integrate into confined spaces, this solution has been proposed by Espiau and Catros [60], Walker et al. [61], and Konstantinova et al. [62], [63], [64]. In [61], the authors integrate 32 fibers, having a diameter of 1 mm, into a disc-like end-effector. The fibers route the reflected light captured around 360° to a 4×8-display. The intensity values on the display are then recorded by a camera. In [62], [63], [64] optical fibers are similarly routed from the tip of a finger to a signal processing module (KEYENCE) for O-RLI-type sensing.

In [65], [66], [67], Koyama et al. developed finger-sized, high-speed proximity sensors mounting twelve photoreflectors on a fingertip as shown in Fig. 11 (a). The sampling time of the sensor outputs is < 1 ms, and the sensor size is thin and compact [68]. As the outputs of the photoreflectors were processed in a grid of resistors, a proximity event can be localized. The same principle is adopted by Arita and Suzuki in [69] for a linear array of sensors. The authors also proposed a simple calibration method using changes in fingertip positioning and reflected light intensity. However, simple calibration methods, e. g. [58], [67], have relatively large errors (with millimeter or sub-millimeter accuracy) due to fingertip position errors or circuit noise. Therefore, reactive preshaping methods (see Sec. IV-B) have not yet reached a high level of accuracy using these calibration schemes.

3) *Reflected Light Intensity-Type Sensors (O-RLI) Using Sensing Modules)*: More recently, some companies have

<figure><img src="image_13.png" /><figcaption>Fig. 12. (a) Gripper-type proximity sensor that can detect contact, force, and distance, (b) Robot skin type. Sensor type: O-RLI (Figures reprinted with kind permission of the authors (a) [72] ©2018 Springer and (b) [73] ©2018 IEEE)</figcaption></figure>

released compact-sized, low-noise proximity sensors with built-in amplifier circuits and $I^2C$ bus connectivity. Multiple sensors can be daisy-chained using the $I^2C$ bus. Researchers can develop fingertip-size proximity sensors and robot skins equipped with multiple ranging sensors. In particular, Vishay Semiconductors released the O-RLI-type sensor, the VCNL4010. The footprint of the VCNL4010 is 3.95 × 3.95 mm², and can measure a range of 1–200 mm within 4 ms (minimum time setting). Although the sensor output is affected by the reflectance of object surfaces, the development of a thin proximity sensor is easily attainable.

Patel et al. [72] have developed a finger-size sensor (Fig. 12 (a)) that can detect distance, contact, and force with the VCNL4010. The sensor consists of multiple VCNL4010 devices covered with a transparent rubber (PDMS silicone). When there is no contact between the sensor and an object, the sensor can measure distance based on the reflected light intensity from an object's surface. The sensor can also detect contact with an object triggered by a sharp change of reflected light intensity. After contact, the contact force can be also estimated by measuring the reflected light from the object surface to the rubber surface. Hughes et al. [73] have proposed flexible robot skin modules (Fig. 12 (b)) using the same sensor structure as in [72]. They realized gesture recognition by combining distance values with a random forest classifier. Originally conceived for oxymetry, the MAX30105 by Maxim Integrated is used for implementing wireless multi-modal sensor for the hand of a humanoid robot (Robonaut 2) in [74].

In [75], the authors repurpose a mouse sensor (ADNS-9500) in a fingertip for proximity sensing, as O-RLI is used in the sensing element. In this case, it is even possible to use the 30 × 30-pixel delivered by the sensor for further processing (e. g. texture recognition and slip detection).

4) *Time-of-Flight-Type (O-ToF) Sensors:* One popular O-ToF sensor is the VL6180X proximity sensor, released by STMicroelectronics. The VL6180X can measure 10–100 mm with 1 mm resolution and a repeat measurement error of ±1–2 mm. The measurement time for one sensor is 7 ms to several tens of ms, depending on the settings. Lancaster et al. [76], [77] developed a fingertip-sized sensor for a parallel jaw gripper on the PR2 and demonstrated a robust manipulation of a Rubik's Cube. They also developed a fingertip-sized sensor comprised of transparent rubber and a O-ToF sensor, which uses reflected light intensity for force
measurement [78]. They designed and evaluated different rubber shapes and optical configurations (flat rubber, rounded rubber, and light blocker configurations). It is reported that a rounded configuration improves the sensitivity of force detection. Sasaki et al. [79] developed a multi-modal proximity sensor, employing both O-RLI and O-ToF sensing. The O-RLI-type detects distance and posture for an object on a table, and O-ToF type measures the distance from the table surface. The robot can adjust the configurations of the fingertips and the end-effector (the hand base) simultaneously using sensor feedback. Tsuji et al. [80] developed a proximity sensor skin using O-ToF sensors for a collaborative robot, which in its layout is comparable to the work by Cheung and Lumelsky (see Fig. 9). In [43], [20], Ding et al. show their developments of a multi-modal proximity sensor, the proximity sensing cuffs, featuring capacitive and O-ToF technology for material recognition and collision avoidance.

Recently, the use of O-ToF has also been proposed for Soft Robotics devices [81], [82]. Even though the module used is the already mentioned VL6180X, which is not deformable, the authors show its integration in a soft circuit, featuring traces of copper wetted with eutectic gallium indium (EGaIn) inside a thin PDMS sheet. The circuit features other sensors ICs: an IMU, barometric pressure, and temperature sensors. The soft sheet is then used to equip a two-jaw gripper with these sensing capabilities.

5) **Triangulation-Type Sensors (O-Tri):** An early example for an O-Tri-Type proximity sensor is due to Fuhrman and Kanade [83]. Using a chip capable of localizing a light spot, they realize several light sources that are evaluated in a time-multiplexed manner, resulting in an object's proximity value as well as orientation and curvature. However, this setup can probably be considered to be too bulky, i. e. not skinlike, for modern applications. In [84], [85], the Ceriani et al. and Avanzini et al. report using the Sharp GP2Y0A02YK0F module, that guarantees a consistent distance output across different reflectivity of surfaces. In their work, they explore the optimal distribution of sensing elements for safe HRI (see also Sec. IV-A3). To measure distance and posture more precisely, Koyama et al. [70], [71] developed a high-speed, high-precision proximity sensor, as shown in Fig. 11 (b). The sensor has the size of a human fingertip (18×28.5×38.5 mm³), and it can detect the distance to and postures of an object surface with a distance error of fewer than 31 µm and a measuring time less than 1 ms. A similar sensor design was explored by Bonen et al. [86], who propose a single emitter and multi-detector architecture for detecting distance as well as object orientation.

6) **Break-Beam-Type Sensors (O-BB):** Teichmann et al. [87] mounted a light-emitting diode at one end of a parallel jaw gripper and detectors at the other end, and switched to reactive motions based on light blocking due to an object. They also describe the application of this approach to a three-fingered hand. In [88] Guo et al. show the integration of an array of O-BB-Type sensors in the jaws of PR2's parallel gripper for reactive preshaping and grasping challenging objects, where other approaches would fail, such as semi-transparent tissues.

7) **Discussion on Optical Sensors:** The devices implementing optical sensing are small in size and have a high-speed response. These advantages are suitable for sensor/actuator integration and automatic grasping using a robot hand, although the sensing has difficulty detecting transparent, black, and shiny objects. To detect all these objects, it is necessary to introduce multi-modal sensing, such as a combination of optical and capacitance sensing.

In terms of spatial resolution, optical sensing elements can be quite small. For example, the Sharp GP2S60 photoreflector used by Mittendorfer et al. [59] has a footprint of 3×4 mm², potentially allowing a density of a few elements per cm². However, on large-area skin (AT-I) such high densities can be impractical in terms of the electronic effort needed (wiring, signaling, etc.). Regarding sensing range, O-RLI-types have been reported to produce a relatively large detection distance of about 300 mm [53] (AT-I), but these sensors are also often used for lower ranges, e. g. [70], where a maximum detection range of 20 mm is reported (AT-II). In both cases, there is a small dead spot near the sensing element. Typical O-ToF-technology used, e. g. [20], [21], can work up to a distance of 4 m, depending on the component used, but this extended range has the cost of having a relatively large dead spot of 10 cm in front of the sensing element. Similarly, the O-Tri-Type sensor used in [84], [85] has a detection range of 1.5 m and a dead spot of 20 cm.

### C. Radar

In recent years, radar sensing technology has become popular in human-centered technologies due to the development of system on chip radar systems reducing the size, which also makes them very attractive for integration on robotic platforms. Recent developments are driven in big part by the automotive industry (see also [1]). Radar sensors withstand harsh weather environments and can augment widely used optical sensor technologies, meaning they have crucial traits for enabling highly automated driving. An important aspect for radar sensors in human-robot interaction is that it is a technology that is widely used in safety related applications in the automotive domain. Consequently, existing expertise from the automotive domain can potentially be utilized towards robotic applications (e. g. [89]). A recent survey by van Berlo et al. summarizes the current application fields of radar technologies [90], including the aforementioned domain of automotive and HCI (tracking, gesture recognition, etc.).

Recently, in a joint effort, Google and the chip manufacturer Infineon boosted this technology as they introduced a 60 GHz radar chip with integrated transmitter and receiver antennas for fine gesture interaction based on frequency modulated continuous wave (FMCW) [91]. The principle of FMCW radars is illustrated in Figure 13. A more detailed description can be found e. g. in [89].

Advantages of FMCW-radar are that they provide distance and velocity measurements simultaneously with a high resolution for close ranges, which makes them suitable for proximity perception, collision avoidance, and HRI, e. g. gesture control, in the field of robotics. In [92], a simulation
<figure><img src="image_14.png" /><figcaption>Fig. 13. Principle of FMCW Radar: A transmitter (Tx) sends out a chirp signal (red), which gets reflected at object boundaries, e. g. a human. The reflected signal (green) at the receiver (Rx) is a delayed, attenuated copy of transmitter signal. The time delay corresponds to the distance and thus the frequency difference between transmitter and receiver is proportional to the distance. This frequency is obtained by mixing the transmitter and receiver signal. For a sequence of chirps, the phases of the received signal changes due to the Doppler shift. An FFT over the time extracts the frequencies, a subsequent FFT over chirps extracts the velocity, such that a 2D range distance map is obtained.</figcaption></figure>

approach for FMCW-radar is proposed and evaluated for
collaborative robotics. The simulation approach aims at fast
development and evaluation of signal processing algorithms as
well as machine learning approaches for FMCW-radar before
deploying it on a robot (sim-to-real learning). Multi-modal
tactile and radar sensing for grasping applications and object
classification was introduced in [93].

Further manufacturers, such as SiliconRadar, Texas Instru-
ments, etc. also provide system on chip radar solutions. In [94],
the integration of a multiple antenna radar sensor based on
160 GHz with flexible waveguides for collaborative robots
was studied. Another operation mode for radar technology
is synthetic aperture radar (SAR). In robotics, it is utilized
for instance for mobile robot navigation. In [95] SAR based
radar imaging was investigated on a PR2 robot system for 2D
and 3D imaging of objects in close proximity to show the
potential of radar sensing to complement optical sensors in
robot perception and manipulation.

Besides proximity and distance sensing, radar systems can
be applied to exploring and constructing subsurface 3D maps
[96]. Those radars are known as ground penetration radar
(GPR) and are applied in different fields such as assessment of
dense underground utilities in urban areas, evaluation of the
subsurface for energy and mineral production operations or
the detection of buried objects, on earth or potentially on other
planets. Its sensing principle is based on emitting radar pulses
and evaluating the propagation velocity into the subsurface and
the soil utilities [97].

D. Other Sensing Principles

In this section, we introduce further sensing principles that
can be used for proximity sensing, i. e. acoustic, inductive, and
whiskers. As of 2021, they can still be considered to be less
mainstream in human-centered robotics than the capacitive,
optical, or radar ones. However, they offer interesting alterna-
tives and can outperform other principles discussed so far in
some scenarios.

1) Acoustic: The widest spread technique for ranging based on acoustic wave propagation is ultrasound (A-US), which can be found in many domains, particularly for under-water

ranging. In robotics, this technology is easily available for the
enthusiast and professional use, such as the MaxSonar-series
by MaxBotix. Higher-end solutions, featuring 3D echolocation
are also available, for instance by Toposens. Nunes et al.
proposed the use of ultrasound in [98] for 3D ranging in
1994. Even though Dario et al. [99] propose an ultrasound
sensor for integration into a fingertip, the sensors usually have
a non-negligible offset or dead-spot for sensing around the
sensing element. Therefore, many of the available solutions are
non-practical for pre-touch applications, i. e. close proximity
(see Fig. 3). Integration is also challenging because sensing
elements do not scale down easily. Thus, oftentimes use-cases
of A-US are more similar to laser-range finders, i. e. mid-range
and long-range sensing. In [100], Fang et al. circumvent the
mentioned difficulties by mounting the sensor at a distance
and tilted while bouncing the waves off a parabolic mirror.
However, the integration remains limited to one acoustic
sensing element. Ultrasound is also widely used in parking
sensors. An example of a combination with capacitive sensors
to overcome detection limitations at short distances is provided
in [101]. The group of Prof. Steckel at University Antwerp has
a strong focus on 3D A-US for robotic applications, e. g. [102],
[103]. The group has achieved remarkable results in areas such
as SLAM for the navigation of mobile platforms. However,
as with other designs, the ultrasound sensing platform is not
very skinlike, thus limiting the pre-touch applications in favor
of longer-range sensing. Furthermore, the authors often make
a point to establish this technology as an alternative to LIDAR
and other mid-range or long-range sensing options for ground
vehicles but also for UAVs. The use of A-US in air-borne
vehicles puts in evidence that this type of sensing can be
considered to be bio-inspired by bats and their echolocation
capabilities.

Another type of acoustic sensing has been proposed by
Jiang et al. [104], [105], which the authors call the *seashell
effect* (A-S). A microphone is placed inside a cavity that is
worked into the structure of a finger. As a surface approaches
the opening, the resonance frequency of the cavity changes.
By analyzing the differences in the spectrum between an
external microphone and the microphone inside the cavity, the
distance can be estimated. This works for very close range
(up to ≈ 4 mm). In their work, this modality is explored,
because it does not suffer from detection difficulties related to
transparency or reflections (optical sensing) or low dielectric
contrast (capacitive sensing).

2) **Inductive**: Inductive sensors utilize alternating magnetic fields to detect objects, as they disturb the generated magnetic field, which can be detected as a change of inductance of a coil, a change of the mutual inductance between several coils or directly by measuring the magnetic field. The objects do not need to be ferromagnetic. In particular, objects with high conductivity such as metals, strongly affect an alternating magnetic field as eddy currents near the surface of such objects prevent deep penetration of the materials by the magnetic fields. Inductive proximity sensors are very robust and commercial sensors provided by a variety of manufactures are widely used in industry as proximity switches, typically detecting conductive objects. However, as these commercial or industrial sensors are not found in robotics, they are not included in this survey. The capabilities of inductive sensors for non-metallic objects are more limited and inductive sensors have therefore been used in combination with other approaches to classify materials, for example.

The sensing system proposed in [40] is stated to combine capacitive force and inductive proximity sensing with a range of up to 150 mm for conductive materials with the help of a layer of carbon micro coils (CMC). The sensor was then enhanced and in [106], having a higher detection range and spatial resolution. The CMC layer was used to form an LCR circuit and enable both tactile and proximity sensing. In a related work [107], an electromagnetic field was formed by exciting a combined co-planar plate capacitor and a coil embedded in a flexible circuit board. The impedance of the resulting LCR circuit was analyzed and the relationship to the distance of different objects was presented as proximity measurement, with a sensing range of up to 300 mm.

In [108] the combination of capacitive and inductive sensing is used to distinguish between humans and other objects such as (grounded) laptops. While the capacitive signals for humans and grounded laptops are very similar, the inductive signal is much different, as the laptop comprises highly conductive metallic parts and thus has a stronger influence on the magnetic field. Even though the setup is not intended for proximity sensing, a range of up to 150 mm is reported. Consequently, the system can also be used to detect non-conductive and conductive objects and offers a very high measurement rate of 25 kHz. With multiple coils, inductive sensors can not only be used to obtain a distance estimate but full 6 DoF information, as discussed in [109].

3) **Whiskers**: Finally, on the fringe of the domain of proximity perception, we can find artificial whiskers that are inspired by mammals, such as rodents, who use them to navigate and explore their environment [110], [111]. These whiskers are beams that bend due to external forces (contacts with walls, wind, etc.) and usually, the resulting force/deformation at the base is measured. These approaches are often featured as part of the tactile perception community, as sensing is actually contact-based, but they are used to probe the nearby

environment much in the same way a proximity sensor is used.

E. Multi-modal and Modular Sensors

The possibility of deploying proximity sensors alongside other sensing modalities on robots (vision, touch, etc.) is a key aspect of the success of this technology. Only if they coexist with the other modalities, can they fill the perception gap that is left by them. This challenge is evidenced by the many existing realizations of multi-modal sensors, especially by designs that include the tactile modality alongside the proximity one. Furthermore, it is common to find that these designs are conceived in a modular manner. The HEX-o-Skin by Mittendorfer et al. [59] is a prominent example of this trend. It is a modular design, which is suited for covering large areas of the robot (see Fig. 5) and includes proximity, tactile, inertial, and temperature sensing in each unit.

The most generic approach for implementing multi-modal sensing is to use a specialized measurement principle for each desired modality. The work by Mittendorfer et al. [59], again, is an example of this approach. Proximity detection is implemented by O-RLI and tactile events are detected with a capacitive sensing element. Stiehl et al. [18], who implement capacitive proximity (C-SE, provided by the MC33794), force and temperature measurement in a pseudo-modular skin, which the authors argue helps in distinguishing social contacts from collisions with the environment. Another, less modular example is due to Guan et al. [112]. In their work, they equip the gripper of a climbing robot with a range finder sensor, two ultrasound modules, and a camera.

In [40], Han et al. show a tactile proximity sensor where the proximity modality principle is inductive and the tactile modality is capacitive. In [62], [63], [64] proximity sensing and tactile (force) sensing is implemented using optical fibers. In proximity sensing, the O-RLI-type is used, for tactile sensing, the reflection that changes inside a movable part is measured. As explained in Sec. III-B3, in [72] the authors show the implementation of a tactile proximity sensor based on O-RLI-type sensing alone. As stated before in Sec. III-A, capacitive sensing is especially attractive for joint tactile and proximity designs. Example designs are shown in [39], [33], [30], [41]. Designs also have been proposed outside of robotics literature, which are nonetheless potentially relevant, e. g. [113]. Finally, there is a subset of approaches that utilize different measurement principles for redundant proximity sensing. This is the case with Ding et al. [43], [20], and Tsuji et al. [21] that use both an O-ToF and C-SE for robustness. Markvicka et al. propose the joint use of O-ToF and O-RLI in [74].

IV. APPLICATIONS AND METHODS IN THE RESEARCH AND
INDUSTRY DOMAINS

In this section, we will review the contributions from the field focusing on the applications and methods presented. We will follow the organization presented in Sec. II-B and Fig. 4, i. e. focusing separately on AT-I and AT-II and going from low-complexity behaviors (BT-I) to high-complexity behaviors (BT-II).
<figure><img src="image_15.png" /><figcaption>Fig. 14. A redundant robot, equipped with proximity sensors on its links, can move towards a target end-effector configuration while simultaneously moving some of the links of the robot away from an obstacle. (Figure adapted from [115])</figcaption></figure>

A. Reactive collision-avoidance and Contour Following (AT-I, BT-I)

Collision-avoidance is regarded as a fundamental skill for autonomous robots as well as for safe human-robot interaction. To start this section, we want to motivate with a quote by Novak et al. from their 1992 work on whole-arm collision-avoidance, which is an elegant statement of this problem:

> "[...] since it is desirable to continue purposeful motion in the presence of obstacles, the sensor system must be able to deliver spatially-resolved proximity data, which reflects the distance to the obstacle, as well as the location along the robot and corresponding robot surface normal. This vector information may then be used to modify trajectories to permit (if possible) continued progress toward the final destination." [14]

The first important wave of interest surrounding proximity perception for collision-avoidance was sparked in the late 1980s and early 1990s [114]. At that time, getting 3D information of the surroundings of the robot via cameras was challenging from a technological point of view, on the accounts of the lack of hardware and lack of performance of the CPUs. Proximity sensors, having desirable properties (low latency, skinlike), were considered an attractive alternative for this challenge.

1) Early Jacobian-type Approaches: A good portion of early works on collision-avoidance get inspiration from the work of Maciejewski and Klein [115], published in 1985. This work introduces the notion of the “obstacle avoidance point Jacobian”, which is analogous to the end-effector Jacobian, i. e. it relates the instantaneous joint velocity to the velocity of an obstacle point on the robot (see Fig. 14). In this figure, we use the following notation: $q_a^i$ are the degrees of freedom of the manipulator arm in joint-space. $x_e$ and $x_o$ are the end-effector configuration and the obstacle point in task-space respectively (boldface indicates these are vector quantities), whereas $c_g$ is the goal configuration established for the end-effector and $p_o$ is the point on the obstacle nearest to the manipulator

arm (also vector quantities). Following this notation, the end-effector Jacobian $J_e$ relates the velocities in the configuration space to the velocities in the task space by:

$$ \dot{x}_e = J_e \dot{q}_a. \qquad (3) $$

Using the obstacle Jacobian $J_o$, The relation in the case of the obstacle-point is likewise:

$$ \dot{x}_o = J_o \dot{q}_a. \qquad (4) $$

Applying the same principles of using the (pseudo-) inverse of the end-effector Jacobian $J_e^+$ for finding desired joint velocities, one can invert the obstacle point Jacobian to find the joint motions to follow a desired trajectory with respect to the obstacle point. In the case of collision-avoidance, the natural choice is a motion away from the obstacle, as indicated by $\dot{x}_o$ in Fig. 14. Also, in the presence of redundancies, this approach allows projecting the avoidance motion into the null-space of a higher-order task, e. g. following a desired end-effector trajectory:

$$ \dot{q}_a = J_e^+ \dot{x}_e + (I - J_e^+ J_e) J_o^+ \dot{x}_o. \qquad (5) $$

Since $\dot{x}_o$ represents the desired motion away from the obstacle in task space, $\dot{q}_a = J_o^+ \dot{x}_o$ is the joint-motion that moves the obstacle-point away from the obstacle. $(I - J_e^+ J_e)$ is the expression that projects this motion into the null-space of the higher-order task, i. e. the desired end-effector trajectory. In this framework, the tasks can be ordered in a different hierarchy as well, i. e. prioritizing the collision-avoidance task over the desired end-effector trajectory.

Among the firsts to apply these ideas to proximity sensor streams were Wegerif et al. [54], [13] as well as Tamasy in [116] at Merrit Systems Inc. in the early 1990s. Wegerif et al. studied the use of several proximity sensing technologies (IR, ultrasound, capacitive), but they ultimately covered three links of a PUMA 600 robot with a total of about 120 IR sender and receiver pairs. They modified the kinematics of the PUMA 600 to have three rotational joints in one plane, introducing a kinematic redundancy in an otherwise non-redundant robot. In their collision-avoidance algorithm, they gave the highest priority to the collision-avoidance task. They report successfully testing the system in an autonomous and a teleoperated scenario with static and dynamic obstacles. In the work by Tamasy [116], the previous work is extended by presenting the realization of a smart sensor network. These kinds of networks provide the base for equipping whole arms with proximity sensors, implementing a bus-system. IR, ultrasound and capacitive sensors can be readily connected to the system, provided they offer digitized data streams. The NASA payload inspection and processing robot (PIPR), featuring 18 DoFs, is shown as an application. A whole control architecture is discussed, with a GUI for user inputs, the generation of low-level commands, as well as a collision-avoidance system based on the previous developments, together with a quadratic programming approach for finding the optimal joint velocities.

Other authors that were inspired by the approach of Maciejewski and Klein are Novak and Feddema [117], [15]. A prior work by the authors that leads up to these results is [14].
In [14] the authors concentrate on the development of the so-
called *whole-arm proximity (WHAP)-sensor*, which is a skin
that is comprised of mutual capacitive sensing elements for
proximity sensing. In this work, the sensor is described and
characterized in depth. Then, the WHAP sensor is installed
and tested on a 2-link planar robot with a total of 8 sensing
elements (two per link). The robot is shown to successfully
circumvent one obstacle made out of concrete and another
metallic one. Later, using a sensor Jacobian in [117], [15] they
concentrate on a teleoperation scenario using a 6-DoF robot
arm (PUMA 560). An obstacle is responsible for a reduction
of the speed of the affected DoFs as the sensors detect the
obstacle approaching. However, the system is designed to not
automatically move away from the obstacle, as the authors
consider that this behavior is not desired by the user, at least
not in a teleoperation scenario.

2) *Early Geometric Approaches:* In contrast to the Jacobian-based approaches, the collision-avoidance can be implemented by a *geometric approach*, i. e. estimating features of the obstacle's surface and following its contour for as long as it obstructs the direct path from the current configuration $c_i$ to a given goal configuration $c_g$ of the robot. We consider this to be a geometric approach because in some way the surface of the obstacle has to be reconstructed. Early, seminal work is due to Lumelsky and Cheung [118], [119], [53], [120] (and more), who propose to move along the tangent plane of the obstacle represented in configuration space. Their hardware is characterized by the use of infra-red sender and receiver pairs that are mounted on flexible printed circuit boards (which would later inspire the work by Wegerif et al. in [54], see above). With the flex-technology, they achieve full integration of the skin onto a manipulator as shown in Fig. 9. In [53], the development of the skin is explained in great detail. In [120] they showcase the methods and technology in the context of teleoperation in the presence of one or more dynamic obstacles.

Nunes et al. [98] propose a collision-avoidance system that
can be regarded as a contour following system that works at the
end-effector level, i. e. in Cartesian space. As before, the idea
is to slide parallel to the obstacle tangential plane. In Fig. 15
the geometric approaches are illustrated and summarized. It
shows the four characteristic configurations $c_s, c_d, c_i, c_e$
and $c_g$ that can be usually be identified. $c_s$ is the starting
configuration, $c_d$ is the configuration where the obstacle is
first detected, $c_i$ is the current robot configuration during the
contour following phase, $c_e$ is the configuration where the
obstacle no longer obstructs the direct path to the goal and
$c_g$ is the goal configuration. The vector showing from the
current configuration $c_i$ to $c_g$ is $\vec{v}$ and its projection on the
surface tangent plane $\Pi$ is $\vec{v}$. The tangent plane is detected
at a point $p_o^i$. The dashed line represents the trajectory of
the robot during the contour following procedure. In general,
during contour following, the movement of the robot will be
parallel to $\vec{v}$, i. e. towards the target configuration.

3) *Recent Jacobian-type Approaches:* In [84] Ceriani et al. discuss the placement of proximity sensors on an industrial manipulator. They present an optimization method that allows to find a suitable arrangement of triangulation type sensors on

the links of the robot based on the concept of a *Danger Field*.
The Danger Field is a distance-based metric for assessing the
danger emanating from a moving robot for a human operator.
Avanzini et al. continue this work by developing a safety
control scheme on top of this concept [85]. They show that
tasks can be deactivated according to a predefined priority to
permit evasive movements. In their case, the tasks are defined
by a Cartesian trajectory split in position and orientation.
Maintaining the orientation is the lower priority task, which
is the first to be abandoned to comply with the collision-
avoidance.

In [16], Schlegl et al. show the use of a capacitive sensor that features both a single-ended and a mutual-capacitive sensing capability, resulting in so-called *Virtual Whiskers*. The use of both modes increases robustness in the detection of conductive and non-conductive objects. An arrangement of 7 electrodes is mounted on a segment near the wrist of a KUKA LWR 4. The hardware has a sample rate of up to 1 kHz. The authors show the combination with the on-line trajectory generation discussed in [121] that is capable of generating smooth trajectories with at least the same frequency. This leads to a highly reactive collision-avoidance prototype. Similar hardware is used in [10] Mülbacher-Karrer et al. to show the contactless control of a 9 DoF redundant manipulator. One link is equipped on two sides with electrodes. A 2D tomographic image can be extracted from each arrangement. The center of the detected event is used to steer the avoidance motion of the link equipped with sensors, while the robot continues to execute a pick-and-place task.

In [20], [122] Ding et al. present a Jacobian-type collision-
avoidance scheme that is based on optimization, also taking
into account the redundancy capabilities of the robot used.
Their 7-DoF robot has three links equipped with 360° sensing
capability provided by so-called proximity-sensing-cuffs [43].
When an obstacle is detected in [20], joint-velocities are

<figure><img src="image_16.png" /><figcaption>Fig. 15. In geometric collision-avoidance approaches, the surface of the obstacle is estimated. From this, a contour following motion can be derived.</figcaption></figure>
calculated according to a mixture of criteria, which are si-
multaneously optimized: distance to the target, manipulability,
deviation from desired task motion, and total magnitude of
joint-velocities. This results in a reflex-like collision-avoidance
system, including movement parallel to the obstacle tangent
plane. This can avoid getting stuck in front of obstacles, which
is a possible failure mode of potential field approaches, like
the one proposed by Khatib et al. [123]. Also following up
on the Jacobian-type approaches is the work presented by
M'Colo et al. in [25]. The capacitive sensing technology of
FOGALE robotics [24] is featured in a robotic system that
has been extensively covered with electrodes to implement a
skin. Using the skin, the robot can avoid static and dynamic
obstacles. Finally, Arita and Suzuki show progress towards
an approach that allows using force control for the seamless
transition between pre-touch and touch states and for achieving
desired contact forces [69].

4) *Recent Geometric Approaches:* For what concerns con-
tour following scenarios, the literature proposes to include
curvature estimates to improve the performance by using a
predictive component, for instance by Baeten and De Schutter
[124]. Relying on the spatial resolution of the sensor, they
can be estimated directly from the current sensor values, as
proposed in the work by Walker et al. [61]. This work uses
an optical proximity sensor attached to an end-effector and
featuring 360° vision in a plane. In [19], these ideas are
generalized by Escaida Navarro et al. for contour following in
3D, i. e. by detecting the 2D curvature of the obstacle surface.

B. *Reactive Preshaping and Grasping (AT-II, BT-I)*

In this section, we describe reactive preshaping using prox-
imity sensor feedback, i. e. closed-loop control (AT-II). The
concept of preshaping has been proposed in psychology in
the context of studies on human grasping, e. g. in [125],
before it was adopted in robotics. In robotics terms, preshap-
ing describes the (preliminary) motions of adjusting finger
joint poses $q_h$ of a robot hand and end-effector pose $\boldsymbol{x}_e$
before grasping, as shown in Figs. 2 and 16. Preshaping is
usually based on visual cues, such as global object shape
or the detection of affordances, as is the case for the hu-
man. However, proximity sensing opens an opportunity in
robotics to implement this behavior by closing the perception
loop to increase robustness and performance, i. e. an ad-hoc
solution. In fact, the traditional, human-like preshaping can
be considered to be a high-level behavior (BT-II) and is
the result of *grasp synthesis* or *grasp planning*, which is
an active research field [126]. In contrast, preshaping using
feedback from proximity sensors is often reflex-like, i. e. BT-
I. We, therefore, call it *reactive preshaping* (see also Fig. 4).
Reactive preshaping has significant potential for HRI, because
the automatic adaption of the robot hand to the object pose
generalizes naturally to the case when the object is not static,
for instance during handover tasks.

Reactive preshaping enhances grasping robustness and per-
formance, from reaching to grasping motions, for the following
reasons (see also Fig. 2):

<figure><img src="image_17.png" /><figcaption>Fig. 16. Reactive control for preshaping is implemented by feeding back the outputs obtained from the proximity sensors mounted on a robot hand, i. e. its palm and fingers, to controllers that affect the pose $q_h^i$ of each finger and the end-effector $\boldsymbol{x}_e$.</figcaption></figure>

* The contact area after grasping is increased and the grasping becomes stable by aligning the normals of the object and fingertip surfaces before establishing contact.

* "[...] ensuring that the fingers contact the object simultaneously can improve the probability of successfully grasping the object" [29], i. e. unwanted object motion is avoided.

* A robot can move fast without moving or damaging an object because sensing is without contact.

* A robot can continuously adjust the end-effector and finger joint poses even when occlusion occurs in the vision sensor.

1) *Reactive Preshaping Control:* To implement reactive preshaping, poses or torques of finger joints are controlled directly based on the signals detected by proximity sensors on the surface of the finger, as illustrated by Fig. 16. An early contribution in this area is due to Espiau and Catros [60], who show closed-loop control of a two-jaw gripper. Mayton et al. [29] realized finger reactive preshaping using the Barrett Hand with mid/short-range electric field (capacitive) sensors. Each finger was controlled independently by PID control of the motor current in the hand using a target proximity sensor target value. They demonstrated reactive preshaping on a banana and a juice bottle on a table, as well as grasping these objects. The developed method also allowed the robust handover of objects with the human and detection of the co-manipulation state from the capacitive signals.

Hsiao et al. [58] proposed a reactive grasping controller, in-
cluding a finger distance controller, with real-time calibration
using optical proximity sensor outputs (see also Fig. 10). To
detect the actual distance and posture of an object’s surface,
they proposed a calibration method based on a probabilistic
model using fingertip positions and reflected light intensity
values. The average distance sensing error was reported as
4 mm, the posture error of pitch rotation was 5.3°, and the
posture error of roll rotation was 17.7° degrees for common
objects. At the beginning of the procedure, the fingertips are
controlled using the raw sensor output as a target value. The
target value is then switched from raw value to estimated
distance and posture once the estimated values are calculated.

Escaida Navarro et al. [31], [32] installed capacitive tactile
<figure><img src="image_18.png" /><figcaption>Fig. 17. Independent joint angle control for preshaping using a high-speed proximity sensor as shown by Koyama et al. [67]. (©2019 Sage Publications)</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Fig. 18. Complementing a pointcloud obtained by the MS-Kinect with O-ToF sensor inside the gripper PR-2, as shown by Lancaster et al. [128]. (©2017 IEEE)</figcaption></figure>

proximity sensors on a parallel jaw gripper, and they realized simultaneous control of six DoF of a two-jaw gripper based on a proximity-feedback control. In [31], there are 2 × 2 sensor areas per finger of the gripper, and the posture and position information of the object are detected simultaneously by comparing the intensity of the sensor values in each area. Somewhat similar is the work of Guan et al. [112], who use reactive preshaping in translation and orientation of a gripper to a pole the robot is climbing. In a sensor arrangement comparable to [31], [32], Guo et al. also use a 2×2 of O-BB sensor arrangement for reactive preshaping in [88].

Koyama et al. [65], [66], [67] realized independent control of joint poses of an 8-DoF robot hand (three fingers) using a high-speed optical proximity sensor. They demonstrated reactive preshaping for an apple, a banana, and moving objects on a conveyor or during handover, with simple joint-angle controls (see Fig. 17), updating the sensor value and control every 5 ms. In these preshaping controls [29], [67], the target value is set in advance from experimental data of an object set.

Furthermore, Koyama et al. [127] realized velocity con-
trol of fingers without relying on a previously established
calibration. The control uses *time-to-contact (TTC)* values
calculated using proximity values from the sensor. TTC is
the remaining time until a collision between an object and
the sensor. TTC is a bio-inspired calculation that does not
depend on the surface characteristics of an object. Therefore,
the relative speed between the fingertip and an object can
be controlled without prior reflectance data. The authors also
realized high-speed catching of soft objects using a high-speed,
high-precision proximity sensor in [70]. As the proposed
design has impressive distance resolution, it is possible to
utilize this value to estimate the contact condition of an object.
Elastic pads of 3 mm-thickness in front of the photodiode
provide deformable spacers (see Fig. 11 b)). Any distance
measurement that is equal or closer to the offset provided
by the elastic pads is thus indicative of a contact situation.
A contact situation with a soft object can be detected with
very low contact force. The contact detection enabled the
catching of very soft objects, namely a marshmallow and a
paper balloon, with negligible deformation [71].

In [36], [37] Erickson et al. apply the principles of reactive

preshaping/contour following to the scenario of dressing and
washing of patients. In [37], a 2×3 array on the end-effector is
used to align the end-effectors distance and orientation to the
patient's limbs, also following their contour. They show this
is a viable approach to automated caregiving tasks (dressing,
washing), as visual occlusions are amortized and the detection
of the human is reliable.

C. *Higher Complexity Methods and Behaviors (Towards BT-II)*

In this subsection, we discuss what contributions can be
found in the literature regarding cognitive and model-based
methods and behaviors (BT-II, see Sec. II-B and Fig. 4).

1) *Pre-touch Exploration (AT-II)*: Proximity perception opens the opportunity of aggregating information to an object model without mechanical contact. Pre-touch exploration means executing a systematic strategy using the robot's tool to acquire and aggregate pre-touch data into an object model. Jiang et al. present an exploration strategy for completing a point cloud obtained from an RGB-D camera in [104], [105]. The object point clouds originating from these cameras are incomplete on two accounts, i. e. the occluded backside of the object and due to translucency. These perception gaps undermine grasp-planning algorithms that have to rely on object geometry knowledge. In their approach, the unknown regions of the objects are explored until stable grasp planning is possible. A similar approach is due to Maldonado et al. [75] who, in addition to completing point clouds for grasp planning tasks, take advantage of the imaging capability of a proposed sensor for the classification of surfaces ( textures). In [88], Guo et al. realized a reactive pre-touch control using an optical proximity sensor. First, in the control loop, the bounding box of the object was detected from Kinect point cloud information, and the initial grasping point was determined. Second, the object shape information was refined by detecting the edge of the object while tracing it with the proximity sensor. Finally, a better grasping position was determined by repeatedly executing grasp planning and pre-touch detection. The researchers realized grasping of tissue paper, a difficult task with only vision/depth and tactile sensor feedback. In [128], Lancaster et al. study the use of deep learning to guide a proximity-based exploration strategy. It yields an improved object model as well as an improved estimate of the object's pose. Fig. 18 shows an example of a point cloud that is being complemented by pre-touch exploration. In [72], Patel et al. use their optical
proximity sensors inside a two jaw gripper to scan objects by moving the gripper around them. The robot's kinematics allows the easy aggregation of the data to point cloud data. A similar result is shown by Markvicka et al. in [74], where they discuss the scanning of a model space shuttle with a robot hand with enough details to capture its most important features (fuselage, wings, etc.).

In [31], Escaida Navarro et al. show object exploration as an application of proximity serving. Edges can be explored continuously by adjusting the gripper pose as the exploration progresses. To obtain the precise location of corners, the strategy is complemented by the acquisition of tactile samples, iteratively, in the regions where corner candidates are detected in proximity mode. In [129] and [130], Kaboli et al. use a multi-modal sensor skin (see [59]) to explore and classify objects according to haptic properties. The number and location of objects on a table in the workspace of the robot are determined by a Bayesian pre-touch exploration strategy. A similar approach for scanning a table workspace for graspable objects, using O-ToF, is followed by Yin et al. in [82]. In [35], Mülbacher-Karrer et al. investigate how capacitive sensing can be used to sense the fill state of bottles using a hand equipped with sensors. The fill state is explored by tilting the bottle with the hand, whereby the decision (full or empty) is decided in a Bayesian framework. Finally, in [64] it is proposed to use proximity sensing is used for increasing robustness in crack detection, although the exploration procedure itself is contact-based.

2) *Bio-inspired Sensing and Behavior (AT-I, AT-II)*: In robotics, the mimicry of the behavior of weakly electric fish has delivered interesting results, which can be considered to be an inspiration for proximity sensing in general. In [48], Boyer et al. study basic control laws of a cylindrical under-water probe, featuring capacitive-like sensing based on a dipole-type arrangement, i. e. a voltage imposed on the tail and current measured on the tip. A model for the electrosense yields justification for a set of basic, reflex-like control laws that govern the behavior of the probe (AT-I, BT-I). In a related work [131], Bazeille et al. show results for recognition of elliptical objects (AT-II, BT-II). A sequence of measurements of an object is obtained as the probe passes by. The material properties (conductive or insulating) and geometrical properties (ellipse parameters, location, and orientation) are found in an optimization framework. The optimization uses a forward model of the electrosense to find the parameter set that best explains the measured sequence. Tackling a similar problem, Bai et al. use active alignment and machine learning to identify spheroids with a biomimetic probe [49] (AT-II, BT-II). In 2013, Neveln et al. presented a survey paper on the subject of biomimetic robotics related to weakly electric fish that significantly goes beyond the scope of what we can summarize here [132].

3) *Teleoperation and VR (AT-I, AT-II)*: As discussed in Sec. IV-A, teleoperation has been seen as an application of proximity sensing from early on [120], [13], [15]. In none of these approaches, however, did the authors rely on a master-device with force or tactile displaying capability. In more recent approaches, using haptic cues that are originating from

proximity sensors has been investigated. As Huang et al. put it in [77]: “Thus it (the haptic feedback) provides the perceptual benefits of touch interaction to the operator, without relying on the negative consequences of the robot actually contacting unknown geometrical structures.” In [77] by Huang et al., point cloud data collected from a finger-tip sensor is used to generate virtual fixtures. It is also shown that proximity sensing promotes the teleoperation-based exploration of moving objects.

Stoelen et al. [133], [134] presented their approaches using whole-arm sensitive robots. The application is a teleoperated robot arm with shared autonomy, which is based on the signals of the proximity sensors. The authors use machine learning to enable the robot to predict collisions from proximity sensor values based on prior experience. Therefore, the velocity of the robot is limited and force feedback through a *PHANTOM Omni* is provided to the operator in [133]. After three days of experiments based on a virtual environment, the authors show that the completion time of tasks, as well as the workload estimated by the users, decreased using the aid of the controller. In further works, authors have extended the visual perception installed on end-effectors with capacitive proximity sensors. In [32], Escaida Navarro et al. generate 6D force-feedback from the proximity signals to aid the user in exploration tasks and Alagi et al. use tactile feedback with spatial resolution to help users in detecting shape cues in [46]. Both works show how proximity sensing can close the perception gap caused by visual distortions or occlusions in teleoperation.

In [135], remote gesture-based control of a mobile manipulator, based on capacitive proximity sensors, was presented. Different operation modes such as control of the end-effector or the mobile platform were demonstrated. Advantages of the capacitive interface acting as a virtual 3D mouse to control the robot are the robustness of the sensor against water, occlusions or even objects covering parts of the sensor interface (this work does not feature force-feedback).

Finally, the new advances in augmented and virtual reality theories provide a new kind of representation of proximity information. Beyond the visual augmentation, one can combine it with force, tactile, audio [136], or even with transcutaneous electrical stimulation [137], addressing different human sensation to increase the level of presence in teleoperation.

4) *Material Classification (AT-II)*: Beyond object exploration based on its geometry, the internal properties, such as the material or electrical properties, are very valuable knowledge for reliable grasping robust object manipulation. For example, using capacitive sensing, the relative permittivity $ε_r$ of an object can be estimated. It is then possible to classify it according to its material. The exciter frequency dependency of $ε_r(ω)$ can be then utilized to identify the material. Kirchner et al. presented an approach to identify material by performing multi-frequency capacitive sensing [42]. The researchers drove the circuit with three different frequencies and were able to classify 7 different materials. A similar approach was presented in [43], driving the electrodes with 290 exciter frequencies between 10 kHz and 300 kHz and analyzing both the amplitude and the phase of the corresponding signals. This method is also known as *capacitive spectroscopy*, referring to the different exciter frequencies used to perform
the measurements. Furthermore, in [44], another approach for material recognition using the flexible spatial resolution of a capacitive sensor array was presented. The sensors were driven with two excitation frequencies at different electrode configurations, in which the size of the electrodes changed. Using GPR, automated mapping of material layers for investigation of soil composition is possible [97]. Based on pulse-echo ultrasound and optoacoustic effects, Fang et al. reported in [100] the feasibility of integrating optical and acoustical measurement systems into a fingertip of a robotic gripper. A preliminary study showed successful material classification with an accuracy over of 87% for three materials (steel, rubber, and acrylic). It is worth mentioning that estimating material properties also plays a big role in the bio-inspired approaches previously addressed (Sec. IV-C2).

5) *Tracking (AT-I, AT-II)*: Profiting from distributed sensing, object tracking from proximity sensing streams also has been investigated to some extent in the literature. In [55], [56], Petryk and Buehler install four O-RLI-type sensors in a gripper and show that an extended Kalman filter can serve to track the 2D-position of a cylindrical object with respect to the gripper. Furthermore, the reflectance of the object is also estimated. The potential for AT-II is discussed. In [138], the authors show an approach for tracking objects detected on a 3×16 array of capacitive sensors. The task is handled like an image processing problem. Also using a Kalman filter, the authors show the capability of tracking two hands and handling occlusions, i. e. targeting HRI tasks (AT-I).

## D. Industrial Technologies and Solutions

In recent years, proximity sensing technology emerged on the market mainly driven by the industry to deploy collaborative robots in production lines, for instance in the automotive industry. The engineering and technology company BOSCH introduced BOSCH APAS for flexible human-robot collaboration (HRC), which is a mobile robot system for industrial applications. The sensor skin of the manipulator utilizes capacitive based proximity sensor technology (C-M) to enable safe HRC in an industrial manufacturing environment [22], [23]. Also recently, FOGALE Robotics [24] presented a smart skin for robots based on capacitive proximity sensing technology (C-SE). The capacitive based multi-modal (tactile and proximity) robot skin reaches a sensor range of up to 300 mm, where the electrodes are arranged in a matrix structure on the surface of the robot manipulator. In [25], the skin was utilized together with a control framework to avoid obstacles for HRI. The KUKA system partner MRK-Systeme provides a sensor skin solely based on capacitive sensor technology for KUKA robots for industrial HRC applications. In [26] MRK-Systeme presented capacitive based proximity perception for HRI (C-SE) in industrial environments, utilizing a sensor front-end with an electrode configuration able to achieve up to 350 mm of sensing range on a KUKA KR6 manipulator.

## V. FUTURE PERSPECTIVES: GRAND CHALLENGES

In this section, we provide an outlook for the domain of human-centered proximity perception in robotics. We have

highlighted what we think are *grand challenges* at the end of each of the three sub-sections.

### A. Human-robot interaction in the industry and service domains

Proximity perception technology is mature enough to be deployed under strict safety requirements, as discussed in Sec. IV-D. However, today, collaborative automation still struggles to be an interesting value proposition, i. e. providing a large increase in efficiency that will justify the cost of investing in this technology. A value proposition that is more likely to be attractive in the near future is that of *fenceless automation*. Here, humans and robots coexist in the same space but do not necessarily share a task. Value is added for instance by the fact that real-estate on the shop-floor is saved or because robotic automation is now possible in spaces that were previously considered to be too small. Nonetheless, safety certification is still a major challenge that is preventing wide-spread use of proximity perception. Today, technologies such as the ones discussed in IV-D need to be certified on a solution level. Certification at the modular sensor level is not yet possible. Therefore, technologies such as radar (see Sec. III-C) can be attractive alternatives for HRI in industrial automation. Radar is more likely to achieve a safety rating on modular level soon, also profiting from all the prior experience coming from the developments in autonomous driving.

Areas such as medical robotics face similar issues for commercialization. The process of certifying the solutions is long and costly. However, we think that the market for automated solutions in health care or service robotics based on proximity perception is there, especially in scenarios where the human is very close to the robot, preventing the sole use of cameras. In this survey, we discussed examples such as grasping of moving objects and handover [29], [66], [67], dressing and washing of patient [36], [37], or assistive robotics based on teleoperation [134], [133]. Overall, we think that medical and service robotics is a promising field for proximity perception, but more research is needed to establish use-cases and the corresponding methods before commercial exploitation appears on the horizon. Furthermore, the study of highly redundant robotic systems that use proximity sensing is scarce. Meanwhile, the exploitation of kinematic redundancy based on proximity sensor feeds is natural, as discussed in Secs. IV-A1 and IV-A3. This includes the use of proximity perception in unusual areas, such as on the legs or feet of robots. More research, like the one done in the group of Prof. Cheng [7], is needed. In summary, we can say the following about the challenges in these domains:

* Lowering the difficulties in achieving safety ratings for proximity sensing technologies will significantly expand the market for fenceless or collaborative automation. An effective procedure for safety certification, elaborated by industrial stakeholders and certification organizations, is needed.

* Modularized technologies, such as radar chips, can have significant advantages in a certification procedure because solutions can be based on safety rated components.
<figure><img src="image_20.png" /><figcaption>Fig. 19. Proximity perception can play an important role in multi-modal exploration: a) Shows a typical scene with objects as captured by an RGBD-camera. b) A perspective shift reveals considerable gaps in the visual perception, having occluded elements that could be explored contactless and contact-based exploration steps, c) and d) respectively. A multi-modal cognitive model controls the exploration and aggregates the information to the object model, which includes geometric, material property, fill-level information as well as potential grasping regions obtained from the exploration steps in c) and d). This aggregated information is illustrated in e). The yellow and red colors on the mug (in c)-d)) illustrate regions that are extracted from the proximity and touch-based perception, respectively.</figcaption></figure>

* In the service, medical and active assisted living domains proximity perception allows for new ways of interaction between humans and robots yet more research is needed before commercially viable applications are established.

* A better understanding of how highly redundant robotic systems can profit from proximity perception is needed.

* The potential of proximity perception for increasing robustness of grasping of moving objects, for instance during handover tasks or in teleoperation, needs to be established further, for example with conclusive user studies.

## B. Cognitive Robotics

One of the hallmarks of cognitive robotics is active perception. It is the paradigm in which perception is enabled by purposeful actions, i. e. exploration). The active perception principle has been studied in many robotics research papers, as discussed in a survey by Bohg et al. [139]. Machine learning is essential in active perception for tasks such as object pose estimation or scene labeling, which can be based on proximity perception, e. g. [128]. Currently, a trend in active perception is the combined perception of vision and touch. We think that extending this trend to include proximity sensing will be a relevant research question in the near future. Therefore, we will see haptic exploration strategies that will be able to explore occluded regions of the workspace combining touch-based and touch-less exploratory movements. We have illustrated this possible multi-modal exploration approach in Fig. 19. Some aspects of it have already been addressed in the literature discussed in this paper, e. g. [75], [35], [140], [128].

Another important domain in this area is the multi-modal modeling of the human for safe interaction and collaboration. In summary, regarding the major challenges we can say that:

* As we mentioned throughout the paper, there is a need for proximity sensing technologies that can easily be combined with vision and/or touch so they can be deployed together. This will make it attractive to include proximity sensing in established and novel active perception/cognitive approaches.

* It can be expected that the trend regarding sim-to-real learning is going to prevail for the next years. Therefore, it is a challenge to implement realistic simulation models for the different measurement principles discussed in Sec. III. In most cases, current approaches to model them are not viable in terms of their temporal performance. Overcoming this is an important challenge.

## C. Soft Robotics

We think that a good portion of the topics relevant for proximity perception will eventually transfer to Soft Robotics. In this way, soft manipulators equipped with proximity sensors can perform collision avoidance in an analogous way as described in Sec. IV-A, i. e. respecting task hierarchies. Similarly, soft robots will be able to execute reactive preshaping, grasping, and exploration tasks using proximity sensors. Arguably more so than in other areas, in Soft Robotics it is of interest to study how to purposefully engage in contacts to achieve the desired task. Proximity perception can help in finding and reaching desired contact states. Overall, the support of simulation frameworks for model-based control and sensing, such as SOFA [141], will also be relevant. Therefore, regarding the major challenges in this domain, we can say that:

* New control methods must be found for soft robots to integrate the information provided by the proximity sensors and adapt the actuation strategy adequately.

* Integration of proximity sensors in deformable structures represents an important challenge. A significant cross-talk between the global deformation and the detection of tactile and proximity events has to be expected. As stated in the previous section, appropriate models for proximity sensors are needed for sim-to-real learning or for interpreting their signals correctly under deformation using interactive simulations [142], [141].

* Except for capacitive sensing, the realization of proximity sensors having deformable or stretchable sensing elements is challenging. Shrinking the size of rigid components, such as ICs, within a stretchable substrate may still allow the realization of deformable circuits, as described e. g. in [143], [81].

# VI. SUMMARY AND CONCLUSIONS

In this paper, we have given an overview of the main aspects of proximity sensing in today's robotic landscape. Considering that the field has not had a significant formalization over the years, we provide a basic scheme for categorization of the robotic applications and technologies based on proximity sensors and we propose a set of traits that characterize proximity
sensors in human-centered robotics. We give an account of the existing technologies and the main measurement principles reported by authors since the early 1970s and have organized the technologies in Table I, including characteristics such as sensing range, update rate, sensing element size, etc. We then proceeded to detail how the technologies have been used for implementing applications such as collision avoidance and human-robot interaction (AT-I) as well as preshaping and grasping (AT-II). We start with the seminal developments of the early years in these domains and cover the progress up to today (2021). The tight integration into the sensory-motor functionality has received constant attention over the years in order to realize highly reactive behavior of robotic systems (BT-I). Meanwhile, as the area of robotics progresses as a whole, we report that more and more approaches begin to have cognitive aspects in them (BT-II). BT-II includes areas such as teleoperation, where the human is in the loop, autonomous object exploration and even bio-inspired approaches that mimic weakly electric fish.

Finally, Sec. V is dedicated to summarizing our projections for the field regarding the grand challenges we have identified. We think that as the technology of proximity sensors is reaching the maturity to coexist with tactile and visual perception in terms of integration, costs, and norm conformity, it will be adopted for a variety of solutions, especially those involving interaction with humans.

## ACKNOWLEDGMENT

This work was supported by the Region Hauts-de-France, the project COMOROS (ANR-17-ERC2-0029), the European Regional Development Fund (ERDF) and the project Inventor (I-SITE ULNE, le programme d'Investissements d'Avenir, Métropole Européenne de Lille).

This work has received funding from the "Kärntner Wirtschaftsförderung Fonds" (KWF) and the "European Regional Development Fund" (EFRE) within the CapSize project 26616/30969/44253.

This work also was supported by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) within the project (Verbundprojekt-Nr.: 16SV7823K.).

The authors would like to thank cartoonist Adriana Filippini for the illustrations.

## REFERENCES

[1] T. Zhou, M. Yang, K. Jiang, H. Wong, and D. Yang, "MMW radar-based technologies in autonomous driving: A review," *Sensors*, vol. 20, no. 24, p. 7283, 2020.

[2] Z. Wang, Y. Wu, and Q. Niu, "Multi-sensor fusion in automated driving: A survey," *IEEE Access*, vol. 8, pp. 2847–2868, 2019.

[3] M. Pérez, D. Gualda, J. Vicente, J. Villadangos, and J. Ureña, "Review of uav positioning in indoor environments and new proposal based on us measurements," in *Proceedings of the 10th International Conference on Indoor Positioning and Indoor NavigationWork-in-Progress (IPIN-WIP) Papers*, Pisa, Italy, 2019, pp. 267–274.

[4] E. Hall, *The Hidden Dimension*. Anchor Books, 1992.

[5] *Panda - Datasheet*, Franka Emika, 2019.

[6] *LBR iiwa, Datasheet*, KUKA, 2017.

[7] G. Cheng, E. Dean-Leon, F. Bergner, J. R. G. Olvera, Q. Leboutet, and P. Mittendorfer, "A comprehensive realization of robot skin: Sensors, sensing, control, and applications," *Proceedings of the IEEE*, vol. 107, no. 10, pp. 2034–2051, 2019.

[8] L. K. Baxter, *Capacitive sensors: design and applications*, 1st ed., ser. IEEE Press series on electronics technology. New York, NY: IEEE Press, 1997.

[9] T. Grosse-Puppendahl, C. Holz, G. Cohn, R. Wimmer, O. Bechtold, S. Hodges, M. S. Reynolds, and J. R. Smith, "Finding common ground: A survey of capacitive sensing in human-computer interaction," in *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems*, 2017, pp. 3293–3315.

[10] S. Mühlbacher-Karrer, M. Brandstötter, D. Schett, and H. Zangl, "Contactless control of a kinematically redundant serial manipulator using tomographic sensors," *IEEE Robotics and Automation Letters*, vol. 2, no. 2, pp. 562–569, April 2017.

[11] Y. Yamada, N. Tsuchida, and M. Ueda, "A Proximity-Tactile Sensor to Detect Obstacles for a Cylindrical Arm," *Journal of the Robotics Society of Japan*, vol. 6, no. 4, pp. 292–300, 1988.

[12] J. M. Vranish, R. L. McConnell, and S. Mahalingam, "Capaciflector collision avoidance sensors for robots," *Computers & Electrical Engineering*, vol. 17, no. 3, pp. 173–179, Jan. 1991. [Online]. Available: http://www.science direct.com/science/article/pii/004579069190033V

[13] D. Wegerif, D. Rosinski, and W. Parton, "Whole arm proximity control system for articulated robots working near space vehicles and flight hardware," in *Space Programs and Technologies Conference and Exhibit*, 1993, p. 4158.

[14] J. L. Novak and J. Feddema, "A Capacitance-based Proximity Sensor for Whole arm Obstacle Avoidance," in *Robotics and Automation, 1992. Proceedings., 1992 IEEE International Conference on*, May 1992, pp. 1307–1314 vol.2.

[15] J. Feddema and J. L. Novak, "Whole Arm Obstacle Avoidance for Teleoperated Robots," in *Robotics and Automation, 1994. Proceedings., 1994 IEEE International Conference on*, May 1994, pp. 3303–3309 vol.4.

[16] T. Schlegl, T. Kroger, A. Gaschler, O. Khatib, and H. Zangl, "Virtual Whiskers - Highly Responsive Robot Collision Avoidance," in *Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on*, Nov 2013, pp. 5373–5379.

[17] S. Mühlbacher-Karrer and H. Zangl, "Object detection based on electrical capacitance tomography," in *IEEE Sensors Applications Symposium*, 2015.

[18] W. Stiehl and C. Breazeal, "A Sensitive Skin for Robotic Companions Featuring Temperature, Force, and Electric Field Sensors," in *Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on*, Oct 2006, pp. 1952–1959.

[19] S. Escaida Navarro, S. Koch, and B. Hein, "3D Contour Following for a Cylindrical End-Effector Using Capacitive Proximity Sensors," in *Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on*, 2016.

[20] Y. Ding, F. Wilhelm, L. Faulhammer, and U. Thomas, "With proximity servoing towards safe human-robot-interaction," in *2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2019, pp. 4907–4912.

[21] S. Tsuji and T. Kohama, "Proximity and contact sensor for human cooperative robot by combining time-of-flight and self-capacitance sensors," *IEEE Sensors Journal*, 2020.

[22] BOSCH, "APAS Human-robot collaboration." [Online]. Available: https://www.boschapas.com/en/human-robot-collaboration/

[23] J. Frangen, "Sensorsystem zur umfeldberwachung an einem mechanischen bauteil und ein verfahren zur ansteuerung und auswertung des sensorsystems," DE Patent DE102 009 029 021A1, 2009.

[24] FOGALE robotics, "Sensitive surfaces for human-robot interactions and cooperation," accessed: 2019-12-10. [Online]. Available: http://www.fogale-robotics.com/

[25] K.-E. M'Colo, B. Luong, A. Crosnier, C. Néel, and P. Fraisse, "Obstacle avoidance using a capacitive skin for safe human-robot interaction," in *2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2019, pp. 6742–6747.

[26] A. Hoffmann, A. Poeppel, A. Schierl, and W. Reif, "Environment-aware proximity detection with capacitive sensors for human-robot-interaction," in *2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2016, pp. 145–150.

[27] J. R. Smith, E. Garcia, R. Wistort, and G. Krishnamoorthy, "Electric field imaging pretouch for robotic grasbers," in *2007 IEEE/RSJ International Conference on Intelligent Robots and Systems*. IEEE, 2007, pp. 676–683.

[28] R. Wistort and J. R. Smith, "Electric field servoing for robotic manipulation," in *IEEE International Conference on Intelligent Robots and Systems*, September 2008, pp. 494 – 499.
[29] B. Mayton, L. LeGrand, and J. R. Smith, "An Electric Field Pretouch System for Grasping and Co-Manipulation," in Robotics and Automation (ICRA), 2010 IEEE International Conference on. IEEE, 2010, pp. 831-838.

[30] D. Göger, H. Alagi, and H. Wörn, "Tactile proximity sensors for robotic applications," in 2013 IEEE International Conference on Industrial Technology (ICIT), Feb 2013, pp. 978-983.

[31] S. Escaida Navarro, M. Schonert, B. Hein, and H. Wörn, "6D Proximity Servoing for Preshaping and Haptic Exploration using Capacitive Tactile Proximity Sensors," in Intelligent Robots and Systems (IROS), 2014 IEEE/RSJ International Conference on, 2014, pp. 7-14.

[32] S. Escaida Navarro, F. Heger, F. Putze, T. Beyl, T. Schultz, and B. Hein, "Telemanipulation with Force-based Display of Proximity Fields," in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, 2015, pp. 4578-4574.

[33] D. Göger, M. Blankertz, and H. Wörn, "A Tactile Proximity Sensor," in IEEE Sensors 2010, 2010, pp. 589-594.

[34] S. Escaida Navarro, H. Alagi, A. Heilig, M. Mende, and B. Hein, "Flexible Spatial Resolution for Preshaping with a Modular Capacitive Tactile Proximity Sensor," in See, Touch, and Hear: 2nd Workshop on multimodal sensor-based robot control for HRI and soft manipulation at IROS 2016, 2016.

[35] S. Mühlbacher-Karrer, A. Gaschler, and H. Zangl, "Responsive fingers - capacitive sensing during object manipulation," in IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Germany, September 28 - October 02 2015.

[36] Z. Erickson, M. Collier, A. Kapusta, and C. C. Kemp, "Tracking human pose during robot-assisted dressing using single-axis capacitive proximity sensing," IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2245-2252, 2018.

[37] Z. Erickson, H. M. Clever, V. Gangaram, G. Turk, C. K. Liu, and C. C. Kemp, "Multidimensional capacitive sensing for robot-assisted dressing and bathing," in 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), 2019, pp. 224-231.

[38] L. Faller, C. Stetco, and H. Zangl, "Design of a novel gripper system with 3d and inkjet-printed multimodal sensors for automated grasping of a forestry robot," in IEEE IROS 2019, Macau, CN, 2019.

[39] H.-K. Lee, S.-I. Chang, and E. Yoon, "Dual-Mode Capacitive Proximity Sensor for Robot Application: Implementation of Tactile and Proximity Sensing Capability on a Single Polymer Platform Using Shared Electrodes," Sensors Journal, IEEE, vol. 9, no. 12, pp. 1748-1755, 2009.

[40] H. S. Han, J. Park, T. D. Nguyen, U. Kim, C. T. Nguyen, H. Phung, and H. R. Choi, "A highly sensitive dual mode tactile and proximity sensor using carbon microcoils for robotic applications," in 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2016, pp. 97-102.

[41] H. Alagi, S. Escaida Navarro, M. Mende, and B. Hein, "A Versatile and Modular Capacitive Tactile Proximity Sensor," in Haptics Symposium 2016, 2016, pp. 290-206.

[42] N. Kirchner, D. Hordern, D. Liu, and G. Dissanayake, "Capacitive sensor for object ranging and material type identification," Sensors and Actuators A: Physical, vol. 148, no. 1, pp. 96-104, 2008.

[43] Y. Ding, H. Zhang, and U. Thomas, "Capacitive proximity sensor skin for contactless material detection," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 7179-7184.

[44] H. Alagi, A. Heilig, S. Escaida Navarro, T. Kroeger, and B. Hein, "Material Recognition Using a Capacitive Proximity Sensor with Flexible Spatial Resolution," in Intelligent Robots and Systems (IROS), 2018 IEEE/RSJ International Conference on, 2018.

[45] S. Mühlbacher-Karrer and H. Zangl, "Electrical capacitance tomography with a variable topology," in 2015 9th International Conference on Sensing Technology (ICST), Dec 2015, pp. 565-569.

[46] H. Alagi, S. Escaida Navarro, J. Hergenhan, S. Music, and B. Hein, "Teleoperation with tactile feedback based on a capacitive proximity sensor array," in IEEE International Instrumentation and Measurement Technology Conference, May 2020.

[47] F. Boyer and V. Lebastard, "Electric sensing for underwater navigation," Living machines: a handbook of research in biomimetics and biohybrid systems. Oxford University Press, Oxford, 2018.

[48] F. Boyer, V. Lebastard, C. Chevallereau, and N. Servagent, "Underwater reflex navigation in confined environment based on electric sense," IEEE transactions on robotics, vol. 29, no. 4, pp. 945-956, 2013.

[49] Y. Bai, J. B. Snyder, M. Peshkin, and M. A. MacIver, "Finding and identifying simple objects underwater with active electrosense," The International Journal of Robotics Research, vol. 34, no. 10, pp. 1255-1277, 2015.

[50] A. Poeppel, A. Hoffmann, M. Siehler, and W. Reif, "Robust distance estimation of capacitive proximity sensors in hri using neural networks," in 2020 Fourth IEEE International Conference on Robotic Computing (IRC). IEEE, 2020, pp. 344-351.

[51] R. A. Lewis and A. K. Bejczy, "Planning Considerations for a Roving Robot with Arm," in The 3rd international joint conference on Artificial intelligence. IJCAI, 1973, p. 308316.

[52] A. R. Johnson, "Optical proximity sensors for manipulators," NASA, Tech. Rep., 1973. [Online]. Available: https://ntrs.nasa.gov/search.jsp?R=19740005038

[53] E. Cheung and V. Lumelsky, "A Sensitive Skin System for Motion Control of Robot Arm Manipulators," Robotics and Autonomous Systems, vol. 10, no. 1, pp. 9-32, 1992. [Online]. Available: http://www.sciencedirect.com/science/article/pii/092188909290012N

[54] D. G. Wegerif and D. J. Rosinski, "Sensor-based whole-arm obstacle avoidance for kinematically redundant robots," in Sensor Fusion V, vol. 1828. International Society for Optics and Photonics, 1992, pp. 417-427.

[55] G. Petryk and M. Buehler, "Dynamic object localization via a proximity sensor network," in 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems. IEEE, 1996, pp. 337-341.

[56] ——, "Robust estimation of pre-contact object trajectories," IFAC Proceedings Volumes, vol. 30, no. 20, pp. 793-799, 1997.

[57] A. Tar, M. Koller, and G. Cserey, "3D geometry reconstruction using Large Infrared Proximity Array for Robotic Applications," in Mechatronics, 2009. ICM 2009. IEEE International Conference on, April 2009, pp. 1-6.

[58] K. Hsiao, P. Nangeroni, M. Huber, A. Saxena, and A. Ng, "Reactive Grasping using Optical Proximity Sensors," in Robotics and Automation, 2009. ICRA '09. IEEE International Conference on, 2009, pp. 2098-2105.

[59] P. Mittendorfer and G. Cheng, "Humanoid Multimodal Tactile-Sensing Modules," Robotics, IEEE Transactions on, vol. 27, no. 3, pp. 401-410, June 2011.

[60] B. Espiau, C. JY et al., "Use of optical reflectance sensors in robotics applications," IEEE Transactions on Systems, Man, and Cybernetics, pp. 903-912, 1980.

[61] S. Walker, K. Loewke, M. Fischer, C. Liu, and K. Salisbury, "An Optical Fiber Proximity Sensor for Haptic Exploration," in Robotics and Automation, 2007 IEEE International Conference on, April 2007, pp. 473-478.

[62] J. Konstantinova, A. Stilli, and K. Althoefer, "Force and proximity fingertip sensor to enhance grasping perception," in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015, pp. 2118-2123.

[63] J. Konstantinova, A. Stilli, A. Faragasso, and K. Althoefer, "Fingertip proximity sensor with realtime visual-based calibration," in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 170-175.

[64] F. Palermo, J. Konstantinova, S. Poslad, K. Althoefer, and I. Farkhatinov, "Automatic fracture characterization using tactile and proximity optical sensing," Frontiers in Robotics and AI, 2020.

[65] K. Koyama, H. Hasegawa, Y. Suzuki, A. Ming, and M. Shimojo, "Preparing for various objects by the robot hand equipped with resistor network structure proximity sensors," in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2013, pp. 4027-4033.

[66] K. Koyama, Y. Suzuki, A. Ming, and M. Shimojo, "Integrated control of a multi-fingered hand and arm using proximity sensors on the fingertips," in 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2016, pp. 4282-4288.

[67] K. Koyama, M. Shimojo, A. Ming, and M. Ishikawa, "Integrated control of a multiple-degree-of-freedom hand and arm using a reactive architecture based on high-speed proximity sensing," The International Journal of Robotics Research, vol. 38, no. 14, pp. 1717-1750, 2019.

[68] H. Hasegawa, Y. Suzuki, A. Ming, K. Koyama, M. Ishikawa, and M. Shimojo, "Net-Structure Proximity Sensor: High-Speed and Free-Form Sensor With Analog Computing Circuit," Mechatronics, IEEE/ASME Transactions on, vol. 20, no. 6, pp. 3232-3241, 2015.

[69] H. Arita and Y. Suzuki, "Contact transition control by adjusting emitting energy of proximity sensor," Advanced Robotics, vol. 35, no. 2, pp. 93-107, 2021.

[70] K. Koyama, M. Shimojo, T. Senoo, and M. Ishikawa, "High-speed high-precision proximity sensor for detection of tilt, distance, and contact," IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3224-3231, 2018.
[71] K. Koyama, K. Murakami, T. Senoo, M. Shimojo, and M. Ishikawa, "High-speed, small-deformation catching of soft objects based on active vision and proximity sensing," *IEEE Robotics and Automation Letters*, vol. 4, no. 2, pp. 578-585, 2019.

[72] R. Patel, R. Cox, and N. Correll, "Integrated proximity, contact and force sensing using elastomer-embedded commodity proximity sensors," *Autonomous Robots*, pp. 1-16, Apr. 2018. [Online]. Available: https://link.springer.com/article/10.1007/s10514-018-9751-4

[73] D. Hughes, J. Lammie, and N. Correll, "A Robotic Skin for Collision Avoidance and Affective Touch Recognition," *IEEE Robotics and Automation Letters*, vol. 3, no. 3, pp. 1386-1393, Jul. 2018.

[74] E. J. Markvicka, J. M. Rogers, and C. Majidi, "Wireless electronic skin with integrated pressure and optical proximity sensing," in 2020 IEEE/RSJ International Conference on Autonomous Robots and Systems (IROS). IEEE, 2020, pp. 8882-8888.

[75] A. Maldonado, H. Alvarez-Heredia, and M. Beetz, "Improving Robot Manipulation Through Fingertip Perception," in *Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on*, oct. 2012, pp. 2947-2954.

[76] B. Yang, P. Lancaster, and J. R. Smith, "Pre-touch sensing for sequential manipulation," in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 5088-5095.

[77] K. Huang, P. Lancaster, J. R. Smith, and H. J. Chizeck, "Visionless tele-exploration of 3d moving objects," in 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2018, pp. 2238-2244.

[78] P. E. Lancaster, J. R. Smith, and S. S. Srinivasa, "Improved proximity, contact, and force sensing via optimization of elastomer-air interface geometry," in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 3797-3803.

[79] K. Sasaki, K. Koyama, A. Ming, M. Shimojo, R. Plateaux, and J.-Y. Choley, "Robotic grasping using proximity sensors for detecting both target object and support surface," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 2925-2932.

[80] S. Tsuji and T. Kohama, "Proximity Skin Sensor Using Time-of-Flight Sensor for Human Collaborative Robot," *IEEE Sensors Journal*, vol. 19, no. 14, pp. 5859-5864, 2019.

[81] T. Hellebrekers, K. B. Ozutemiz, J. Yin, and C. Majidi, "Liquid metal-microelectronics integration for a sensorized soft robot skin," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 5924-5929.

[82] J. Yin, T. Hellebrekers, and C. Majidi, "Closing the loop with liquid-metal sensing skin for autonomous soft robot gripping," in 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 2020, pp. 661-667.

[83] M. Fuhrman, and T. Kanade, "Optical Proximity Sensor Using Multiple Cones Of Light For Measuring Surface Shape," *Optical Engineering*, vol. 23, no. 5, pp. 9-32, 1984.

[84] N. M. Ceriani, G. B. Avanzini, A. M. Zanchettin, L. Bascetta, and P. Rocco, "Optimal placement of spots in distributed proximity sensors for safe human-robot interaction," in 2013 IEEE International Conference on Robotics and Automation. IEEE, 2013, pp. 5858-5863.

[85] G. B. Avanzini, N. M. Ceriani, A. M. Zanchettin, P. Rocco, and L. Bascetta, "Safety control of industrial robots based on a distributed distance sensor," *IEEE Transactions on Control Systems Technology*, vol. 22, no. 6, pp. 2127-2140, 2014.

[86] A. Bonen, R. E. Saad, K. C. Smith, and B. Benhabib, "A novel electrooptical proximity sensor for robotics: calibration and active sensing," in *IEEE Transactions on Robotics and Automation*, vol. 13, no. 3, pp. 377-386, 1997.

[87] M. Teichmann and B. Mishra, "Reactive robotics i: Reactive grasping with a modified gripper and multifingered hands." *The International Journal of Robotics Research*, vol. 19, no. 7, p. 697708, 2000.

[88] D. Guo, P. Lancaster, L.-T. Jiang, F. Sun, and J. R. Smith, "Transmissive Optical Pretouch Sensing for Robotic Grasping," in *Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on*. IEEE, 2015, pp. 5891-5897.

[89] M. Gerstmair, A. Melzer, A. Onic, and M. Huemer, "On the safe road toward autonomous driving: Phase noise monitoring in radar sensors for functional safety compliance," *IEEE Signal Processing Magazine*, vol. 36, no. 5, pp. 60-70, 2019.

[90] B. van Berlo, A. Elkelany, T. Ozcelebi, and N. Meratnia, "Millimeter wave sensing: A review of application pipelines and building blocks," *arXiv preprint arXiv:2012.13664*, 2020.

[91] J. Lien, N. Gillian, M. E. Karagozler, P. Amihood, C. Schwesig, E. Olson, H. Raja, and I. Poupyrev, "Soli: Ubiquitous gesture sensing with millimeter wave radar," *ACM Trans. Graph.*, vol. 35, no. 4, pp. 142:1-142:19, Jul. 2016. [Online]. Available: http://doi.acm.org/10.1145/2897824.2925953

[92] C. Stetco, B. Ubezio, S. Mühlbacher-Karrer, and H. Zangl, "Radar Sensors in Collaborative Robotics: Fast Simulation and Experimental Validation," in *Accepted for publication in IEEE International Conference on Robotics and Automation (ICRA)*. IEEE, 2020.

[93] Z. Flintoff, B. Johnston, and M. Liarokapis, "Single-grasp, model-free object classification using a hyper-adaptive hand, google soli, and tactile sensors," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 1943-1950.

[94] M. Geiger and C. Waldschmidt, "160-ghz radar proximity sensor with distributed and flexible antennas for collaborative robots," *IEEE Access*, vol. 7, pp. 14977-14984, 2019.

[95] C. M. Watts, P. Lancaster, A. Pedross-Engel, J. R. Smith, and M. S. Reynolds, "2d and 3d millimeter-wave synthetic aperture radar imaging on a pr2 platform," in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 4304-4310.

[96] "Ground Penetrating Radar (GPR) Equipment | GSSI Inc. | Georadar." [Online]. Available: https://www.geophysical.com/

[97] G. Kouros, I. Kotavelis, E. Skartados, D. Giakoumis, D. Tzovaras, A. Simi, and G. Manacorda, "3d underground mapping with a mobile robot and a gpr antenna," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 3218-3224.

[98] U. Nunes, P. Faia, and A. de Almeida, "Sensor-based 3-D Autonomous Contour-Following Control," in Intelligent Robots and Systems '94. 'Advanced Robotic Systems and the Real World', IROS '94. Proceedings of the IEEE/RSJ/GI International Conference on, vol. 1, Sep 1994, pp. 172-179 vol.1.

[99] P. Dario, A. Sabatini, B. Allotta, M. Bergamasco, and G. Buttazzo, "A Fingertip Sensor with Proximity, Ttactile and Force Sensing Capabilities," in Intelligent Robots and Systems '90. 'Towards a New Frontier of Applications', Proceedings. IROS '90. IEEE International Workshop on, 1990, pp. 883-889 vol.2.

[100] C. Fang, D. Wang, D. Song, and J. Zou, "Toward fingertip non-contact material recognition and near-distance ranging for robotic grasping," in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 4967-4974.

[101] T. Schlegl, T. Bretterklieber, M. Neumayer, and H. Zangl, "Combined capacitive and ultrasonic distance measurement for automotive applications," *IEEE Sensors Journal*, vol. 11, no. 11, pp. 2636-2642, 2011.

[102] J. Steckel and H. Peremans, "Acoustic flow-based control of a mobile platform using a 3d sonar sensor," *IEEE Sensors Journal*, vol. 17, no. 10, pp. 3131-3141, 2017.

[103] T. Verellen, R. Kerstens, and J. Steckel, "High-resolution ultrasound sensing for robotics using dense microphone arrays," *IEEE Access*, vol. 8, pp. 190 083-190 093, 2020.

[104] L.-T. Jiang and J. R. Smith, "Seashell Effect Pretouch Sensing for Robotic Grasping," in *Robotics and Automation (ICRA), 2012 IEEE International Conference on*. IEEE, 2012, pp. 2851-2858.

[105] —, "A Unified Framework for Grasping and Shape Acquisition via Pretouch Sensing," in *Robotics and Automation (ICRA), 2013 IEEE International Conference on*. IEEE, 2013, pp. 999-1005.

[106] T. D. Nguyen, H. S. Han, H.-Y. Shin, C. T. Nguyen, H. Phung, H. V. Hoang, and H. R. Choi, "Highly sensitive flexible proximity tactile array sensor by using carbon micro coils," *Sensors and Actuators A: Physical*, vol. 266, pp. 166-177, Oct. 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0924424717303837

[107] T. D. Nguyen, T. Kim, J. Noh, H. Phung, G. Kang, and H. R. Choi, "Skin-type proximity sensor by using the change of electromagnetic field," *IEEE Transactions on Industrial Electronics*, vol. 68, no. 3, pp. 2379-2388, 2020.

[108] B. George, H. Zangl, T. Bretterklieber, and G. Brasseur, "A combined inductivecapacitive proximity sensor for seat occupancy detection," *IEEE Transactions on Instrumentation and Measurement*, vol. 59, no. 5, pp. 1463-1470, 2010.

[109] H. Gietler, H. Ammari, and H. Zangl, "Robust electromagnetic pose estimation for robotic applications," *IEEE Transactions on Instrumentation and Measurement*, pp. 1-1, 2019.

[110] R. A. Russell and J. A. Wijaya, "Object location and recognition using whisker sensors," in *Australasian Conference on Robotics and Automation*. Citeseer, 2003, pp. 761-768.

[111] N. F. Lepora, M. Pearson, and L. Cramphorn, "Tacwhiskers: Biomimetic optical tactile whiskered robots," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 7628-7634.
[112] Y. Guan, Z. Xiao, W. Wu, J. Wu, H. Zhu, M. Su, and H. Li, "Gripper self-alignment for autonomous pole-grasping with a biped climbing robot," in Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on. IEEE, 2012, pp. 181–186.

[113] B. Zhang, Z. Xiang, S. Zhu, Q. Hu, Y. Cao, J. Zhong, Q. Zhong, B. Wang, Y. Fang, B. Hu, J. Zhou, and Z. Wang, "Dual Functional Transparent Film for Proximity and Pressure Sensing," Nano Research, vol. 7, no. 10, pp. 1488–1496, 2014.

[114] C. Samson and B. Espiau, "Application of the task-function approach to sensor-based control of robot manipulators," IFAC Proceedings Volumes, vol. 23, no. 8, pp. 269–274, 1990.

[115] A. A. Maciejewski and C. A. Klein, "Obstacle avoidance for kinematically redundant manipulators in dynamically varying environments," The international journal of robotics research, vol. 4, no. 3, pp. 109–117, 1985.

[116] G. J. Tamasy, "Smart sensor networks for robotic sensor skins," Sensor Review, vol. 17, no. 3, pp. 232–239, 1997.

[117] J. L. Novak, J. T. Feddema, N. E. Miner, and S. A. Stansfield, "Collision avoidance during teleoperation using whole-arm proximity sensors coupled to a virtual environment," in Telemanipulator Technology and Space Telerobotics, vol. 2057. International Society for Optics and Photonics, 1993, pp. 2–10.

[118] E. Cheung and V. Lumelsky, "Motion planning for robot arm manipulators with proximity sensing," in Proceedings. 1988 IEEE International Conference on Robotics and Automation. IEEE, 1988, pp. 740–745.

[119] ——, "Development of sensitive skin for a 3d robot arm operating in an uncertain environment," in Proceedings, 1989 International Conference on Robotics and Automation. IEEE, 1989, pp. 1056–1061.

[120] V. J. Lumelsky and E. Cheung, "Real-Time Collision Avoidance in Teleoperated Whole-Sensitive Robot Arm Manipulators," Systems, Man and Cybernetics, IEEE Transactions on, vol. 23, no. 1, pp. 194–203, Jan 1993.

[121] T. Kröger, On-Line Trajectory Generation in Robotic Systems: Basic Concepts for Instantaneous Reactions to Unforeseen (Sensor) Events. Springer, 2010, vol. 58.

[122] Y. Ding and U. Thomas, "Collision avoidance with proximity serving for redundant serial robot manipulators," in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020.

[123] O. Khatib, "Real-time obstacle avoidance for manipulators and mobile robots," in Autonomous robot vehicles. Springer, 1986, pp. 396–404.

[124] J. Baeten and J. De Schutter, "Improving force controlled planar contour following using online eye-in-hand vision based feedforward," in Advanced Intelligent Mechatronics, 1999. Proceedings. 1999 IEEE/ASME International Conference on. IEEE, 1999, pp. 902–907.

[125] J. W. Pellegrino, R. L. Klatzky, and B. P. McCloskey, "Timecourse of preshaping for functional responses to objects," Journal of Motor Behavior, vol. 21, no. 3, pp. 307–316, 1989.

[126] J. Bohg, A. Morales, T. Asfour, and D. Kragic, "Data-driven grasp synthesis survey," IEEE Transactions on Robotics, vol. 30, no. 2, pp. 289–309, 2013.

[127] K. Koyama, Y. Suzuki, A. Ming, and M. Shimojo, "Grasping control based on time-to-contact method for a robot hand equipped with proximity sensors on fingertips," in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015, pp. 504–510.

[128] P. Lancaster, B. Yang, and J. R. Smith, "Improved object pose estimation via deep pre-touch sensing," in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 2448–2455.

[129] M. Kaboli, D. Feng, K. Yao, P. Lanillos, and G. Cheng, "A tactile-based framework for active object learning and discrimination using multimodal robotic skin," IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 2143–2150, 2017.

[130] M. Kaboli, D. Feng, and G. Cheng, "Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin," International Journal of Humanoid Robotics, vol. 15, no. 01, p. 1850001, 2018.

[131] S. Bazeille, V. Lebastard, S. Lanneau, and F. Boyer, "Model based object localization and shape estimation using electric sense on underwater robots," IFAC-PapersOnLine, vol. 50, no. 1, pp. 5047–5054, 2017.

[132] I. D. Neveln, Y. Bai, J. B. Snyder, J. R. Solberg, O. M. Curet, K. M. Lynch, and M. A. MacIver, "Biomimetic and bio-inspired robotics in electric fish research," Journal of experimental Biology, vol. 216, no. 13, pp. 2501–2514, 2013.

[133] M. F. Stoelen, V. F. Tejada, A. Jardn, F. Bonsignorio, and C. Balaguer, "Adaptive Aid on Targeted Robot Manipulator Movements in TeleAssistance," Paladyn, Journal of Behavioral Robotics, vol. 7, no. 1, Jan. 2016. [Online]. Available: https://www.degruyter.com/view/j/pjbr.2016.7.issue-1/pjbr-2016-0002/pjbr-2016-0002.xml

[134] M. F. Stoelen, V. F. Tejada, J. G. Victores, A. J. Huete, F. Bonsignorio, and C. Balaguer, "Adaptive collision-limitation behavior for an assistive manipulator," in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nov. 2013, pp. 1143–1148.

[135] C. Stetco, S. Mühlbacher-Karrer, M. Lucchi, M. Weyrer, L. Faller, and H. Zangl, "Gesture-based contactless control of mobile manipulators using capacitive sensing," in IEEE International Instrumentation and Measurement Technology Conference, May 2020.

[136] A. Marquardt, E. Kruijff, C. Trepkowski, J. Maiero, A. Schwandt, A. Hinkenjann, W. Stuerzlinger, and J. Schning, "Audio-tactile proximity feedback for enhancing 3D manipulation," in Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, ser. VRST '18. New York, NY, USA: Association for Computing Machinery, Nov. 2018, pp. 1–10. [Online]. Available: https://doi.org/10.1145/3281505.3281525

[137] Z. Zhao, M. Yeo, S. Manoharan, S. C. Ryu, and H. Park, "Electrically-Evoked Proximity Sensation Can Enhance Fine Finger Control in Telerobotic Pinch," Scientific Reports, vol. 10, no. 1, p. 163, Jan. 2020, number: 1 Publisher: Nature Publishing Group. [Online]. Available: https://www.nature.com/articles/s41598-019-56985-9

[138] S. Escaida Navarro, M. Marufo, Y. Ding, S. Puls, D. Göger, B. Hein, and H. Wörn, "Methods for Safe Human-Robot-Interaction Using Capacitive Tactile Proximity Sensors," in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp. 1149–1154.

[139] J. Bohg, K. Hausman, B. Sankaran, O. Brock, D. Kragic, S. Schaal, and G. S. Sukhatme, "Interactive perception: Leveraging action in perception and perception in action," IEEE Transactions on Robotics, vol. 33, no. 6, pp. 1273–1291, 2017.

[140] J. Varley, C. DeChant, A. Richardson, J. Ruales, and P. Allen, "Shape completion enabled robotic grasping," in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 2442–2447.

[141] S. Escaida Navarro, S. Nagels, H. Alagi, L.-M. Faller, O. Goury, T. Morales-Bieze, H. Zangl, B. Hein, R. Ramakers, W. Deferme et al., "A model-based sensor fusion approach for force and shape estimation in soft robotics," IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5621–5628, 2020.

[142] H.-S. Shin and S. Bergbreiter, "Effect of finger geometries on strain response of interdigitated capacitor based soft strain sensors," Applied Physics Letters, vol. 112, no. 4, p. 044101, Jan. 2018. [Online]. Available: http://aip.scitation.org/doi/10.1063/1.4998440

[143] S. Nagels, R. Ramakers, K. Luyten, and W. Deferme, "Silicone devices: A scalable diy approach for fabricating self-contained multi-layered soft circuits using microfluidics," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–13.

[144] F. Boyer, P. B. Gossiaux, B. Jawad, V. Lebastard, and M. Porez, "Model for a sensor inspired by electric fish," IEEE transactions on robotics, vol. 28, no. 2, pp. 492–505, 2011.

<figure><img src="image_21.png" /></figure>

Stefan Escaida Navarro received his diploma in computer science (Dipl.-Informatiker) from Karl-ruhe Institute of Technology (KIT), Germany in 2010. He obtained is PhD degree (Dr.-Ing.) from the same institution in 2016. During his doctoral phase he researched on haptic object recognition and grasping as well as the technology and the applications for capacitive tactile proximity sensors, such as preshaping, collision avoidance and teleoperation. He is currently a postdoctoral researcher at Inria Lille - Nord Europe, France, where he is working

on model-based force and shape sensing for Soft Robotics.
<figure><img src="image_22.png" /></figure>

**Stephan Mühlbacher-Karrer** received the Dipl.Ing. degree in telematics from the Graz University of Technology (TUG), Graz, Austria, in 2012, where he focused on Autonomous Robots and Telecommunications. In 2017 he received his Dr. tech. degree in information and communications engineering from the Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria, focusing on capacitive sensing for robotic applications. From 2012 to 2013, he was with Gigatronik Ingolstadt GmbH, Gaimersheim, Germany, working for Audi

Electronics Venture GmbH, Gaimersheim, where he was involved in the pre-development for autonomous driving. Since 2017 he is a senior researcher with JOANNEUM RESEARCH ROBOTICS, Institute for Robotics and Mechatronics in Klagenfurt Austria. His current research interests include near field sensor technology, signal processing, and autonomous robots.

<figure><img src="image_23.png" /></figure>

**Keisuke Koyama** received the B.E., M.E. and Ph.D. in Eng. degrees in mechanical engineering and intelligent Systems from the University of Electro-Communications (UEC), in 2013, 2015 and 2017, respectively. He was a research fellow of Japan Society for the Promotion of Science (JSPS) from 2015 to 2017. He researched on high-speed proximity sensor for pre-grasping and integrated control of multi-degree-of-freedom robot hand and arm. From 2017 to 2019, he was project Assistant Professor, Department of Information Physics and Computing,

Graduate School of Information Science and Technology, The University of Tokyo. Since 2019, he has been Assistant Professor, Department of Systems Innovation, Graduate School of Engineering Science, Osaka University. And, he has been Visiting Researcher, Department of Information Physics and Computing, Graduate School of Information Science and Technology, The University of Tokyo. His current research focuses on high-speed, high-precision proximity sensing for high-speed robotic manipulation and assembly.

<figure><img src="image_24.png" /></figure>

**Hosam Alagi** received his M.Sc. degree in electrical engineering and information technology at the Karlsruhe Institute of Technology in Germany with a specialization in System-on-Chip. Since August 2015 he is working as a research assistant at the Institute for Anthropomatics and Robotics (IAR) at Intelligent Process Automation and Robotics Lab (IPR). Focusing on human-robot interaction, he develops capacitive multi-modal sensors and evaluates methods that allow robots to perceive their near environment. Within the scope of the PhD Studies

both the sensors and sensor applications are being implemented, addressing the industrial and domestic domains.

<figure><img src="image_25.png" /></figure>

**Christian Duriez** is Research Director at Inria Lille - Nord Europe. He received the engineering degree from the Institut Catholique d'Arts et Mtiers of Lille, France and a PhD degree in robotics from University of Evry and CEA in France in 2004. He had a postdoctoral position at the CIMIT SimGroup in Boston. He arrived at INRIA in 2006 and worked on interactive simulation of deformable objects and haptic rendering with focus on surgical simulation. He is now the head of DEFROST team, created in January 2015. In 2018 he was invited researcher

at Stanford University. His research topics are Soft Robot models and control, Fast Finite Element Methods, simulation of contact response, new algorithms for haptics He participated to the creation of the open-source SOFA framework. He was also one of the founders of the start-up company InSimo.

<figure><img src="image_26.png" /></figure>

**Björn Hein** studied electrical engineering with a focus on control theory and received his PhD in 2003 concerning automatic collision-free path planning at the Karlsruhe Institute of Technology (KIT). May 2010 he finished his post-doctoral lecture qualification for computer science with stress on human-robot interaction. 2012-2018 he was professor for Interaction Technologies for Robot Systems at the Institute for Anthropomatics and Robotics (IAR) - Intelligent Process Automation and Robotics Lab (IPR). Currently he holds a professorship at the

University of Applied Sciences in Karlsruhe. His research focus comprises: algorithms for collision free motion planning and path optimization, methods for intuitive and automatic programming of robots, human-robot interaction, novel sensor-technologies for enhancing capabilities of robot systems.

<figure><img src="image_27.png" /></figure>

**Hubert Zangl** received the Dipl.Ing. degree in telematics, the Dr. Tech. degree in electrical engineering, and the Venia Docendi degree in sensors and instrumentation from the Graz University of Technology (TUG), Graz, Austria, in 2001, 2005, and 2009, respectively. From 2010 to 2013, he was an Associate Professor of Sensors and Instrumentation with the Institute of Electrical Measurement and Measurement Signal Processing, TUG. Since 2013, he has been a Professor chairing Sensors and Actuators with the Institute of Smart System

Technologies, Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria. His current research interests include the design and optimization of smart sensors and actuators, robustness, and reliability of sensors and actuators, sensor signal processing, autarkic wireless sensors, and energy harvesting with target applications in the field of IoT and robotics.

<figure><img src="image_28.png" /></figure>

**Joshua R. Smith** is the Milton and Delia Zeutschel Professor, jointly appointed in the Allen School of Computer Science and Engineering, and the Department of Electrical Engineering, at the University of Washington, Seattle. He received BA degrees in Computer Science and Philosophy from Williams College in 1991, an MA in Physics from the University of Cambridge in 1997, and SM and PhD degrees in 1995 and 1999 from the MIT Media Lab's Physics and Media Group, where he began working on Electric Field Sensing for input device

applications. He began applying these techniques to robotics while he was a researcher at Intel Labs Seattle, between 2004 and 2010. In recent years his group at UW has developed a number of new approaches to proximity perception for robotics.
# Appendix

## ACRONYMS

**A-S** seashell effect. 10

**A-US** ultrasound. 10

**AT-I** applications of type I. 1–3, 5–7, 9, 11, 12, 16, 17, 19

**AT-II** applications of type II. 1–3, 5–7, 9, 11, 14–17, 19

**BT-I** reactive or reflex-like behavior (low complexity). 3, 5, 11, 12, 14, 16, 19

**BT-II** cognitive or model-based behavior (high complexity). 3, 5, 11, 14–16, 19

**C-M** mutual-capacitance mode. 4–6, 17

**C-SE** capacitive single-ended mode. 4, 5, 11, 17

**CMC** carbon micro coils. 11

**ECT** electrical capacitance tomography. 4

**FMCW** frequency modulated continuous wave. 9, 10

**GPR** ground penetration radar. 10, 17

**HCI** human-computer interaction. 4, 9

**HRC** human-robot collaboration. 3, 17

**HRI** human-robot interaction. 3, 5, 6, 9, 14, 17

**IR** infrared. 5, 6

**O-BB** break-beam. 5–7, 9, 15

**O-RLI** reflected light intensity. 5–9, 11

**O-ToF** time-of-flight. 5–9, 11, 15, 16

**O-Tri** triangulation. 5–7, 9

**PIPR** payload inspection and processing robot. 12

**SAR** synthetic aperture radar. 10

**SNN** smart sensor network. 12

**TTC** time-to-contact. 15

**UAVs** unmanned aerial vehicles. 2, 10

**WHAP** whole-arm proximity. 13
<table><caption>Overview - Sorted by year - Table I</caption><thead><tr><th>Measurement Principle</th><th>Reference</th><th>Year</th><th>Reported Min. Working Distance [mm]</th><th>Reported Max. Range [mm]</th><th>Field of View [degree]</th><th>Measurement Rate [Hz]</th><th>Sensing Element Dimension [mm / mm<sup>2</sup> / mm<sup>3</sup>]</th><th>Multiple-Obstacles</th><th>Commercially Available Core Components</th><th>Categorization</th><th>Basis reference</th></tr></thead><tbody><tr><td>1</td><td>O-Tri</td><td>[52]</td><td>1973</td><td>2</td><td>20</td><td>n/s</td><td>n/s</td><td>3x5x5</td><td>not considered</td><td>no</td><td>AT-II, Manipulation and Grasping</td><td>-</td></tr><tr><td>2</td><td>O-Tri</td><td>[51]</td><td>1973</td><td>2</td><td>20</td><td>n/s</td><td>n/s</td><td>3x5x5</td><td>not considered</td><td>no</td><td>AT-II, Grasping, Teleoperation</td><td>[52]</td></tr><tr><td>4</td><td>O-Tri</td><td>[83]</td><td>1984</td><td>n/s</td><td>v1: focal point 4.5 +0.56 cm, v2: focal point...</td><td>13</td><td>1000</td><td>110</td><td>no</td><td>no</td><td>AT-II, measure distance, orientation and curvature...</td><td>-</td></tr><tr><td>7</td><td>O-RLI</td><td>[118]</td><td>1988</td><td>n/s</td><td>250</td><td>60</td><td>16 whole skin</td><td>min. one sensor pair</td><td>yes</td><td>no</td><td>AT-I, Collision Avoidance</td><td>[53]</td></tr><tr><td>6</td><td>C-M</td><td>[11]</td><td>1988</td><td>40</td><td>100</td><td>n/s</td><td>n/s</td><td>80x210</td><td>not tested</td><td>no</td><td>AT-I, Obstacle Detection</td><td>-</td></tr><tr><td>8</td><td>O-RLI</td><td>[119]</td><td>1989</td><td>n/s</td><td>250</td><td>60</td><td>16 whole skin</td><td>min. one sensor pair</td><td>yes</td><td>no</td><td>AT-I, Collision Avoidance</td><td>[53]</td></tr><tr><td>9</td><td>A-US</td><td>[99]</td><td>1990</td><td>n/s</td><td>200-300</td><td>180</td><td>n/s</td><td>20x20x20</td><td>n/s</td><td>n/s</td><td>AT-II</td><td>-</td></tr><tr><td>10</td><td>C-M</td><td>[12]</td><td>1991</td><td>0</td><td>304.8 human and alu, 127 graphite lead</td><td>n/s</td><td>n/s</td><td>355,6x190,5</td><td>no</td><td>n/s</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>13</td><td>O-RLI</td><td>[53]</td><td>1992</td><td>n/s</td><td>250</td><td>60</td><td>16 whole skin</td><td>min. one sensor pair</td><td>yes</td><td>OD8810 and SFH205</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>14</td><td>O-RLI</td><td>[54]</td><td>1992</td><td>n/s</td><td>200</td><td>n/s</td><td>30 whole skin</td><td>n/s</td><td>n/s</td><td>no</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>12</td><td>C-M</td><td>[14]</td><td>1992</td><td>n/s</td><td>400</td><td>n/s</td><td>100</td><td>n/s</td><td>not tested</td><td>n/s</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>16</td><td>O-RLI</td><td>[13]</td><td>1993</td><td>n/s</td><td>200</td><td>n/s</td><td>30 whole skin</td><td>n/s</td><td>n/s</td><td>no</td><td>AT-I, Collision Avoidance</td><td>[54]</td></tr><tr><td>17</td><td>O-RLI</td><td>[120]</td><td>1993</td><td>n/s</td><td>250</td><td>60</td><td>16 whole skin</td><td>min. one sensor pair</td><td>yes</td><td>no</td><td>AT-I, Collision Avoidance</td><td>[53]</td></tr><tr><td>15</td><td>C-M</td><td>[117]</td><td>1993</td><td>n/s</td><td>400</td><td>n/s</td><td>100</td><td>n/s</td><td>not tested</td><td>n/s</td><td>AT-I, Collision Avoidance</td><td>[14]</td></tr><tr><td>19</td><td>C-M</td><td>[15]</td><td>1994</td><td>n/s</td><td>400</td><td>n/s</td><td>100</td><td>n/s</td><td>not tested</td><td>n/s</td><td>AT-I, Collision Avoidance</td><td>[14]</td></tr><tr><td>18</td><td>A-US</td><td>[98]</td><td>1994</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>STM</td><td>AT-II, Contour Following</td><td>-</td></tr><tr><td>20</td><td>O-RLI</td><td>[55]</td><td>1996</td><td>5</td><td>75</td><td>100 (+-50)</td><td>300</td><td>5x5x13</td><td>no</td><td>no</td><td>AT-II, Manipulation and Grasping</td><td>-</td></tr><tr><td>22</td><td>O-RLI</td><td>[86]</td><td>1997</td><td>n/s</td><td>40</td><td>60 (+-30)</td><td>10000</td><td>32</td><td>no</td><td>no</td><td>AT-II, Manipulation and Grasping</td><td>-</td></tr><tr><td>24</td><td>O-RLI</td><td>[56]</td><td>1997</td><td>n/s</td><td>200</td><td>130 (+-65)</td><td>n/s</td><td>2020-08-07 00:00:00</td><td>no</td><td>no</td><td>AT-II, Object Tracking for Grasping</td><td>-</td></tr><tr><td>23</td><td>A-US,O-RLI</td><td>[116]</td><td>1997</td><td>25 (A-US), 50 (O-RLI)</td><td>3000 (A-US), 450 (O-RLI)</td><td>n/s</td><td>40 (A-US), 100 (O-RLI)</td><td>63.5x44.5x20</td><td>yes</td><td>no</td><td>AT-I, Sensor Skin</td><td>-</td></tr><tr><td>25</td><td>O-BB</td><td>[87]</td><td>2000</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>no</td><td>no</td><td>AT-II, BT-I, Sensing for Preshaping</td><td>-</td></tr><tr><td>27</td><td>C-SE</td><td>[18]</td><td>2006</td><td>0</td><td>n/s</td><td>n/s</td><td>200</td><td>15x5</td><td>no</td><td>MC33794</td><td>AT-I, Sensor Skin</td><td>-</td></tr><tr><td>29</td><td>O-RLI</td><td>[61]</td><td>2007</td><td>0</td><td>50</td><td>360</td><td>48</td><td>1</td><td>yes</td><td>no</td><td>AT-I, BT-I, Contour Following</td><td>-</td></tr><tr><td>28</td><td>C-M</td><td>[27]</td><td>2007</td><td>n/s</td><td>50</td><td>180</td><td>n/s</td><td>50</td><td>no</td><td>no</td><td>AT-II, Sensing for Preshaping</td><td>-</td></tr><tr><td>31</td><td>C-M</td><td>[42]</td><td>2008</td><td>n/s</td><td>n/s</td><td>n/s</td><td>500</td><td>n/s</td><td>n/s</td><td>no</td><td>AT-II, Material recognition</td><td>-</td></tr><tr><td>32</td><td>C-M</td><td>[28]</td><td>2008</td><td>n/s</td><td>190</td><td>n/s</td><td>30</td><td>50</td><td>n/s</td><td>no</td><td>AT-II, Proximity Servoing</td><td>-</td></tr><tr><td>33</td><td>O-Tri</td><td>[57]</td><td>2009</td><td>10</td><td>60</td><td>n/s</td><td>224</td><td>90 x 90</td><td>yes</td><td>TCRT1000</td><td>AT-I, AT-II, Sensor Array</td><td>-</td></tr><tr><td>36</td><td>O-RLI</td><td>[58]</td><td>2009</td><td>2</td><td>40</td><td>4 alnaog sensors per finger = 90</td><td>n/s</td><td>6x3.7x3.7</td><td>n/s</td><td>Vishay TCND5000</td><td>AT-II, Preshaping and Grasping</td><td>-</td></tr><tr><td>35</td><td>C-M</td><td>[39]</td><td>2009</td><td>n/s</td><td>170</td><td>n/s</td><td>n/s</td><td>22x22</td><td>no</td><td>no</td><td>AT-I, AT-II, Sensor</td><td>-</td></tr><tr><td>38</td><td>C-SE, C-M</td><td>[33]</td><td>2010</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40x40</td><td>yes</td><td>no</td><td>Preshaping, Grasping, Exploration, Tele-operati...</td><td>-</td></tr><tr><td>37</td><td>C-M</td><td>[29]</td><td>2010</td><td>0</td><td>150</td><td>n/s</td><td>20 (update cycle for control)</td><td>Fingertip of Barret Hand</td><td>not tested</td><td>no</td><td>AT-II, Preshaping, Grasping and Co-manipulation</td><td>-</td></tr><tr><td>40</td><td>C-M</td><td>[23]</td><td>2010</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>yes</td><td>BOSCH (complete solution)</td><td>AT-I, Sensor</td><td>-</td></tr><tr><td>42</td><td>C-M</td><td>[108]</td><td>2010</td><td>0</td><td>160</td><td>180</td><td>25000</td><td>220x150</td><td>no</td><td>no</td><td>Automotive Seat Occupancy</td><td>-</td></tr><tr><td>43</td><td>O-RLI</td><td>[59]</td><td>2011</td><td>n/s</td><td>3</td><td>n/s (180)</td><td>1000</td><td>3x4</td><td>n/s</td><td>GP2S60</td><td>AT-I, AT-II, Modular Multi-Modal Skin</td><td>-</td></tr><tr><td>44</td><td>C-M</td><td>[144]</td><td>2011</td><td>0</td><td>25</td><td>360</td><td>1000</td><td>whole body</td><td>n/s</td><td>no</td><td>AT-I, AT-II, bioinspired</td><td>-</td></tr><tr><td>46</td><td>C-M</td><td>[101]</td><td>2011</td><td>0</td><td>2000</td><td>360</td><td>n/s</td><td>30 x 570</td><td>yes</td><td>AD7143</td><td>Automotive parking</td><td>-</td></tr><tr><td>48</td><td>O-RLI</td><td>[75]</td><td>2012</td><td>1</td><td>10</td><td>narrow</td><td>n/s</td><td>15x15</td><td>no</td><td>ADNS-9500</td><td>AT-II, Slip Detection, Object Reconstruc-tion....</td><td>-</td></tr><tr><td>49</td><td>A-US</td><td>[112]</td><td>2012</td><td>20</td><td>400</td><td>15</td><td>40</td><td>45 x 20 x 15</td><td>no</td><td>HC-SR04</td><td>AT-II legged climbing robot</td><td>-</td></tr><tr><td>47</td><td>A-S</td><td>[104]</td><td>2012</td><td>n/s</td><td>2020-04-03 00:00:00</td><td>n/s</td><td>20</td><td>6</td><td>no</td><td>yes</td><td>AT-II, Reactive Grasping, Object Exploration</td><td>-</td></tr><tr><td>57</td><td>O-Tri</td><td>[84]</td><td>2013</td><td>200</td><td>1500</td><td>n/s</td><td>250 (20 sensors)</td><td>20 x 40</td><td>not tested</td><td>Sharp GP2Y0A02YK</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>52</td><td>O-RLI</td><td>[134]</td><td>2013</td><td>10</td><td>400</td><td>10</td><td>50</td><td>simulated</td><td>yes</td><td>TCND5000,GP2D120</td><td>AT-I, AT-II, Teleoperation, Shared Autonomy, S...</td><td>-</td></tr></tbody></table>
<table><thead><tr><th>Measurement Principle</th><th>Reference</th><th>Year</th><th>Reported Min. Working Distance [mm]</th><th>Reported Max. Range [mm]</th><th>Field of View [degree]</th><th>Measurement Rate [Hz]</th><th>Sensing Element Dimension [mm² / mm² / mm³]</th><th>Multiple-Obstacles</th><th>Commercially Available Core Components</th><th>Categorization</th><th>Basis reference</th></tr></thead><tbody><tr><td>59</td><td>O-RLI</td><td>[65]</td><td>0</td><td>50</td><td>90</td><td>1000</td><td>22x24x40</td><td>n/s</td><td>EE-SY1200</td><td>AT-II, Preshaping</td><td>-</td></tr><tr><td>54</td><td>C-SE, C-M</td><td>[30]</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40 x 40</td><td>yes</td><td>no</td><td>AT-II, Multi-Modal Sensor System</td><td>-</td></tr><tr><td>55</td><td>C-SE</td><td>[138]</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40 x 40</td><td>yes</td><td>no</td><td>AT-II, Hand and Object Tracking, Collision Pre...</td><td>[30]</td></tr><tr><td>51</td><td>C-M</td><td>[16]</td><td>2013</td><td>0</td><td>>100</td><td>180</td><td>6250</td><td>1500x150x0.35</td><td>no</td><td>AT-I, BT-I, Reactive Collision Avoidance</td><td>-</td></tr><tr><td>60</td><td>C-M</td><td>[48]</td><td>2013</td><td>0</td><td>25</td><td>360</td><td>1000</td><td>whole body</td><td>n/s</td><td>AT-I, AT-II, Bio-inspired</td><td>[144]</td></tr><tr><td>61</td><td>A-S</td><td>[105]</td><td>2013</td><td>n/s</td><td>2020-04-03 00:00:00</td><td>n/s</td><td>20</td><td>6</td><td>no</td><td>AT-II, Reactive Grasping, Object Exploration</td><td>[104]</td></tr><tr><td>63</td><td>O-Tri</td><td>[85]</td><td>2014</td><td>200</td><td>1500</td><td>n/s</td><td>250 (20 sensors)</td><td>20x40</td><td>not tested</td><td>Sharp GP2Y0A02YK</td><td>AT-I, Collision Avoidance</td><td>[84]</td></tr><tr><td>62</td><td>C-SE</td><td>[31]</td><td>2014</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40x40</td><td>yes</td><td>no</td><td>AT-II, Preshaping, Grasping and Exploration</td><td>[30]</td></tr><tr><td>65</td><td>C-M</td><td>[113]</td><td>2014</td><td>0</td><td>90</td><td>n/s</td><td>n/s</td><td>5x5</td><td>yes</td><td>no</td><td>Sensor System</td><td>-</td></tr><tr><td>66</td><td>O-RLI</td><td>[62]</td><td>2015</td><td>0</td><td>20</td><td>n/s</td><td>1000</td><td>3 x 1</td><td>no</td><td>KEYENCE fiber optical converter</td><td>AT-II, Multi-Modal Sensor System</td><td>-</td></tr><tr><td>67</td><td>O-RLI</td><td>[127]</td><td>2015</td><td>0</td><td>50</td><td>90</td><td>1000</td><td>22x24x40</td><td>n/s</td><td>EE-SY1200</td><td>AT-II</td><td>[65]</td></tr><tr><td>72</td><td>O-RLI</td><td>[68]</td><td>2015</td><td>n/s</td><td>300</td><td>n/s</td><td>1000</td><td>100x100</td><td>not considered</td><td>no</td><td>AT-I, AT-II</td><td>-</td></tr><tr><td>73</td><td>O-BB</td><td>[88]</td><td>2015</td><td>0</td><td>>84</td><td>n/s</td><td>n/s</td><td>32x19x6.5</td><td>no</td><td>-</td><td>AT-II, Preshaping and Grasping</td><td>-</td></tr><tr><td>68</td><td>C-M</td><td>[49]</td><td>2015</td><td>0</td><td>250</td><td>360</td><td>n/s</td><td>whole body</td><td>no</td><td>no</td><td>AT-I, AT-II, Bio-Inspired</td><td>-</td></tr><tr><td>69</td><td>C-M</td><td>[32]</td><td>2015</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40x40</td><td>yes</td><td>no</td><td>AT-II, Teleoperation, Exploration</td><td>[30]</td></tr><tr><td>70</td><td>C-M</td><td>[17]</td><td>2015</td><td>5</td><td>50</td><td>Cross Section</td><td>n/s</td><td>200x200</td><td>yes</td><td>no</td><td>AT-I, BT-II Object detection, tomography</td><td>-</td></tr><tr><td>71</td><td>C-M</td><td>[35]</td><td>2015</td><td>0</td><td>n/s</td><td>180</td><td>quasi-simultaneously</td><td>Meka H2 Finger</td><td>no</td><td>no</td><td>AT-II, BT-II Active Object Categorization, Gra...</td><td>-</td></tr><tr><td>83</td><td>R</td><td>[95]</td><td>2016</td><td>n/s</td><td>500 (for given resolution)</td><td>58 (E) 57 (H)</td><td>n/s</td><td>13,2 (antenna diameter)</td><td>yes</td><td>no</td><td>BT-II, Object exploration</td><td>-</td></tr><tr><td>76</td><td>O-RLI</td><td>[63]</td><td>2016</td><td>0</td><td>20</td><td>n/s</td><td>1000</td><td>3 x 1</td><td>no</td><td>KEYENCE fiber optical converter</td><td>AT-II, Multi-Modal Sensor System</td><td>[62]</td></tr><tr><td>77</td><td>O-RLI</td><td>[66]</td><td>2016</td><td>0</td><td>50</td><td>90</td><td>1000</td><td>22x24x40</td><td>n/s</td><td>EE-SY1200</td><td>AT-II, Preshaping and Grasping</td><td>[65]</td></tr><tr><td>81</td><td>O-RLI</td><td>[133]</td><td>2016</td><td>10</td><td>400</td><td>10</td><td>50</td><td>simulated</td><td>yes</td><td>TCND5000,GP2D120</td><td>AT-I, AT-II, Teleoperation, Shared Autonomy, S...</td><td>[134]</td></tr><tr><td>78</td><td>C-SE, I</td><td>[40]</td><td>2016</td><td>n/s</td><td>150</td><td>n/s</td><td>5</td><td>30x30</td><td>n/s</td><td>no</td><td>AT-I, AT-II, Multi-Modal Sensor System</td><td>-</td></tr><tr><td>80</td><td>C-SE, C-M</td><td>[41]</td><td>2016</td><td>0</td><td>100</td><td>n/s</td><td>22-380</td><td>20x20 to 40x40</td><td>yes, but only x/y</td><td>no</td><td>AT-I, AT-II, Multi-Modal Sensor System</td><td>-</td></tr><tr><td>75</td><td>C-SE</td><td>[19]</td><td>2016</td><td>n/s</td><td>100</td><td>n/s</td><td>30-100</td><td>40x40</td><td>yes</td><td>no</td><td>AT-I, Contour Following</td><td>[30]</td></tr><tr><td>79</td><td>C-SE</td><td>[26]</td><td>2016</td><td>0</td><td>350</td><td>n/s</td><td>40</td><td>n/s</td><td>yes</td><td>MRK-Systeme (complete solution)</td><td>AT-II, HRI</td><td>-</td></tr><tr><td>74</td><td>C-M</td><td>[10]</td><td>2016</td><td>0</td><td>50/100</td><td>180</td><td>n/s</td><td>100x150</td><td>yes</td><td>no</td><td>AT-II, Gesture Control, Grasping & Object Man...</td><td>-</td></tr><tr><td>82</td><td>C-M</td><td>[34]</td><td>2016</td><td>0</td><td>100</td><td>n/s</td><td>22-380</td><td>20x20 to 40x40</td><td>yes, but only x/y</td><td>no</td><td>AT-II, Preshaping and Grasping</td><td>[41]</td></tr><tr><td>86</td><td>O-ToF, C-SE</td><td>[76]</td><td>2017</td><td>10</td><td>100</td><td>180 for whole fipgertip (6 module)</td><td>30</td><td>4.8 x 2.8 x 1</td><td>n/s</td><td>VL6180x</td><td>AT-II, Object Exploration</td><td>-</td></tr><tr><td>85</td><td>O-ToF</td><td>[128]</td><td>2017</td><td>0</td><td>70</td><td>42</td><td>10</td><td>4.8 x 2.8 x 1</td><td>not tested</td><td>VL6180x ToF sensor</td><td>AT-II, Preshaping and Grasping</td><td>[76]</td></tr><tr><td>84</td><td>O-RLI</td><td>[129]</td><td>2017</td><td>n/s</td><td>3</td><td>n/s (180)</td><td>1000</td><td>3x4</td><td>n/s</td><td>GP2S60</td><td>AT-II, BT-II</td><td>[59]</td></tr><tr><td>88</td><td>C-SE, I</td><td>[106]</td><td>2017</td><td>n/s</td><td>150</td><td>n/s</td><td>5</td><td>30x30</td><td>n/s</td><td>no</td><td>AT-I, AT-II, Multi-Modal Sensor System</td><td>[40]</td></tr><tr><td>87</td><td>A-US</td><td>[102]</td><td>2017</td><td>n/s</td><td>6000</td><td>180</td><td>30</td><td>5.7 x 4.6</td><td>yes</td><td>no</td><td>AT-II, Navigation</td><td>-</td></tr><tr><td>96</td><td>R</td><td>[93]</td><td>2018</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>yes</td><td>yes</td><td>AT-II, Grasping</td><td>-</td></tr><tr><td>97</td><td>O-Tri</td><td>[70]</td><td>2018</td><td>2.85</td><td>20</td><td>90</td><td>1000</td><td>18 x 28.5 x 38.5</td><td>not tested</td><td>VSMY1850, TEMD7500X01</td><td>AT-II Dynamic grasping</td><td>-</td></tr><tr><td>98</td><td>O-ToF</td><td>[81]</td><td>2018</td><td>5</td><td>200</td><td>-</td><td>30</td><td>4.8 x 2.8 x 1</td><td>n/s</td><td>VL6180x</td><td>AT-II, Object Exploration</td><td>-</td></tr><tr><td>100</td><td>O-ToF</td><td>[77]</td><td>2018</td><td>0</td><td>70</td><td>42</td><td>10</td><td>4.8 x 2.8 x 1</td><td>not tested</td><td>VL6180x</td><td>AT-II, Teleoperation</td><td>[76]</td></tr><tr><td>91</td><td>O-RLI, O-ToF</td><td>[79]</td><td>2018</td><td>0</td><td>150</td><td>n/s</td><td>1000, 100</td><td>3.2 x 1.9 x 1.1, 4.8 x 2.8 x 1</td><td>no</td><td>EE-SY 1200, VL6180x</td><td>AT-II, Preshaping and Grasping</td><td>-</td></tr><tr><td>90</td><td>O-RLI</td><td>[130]</td><td>2018</td><td>n/s</td><td>3</td><td>n/s (180)</td><td>1000</td><td>3x4</td><td>n/s</td><td>GP2S60</td><td>AT-II, Object Exploration</td><td>[59]</td></tr><tr><td>93</td><td>O-RLI</td><td>[73]</td><td>2018</td><td>5</td><td>200 possible, 60-70 in their work due to PDMS</td><td>60</td><td>20 (whole skin)</td><td>4x4</td><td>not tested</td><td>Vishay VCNL4010</td><td>AT-I, AT-II, Preshaping, Grasping and Gesture ...</td><td>-</td></tr><tr><td>94</td><td>O-RLI</td><td>[72]</td><td>2018</td><td>5</td><td>200 possible, 60-70 in their work due to PDMS</td><td>60</td><td>20 (whole skin)</td><td>4x4</td><td>not tested</td><td>Vishay VCNL4010</td><td>AT-I, AT-II, Preshaping, Grasping and Gesture ...</td><td>-</td></tr><tr><td>95</td><td>C-SE</td><td>[36]</td><td>2018</td><td>0</td><td>100</td><td>n/s</td><td>200</td><td>115 x 85 x 1</td><td>n/s</td><td>MPR121</td><td>AT-II, Contour Following, Health Care</td><td>-</td></tr></tbody></table>

Continued on next page
<table><caption>TABLE I: Overview - Sorted by year</caption><thead><tr><th>Measurement Principle</th><th>Reference</th><th>Year</th><th>Reported Min. Working Distance [mm]</th><th>Reported Max. Range [mm]</th><th>Field of View [degree]</th><th>Measurement Rate [Hz]</th><th>Sensing Element Dimension [mm / mm² / mm³]</th><th>Multiple-Obstacles</th><th>Commercially Available Core Components</th><th>Categorization</th><th>Basis reference</th></tr></thead><tbody><tr><td>89</td><td>C-M</td><td>[43]</td><td>n/s</td><td>500</td><td>180</td><td>Capacitive: 1000, ToF: limited bei the I2C 400k</td><td>40x40x4</td><td>yes</td><td>no</td><td>AT-II, Material Recognition</td><td>[103]</td></tr><tr><td>92</td><td>C-M</td><td>[44]</td><td>0</td><td>100</td><td>n/s</td><td>22-380</td><td>20x20 to 40x40</td><td>yes, but only x/y</td><td>no</td><td>AT-II, Material Recognition</td><td>[41]</td></tr><tr><td>99</td><td>A-US</td><td>[136]</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>n/s</td><td>AT-II, Teleoperation</td><td>-</td></tr><tr><td>104</td><td>R</td><td>[94]</td><td>300</td><td>n/s</td><td>n/s</td><td>n/s</td><td>300 (waveguide length)</td><td>yes</td><td>no</td><td>AT-I</td><td>-</td></tr><tr><td>110</td><td>O-Tri</td><td>[71]</td><td>2.85</td><td>20</td><td>90</td><td>1000</td><td>18 x 28.5 x 38.5</td><td>not tested</td><td>VSMY1850, TEMD7500X01</td><td>AT-II, Preshaping and Grasping</td><td>[70]</td></tr><tr><td>102</td><td>O-ToF</td><td>[78]</td><td>0</td><td>70</td><td>42</td><td>10</td><td>4.8 x 2.8 x 1</td><td>not tested</td><td>VL6180x ToF sensor</td><td>AT-II, Multi-Modal Sensor System</td><td>[76]</td></tr><tr><td>108</td><td>O-RLI</td><td>[7]</td><td>n/s</td><td>3</td><td>n/s (180)</td><td>1000</td><td>3x4</td><td>n/s</td><td>GP2S60</td><td>AT-I, AT-II, Whole-Body Control</td><td>[59]</td></tr><tr><td>109</td><td>O-RLI</td><td>[67]</td><td>0</td><td>50</td><td>90</td><td>1000</td><td>22x24x40</td><td>n/s</td><td>EE-SY1200</td><td>AT-II, Preshaping and Grasping</td><td>[65]</td></tr><tr><td>101</td><td>C-SE, O-ToF</td><td>[80]</td><td>10</td><td>100</td><td>180 for whole</td><td>30</td><td>4.8 x 2.8 x 1</td><td>n/s</td><td>VL6180x</td><td>AT-I, Multi-Modal Skin</td><td>-</td></tr><tr><td>103</td><td>C-SE</td><td>[37]</td><td>0</td><td>n/s</td><td>n/s</td><td>100</td><td>30 x 30</td><td>no</td><td>Teensy 3.2</td><td>AT-II, Contour Following, Health Care</td><td>-</td></tr><tr><td>106</td><td>C-SE</td><td>[25]</td><td>0</td><td>300</td><td>n/s</td><td>125</td><td>50 x 50 to 100 x 100</td><td>yes</td><td>FOGALE Robotics (complete solution)</td><td>AT-I, Collision Avoidance</td><td>-</td></tr><tr><td>111</td><td>C-SE</td><td>[38]</td><td>0</td><td>50</td><td>180</td><td>n/s</td><td>5 x 5</td><td>no</td><td>no</td><td>AT-II, Grasping in Harsh environments, Industr...</td><td>-</td></tr><tr><td>107</td><td>C-M, O-ToF</td><td>[20]</td><td>n/s</td><td>500</td><td>180</td><td>Capacitive: 1000, ToF: limited bei the I2C 400k</td><td>40x40x4</td><td>yes</td><td>no</td><td>AT-I, Collision Avoidance</td><td>[43]</td></tr><tr><td>105</td><td>A-US</td><td>[100]</td><td>2020</td><td>0</td><td>8</td><td>n/s</td><td>30x14x14</td><td>no</td><td>no</td><td>AT-II, Material Recognition</td><td>-</td></tr><tr><td>113</td><td>R</td><td>[92]</td><td>n/s</td><td>>1200</td><td>160</td><td>40</td><td>9x13</td><td>yes</td><td>yes</td><td>AT-I, BT-II</td><td>-</td></tr><tr><td>118</td><td>O-ToF, O-RLI</td><td>[74]</td><td>2020</td><td>5</td><td>200</td><td>-</td><td>30</td><td>4.8 x 2.8 x 1</td><td>n/s</td><td>VL6180x, MAX30105</td><td>AT-II, Object Exploration</td><td>-</td></tr><tr><td>116</td><td>O-ToF</td><td>[82]</td><td>2020</td><td>5</td><td>200</td><td>-</td><td>30</td><td>4.8 x 2.8 x 1</td><td>n/s</td><td>VL6180x</td><td>AT-II, Object Exploration</td><td>[81]</td></tr><tr><td>119</td><td>O-RLI</td><td>[137]</td><td>2020</td><td>n/s</td><td>n/s</td><td>n/s</td><td>7.1 x 2.75 x 2.7</td><td>no</td><td>Broadcom Limited, HSDL-9100-021</td><td>AT-II, Teleoperation, VR, Transcutaneous Elect...</td><td>-</td></tr><tr><td>115</td><td>C-SE, O-ToF</td><td>[21]</td><td>2020</td><td>0</td><td>350</td><td>180</td><td>50</td><td>27x27</td><td>no</td><td>no</td><td>AT-II, HRI</td><td>-</td></tr><tr><td>117</td><td>C-SE, I</td><td>[107]</td><td>2020</td><td>0</td><td>300</td><td>n/s</td><td>100</td><td>100 x 100 x 2.75</td><td>n/s</td><td>no</td><td>AT-I, AT-II, Multi-Modal Sensor System</td><td>[40]</td></tr><tr><td>114</td><td>C-SE</td><td>[46]</td><td>2020</td><td>0</td><td>100</td><td>n/s</td><td>22-380</td><td>20x20 to 40x40</td><td>yes, but only x/y</td><td>no</td><td>AT-II, Teleoperation, Tactile Feedback</td><td>[41]</td></tr><tr><td>121</td><td>C-SE</td><td>[50]</td><td>2020</td><td>0</td><td>350</td><td>180</td><td>108</td><td>175 x 80</td><td>no</td><td>AD7147</td><td>AT-I, Collision Avoidance</td><td>[26]</td></tr><tr><td>112</td><td>C-M, ToF</td><td>[122]</td><td>2020</td><td>n/s</td><td>500</td><td>180</td><td>Capacitive: 1000, ToF: limited bei the I2C 400k</td><td>40x40x4</td><td>yes</td><td>no</td><td>AT-I Collision Avoidance</td><td>[43]</td></tr><tr><td>120</td><td>A-US</td><td>[103]</td><td>2020</td><td>n/s</td><td>6000</td><td>180</td><td>30</td><td>5.7 x 4.6</td><td>yes</td><td>no</td><td>AT-II, Navigation</td><td>-</td></tr></tbody></table>
CONTENTS

I **Introduction** 1

II **Proximity Sensor: Characterization, Applications and Safety Considerations** 2

II-A Characterization 2

II-B Application and Behavior Types 3

II-C Safety Considerations and Norm Compliance 3

III **Measurement Principles for Proximity Sensing** 4

III-A Capacitive Sensing 4

III-A1 Early Capacitive Technologies in Robotics 5

III-A2 Capacitive Sensing for AT-I 5

III-A3 Capacitive Sensing in AT-II 5

III-A4 Further Aspects of Capacitive Proximity Sensing Technologies in Robotics 6

III-A5 Spatial Resolution and Sensing Range for Capacitive Sensors 6

III-B Optical Sensing 6

III-B1 Reflected Light Intensity-Type Sensors (O-RLI) at an Early Stage 7

III-B2 Reflected Light Intensity-Type Sensors (O-RLI) Using Photoreflectors and Custom Electronics 7

III-B3 Reflected Light Intensity-Type Sensors (O-RLI) Using Sensing Modules) 8

III-B4 Time-of-Flight-Type (O-ToF) Sensors 8

III-B5 Triangulation-Type Sensors (O-Tri) 9

III-B6 Break-Beam-Type Sensors (O-BB) 9

III-B7 Discussion on Optical Sensors 9

III-C Radar 9

III-D Other Sensing Principles 10

III-D1 Acoustic 10

III-D2 Inductive 11

III-D3 Whiskers 11

III-E Multi-modal and Modular Sensors 11

IV **Applications and Methods in the Research and Industry Domains** 11

IV-A Reactive collision-avoidance and Contour Following (AT-I, BT-I) 12

IV-A1 Early Jacobian-type Approaches 12

IV-A2 Early Geometric Approaches 13

IV-A3 Recent Jacobian-type Approaches 13

IV-A4 Recent Geometric Approaches 14

IV-B Reactive Preshaping and Grasping (AT-II, BT-I) 14

IV-B1 Reactive Preshaping Control 14

IV-C Higher Complexity Methods and Behaviors (Towards BT-II) 15

IV-C1 Pre-touch Exploration (AT-II) 15

IV-C2 Bio-inspired Sensing and Behavior (AT-I, AT-II) 16

IV-C3 Teleoperation and VR (AT-I, AT-II) 16

IV-C4 Material Classification (AT-II) 16

IV-C5 Tracking (AT-I, AT-II) 17

IV-D Industrial Technologies and Solutions 17

V **Future Perspectives: Grand Challenges** 17

V-A Human-robot interaction in the industry and service domains 17

V-B Cognitive Robotics 18

V-C Soft Robotics 18

VI **Summary and Conclusions** 18

References 19
Biographies

<table><tr><td>Stefan Escaida Navarro</td><td>22</td></tr><tr><td>Stephan Mühlbacher-Karrer</td><td>22</td></tr><tr><td>Hosam Alagi</td><td>23</td></tr><tr><td>Björn Hein</td><td>23</td></tr><tr><td>Hubert Zangl</td><td>23</td></tr><tr><td>Keisuke Koyama</td><td>23</td></tr><tr><td>Christian Duriez</td><td>23</td></tr><tr><td>Joshua R. Smith</td><td>23</td></tr></table>