# Decentralized control of a heterogeneous human-robot team for exploration and patrolling

Marco Aggravi, Giuseppe Sirignano, Paolo Robuffo Giordano, Claudio

Pacchierotti

► To cite this version:

Marco Aggravi, Giuseppe Sirignano, Paolo Robuffo Giordano, Claudio Pacchierotti. Decentralized control of a heterogeneous human-robot team for exploration and patrolling. IEEE Transactions on Automation Science and Engineering, 2022, 19 (4), pp.3109-3125. 10.1109/TASE.2021.3106386. hal-03321834

HAL Id: hal-03321834

https://inria.hal.science/hal-03321834

Submitted on 18 Aug 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Decentralized Control of a Heterogeneous Human-Robot Team for Exploration and Patrolling

Marco Aggravi, Member, IEEE, Giuseppe Sirignano, Paolo Robuffo Giordano, Senior Member, IEEE and Claudio Pacchierotti, Senior Member, IEEE

**Abstract**—We present a decentralized connectivity-maintenance control framework for an heterogeneous human-robot team. The algorithm is able to manage a team composed of an arbitrary number of mobile robots (drones and ground robots in our case) and humans, for collaboratively achieving exploration and patrolling tasks. Differently from other works on the subject, here the human user physically becomes part of the team, moving in the same environment of the robots and receiving information about the team connectivity through wearable haptics or audio feedback. While the human explores the environment, the robots move so as to keep the team connected via a connectivity-maintenance algorithm; at the same time, each robot can also be assigned with a specific target to visit. We carried out three human subject experiments, both in virtual and real environments. Results show that the proposed approach is effective in a wide range of scenarios. Moreover, providing either haptic or audio feedback for conveying information about the team connectivity significantly improves the performance of the considered tasks, although users significantly preferred receiving haptic stimuli w.r.t. the audio ones.

**Note to Practitioners**— Exploration, patrolling, and Search-and-Rescue are highly-dynamic and unstructured scenarios. When considering the operative conditions of such environments, the benefits of multi-robot systems are evident. Most tasks can be carried out faster and more robustly by a team of robots with respect to a single unit. There are also situations explicitly requiring the presence of a multi-robot team, e.g., using one drone for surveillance of the ground team and one ground mobile robot for carrying supplies. Of course, if the operator(s) in charge of the operation could share the same environment of the robots (i.e., be together with the robots in the field), they would be provided with a level of situational awareness that no teleoperation technology can match as of today. This work presents a framework for controlling heterogeneous teams composed of one human operator and an arbitrary number of aerial and ground mobile robots. The operator moves together with the robotic team and, at the same time, he or she receives meaningful information about the status of the formation. The algorithm only uses the relative position of the drones and humans with respect to each other, and all computations are designed in a decentralized fashion. Decentralization avoids relying on any absolute positioning system (e.g., GPS) or centralized command centers. These features make the proposed framework ready for deployment in different high-impact applications, such as in surveillance, search-and-rescue, and disaster response scenarios.

**Index Terms**—Heterogeneous human-robot teams, Multi-robot systems, Haptic interfaces, Human-centered robotics.

This research was partially supported by Rennes Métropole, under the "Allocation d'Installation Scientifique (AIS) 2017" programme, by the Région Bretagne, with the "Stratégie d'attractivité durable (SAD) 2018" programme (project WH-DRONE), and by the ANR-20-CHIA-0017-01 project "MULTISHARED".

M. Aggravi, P. Robuffo Giordano, and C. Pacchierotti are with CNRS, Univ Rennes, Inria, IRISA, Rennes, France. E-mail: {marco.aggravi, claudio.pacchierotti, prg}@irisa.fr

G. Sirignano is with the University of Salerno, Dept. Information and Electrical Engineering and Applied Mathematics, Salerno, Italy. E-mail: giuseppe.sirignano18@outlook.it

## I. INTRODUCTION

TEAMS of coordinated robots have been successfully used in a plethora of different applications, including disaster response, exploration, patrolling, and surveillance. Indeed, the use of robots in disaster environments has rapidly increased in the last decade, thanks to their expendability, flexibility, possibility to adapt to different situations and tasks and to exploit the onboard sensors for obtaining information (e.g., 3D maps) of the surroundings [1], [2]. In this respect, grounded Urban Search-And-Rescue (USAR) robots are already widely used [3], [4], while aerial solutions are only recently gaining interest [5], [6]. For example, since 2011, there have been more than fifty documented robot deployments in disaster relief scenarios, and various USAR operations have been carried out in more than fifteen countries [7]. Notable examples are the USAR operations at the World Trade Center site [3] and during Hurricane Katrina [8]. Unfortunately, natural disasters are frighteningly on the rise [9], doubling over the past forty years. It is therefore vital to work on solutions able to mitigate the tragic aftermath of these events. Another relevant application for robotic teams is surveillance and patrolling. Counter terrorism, border control, and city surveillance are indeed top priorities of several governments nowadays, and they have played a significant role in many recent political campaigns. Notable commercial solutions for robotic surveillance are provided by SMP Robotics (Canada), Knightscope (USA), and OTSAW (Singapore). Also in this context, most of the robots employed are grounded, while aerial solutions are less common [10], [11].

As the above scenarios are generally highly dynamic and unstructured, it is often important to enable human operators to control the robotic systems in a reactive, effective, and intuitive way [3], [12], [13]. For example, most USAR robots are nowadays fully teleoperated [14], [15], [7] while autonomous solutions are scarce [16], [17], [18]. On the other hand, autonomous surveillance robots are more common. However, also in this case, a human operator can usually remotely access the robots. While these solutions are already widely employed, having the expert operator in the target environment has been proven to significantly improve the response time and effectiveness with respect to remotely teleoperated solutions [19], [20]. Indeed, sharing the environment with the robots provides the human operator with a level of situational awareness that no teleoperation technology can match as of today.

In this context, we propose a novel paradigm for the control of an heterogeneous team composed of both robots and humans, suitable for applications involving exploration, mapping, patrolling, surveillance, and USAR operations. The main contributions of our work can be summarized as follows:
* we present the design of a decentralized connectivity-maintenance algorithm for the control of an heterogeneous team of robots and humans, inspired by Robuffo Giordano et al. [21], [22];

* on top of it, we propose the design of a decentralized exploration algorithm, inspired by Nestmeyer et al. [23], which enables the robots to visit specific targets in the environment while keeping the team connected;

* we introduce different methods enabling a human operator to control the formation and receive information about the connectivity of the team, exploring the possibility of employing wearable feedback techniques (either haptic or audio);

* we carry out three principled human subject experiments in simulated and real environments, comparing different feedback techniques and scenarios, involving groups of either six or four agents (drones, humans, and ground mobile robots).

Differently from other works on this subject, which usually consider teleoperated solutions, here the human agent is allowed to physically become part of the team, moving in the same environment of the robots and receiving information about the connectivity of the team through wearable feedback. As the human agent explores the environment, the robots autonomously move so as to keep the team connected (via the connectivity-maintenance algorithm) and to visit specific targets (thanks to the exploration algorithm). All these actions are carried out in a decentralized way, transparently with respect to the human operator. Our objective is indeed to bring the human agent back into the scene, by augmenting his actions through a team of fast-responding robots, foreseeing improvements in the efficacy, flexibility, and situational awareness of the response. We see the robotic part of the team as an extension of the human user. As the operator explores the environment, her/his robotic counterparts extend her/his reach beyond what is naturally possible, enabling her/him to explore and surveil a much larger part of the scene. Of course, this paradigm may only work if the operator is able to control and receive feedback from the rest of the team in an intuitive and natural way.

The rest of the paper is organized as follows. Section II briefly reviews the state of the art, focusing on heterogeneous human-robot teams, multi-robot formations, and on how to (tele)operate them. Section III presents our decentralized control framework, which comprises an exploration policy on top of a connectivity-maintenance algorithm. This Section also presents the two proposed feedback techniques. Section IV describes the three human subject experiments. We enrolled 32 subjects to evaluate the proposed framework in three different scenarios, for a total of more than 15 hours of testing. Section V discusses the results of this evaluation as well as the benefits and limitations of the proposed approach. Finally, Section VI addresses the concluding remarks and perspectives of this work.

## II. RELATED WORKS

**Heterogeneous human-robot teams:** There exist several solutions for achieving an effective interaction between man and machines sharing the same environment. We can highlight two main interaction modalities: (i) one-way, where the human

provides one or more robots with local information (e.g., his or her position), and (ii) two-way, where the human shares local information with one or more robots and receives back information regarding their state. Examples of one-way interaction have been presented by [24], [25], [26]. However, for our application, two-way interactions are far more interesting. In this respect, we can identify two main categories of two-way interaction: physical and non physical. Direct physical interaction comprises physically exchanging information with the robot [27], [28]. For example, Galatas et al. [27] presented a robotic guide dog which helps the visually impaired navigating an unknown environment. The robotic dog moves autonomously using a camera and a LIDAR sensor, while the user follows using a standard dog leash. More recently, Ranasinghe et al. [28] proposed a guidance system for people with limited auditory and visual capabilities, composed of a hard rein steering the user along the given path. Results showed that the user can be modeled as a dumped inertial system, enabling the trust level to be adjusted by simply modifying the virtual damping coefficient. On the other hand, non-physical interaction comprises exchanging information without a direct contact between the human and the robot(s). In this case, information coming from the robot(s) is provided to the operator through an additional (wearable) device. Thanks to their flexibility and effectiveness, haptic stimuli are often a popular choice. For example, Scheggi et al. [29], [30] developed a haptic-enabled navigation strategy for heterogeneous human-robot teams. A mobile robot navigates in a known environment and provides the human agent with wearable vibrotactile feedback regarding the path to follow.

In a recent work of ours [31], we started to investigate the introduction of the human user in the robotic team from an application point of view. There, we had the augmented human-robot team employed in virtual USAR operations, where the human played the role of either a civilian survivor to be rescued or an expert rescuer looking for survivors and assessing damages. The user received wearable haptic feedback about the team connectivity level and/or the direction towards a safe path. With respect to [31], here we focus on *how* to integrate human users in the team, carefully studying and evaluating how to best provide them with meaningful information about the status of the team and any higher-level task. Conversely, [31] focused on devising an effective technique to guide a human user and its robotic team through a virtual disartered environment using wearable vibrotactile guidance, showing the great potential of this technology for SAR operations.

**Teleoperated robotic teams:** In all the aforementioned works, the interaction always happens between one human operator and one robot sharing the same environment. To the best of our knowledge, there are no examples of complex interactions between multiple humans and robots in this scenario. On the other hand, we can find several examples of human operators teleoperating one or more mobile robots [13], [32], [12], [33], [34], [35], [36]. For example, Setter et al. [33] proposed a multi-agent manipulability concept, mapping the forces commanded by the operator through a grounded haptic interface to the motion of a swarm of mobile robots and viceversa. Recchiuto et al. [34] evaluated the users' capabilities of controlling a swarm of UAVs with a joypad, testing different
tasks and providing visual feedback from different point of views. Results showed that providing an exocentric point of view led to the best performance, reducing completion time and trajectory length. Masone et al. [35] presented a bilateral shared control framework for online trajectory generation in mobile robotics applications. The user controls the robot by modifying, at runtime, the trajectory along which the robot is navigating, while an autonomous algorithm ensures that no collisions occur, the path remains smooth, and some points of interest are visited. In [36], we proposed a decentralized connectivity-maintenance algorithm for the teleoperation of a team of multiple UAVs. With respect to this work, [36] considered a pure teleoperation approach, where the human operator is remotely controlling the formation. Here, in contrast, we focus on the inclusion of the operator in the team of robots, aiming at achieving increased situational awareness and faster reaction time from the human agent.

**Multi-robot tasks:** Using multiple coordinated robots to accomplish an exploration or mapping task is an old problem [37], [38], [39], [40]. A recent survey of Robin et al. [41] presents a wide overview on target detection and tracking for multi-robot systems, while other surveys on multi-robot organization and coordination have been presented by [42], [43]. Recently, Hashemipour et al. [44] described how to optimize first-response robotic teams in terms of their size, teamwork skills, and robot reliability. They ran extensive simulations on an agent-based system, checking whether different configuration setups could bring to a lower operation completion time. The results showed that clustering microtasks could indeed improve the operation performance. Recently, Seraj and Gombolay [45] proposed a distributed control framework to coordinate a team of unmanned aerial vehicles (UAVs) for a human-centered active sensing of wildfires. The results show improvements in reducing environments cumulative uncertainty residual in forefront coverage performance to support human-robot teaming for firefighting. The opportunity of using multi-robot teams in SAR operations has also captured the attention of our Institutions. For example, the European Union funded the collaborative project ICARUS [46] to develop robotic tools able to assist human-led intervention teams. Local efforts, such as the Italian project PRISMA [47], share similar objectives. On this topic, Recchiuto et al. [48] prepared a survey focusing on practical uses of UAVs in post-disaster assessment scenarios.

**Human guidance:** Providing the human operator with information regarding the status of the team is very important. As it will be clear later on, without an effective communication between the robots and the human, the team can hardly function properly. Audio cues have been often used to provide guidance information, especially in those situations where vision is compromised. For example, Boer et al. [49] tested an auditory guidance system in low-visibility situations, e.g., smoke-filled tunnels, where sound stimuli were used to guide an operator toward the exit. Data from walking behaviors and completion times suggest that using directional sound stimuli is a viable and effective solution to ensure a fast and safe escape. Reagan et al. [50] proposed an auditory route guidance system (RGS) able to facilitate wayfinding and reduce mental workload in drivers. The system added salient spatial cues to standard auditory RGS, providing information such as distance to the

<table><caption>TABLE I
RELEVANT NOMENCLATURE</caption><thead><tr><th>Var.</th><th>∈</th><th>Meaning</th></tr></thead><tbody><tr><td>N</td><td>N \ {0}</td><td>number of robots in the team</td></tr><tr><td>x<sub>i</sub>, x&#x0307;<sub>i</sub>, x&#x0308;<sub>i</sub></td><td>R<sup>3</sup></td><td>position, velocity, and acceleration of robot i</td></tr><tr><td>O</td><td>—</td><td>set of obstacle points</td></tr><tr><td>d<sub>ij</sub></td><td>R<sup>+</sup></td><td>distance between robot i and robot j</td></tr><tr><td>d<sub>io</sub></td><td>R<sup>+</sup></td><td>distance between robot i and obstacle point o<sub>k</sub></td></tr><tr><td>N<sub>i</sub></td><td>—</td><td>neighborhood set for robot i</td></tr><tr><td>S<sub>i</sub></td><td>—</td><td>sensing set for robot i</td></tr><tr><td>d<sub>ijk</sub></td><td>R<sup>+</sup></td><td>distance from o<sub>k</sub> of line-of-sight between i and j</td></tr><tr><td>G</td><td>—</td><td>graph representing the group of robots</td></tr><tr><td>A</td><td>R<sup>N×N</sup></td><td>adjacency matrix for graph G</td></tr><tr><td>L</td><td>R<sup>N×N</sup></td><td>Laplacian matrix for A</td></tr><tr><td>λ<sup>min</sup><sub>2</sub></td><td>R</td><td>second smallest eigenvalue of L</td></tr><tr><td>F&#x0307;<sup>λ</sup><sub>i</sub></td><td>R<sup>3</sup></td><td>generalized connectivity force acting on robot i</td></tr><tr><td>F&#x0307;<sup>e</sup><sub>i</sub></td><td>R<sup>3</sup></td><td>external force acting on robot i</td></tr><tr><td>F&#x0307;<sup>b</sup><sub>i</sub></td><td>R<sup>3</sup></td><td>damping force acting on robot i</td></tr><tr><td>F&#x0307;<sup>t</sup><sub>i</sub></td><td>R<sup>3</sup></td><td>travel force acting on robot i</td></tr><tr><td>m<sub>i</sub></td><td>N</td><td>number of targets for robot i</td></tr><tr><td>z<sub>i</sub></td><td>R<sup>3×m<sub>i</sub></sup></td><td>target list for robot i</td></tr><tr><td>z&#x0308;<sup>*</sup><sub>i</sub></td><td>R<sup>3</sup></td><td>current target for robot i inside z<sub>i</sub></td></tr><tr><td>R<sub>t</sub></td><td>R<sup>+</sup></td><td>range of detection for new targets of robot i</td></tr><tr><td>x<sub>h</sub></td><td>R<sup>3</sup></td><td>position of the human agent</td></tr><tr><td>θ<sub>h</sub></td><td>[0, 2π]</td><td>orientation of the human agent</td></tr><tr><td>F&#x0307;<sup>λ</sup><sub>h</sub></td><td>R<sup>3</sup></td><td>generalized connectivity force acting on the human agent</td></tr></tbody></table>

next turn, turn direction, and presence of landmarks. Results showed that participants took fewer navigation errors with respect to using standard RGS approaches. Holland et al. [51] developed a spatial audio user interface for GPS system, allowing users to carry out location tasks while their attention is otherwise engaged. Information on the distance to the waypoint was indicated by changing the time between audio pulses. The results suggested that such simple and computationally-inexpensive spatial mappings are rather effective at helping users finding locations. A similar approach has also been pursued by Strachan et al. [52]. Vazquez et al. [53] proposed an urban sound garden featuring overlapping proximity zones and spatialized animal sounds to attract the user's attention. The results showed that the proposed sound garden design could provide a greater sense of immersion, discovery, and playfulness with respect to other standard approaches.

Haptic cues represent another promising approach to provide guidance information, especially considering that the haptic channel is often significantly underexploited. Many wearable devices have been developed for providing guidance feedback to the user, including vests [54], bracelets and armbands [55], [56], [57], [58], [59], [60], [61], [62], belts [63], [64], [65], [66], chairs [67], and shoes [68]. For example, Bosman et al. [55] proposed two haptic bracelets for providing the analogue of a little nudge one gives to a disoriented companion. Each bracelet contains a vibrating motor informing the users about turning directions. The results showed that all participants successfully reached the final destination. Heuten et al. [69] developed a belt comprising six vibrating motors for navigating users in an urban environment. It provided information regarding deviations from the target path. Experiments on an open field
showed the validity of the proposed technique. The same device is also used by Pielot et al. [66]. More recently, Dobbelstein et al. [59] presented a bearing-based navigation system using vibrotactile feedback to convey information about the target direction. They employed an ergonomic elastic wristband with four vibrating motors, on top of which the user was wearing a smartwatch. The directions were coded into six areas, and the duration of the vibration conveyed the angular offset between the user and the target. The results suggested that, while the detection of the exact offset angle requires some concentration, finding the target is generally easy.

**Connectivity Maintenance and Control:** Connectivity maintenance is a well-studied topic in the multi-robot community: indeed, group connectivity is a pre-requisite for the convergence of cooperative estimation/formation control schemes, and sensing/communication constraints can result in group disconnection during motion if appropriate strategies are not employed.

Some initial attempts addressed the connectivity maintenance problem by enforcing the preservation of a fixed graph topology during motion, i.e., any initially connected robot pair was not allowed to disconnect thanks to an appropriate control action [70]. This kind of solutions solves the problem but is also clearly not very flexible, since the robot group is not allowed to dynamically change its internal topology for better maneuvering in space. Therefore, more global approaches that allow for losing edges among robot pairs but still guarantee maintenance of the global group connectivity have been extensively studied [71], [21], [22], [72]. These approaches can accommodate complex sensing/communication constraints among robot pairs, and is some cases also other requirements of interest such as collision avoidance with obstacles or other robots. Many extensions on these themes have then been considered over the last years. For example. in [73] the case of intermittent communication is addressed, in which the robots do not need to communicate continuously and connectivity is considered in an “integral” sense. A similar setting is considered in [74] but in the context of distributed state estimation. In [75] the problem of planning energy-optimal paths for robots that seek connection with a larger group is studied.

In this work we exploit the connectivity maintenance control strategy originally proposed in [21] and later exploited in several works, e.g., [36]. Indeed, this strategy is general enough for allowing the inclusion of many constraints/requirements of interest into a single generalized connectivity maintenance action, and it also easily allows for the presence of additional external inputs that can be used to steer the individual robots in the group in performing their own mission besides the sole connectivity maintenance constraint.

### III. PROBLEM DESCRIPTION

The multi-robot exploration concept and the connectivity-maintenance strategy are inspired by [23], [21], [22]. Secs. III-A and III-B briefly summarize these works, while Sec. III-C describes how they have been extended for considering the presence of the human operator in the team. Moreover, for the reader's convenience Table I summarizes the main variables used in the following Sections.

#### A. Connectivity-maintenance strategy

Let us consider a group of N mobile robots, either grounded or aerial, operating in a 3D environment with position $\mathbf{x}_i \in \mathbb{R}^3$, $i \in \{1, \dots, N\}$, and let each robot be modeled as a double integrator

$$ \ddot{\mathbf{x}}_i = \mathbf{u}_i, \quad i \in \{1, \dots, N\}, \qquad (1) $$

with $\mathbf{u}_i \in \mathbb{R}^3$ being the acceleration inputs (for grounded mobile robots, acceleration along the $z$ (vertical) axis are neglected). Let us also model the environment surrounding the agents as a collection of $N_o$ 3D points $\mathcal{O} = \{\mathbf{o}_k \in \mathbb{R}^3, k = 1, \dots, N_o\}$, and let us assume that each robot can measure the distance $d_{ij} = \|\mathbf{x}_j - \mathbf{x}_i\|$ between itself and any other agent $j$ in visibility, and between itself and the obstacles in the environment, i.e., $d_{io} = \|\mathbf{x}_i - \mathbf{o}_k\|$, for all sensed obstacle points $\mathbf{o}_k$. Any pair of robots is assumed able to interact (i.e., to sense and communicate with each other) (i) if their distance $d_{ij}$ is smaller than a maximum sensing/communication range $D \in \mathbb{R}^+$, and (ii) if their line-of-sight is not occluded by any obstacle, that is, $d_{ijk} = \|\sigma\mathbf{x}_j + (1-\sigma)\mathbf{x}_i - \mathbf{o}_k\| \ge d_{min}^o, \forall \sigma \in [0, 1], \forall \mathbf{o}_k \in \mathcal{O}$, where $d_{min}^o \in \mathbb{R}^+$ is a minimum visibility range. These two sensing/communication constraints are meant to model the characteristics of common exteroceptive sensors which we assume to be installed on the considered robots<fnref n="1" />. The group interactions are then captured by an undirected graph $\mathcal{G} = (V, E)$, where $V = \{1, \dots, N\}$ is a finite set comprising all agents (vertex set), and $E = \{e_{ij} | i \in V, j \in V, i \neq j\}$ is a finite set comprising edges connecting agents in $V$ (edge set). We then finally let $S_i(t) \subset V$ be the time-varying set of sensing/communication neighbors of agent $i$ as per the sensing model described above (i.e., at runtime an agent can gain/lose neighbors during motion).

A measure of the connectivity of the robot group is, as well-known, the second smallest eigenvalue $\lambda_2 \ge 0$ of the Laplacian matrix $\mathbf{L}$ associated to graph $\mathcal{G}$: the graph is connected iff $\lambda_2 > 0$, [76]. By properly controlling the robots, the connectivity-maintenance algorithm proposed in [21] guarantees that $\mathcal{G}$ can never become disconnected, i.e., that $\lambda_2(t) \ge \lambda_{min} > 0$ at all times, where $\lambda_{min} > 0$ is a minimum connectivity threshold. This control strategy accounts not only for the sensing/communication constraints among the robot pairs ($i, j$) (e.g., maximum range and line-of-sight occlusions in our case), but also for other requirements such as inter-robot and obstacle avoidance and formation control. In particular, the strategy is based on a (decentralized) gradient descent action of a connectivity potential function $V^\lambda(\lambda_2)$ with the properties that

* $V^\lambda(\lambda_2) \to +\infty$ when $\lambda_2 \to \lambda_2^{min}+$,

* $V^\lambda(\lambda_2) \to 0$ when $\lambda_2 \to \lambda_2^{max}$, and

* $V^\lambda(\lambda_2) = 0$ for $\lambda_2 > \lambda_2^{max}$,

see also Fig. 1. The negative gradient of $V^\lambda(\lambda_2)$ w.r.t. the agent position $\mathbf{x}_i$

$$ \mathbf{F}_i^\lambda = - \frac{\partial V^\lambda(\lambda_2)}{\partial \mathbf{x}_i} \qquad (2) $$

is taken as a connectivity force and, as explained in [21], can be evaluated in a decentralized way by only resorting to local

<footnote n="1">¹Of course, more complex sensing models could be considered.</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1. Potential function $V^\lambda(\lambda_2)$ used to compute the connectivity-maintenance force applied by the robots. The asymptote for $\lambda_2$ is set at $\lambda_2^{\min} \in \mathbb{R}^+$, while $V^\lambda(\lambda_2) = 0$ for $\lambda_2 > \lambda_2^{\max} = \lambda_2^{\min} + \eta$.</figcaption></figure>

and 1-hop information. The force $\mathbf{F}_i^\lambda$ is then plugged in (1) by letting

$$ \mathbf{u}_i = \mathbf{F}_i^\lambda - \mathbf{B}_i \dot{\mathbf{x}}_i, \quad (3) $$

where $\mathbf{B}_i > 0$ is a positive definite matrix representing a damping term included for stabilization purposes. When implemented by each agent, the action (3) will guarantee connectivity maintenance and fulfilment of the other constraints/requirements considered in the machinery illustrated in [21] (in particular, obstacle and inter-robot collision avoidance).

## B. Multi-robot exploration

In [23] the connectivity-maintenance algorithm of [21], [22] has been extended for integrating an exploration task so as to enable the robots exploring certain target positions in the environment while maintaining the group connected. We then now briefly review the main steps of this machinery.

In order to include this exploration task while preserving graph connectivity, the control input (3) is extended by adding a travel force $\mathbf{F}_i^t \in \mathbb{R}^3$

$$ \mathbf{u}_i = \mathbf{F}_i^\lambda + \mathbf{F}_i^t - \mathbf{B}_i \dot{\mathbf{x}}_i. \quad (4) $$

This new travel force is exploited for letting the robots fulfilling their exploratory task. In the original work of [23], each of the N robots is assigned with a full list of targets to visit since the very beginning. On the contrary, in our implementation, we update this list at runtime, as the heterogeneous team explores the environment. This different approach leads to more flexibility as well as a higher scalability of the system for large environments and/or several targets. More importantly, it enables us to change at runtime which robot should visit which target. Towards this objective, we assumed that each robot i is able to detect targets closer than a sensing range $R_t \in \mathbb{R}^+$. We call this list $\mathbf{z}_i$, defined as $\mathbf{z}_i = \{\mathbf{z}_i^1, \dots, \mathbf{z}_i^{m_i}\} \in \mathbb{R}^{3 \times m_i}$, where $||\mathbf{z}_i^k - \mathbf{x}_i|| \le R_t, \forall k \in [1, \dots, m_i]$. This list takes also into account any height requirement, i.e., any target higher than $h_{\max}^t > 0$ will be ignored by the grounded mobile robots while any target on the ground will be ignored by the UAVs.

As soon as a robot i has a non-empty list $\mathbf{z}_i$ (i.e., it is close to at least one target), it selects the first entry of $\mathbf{z}_i$ as its next position to visit, denoted by $\mathbf{z}_i^*$. We then compute the $\mathcal{C}^2$ shortest geometric path from the center of robot i to $\mathbf{z}_i^*$.

This path is generated using an $A^*$ algorithm, which outputs a list of points that the robot has to visit to arrive to $\mathbf{z}_i^*$. Before commanding this path to the robot, the trajectory is smoothed by using the $A^*$ output as control points of a cubic B-spline [77]. For grounded mobile robots this spline will lie on a 2D plane, while for UAVs it will be 3 dimensional. Finally, the travel force $\mathbf{F}_i^t$ is computed to drive the robot along this smoothed trajectory towards $\mathbf{z}_i^*$ similarly to [23].

As mentioned before, this target exploration task is carried out on top of the connectivity-maintenance algorithm described in Sec. III-A. The robots are thus asked to visit the considered targets only if their actions do not disconnect $\mathcal{G}$. For this reason, [23] identify four roles for the robots in the team:

* **Prime traveler:** a robot in this state travels towards its current target $\mathbf{z}_i^*$ driven by the travel force $\mathbf{F}_i^t$. Only one robot at a time can be in this state. A prime traveler becomes an anchor robot when it reaches $\mathbf{z}_i^*$;

* **Secondary traveler:** a robot in this state travels towards its current target $\mathbf{z}_i$ driven by the travel force $\mathbf{F}_i^t$, but it can slow down (or even stop) its motion in case of conflicts with the connectivity maintenance action;

* **Connector:** a robot in this state has no assigned target (i.e., empty $\mathbf{z}_i$), and therefore its only task is to keep $\mathcal{G}$ connected. $\mathbf{F}_i^t$ is set to zero, making the robot subject only to the damping $\mathbf{F}_i^b$ and the connectivity force $\mathbf{F}_i^\lambda$ (see (4)). A connector robot becomes a prime traveler if new targets are detected within its sensing range when no other robot is prime traveler, or secondary traveler if new targets are detected and one prime traveler already exists in the team;

* **Anchor:** a robot in this state has just reached its target $\mathbf{z}_i^*$. The traveling force $\mathbf{F}_i^t$ is used to maintain the robot close to $\mathbf{z}_i^*$ for a fixed amount of time (e.g., the time needed to take a picture, map a certain area, or send a signal to the ground). Once this period of time has elapsed, the target is considered visited and the robot returns to be a connector. Of course, if there are still targets in its $\mathbf{z}_i$ list, the robot becomes again a prime or secondary traveler (see above).

This approach worked quite well for the case study addressed in [23]. However, no human input was considered, not even through teleoperation: the robot team consisted solely of robots, which received all the information (full map and targets assignments) at the beginning of the task. Once the mission started the robotic team moved autonomously to visit the targets while maintaining the graph connected – no external input was allowed. In this work, we instead aim at extending this approach to include the presence of a human operator, who shares the same environment as the robots’ and moves freely in it. Moreover, the operator has here the possibility to update the targets at runtime.

## C. Human model and human-in-the-formation

This Section describes how we extend the strategy discussed above when a human user becomes part of the robotic team.

1) **The human agent as a unicycle robot:** Let us consider a human operator walking in the same 3D environment of the N robots discussed above. Recent studies have shown
a close relationship between the shape of human locomotor paths in goal-directed movements and the simplified kinematic model of a wheeled mobile robot [78], [79]. In particular, it has been shown that human's shoulders can be considered as a steering wheel driving the whole body. This observation indicates that humans mostly perform smooth paths and that the direction of the body is always tangent to its trajectory. As mentioned in Sec. I, several studies have exploited these results for effectively guiding human along various paths [29], [30].

These results enable us to directly integrate the human agent into our multi-robot team. In fact, by modeling and planning the human locomotion as that of a unicycle robot, we can easily incorporate the human agent into the formation. Nevertheless, modeling the human as a unicycle robot does come at a price: the human's locomotion can be modeled as that of a mobile robot, but cannot be controlled as if the human actually were a robot. While a variety of motion controllers are available for unicycle systems, it is not possible to simply provide the human user with control inputs for guiding her/his motion: the person is, in fact, generally free to move wherever she/he pleases. The best we can do is provide her/him with suggestions/guidance feedback stimuli, as described in the following Sections.

**2) Guiding the human using the connectivity measure:** Since both [21] and [23] already consider the possibility of including planar robots in the formation, we can directly enlarge the group of N robots including the human as the last element of V for G. We then indicate the position of the human on the x-y plane as $x_{N+1} = x_h = (x_h, y_h, 0) \in \mathbb{R}^3$, while $\theta_h \in [0, 2\pi]$ is his orientation with respect to the environment's base frame. From now on, N represents the new group of agents, including mobile robots, UAVs, and the human agent, who is identified with index h.

Nonetheless, conversely to the other robots, as explained we cannot directly control the motion of the human agent, who is free to move wherever she/he wants and may therefore not comply with the connectivity requirements of the formation. This behavior may lead to situations when $\lambda_2^{\min} = 0$ (unconnected graph G), which are possibly very dangerous for the team. To address this issue, we use guidance feedback for providing the human user with the information carried by the connectivity-maintenance force $\mathbf{F}_h^{\lambda}$ that would act on her/him, were her/him a robot. In particular, the magnitude of the connectivity force acting on the human $||\mathbf{F}_h^{\lambda}||$ indicates "how much" she/he is deviating from an ideal fully-connected situation, whereas the direction of $\mathbf{F}_h^{\lambda}$ indicates "where" the human should move for improving the connectivity level of the team. We assume that the human is capable of "computing" its generalized connectivity force $\mathbf{F}_i^{\lambda}$ in a decentralized way, e.g., by wearing a laptop in a backpack that actually exchanges information with the other robots in the team.

**Magnitude.** To convey how much the user would be subjected to the connectivity-maintenance action, we define two thresholds, $^mF_h^{\lambda} \in \mathbb{R}^+$ and $^M F_h^{\lambda} \in \mathbb{R}^+$. The former is used to avoid unnecessary feedback when $\mathbf{F}_h^{\lambda}$ is very small. On the other hand, the latter is used to convey a sense of emergency or imminent danger when $\mathbf{F}_h^{\lambda}$ is very high and,

therefore, $\lambda_2^{\min}$ is dangerously approaching zero, i.e.,

$$ \left\{ \begin{array}{ll} \text{No feedback} & \text{if } ||\mathbf{F}_h^{\lambda}|| \le {}^mF_h^{\lambda} \\ \text{Standard feedback} & \text{if } {}^mF_h^{\lambda} < ||\mathbf{F}_h^{\lambda}|| \le {}^M F_h^{\lambda} \\ \text{"Emergency" feedback} & \text{if } ||\mathbf{F}_h^{\lambda}|| > {}^M F_h^{\lambda} \end{array} \right. \qquad (5) $$

In our implementation, to convey a sense of imminent danger, we increase the frequency of the stimuli bursts between standard and "emergency" feedback modes.

**Direction.** The guidance is directed as the vector $\mathbf{F}_h^{\lambda}$. By following $\mathbf{F}_h^{\lambda}$, the agent improves the connectivity level of the team, increasing $\lambda_2^{\min}$ and lowering $||\mathbf{F}_h^{\lambda}||$.

**3) Guidance feedback stimuli:** This guidance feedback can be provided in many different ways, as long as it conveys the two pieces of information described above and it is delivered in a wearable and unobtrusive way, leaving the user free of moving in the environment. In our experimental evaluation, we choose to provide such guidance either through haptic feedback (using vibratory cues provided by two wearable armbands) or auditory feedback (using sound signals via a circumaural noise-cancelling headphones).

Fig. 2 summarizes how we map $\mathbf{F}_h^{\lambda}$ into these feedback cues. It implements an attractive paradigm: stimuli coming from the left device (either the armband or the speaker) instruct the user to turn left and viceversa. Given $\theta_h$ as the current orientation of the human, let us consider two circular sectors of 270 degrees: one defined in the counter-clockwise direction (green, Fig. 2(a)) and one defined in the clockwise direction (red, Fig. 2(b)), each divided into eight areas of 33.75 degrees. Such areas represent the zones upon which we activate our feedback, which comprises four intensity levels: low (L), mid-low (ML), mid-high (MH), and high (H). Starting from $\theta_h$ and proceeding counter-clockwise, areas 1 to 4 show a gradual increase of the feedback provided on the left device (from L to H), while areas 5 to 8 show a gradual decrease for the same device (from H to L, see Fig. 2(a)). A similar behavior is defined for the right device. Starting from $\theta_h$ and proceeding clockwise, areas 1 to 4 show a gradual increase of the feedback (from L to H), while areas 5 to 8 show a gradual decrease (from H to L, see Fig. 2(b)).

Whenever $\mathbf{F}_h^{\lambda} > {}^mF_h^{\lambda}$, the direction of $\mathbf{F}_h^{\lambda}$ w.r.t. the current user orientation $\theta_h$ determines the feedback sector, while its magnitude $||\mathbf{F}_h^{\lambda}||$ determines the type of feedback (standard or "emergency," see (5)). An example is shown in Fig. 2(c). The connectivity-maintenance algorithm of Sec. III-A generates a $\mathbf{F}_h^{\lambda}$ pointing in the backward left direction, in area 4 (level H) for the left device and area 7 (level ML) for the right device, with $||\mathbf{F}_h^{\lambda}|| > {}^M F_h^{\lambda}$, thus producing an "emergency" mode feedback on both devices. This guidance approach is inspired by the work of [30], who used vibrotactile feedback to provide human users with information regarding the status of a mobile robot preceding them. Discrete radial-based vibrotactile feedback was preferred over more standard continuous vibrotactile feedback because it was found easier to recognize and differentiate. In a similar work, authors of [29] reported a Just-Noticeable Difference (JND) reference for their armband to be around 30% on the given stimuli. Since our armbands can be seen as an improved version of that employed in [29], we hypothesized that our users could understand a
<figure><img src="image_3.png" /><figcaption>Fig. 2. Guidance feedback stimuli provided to the human user. (a) From the current orientation of the human $\theta_h$, we define a circular sector of 270 degrees in the counter-clockwise direction, divided into eight equal areas. Each area is assigned with a specific level of feedback stimuli: low (L, areas 1 and 8), mid-low (ML, areas 2 and 7), mid-high (MH, areas 3 and 7), and high (H, areas 4 and 5). These different feedback levels result in different feedback intensities provided through the left device. (b) From the current orientation of the human $\theta_h$, we define a circular sector of 270 degrees in the clockwise direction, divided into eight equal areas. Each area is again assigned with a specific level of feedback stimuli: low (L, areas 1 and 8), mid-low (ML, areas 2 and 7), mid-high (MH, areas 3 and 7), and high (H, areas 4 and 5). These different feedback levels result in different feedback intensities provided through the right device. (c) An example. The direction of $F_h^\lambda$ falls into two areas of the circular sector (4 for the left device and 7 for the right one), determining the feedback intensity for the left and right devices (H and ML in this case, respectively). The magnitude of $F_h^\lambda$ determines the duration of the stimuli bursts ("emergency" mode, 200 ms).</figcaption></figure>

<table><caption>TABLE II
AUDIO AND HAPTIC FREQUENCIES AND AMPLITUDE FOR THE FOUR FEEDBACK LEVELS AND TWO FEEDBACK MODES</caption><thead><tr><th>Feedback level</th><th>Audio freq. [Hz]</th><th>Audio amp. [dB]</th><th>Vibration freq. [Hz]</th><th>Vibration amp. [G]</th></tr></thead><tbody><tr><td>H</td><td>800</td><td>80.0</td><td>255</td><td>8.5</td></tr><tr><td>MH</td><td>500</td><td>66.4</td><td>189</td><td>5.7</td></tr><tr><td>ML</td><td>250</td><td>46.4</td><td>126</td><td>1.7</td></tr><tr><td>L</td><td>150</td><td>32.8</td><td>63</td><td>1.2</td></tr></tbody></table>

<table><thead><tr><th>Feedback mode</th><th>Stimuli burst duration v [ms]</th></tr></thead><tbody><tr><td>Standard</td><td>500</td></tr><tr><td>"Emergency"</td><td>200</td></tr></tbody></table>

difference 25% on the given stimuli, which leads us to our four level of intensity (L, ML, MH, and H).

As mentioned at the beginning of this Section, in our work we provide guidance either using a pair of vibrotactile armbands or a pair of headphones, as detailed below.

**Haptic feedback.** Vibrotactile feedback stimuli are provided via a pair of custom haptic armbands, worn on the right and left upper arms. Each armband comprises four 307-100 Pico Vibe vibrating motors (Precision Microdrives Ltd., UK), positioned evenly around the arm at 90 degrees from each other. They have an effective vibration frequency range of [0–255] Hz (the maximal sensitivity is achieved at around [200–300] Hz [80]) and a typical normalized amplitude of 7 G. Each armband vibrates with frequency $f^h \in [0–255]$ Hz in square-wave-like vibration pattern of $v \in \mathbb{R}^+$, i.e., it vibrates for a period of $2v$ ms with duty cycle 50%. The four motors of one armband always vibrate together.

The connectivity-maintenance force $F_h^\lambda$ is rendered as described at the beginning of this Section. The frequency and amplitude inputs for the vibrotactile armband at each feedback level are indicated in Table II, together with the duration of the stimuli bursts $v$ between standard and “emergency” mode

feedback. To convey a sense of imminent danger, we more than double the frequency of the stimuli bursts between these two modes. For example, in the case of Fig. 2(c): the motors on the left armband will vibrate at 255 Hz (H), with $v = 200$ ms (“emergency”); while the motors on the right armband will vibrate at 126 Hz (ML), with $v = 200$ ms (“emergency”).

To ensure that users are really able to discriminate the direction and amplitude of our feedback methodology, we carried out a preliminary perceptual test. We asked four subjects to wear the two armbands and sit in front of a table. Before starting the experiment, we explained them the feedback division and policy of Fig. 2. We also make them try the different feedback levels (H, MH, ML, L) and modes (standard and “emergency”). Then, we generated 50 random $F_h^\lambda$ vectors per subject. For each vector, the vibrotactile armbands provided the user with the corresponding vibrotactile guidance feedback, and subjects had to indicate the feedback level and mode. Fig. 3a shows the resultant confusion matrix (feedback mode recognition rate 100%). Results show that the stimuli provided are rather easy to understand and differentiate. All errors led to confuse neighboring levels, and the two modes were always recognized correctly.

**Audio feedback.** Auditory feedback stimuli are provided via a pair of Sony WH-1000XM2 noise-cancelling wireless headphones, which can generate feedback signals on each speaker separately. Each audio signal is represented by a sinusoidal-shape sound wave, provided in bursts of $v$ ms and specified in frequency: higher the frequency of the wave, sharper the sound produced. When a speaker is activated, it can generate a sound wave of frequency $f^s \in [0–800]$ Hz and amplitude $\in [0–80]$ dB. The connectivity-maintenance force $F_h^\lambda$ is again rendered as described at the beginning of this Section. The frequency and amplitude inputs for the speakers at each feedback level as well as the sound wave period $v$ for each feedback mode are indicated in Table II. For example, in the case depicted in Fig. 2(c): the left speaker will produce a sound wave at 800 Hz and 80 dB (H), with
<figure><img src="image_4.png" /><figcaption>Fig. 3. Confusion matrices for haptic (a) and audio (b) feedback. Rows indicate given stimuli (see Fig. 2(c)), columns indicate users' guesses. Feedback modes recognition rate are 100% and 86%, for haptic and audio medium, respectively.</figcaption></figure>

$\nu = 200$ ms (“emergency”); while the right speaker will produce a sound wave at 250 Hz and 46.4 dB (ML), with $\nu = 200$ ms (“emergency”).

As before, we carried out a preliminary perceptual test. The same four subjects above were asked to wear the wireless headphones. We made them try the different feedback levels (H, MH, ML, L) and modes (standard and emergency) in this other condition. Then, again, we generated $n = 50$ random $F_h^\lambda$ vectors per subject. For each vector, the headphones provided the user with the corresponding audio guidance feedback, and subjects had to indicate the feedback level and mode. Fig. 3b shows the resultant confusion matrix (feedback mode recognition rate 86%). Results show again that the stimuli provided are rather easy to understand.

## IV. EXPERIMENTAL EVALUATION

To evaluate the effectiveness and viability of the proposed approach, we carried out three human subjects experiments.

In the first experiment, fifteen naïve subjects control an heterogeneous team composed of six virtual agents: one human, one (ground) mobile robot, and four quadrotors. Subjects

control the motion of the virtual human using a joystick. All agents move in a virtual environment.

In the second experiment, the subjects enter directly into play, becoming the human agent. Sixteen naïve subjects control an heterogeneous team composed of one real human agent (themselves), one virtual mobile robot, and four virtual quadrotors. Subjects move in an instrumented room, tracked by an optical tracking system, while the other five agents move in a virtual environment.

Finally, in the third experiment, one expert operator controls an heterogeneous team composed of four real agents: one human (the operator himself), one mobile robot, and two quadrotors. All agents move in an instrumented room, tracked by an optical tracking system.

### A. Experiment in a simulated environment

In this first experiment, users control the heterogeneous team in a virtual environment simulated using V-REP. A video is available at https://youtu.be/21qk_Y95gMw?t=61.

1) Environment and task: The virtual environment is composed of a city-like scene including a circular road and several scattered structures such as buildings, trees, pedestals, and open walls (see Fig. 4 and the video). The team is composed of six virtual agents: one human, one ground mobile robot, and four quadrotors (see Fig. 4b).

Users are asked to steer the human agent along the road, until it has performed a complete lap. At the same time, the robots are autonomously controlled as described in Secs. III-A and III-B, keeping the team connected while visiting nine targets $[z_1, ..., z_9]$ (see Fig. 4c). Five out of nine targets $(z_i, i = 2, 5, 7, 8, 9)$ are located very close to the ground, making them reachable only by the mobile robot. The remaining targets $(z_i, i = 1, 3, 4, 6)$ are placed well above the ground, making them reachable only by the quadrotors. Two targets, $z_1$ and $z_9$, deserve special attention, as they are placed in positions difficult to reach by the formation: $z_1$ is placed on the top of a building, 5.2 m from the ground, meaning that the team has to significantly stretch in the upper direction to maintain its connectivity; $z_9$ is placed inside a small room, away from the street, meaning that the team has again to significantly stretch to maintain its connectivity (see bottom of Fig. 4b).

The virtual scene is displayed on an LCD screen posed in front of the user. It shows the environment from different points of view (see Fig. 4a), including a top view of the map (Top), a first-person view of the human agent (PoV), a bird's-eye prospective (B), the role of each robot (S, e.g., if they are connectors/travelers and the assigned target), and the evolution of the connectivity eigenvalue $\lambda_2$ (C). The virtual scene is managed by one computer (6 GB RAM, $4 \times 3.07$ GHz Intel Xeon CPU, Gallion 0.4 on NV94 graphic card), while the control loop described in Secs. III-A and III-B runs on another computer (16 GB RAM, $4 \times 2.60$ GHz Intel Core i7-6600U CPU, Intel HD Graphics 520 graphic card). The communication between the two machines is done through Ethernet, with ROS acting as bridge. Both V-REP simulation and the control loop run at 200 Hz. The control parameters used in this evaluation are reported in Table III.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Experiment in a simulated environment. An heterogeneous team, composed of six virtual agents (upper (b)), explores a virtual environment representing a city-like scene (c). The team has to visit nine targets [z₁, ..., z₉] (c). The virtual scene is displayed on an LCD screen (a), showing the environment from different points of view: a top view of the map (Top), a first-person view of the human agent (PoV), a bird's-eye prospective (B), the role of each robot (S, e.g., if they are connectors/travelers and the assigned target), and the evolution of the connectivity eigenvalue λ₂ (C).</figcaption></figure>

<table><caption>TABLE III
PARAMETERS SETTING FOR ALL EXPERIMENTS</caption><tr><td>λ<sub>2</sub><sup>m</sup> =</td><td>0.01</td><td>R<sub>t</sub> =</td><td>1.5 m</td></tr><tr><td>v<sub>i</sub><sup>cruise</sup> =</td><td>0.51 m/s</td><td>T =</td><td>4 s</td></tr><tr><td>e<sup>m</sup> =</td><td>0.1</td><td>e<sup>M</sup> =</td><td>0.35</td></tr><tr><td><sup>m</sup>F<sub>h</sub><sup>λ</sup> =</td><td>3.5 N</td><td><sup>M</sup>F<sub>h</sub><sup>λ</sup> =</td><td>4.5 N</td></tr><tr><td>α<sub>Δ</sub> =</td><td>0</td><td>ψ =</td><td>2</td></tr></table>

2) **Master interfaces:** Subjects control the motion of the human agent in the virtual scene through an off-the-shelf joypad Logitech Gamepad F310 (Logitech, Switzerland). The right analog thumbstick commands a linear velocity $v_h \in \mathbb{R}$ to the agent, whereas the left stick rotates it with $\omega_h \in S$.

3) **Feedback modalities:** We consider three different ways of providing the operator with feedback information regarding the connectivity of the team: no feedback at all (N), vibrotactile haptic feedback (T), and auditory feedback (A). In all conditions, users always see the virtual scene, as described in Sec. IV-A1, and use the joystick to control the human agent, as described in Sec. IV-A2.

**Condition N.** In this condition, the user receives no feedback about the connectivity force.

**Condition T.** Vibrotactile feedback stimuli about the connectivity force are provided via a pair of custom haptic armbands, as described in Sec. III-C. Noise-cancelling headphones are used to mask any sound coming from the vibrating motors.

**Condition A.** Auditory feedback stimuli about the connectivity force are provided via a pair of Sony WH-1000XM2 noise-cancelling wireless headphones, as described in Sec. III-C.

4) **Subjects:** Fifteen participants took part in this first experiment in a simulated environment, including two women and thirteen men (women: 23 and 24 years old; men: 26.4±2.8 years old). Four of them had previous experience with haptic interfaces, twelve had prior engineering experience. None of them works with haptic or cutaneous interfaces on a daily basis. The experimenter explained to each user, in detail, the connectivity-control algorithm, the multi-robot exploration concept, and the meaning of each feedback. Then, the experimenter explained the task and adjusted the setup to be comfortable

for the user. One practice trial was allowed. Users were asked to complete the task as fast as possible, taking however into account the feedback received. Each user carried out the task in the three feedback conditions, N, T, and A. Each session lasted around 30 minutes. We did not record any sensitive, protected, or identifiable data from the participants.

5) **Results:** As a measure of performance, we registered (i) the average task completion time, (ii) the average connectivity force commanded to the human operator, (iii) the average total force commanded to the robots, (iv) the maximum total force commanded to the robots, and (v) the number of targets reached. To compare them, we ran one-way repeated-measures ANalysis Of VAriance (ANOVA) tests ($a = 0.05$). Feedback conditions (N, A, T) were treated as within-subject factors. All data passed the Shapiro-Wilk normality test.

Fig. 5a shows the average completion time. The task starts when the human operator moves for the first time and finishes when the team reaches the end of the circuit. The data passed the Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant change in this metric ($F(2, 28) = 85.391, p < 0.001$, partial $\eta^2 = 0.859$). Post hoc analysis with Bonferroni adjustments revealed a significant difference between conditions N vs. A ($p < 0.001$) and N vs. T ($p < 0.001$). Fig. 5b shows the average total connectivity force provided to the human operator. It is calculated as the mean over time of $||F_h^\lambda||$. In condition A, this information is provided via auditory stimuli; in condition T, it is provided via vibrotactile stimuli; in condition N, the user does not receive this information (see Sec. IV-A3). Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated ($\chi^2(2) = 62.055, p < 0.001$). A Greenhouse-Geisser correction was therefore applied to these data [81]. The one-way repeated-measure ANOVA with Greenhouse-Geisser correction revealed a statistically significant change in this metric ($F(1.004, 14.059) = 43.687, p < 0.001$, partial $\eta^2 = 0.757$). Post hoc analysis with Bonferroni adjustments revealed a significant difference between conditions N vs. A ($p < 0.001$) and N vs. T ($p < 0.001$). Fig. 5c shows the average total force commanded to the robots. It is calculated
<figure><img src="image_6.png" /><figcaption>Fig. 5. Experiment in a simulated environment. Objective measures. Mean and 95% confidence interval of the (a) average task completion time, (b) average connectivity force, (c) average total commanded force, (d) maximum commanded force, and (e) number of targets reached.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6. Experiment in a simulated environment. Subjective measures. Mean and 95% confidence interval of the (a) perceived effectiveness and (b) comfort.</figcaption></figure>

as the mean over time of $\sum_{i=1}^{N} \|\ddot{x}_i\|/N$. Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated ($\chi^2(2) = 45.950, p < 0.001$). The one-way repeated-measure ANOVA with Greenhouse-Geisser correction revealed a statistically significant change in this metric ($F(1.015, 14.207) = 16.166, p = 0.001$, partial $\eta^2 = 0.536$). Post hoc analysis with Bonferroni adjustments revealed a

significant difference between conditions N vs. A ($p = 0.031$), N vs. T ($p = 0.001$), and A vs. T ($p < 0.001$). Fig. 5d shows the maximum force commanded to the robots. It is calculated as the maximum value registered during each trial for $\ddot{x}_i$. Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated ($\chi^2(2) = 11.425, p = 0.003$). The one-way repeated-measure ANOVA with Greenhouse-Geisser correction revealed a statistically significant change in this metric ($F(1.262, 17.668) = 61.481, p < 0.001$, partial $\eta^2 = 0.815$). Post hoc analysis with Bonferroni adjustments revealed a significant difference between conditions N vs. A ($p < 0.001$) and N vs. T ($p < 0.001$). This metric is particularly relevant, as it provides information about the maximum acceleration/thrust requested to the robots. A failure in providing such acceleration, even for a short period of time, may result in a loss of connectivity for the team. While this situation cannot happen in our simulated scenario, where the drones can apply infinite thrusts, it may happen when implementing the system in a real scenario. This point is also discussed in Sec. V. Fig. 5e shows the number of targets visited by the robots during the task. A Friedman test showed a statistically significant difference between the means of the four feedback conditions ($\chi^2(2) = 30, p < 0.001, a = 0.05$). The Friedman test is the non-parametric equivalent of the more popular repeated-measures ANOVA. The latter is not appropriate here since the dependent variable was measured at the ordinal level. Post hoc analysis with Bonferroni adjustments revealed a statistically significant difference between N vs. A ($p < 0.001$) and N vs. T ($p < 0.001$).

Immediately after the experiment, we also measured users' experience. Participants were asked to rate each feedback using a slider that ranged from 1 ("very low") to 21 ("very high"). We ran again one-way repeated-measures ANOVA tests ($a = 0.05$). All data passed the Shapiro-Wilk normality test.

Figure 6a shows the user's perceived effectiveness. The data passed the Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant difference between the means ($F(2, 28) = 37.893, p < 0.001$, partial $\eta^2 = 0.730$). Post hoc analysis with Bonferroni adjustments revealed a significant difference between N vs. A ($p < 0.001$), N vs. T ($p < 0.001$), and A vs. T ($p = 0.017$). Figure 6b shows the user's comfort. The data passed the Mauchly's Test of Sphericity. The one-way repeated-measure ANOVA revealed a statistically significant difference between the means ($F(2, 28) = 41.159, p < 0.001$, partial $\eta^2 = 0.746$). Post hoc analysis with Bonferroni adjustments revealed a significant difference between N vs. A ($p < 0.001$), N vs. T ($p = 0.010$), and A vs. T ($p = 0.001$).

## B. Experiment in an augmented environment

In this second experiment, users become the human agent. They physically move in an instrumented room wearing the feedback devices, while the rest of the robotic team is simulated using V-REP. A video of this experiment is available at https://youtu.be/21qk_Y95gMw?t=159.

1) Environment and task: The team is composed of four virtual quadrotors, one virtual ground mobile robot, and one (real) human operator. The virtual agents move in a virtual
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 7. Experiment in an augmented environment. The team, composed of four virtual quadrotors, one virtual ground mobile robot, and one virtual human, move in a virtual environment composed of three large corridors, while keeping the team connected and visiting five targets [z₁, ..., z₅] (b). A real human operator moves in an instrumented, empty 8.5 × 4.8 m room, tracked by a 12-cameras optical sensing system (upper (a)). The position of the real operator directly controls that of the virtual human avatar (lower (a) and (b)). Users are asked to roam the room, visiting three locations which correspond to the thresholds of the three corridors composing the virtual scene (upper (a)). Zoom: Target z₃ is put behind a wall (t = 110 s, condition A); for exploring it, the team has to significantly stretch to maintain its connectivity.</figcaption></figure>

environment composed of three large corridors (see Fig. 7 and the video). On the other hand, the human operator moves in an instrumented, empty 8.5 × 4.8 m room, tracked by a 12-cameras optical sensing system (Vicon, UK). The room where the operator moves represents part of the virtual scene (see again Fig. 7), and a virtual human avatar shows the position of the real human operator in the virtual environment. While the operator (and the corresponding avatar) can obviously only move in the instrumented room (dashed green in Fig. 7), the rest of the team can freely move everywhere in the virtual scene.

Users are asked to roam the room, visiting three locations which correspond to the thresholds of the three corridors

composing the virtual scene. At the same time, the robots are autonomously controlled as described in Secs. III-A and III-B, keeping the team connected while visiting five targets [z₁, ..., z₅] (see Fig. 7). Two out of five targets (zᵢ, i = 1, 5) are located very close to the ground, making them reachable only by the mobile robot. The remaining targets (zᵢ, i = 2, 3, 4) are placed well above the ground, making them reachable only by the quadrotors. Target z₃ deserves special attention, as it is placed behind a wall, meaning that the team has to significantly stretch to maintain its connectivity (similarly to what happened for z₁ and z₉ in Sec. IV-A).

The virtual scene is not shown to the user, who is only provided with information regarding $F_h^λ$ via the considered feedback devices. The virtual scene is managed by the same computing system as in Sec. IV-A. The control parameters used in this evaluation are reported in Table. III.

2) *Feedback modalities:* We consider two different ways of providing the operator with feedback information regarding the connectivity of the team: vibrotactile haptic feedback (T) and auditory feedback (A), as already described in Sec. IV-A3. In all conditions, users moved in the instrumented room as described in Sec. IV-B1. Differently from Sec. IV-A, in this experiment we did not consider condition N, since providing no feedback already showed a performance significantly worse than T or A.

3) *Subjects:* Sixteen participants took part in this experiment in an augmented environment, including two women and fourteen men (women: 23 and 24 years old; men: $26.2 \pm 2.8$ years old). Five of them had previous experience with haptic interfaces, thirteen had prior engineering experience. None of them works with haptic or cutaneous interfaces on a daily basis. Ten of them had already participated in the first experiment. The experimenter explained to each user, in detail, the connectivity-control algorithm, the multi-robot exploration concept, and the meaning of each feedback. Then, the experimenter explained the task and adjusted the devices to be comfortable for the user. One practice trial was allowed. Users were asked to complete the task as fast as possible, taking however into account the feedback received. Each user carried out the task in the two feedback conditions, T and A. Each session lasted around 30 minutes. We did not record any sensitive, protected, or identifiable data from the participants.

4) *Results:* As for the previous experiment, also here we registered (i) the average task completion time, (ii) the average connectivity force commanded to the human operator, (iii) the average total force commanded to the robots, (iv) the maximum total force commanded to the robots, and (v) the number of targets reached. Since we only have two conditions to compare, here we ran paired Student's t-tests ($a = 0.05$). Feedback conditions (A, T) were treated as within-subject factors. All data passed the Shapiro-Wilk normality test. All metrics are calculated as in Sec. IV-A.

Fig. 8a shows the average completion time. A parametric two-tailed paired t-test revealed no statistically significant difference between conditions. Fig. 8b shows the average total connectivity force provided to the human operator. A parametric two-tailed paired t-test revealed a statistically significant difference between conditions ($t(15) = 2.513, p = 0.024$). Fig. 8c shows the average total force commanded to the robots. A parametric
<figure><img src="image_10.png" /><figcaption>Fig. 8. Experiment in an augmented environment. Objective measures. Mean and 95% confidence interval of the (a) average task completion time, (b) average connectivity force, (c) average total commanded force, (d) maximum commanded force, and (e) number of targets reached.</figcaption></figure>

two-tailed paired t-test revealed a statistically significant difference between conditions ($t(15) = 2.130, p = 0.050$). Fig. 8d shows the maximum force commanded to the robots. A parametric two-tailed paired t-test revealed no statistically significant difference between conditions. Fig. 8e shows the number of targets visited by the robots during the task. No difference between conditions was registered.

As in Sec. IV-A, immediately after the experiment, we also measured users' experience. Participants were asked to rate each feedback using a slider that ranged from 1 to 21, where a score of 1 meant "very low" and a score of 21 meant "very high". We ran again two-tailed paired t-tests ($a = 0.05$). All data passed the Shapiro-Wilk normality test.

Fig. 9a shows the user's perceived effectiveness. A parametric two-tailed paired t-test revealed a statistically significant difference between conditions ($t(15) = -3.860, p = 0.002$). Fig. 9b shows the user's comfort. A parametric two-tailed paired t-test revealed a statistically significant difference between conditions ($t(15) = -6.130, p < 0.001$).

### C. Experiment in a real environment

Finally, we carried out an experiment in a real environment. All agents move in an instrumented room, tracked by an optical

<figure><img src="image_11.png" /></figure>

We use two quadrotors from MikroKopter GmbH (Germany) and a Pioneer P3-DX mobile robot from Omron Adept (CA, USA), all controlled at 50 Hz using three computers. The first machine (16 GB RAM, 4 × 2.80 GHz Intel Xeon CPU E5-1603 CPU, NVIDIA Quadro K2000) controls the quadrotors using packages Genom3 and Robotpkg, (CNRS/LAAS, France) [82] over Wi-Fi; the second machine (same specifics as the first one) controls the mobile robot using a dedicated ROS node; finally, the third machine (16 GB RAM, 4 × 2.60 GHz Intel Core i7-6600U CPU, Intel HD Graphics 520) computes the robots position references as per our control algorithm of Secs. III-A and III-B and sends them to the other two computers. The three machines communicate through the Robot Operating System (ROS). The control parameters used in this evaluation are reported in Table III.

While the measurement of the needed relative positions (drone-drone, drone-human, and drone-obstacles) was retrieved by using the motion capture system, each robot was provided *only* with the *relative* positions of the surrounding agents and obstacles, thus simulating the presence of an onboard sensor.

2) *Feedback modalities:* We only considered one way of providing the operator with feedback information regarding the connectivity of the team: vibrotactile haptic feedback (T), as already described in Sec. IV-A3.

3) *Subject:* One male expert operator took part in this experiment (age 30). The user was knowledgeable of the connectivity control algorithm, the multi-robot exploration concept, and the meaning of the feedback. He wore a non-reflective helmet for tracking purpose, as well as safety glasses, garments, and gloves. We did not record any sensitive, protected, or identifiable data from the participant.
4) Results: Fig. 10 shows three moments of this experiment, at $t = 0$, $T_f/2$, $T_f$ with $T_f = 103$ s. Specifically, Fig. 10a shows the trajectories of the four agents (blue: human, cyan: mobile robot, red and green: quadrotors) as well as the location of the three targets (grey circles). Fig. 10b shows the three moments from two different points of view. The colors of the circles around the robots match the colors of the trajectories in Fig. 10a.

## V. DISCUSSION

To evaluate the effectiveness and viability of the proposed control and feedback framework, we carried out three human subjects experiment.

In the first experiment, fifteen naïve subjects control an heterogeneous team composed of six virtual agents: one human, one (ground) mobile robot, and four quadrotors. Subjects control the motion of the virtual human avatar using a joystick, and all the agents move in a virtual environment. We considered three ways of providing the operator with feedback information regarding the connectivity of the team: no feedback at all (N), vibrotactile haptic feedback (T), and auditory feedback (A). Fig. 5 shows that providing feedback information about the team connectivity significantly improved all the considered metrics. On the other hand, the difference between vibrotactile and auditory feedback was found very subtle. Vibrotactile feedback outperformed auditory stimuli only in one metric out of five (average total force). It is also important to notice that subjects were provided with very rich visual information during the experiment (e.g., three views of the scene and a graph of the connectivity eigenvalue over time). In cases where less information is available, we expect audio or haptic feedback on the status of the team to be even more valuable.

Among the considered metrics, the maximum total force holds a very important status. As it provides information about the maximum acceleration/thrust requested to the robots, a failure in providing such acceleration may result in a loss of connectivity for the team. While this undesired behavior cannot happen in this first simulated scenario, where the virtual robots can apply infinite thrusts, it can obviously happen when implementing the system in a real environment. The world fastest drone can accelerate up to $35 \text{ m/s}^2$ [83], while the more common DIJ Phantom 3 Pro drone can accelerate up to $31 \text{ m/s}^2$ [84]. From Fig. 5d, we can see that a team of DIJ Phantom 3 Pro drones would have not been able to guarantee the connectivity of the team in condition N, as the peak thrust requested in those condition is higher than the one the robots can actuate. On the other hand, conditions providing haptic or audio feedback demanded lower acceleration values. Of course, there are many commercially-available drones that cannot even accelerate more than $10 \text{ m/s}^2$. Therefore, it is very important to tune the gains of the feedback to match the maximum applicable thrust. Higher gains will lead to higher connectivity forces provided to the user, which should in turn lead to a lower thrust requested to the robots. In the future, we plan to study how to express the feedback control actions of Sec. III-C3 in terms of the maximum thrust robots can provide.

We also measured the user's experience during this experiment, evaluating the perceived effectiveness and comfort of each condition. Results are shown in Fig. 6. Users found the

condition providing haptic feedback to be the most effective, followed by the one conveying audio feedback and then the one providing no feedback at all. On the other hand, providing no feedback was found the most comfortable solution, followed by the haptic and then the audio conditions. Five subjects reported that being so often provided with audio signals was rather annoying and partially impaired their concentration. For this reason, in the future, we plan to study less obtrusive solutions to provide feedback through the headphones.

In the second experiment, subjects enter directly into play, becoming the human agent. Sixteen naïve subjects control an heterogeneous team composed of one real human agent (themselves), one virtual mobile robot, and four virtual quadrotors. Subjects move in an instrumented room, tracked by an optical tracking system, while the five robot agents move in a virtual environment. Fig. 8 shows again little difference between vibrotactile and auditory feedback. Providing vibrotactile feedback outperformed providing auditory stimuli in two metrics out of five (average connectivity force and average total force). We again measured the user's experience as well, evaluating the perceived effectiveness and comfort of each condition. Fig. 9 shows that the condition providing haptic feedback was considered the most effective and comfortable.

Finally, in the third experiment, one expert operator controls an heterogeneous team composed of four real agents: one human (himself), one mobile robot, and two quadrotors. All agents move in an instrumented room, tracked by an optical tracking system. Fig. 10 shows the experiment. As the human operator moves in the room, the robotic team moves accordingly to keep the team connected and visit the three target locations. The operator was provided with haptic feedback through the wearable vibrotactile bracelets and was able to easily follow their indications regarding the connectivity level of the team.

In addition to the great importance of providing feedback information (either via haptic and audio stimuli), all experiments show that the overall framework works quite well and it is robust to very different conditions. The team remained well connected in all scenarios, providing the user with meaningful and easy-to-follow guidance information. Considering that we tested our system in three different experiments, enrolling 32 subjects for more than 15 hours of testing, in our opinion this work represents an extensive evaluation on the topic. Finally, it is also important to notice that the framework proposed is rather flexible, as it can support an arbitrary amount of mobile robots and human agents. In fact, although in this work we only considered one human, nothing prevents the system from handling multiple human agents in the same team.

## VI. CONCLUSIONS

This paper presents a decentralized, connectivity-maintenance, exploration framework for an heterogeneous human-robot team, together with an extensive experimental evaluation both in virtual and real environments. The work has been originally inspired by the connectivity-maintenance framework of [21] and [22], extending it to consider heterogeneous teams, concurrent environment exploration, and human feedback guidance. With respect to the above works, here the human operator comes directly into play. He physically becomes part of the team, moving in the same
<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>(a) From left to right: robot and human initial positions at time $t = 0$, grid of obstacles, and targets locations; robot and human trajectories at time $t = T_f/2$; robot and human trajectories at time $t = T_f$.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>t = 0</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>t = T_f/2</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>t = T_f</figcaption></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /><figcaption>(b) From left to right, shots of the experiment, from two cameras, at time: $t = 0$, $t = T_f/2$, and $t = T_f$.</figcaption></figure>

<caption>Fig. 10. Evaluation of our approach in a real-setting scenario, employing two quadrotors and a mobile robot. Each robot is identified with a color: red and green are the two quadrotors while cyan is the mobile robot. $T_f = 103$ s.</caption>

environment of the robots and receiving information about the connectivity of the team through wearable haptic or audio feedback stimuli. While the human operator walks around, the robots move to keep the team connected according to the implemented connectivity-maintenance algorithm. At the same time, each robot can be assigned with a specific target position to visit. The robotic part of the team is thus endowed with a twofold role: keeping the heterogeneous team connected and fulfilling given exploration tasks. On the other hand, the human operator acts as the team leader, moving freely to steer the team forward.

Results prove the effectiveness and applicability of the proposed framework system for the intuitive control of heterogeneous robotic teams, which can be useful in various applications of exploration, patrolling, and disaster response. The framework is able to manage a broad range of teams, including an arbitrary number of drones, grounded mobile robots, and humans. Results also show a significant role of the two considered feedback modalities. Providing either haptic or audio stimuli indeed significantly improved the given tasks. Haptic feedback performed significantly better than audio feedback in few objective metrics. However, it performed significantly better in all the subjective ones, showing an important preference of users towards this type of feedback.

In the next future, we plan to significantly extend the experimental evaluation, so as to include scenarios closer to real-world situations (e.g., presence of dynamic targets and obstacles, error in the agent's localization, noise in the communication, use of onboard sensors). Moreover, we will also consider heterogeneous teams with more than one human agent as well as working to extend the connectivity algorithm to allow for temporary loss of the team connectivity [85], [86]. This latter feature will enable the team to temporarily split, e.g., to cover more ground, and then join again, e.g., to integrate new agents. Finally, we plan to recruit subjects from more different backgrounds and age groups, as well as include more women.

## ACKNOWLEDGEMENTS

The authors would like to thank Pol Mordel and Fabien Splinder for their help with the experimental setup.

## REFERENCES

[1] R. R. Murphy, "Human-robot interaction in rescue robotics," *IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Reviews*, vol. 34, no. 2, pp. 138-153, 2004.

[2] Y. Liu and G. Nejat, "Robotic urban search and rescue: A survey from the control perspective," *J. Intelligent & Robotic Systems*, vol. 72, no. 2, pp. 147-165, 2013.
[3] J. Casper and R. R. Murphy, "Human-robot interactions during the robot-assisted Urban Search and Rescue response at the World Trade Center," *IEEE Trans. Systems, Man, and Cybernetics, Part B (Cybernetics)*, vol. 33, no. 3, pp. 367-385, 2003.

[4] A. Davids, "Urban search and rescue robots: from tragedy to technology," *IEEE Intelligent systems*, vol. 17, no. 2, pp. 81-83, 2002.

[5] L. Marconi, C. Melchiorri, M. Beetz, D. Pangercic, R. Siegwart, S. Leutenegger, R. Carloni, S. Stramigioli, H. Bruyninckx, P. Doherty, A. Kleiner, V. Lippiello, A. Finzi, B. Siciliano, A. Sala, and N. Tomatis, "The SHERPA project: Smart collaboration between humans and ground- aerial robots for improving rescuing activities in alpine environments," in *IEEE Int. Symp. on Safety, Security, and Rescue Robotics (SSRR)*, 2012, pp. 1-4.

[6] M. A. Goodrich, B. S. Morse, D. Gerhardt, J. L. Cooper, M. Quigley, J. A. Adams, and C. Humphrey, "Supporting wilderness search and rescue using a camera-equipped mini uav," *J. Field Robotics*, vol. 25, no. 1-2, pp. 89-110, 2008.

[7] R. Bogue, "Search and rescue and disaster relief robots: has their time finally come?" *Industrial Robot: An International Journal*, vol. 43, no. 2, pp. 138-143, 2016.

[8] L. E. Davis, J. Rough, G. Cecchine, A. G. Schaefer, L. L. Rohn, and L. L. Zeman, *Hurricane Katrina: Lessons for army planning and operations*. Rand Corporation, 2007.

[9] D. Guha-Sapir, R. Below, and P. Hoyois, "The OFDA/CRED International Disaster Database. Université Catholique de Louvain, Belgium," 2016.

[10] S. J. Kim and G. J. Lim, "Drone-aided border surveillance with an electrification line battery charging system," *J. Intelligent & Robotic Systems*, vol. 92, no. 3-4, pp. 657-670, 2018.

[11] R. L. Finn and D. Wright, "Unmanned aircraft systems: Surveillance, ethics and privacy in civil applications," *Computer Law & Security Review*, vol. 28, no. 2, pp. 184-194, 2012.

[12] G.-J. M. Kruijff, M. Janicek, S. Keshavdas, B. Larochelle, H. Zender, N. J. Smets, T. Mioch, M. A. Neerincx, J. Diggelen, F. Colas et al., "Experience in system design for human-robot teaming in urban search and rescue," in *Field and Service Robotics*. Springer, 2014, pp. 111-125.

[13] I. R. Nourbakhsh, K. Sycara, M. Koes, M. Yong, M. Lewis, and S. Burion, "Human-robot teaming for search and rescue," *IEEE Pervasive Computing*, vol. 4, no. 1, pp. 72-79, 2005.

[14] M. Zucker, S. Joo, M. X. Grey, C. Rasmussen, E. Huang, M. Stilman, and A. Bobick, "A General-purpose System for Teleoperation of the DRC-HUBO Humanoid Robot," *J. Field Robotics*, vol. 32, no. 3, pp. 336-351, 2015.

[15] R. R. Murphy, S. Tadokoro, and A. Kleiner, "Disaster robotics," in *Springer Handbook of Robotics*. Springer, 2016, pp. 1577-1604.

[16] B. Doroodgar, M. Ficocelli, B. Mobedi, and G. Nejat, "The search for survivors: Cooperative human-robot interaction in search and rescue environments using semi-autonomous robots," in *Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)*, 2010, pp. 2858-2863.

[17] M. Bernard, K. Kondak, I. Maza, and A. Ollero, "Autonomous transportation and deployment with aerial robots for search and rescue missions," *J. Field Robotics*, vol. 28, no. 6, pp. 914-931, 2011.

[18] T. Tomic, K. Schmid, P. Lutz, A. Domel, M. Kassecker, E. Mair, I. L. Grixa, F. Ruess, M. Suppa, and D. Burschka, "Toward a fully autonomous UAV: Research platform for indoor and outdoor urban search and rescue," *IEEE Robotics & Automation Magazine*, vol. 19, no. 3, pp. 46-56, 2012.

[19] D. King, "Organisations in disaster," *Natural Hazards*, vol. 40, no. 3, pp. 657-665, 2007.

[20] M. Statheropoulos, A. Agapiou, G. Pallis, K. Mikedi, S. Karma, J. Vamvakari, M. Dandoulaki, F. Andritsos, and C. P. Thomas, "Factors that affect rescue time in urban search and rescue (USAR) operations," *Natural Hazards*, vol. 75, no. 1, pp. 57-69, 2015.

[21] P. Robuffo Giordano, A. Franchi, C. Secchi, and H. H. Bülthoff, "A passivity-based decentralized strategy for generalized connectivity maintenance," *The International J. Robotics Research*, vol. 32, no. 3, pp. 299-323, 2013.

[22] C. Secchi, A. Franchi, H. H. Bülthoff, and P. Robuffo Giordano, "Bilateral Control of the Degree of Connectivity in Multiple Mobile-robot Teleoperation," in *Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)*, 2013, pp. 3645-3652.

[23] T. Nestmeyer, P. Robuffo Giordano, H. H. Bülthoff, and A. Franchi, "Decentralized simultaneous multi-target exploration using a connected network of multiple robots," *Autonomous Robots*, vol. 41, no. 4, pp. 989-1011, 2017.

[24] G. Podevijn, R. O'Grady, and M. Dorigo, "Self-organised feedback in human swarm interaction," in *Proc. IEEE Int. Workshop on Robot and Human Interaction (ROMAN)*, workshop "how to make a robot readable for a human interaction partner", 2012.

[25] T. Iqbal and L. D. Riek, "Human coordination dynamics with heterogeneous robots in a team," in *Proc. ACM/IEEE Int. Conf. on Human Robot Interaction (HRI)*, 2016, pp. 619-620.

[26] A. Cosgun, D. A. Florencio, and H. I. Christensen, "Autonomous person following for telepresence robots," in *Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)*, 2013, pp. 4335-4342.

[27] G. Galatas, C. McMurrough, G. L. Mariottini, and F. Makedon, "iDog: an assistive-guide robot for the visually impaired," in *Proc. ACM Int. Conf. on PErvasive Technologies Related to Assistive Environments (PETRA)*, 2011, p. 58.

[28] A. Ranasinghe, P. Dasgupta, K. Althoefer, and T. Nanayakkara, "Identification of haptic based guiding using hard reins," *PloS one*, vol. 10, no. 7, pp. 1-22, 2015.

[29] S. Scheggi, F. Morbidi, and D. Prattichizzo, "Human-robot formation control via visual and vibrotactile haptic feedback," *IEEE Trans. Haptics*, vol. 7, no. 4, pp. 499-511, 2014.

[30] S. Scheggi, M. Aggravi, and D. Prattichizzo, "Cooperative Navigation for Mixed Human-Robot Teams Using Haptic Feedback," *IEEE Trans. Human-Machine Systems*, vol. 47, no. 4, pp. 462-473, 2017.

[31] M. Aggravi, A. A. S. Elsherif, P. Robuffo Giordano, and C. Pacchierotti, "Haptic-Enabled Decentralized Control of a Heterogeneous Human-Robot Team for Search and Rescue in Partially-known Environments," *IEEE Robotics and Automation Letters*, 2021.

[32] C. Pacchierotti, S. Scheggi, D. Prattichizzo, and S. Misra, "Haptic feedback for microrobotics applications: A review," *Frontiers in Robotics and AI*, vol. 3, p. 53, 2016.

[33] T. Setter, A. Fouraker, H. Kawashima, and M. Egerstedt, "Haptic interactions with multi-robot swarms using manipulability," *J. Human-Robot Interaction*, vol. 4, no. 1, pp. 60-74, 2015.

[34] C. T. Recchiuto, A. Sgorbissa, and R. Zaccaria, "Usability evaluation with different viewpoints of a human-swarm interface for uavs control in formation," in *Proc. IEEE Int. Workshop on Robot and Human Interaction (ROMAN)*, 2015, pp. 523-528.

[35] C. Masone, M. Mohammadi, P. Robuffo Giordano, and A. Franchi, "Shared planning and control for mobile robots with integral haptic feedback," *The International J. Robotics Research*, vol. 37, no. 11, pp. 1395-1420, 2018.

[36] M. Aggravi, C. Pacchierotti, and P. Robuffo Giordano, "Connectivity-Maintenance Teleoperation of a UAV Fleet With Wearable Haptic Feedback," *IEEE Trans. Automation Science and Engineering*, vol. 18, no. 3, pp. 1243-1262, 2020.

[37] W. Burgard, M. Moors, D. Fox, R. Simmons, and S. Thrun, "Collaborative multi-robot exploration," in *Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)*, vol. 1, 2000, pp. 476-481.

[38] W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, "Coordinated multi-robot exploration," *IEEE Trans. Robotics*, vol. 21, no. 3, pp. 376-386, 2005.

[39] A. Howard, L. E. Parker, and G. S. Sukhatme, "Experiments with a large heterogeneous mobile robot team: Exploration, mapping, deployment and detection," *The International J. Robotics Research*, vol. 25, no. 5-6, pp. 431-447, 2006.

[40] E. G. Jones, B. Browning, M. B. Dias, B. Argall, M. Veloso, and A. Stentz, "Dynamically formed heterogeneous robot teams performing tightly-coordinated tasks," in *Proc. IEEE Int. Conf. on Robotics and Automation (ICRA)*, 2006, pp. 570-575.

[41] C. Rubin and S. Lacroix, "Multi-robot target detection and tracking: taxonomy and survey," *Autonomous Robots*, vol. 40, no. 4, pp. 729-760, 2016.

[42] M. Juliá, A. Gil, and O. Reinoso, "A comparison of path planning strategies for autonomous exploration and mapping of unknown environments," *Autonomous Robots*, vol. 33, no. 4, pp. 427-444, 2012.

[43] F. Amigoni, J. Banfi, and N. Basilico, "Multirobot Exploration of Communication-Restricted Environments: A Survey," *IEEE Intelligent Systems*, vol. 32, no. 6, pp. 48-57, 2017.

[44] M. Hashemipour, S. Stuban, and J. Dever, "A disaster multiagent coordination simulation system to evaluate the design of a first-response team," *Systems Engineering*, vol. 21, no. 4, pp. 322-344, 2018.

[45] E. Seraj and M. Gombolay, "Coordinated Control of UAVs for Human-Centered Active Sensing of Wildfires," in *American Control Conference (ACC)*, 2020, pp. 1845-1852.

[46] G. D. Cubber, D. Doroftei, K. Rudin, K. Berns, D. Serrano, J. Sanchez, S. Govindaraj, J. Bedkowski, and R. Roda, *Search and Rescue Robotics-From Theory to Practice*. InTechOpen, 2017.

[47] C. T. Recchiuto and A. Sgorbissa, "The Project PRISMA: Post-Disaster Assessment with UAVs," in *International Conference on Intelligent Autonomous Systems*. Springer, 2016, pp. 199-211.
[48] —, "Post-disaster assessment with unmanned aerial vehicles: A survey on practical implementations and research approaches," *J. Field Robotics*, vol. 35, no. 4, pp. 459-490, 2018.

[49] L. C. Boer and D. J. Withington, "Auditory guidance in a smoke-filled tunnel," *Ergonomics*, vol. 47, no. 10, pp. 1131-1140, 2004.

[50] I. Reagan and C. L. Baldwin, "Facilitating route memory with auditory route guidance systems," *J. Environmental Psychology*, vol. 26, no. 2, pp. 146-155, 2006.

[51] S. Holland, D. R. Morse, and H. Gedenryd, "AudioGPS: Spatial audio navigation with a minimal attention interface," *Personal and Ubiquitous computing*, vol. 6, no. 4, pp. 253-259, 2002.

[52] S. Strachan, P. Eslambolchilar, R. Murray-Smith, S. Hughes, and S. O'Mohrain, "GpsTunes: controlling navigation via audio feedback," in *Proc. ACM Int. Conf. on Human Computer Interaction with Mobile Devices & Services*, 2005, pp. 275-278.

[53] Y. Vazquez-Alvarez, I. Oakley, and S. Brewster, "Urban sound gardens: Supporting overlapping audio landmarks in exploratory environments," in *Proc. Multimodal Location Based Techniques for Extreme Navigation workshop, Int. Conf. on Pervasive Computing*, 2010.

[54] S. Ertan, C. Lee, A. Willets, H. Tan, and A. Pentland, "A wearable haptic navigation guidance system," in *Proc. IEEE Int. Symp. on Wearable Computers*, 1998, pp. 164-165.

[55] S. Bosman, B. Groenendaal, J.-W. Findlater, T. Visser, M. de Graaf, and P. Markopoulos, "Gentleguide: An exploration of haptic output for indoors pedestrian guidance," in *Human-computer interaction with mobile devices and services*. Springer, 2003, pp. 358-362.

[56] S. Scheggi, F. Chinello, and D. Prattichizzo, "Vibrotactile haptic feedback for human-robot interaction in leader-follower tasks," in *Proc. ACM Int. Conf. on PErvasive Technologies Related to Assistive Environments (PETRA)*, 2012, pp. 51-54.

[57] F. Chinello, C. Pacchierotti, J. Bimbo, N. G. Tsagarakis, and D. Prattichizzo, "Design and evaluation of a wearable skin stretch device for haptic guidance," *IEEE Robotics and Automation Letters*, vol. 3, no. 1, pp. 524-531, 2017.

[58] F. Abi-Farraj, C. Pacchierotti, O. Arenz, G. Neumann, and P. R. Giordano, "A haptic shared-control architecture for guided multi-target robotic grasping," *IEEE Trans. Haptics*, vol. 13, no. 2, pp. 270-285, 2019.

[59] D. Dobbelstein, P. Henzler, and E. Rukzio, "Unconstrained Pedestrian Navigation based on Vibro-tactile Feedback around the Wristband of a Smartwatch," in *Proc. ACM Conf. Extended Abstracts on Human Factors in Computing Systems (CHI)*, 2016, pp. 2439-2445.

[60] M. Aggravi, G. Salvietti, and D. Prattichizzo, "Haptic wrist guidance using vibrations for Human-Robot teams," in *Proc. IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN)*, 2016, pp. 113-118.

[61] M. Aggravi, F. Pausé, P. Robuffo Giordano, and C. Pacchierotti, "Design and Evaluation of a Wearable Haptic Device for Skin Stretch, Pressure, and Vibrotactile Stimuli," in *IEEE Robotics and Automation Letters*, vol. 3, no. 3, 2018, pp. 2166-2173.

[62] L. Devigne, M. Aggravi, M. Bivaud, N. Balix, C. S. Teodorescu, T. Carlson, T. Spreters, C. Pacchierotti, and M. Babel, "Power wheelchair navigation assistance using wearable vibrotactile haptics," *IEEE Trans. Haptics*, vol. 13, no. 1, pp. 52-58, 2020.

[63] K. Tsukada and M. Yasumura, "ActiveBelt: Belt-Type Wearable Tactile Display for Directional Navigation," in *UbiComp: Ubiquitous Computing*, ser. Lecture Notes in Computer Science, 2004, vol. 3205, pp. 384-399.

[64] J. B. F. Van Erp, H. A. Van Veen, C. Jansen, and T. Dobbins, "Waypoint navigation with a vibrotactile waist belt," *ACM Trans. Applied Perception*, vol. 2, no. 2, pp. 106-117, 2005.

[65] R. W. Lindeman, J. L. Sibert, E. Mendez-Mendez, S. Patil, and D. Phifer, "Effectiveness of directional vibrotactile cueing on a building-clearing task," in *Proc. ACM Conf. on Human Factors in Computing Systems (CHI)*. ACM, 2005, pp. 271-280.

[66] M. Pielot and S. Boll, "Tactile wayfinder: comparison of tactile waypoint navigation with commercial pedestrian navigation systems," in *Pervasive computing*. Springer, 2010, pp. 76-93.

[67] H. Tan, R. Gray, J. J. Young, and R. Taylor, "A haptic back display for attentional and directional cueing," *Haptics-e, The electronic J. haptics research*, 2003.

[68] M. Schirmer, J. Hartmann, S. Bertel, and F. Echtler, "Shoe me the Way: A Shoe-Based Tactile Interface for Eyes-Free Urban Navigation," in *Proc. ACM Int. Conf. on Human-Computer Interaction with Mobile Devices & Services*, 2015, pp. 327-336.

[69] W. Heuten, N. Henze, S. Boll, and M. Pielot, "Tactile wayfinder: A non-visual support system for wayfinding," in *Proc. Nordic conference on Human-computer interaction: building bridges*, 2008, pp. 172-181.

[70] M. Ji and M. Egerstedt, "Distributed coordination control of multiagent systems while preserving connectedness," *IEEE Trans. Robotics*, vol. 23, no. 4, pp. 693-703, 2007.

[71] P. Yang, R. Freeman, G. Gordon, K. Lynch, S. Srinivasa, and R. Sukthankar, "Decentralized estimation and control of graph connectivity for mobile sensor networks," *Automatica*, vol. 46, pp. 390-396, 2010.

[72] L. Sabattini, C. Secchi, N. Chopra, and A. Gasparri, "Distributed control of multirobot systems with global connectivity maintenance," *IEEE Trans. Robotics*, vol. 29, no. 5, pp. 1326-1332, 2013.

[73] Y. Kantaros and M. M. Zavlanos, "Distributed intermittent connectivity control of mobile robot networks," *IEEE Trans. Affective Computing*, vol. 62, no. 7, pp. 3109-3121, 2017.

[74] R. Khodayi-mehr, Y. Kantaros, and M. M. Zavlanos, "Distributed state estimation using intermittently connected robot networks," *IEEE Trans. Robotics*, vol. 35, no. 3, pp. 709-724, 2019.

[75] A. Muralidharan and Y. Mostofi, "Path planning for a connectivity seeking robot," in *Proc. of the IEEE Globecom, Workshop on Wireless Networking for Unmanned Autonomous Vehicles*, 2017.

[76] M. Mesbahi and M. Egerstedt, *Graph theoretic methods in multiagent networks*. Princeton University Press, 2010.

[77] L. Biagiotti and C. Melchiorri, *Trajectory planning for automatic machines and robots*. Springer Science & Business Media, 2008.

[78] G. Arechavaleta, J.-P. Laumond, H. Hicheur, and A. Berthoz, "On the nonholonomic nature of human locomotion," *Autonomous Robots*, vol. 25, no. 1-2, pp. 25-35, 2008.

[79] —, "An optimality principle governing human walking," *IEEE Trans. Robotics*, vol. 24, no. 1, pp. 5-14, 2008.

[80] A. Riener, *Sensor Actuator Supported Implicit Interaction in Driver Assistance Systems*. Springer, 2010.

[81] S. W. Greenhouse and S. Geisser, "On methods in the analysis of profile data," *Psychometrika*, vol. 24, no. 2, pp. 95-112, 1959.

[82] Robotics Open Source Software, "CNRS/LAAS," https://web.archive.org/web/20190507150135/https://www.openrobots.org/wiki, 2018.

[83] D. Brunner, "Extreme high power drone with highest climb rate," https://web.archive.org/web/20180814195214/https://www.research-drone.com/en/extreme_climb_rate.html, 2018.

[84] The Half Chrome Team, "Drone thrust testing," https://web.archive.org/web/20180925134358/https://www.halfchrome.com/drone-thrust-testing, 2018.

[85] G. A. Hollinger and S. Singh, "Multirobot coordination with periodic connectivity: Theory and experiments," *IEEE Trans. Robotics*, vol. 28, no. 4, pp. 967-973, 2012.

[86] J. Banfi, A. Q. Li, I. Rekleitis, F. Amigoni, and N. Basilico, "Strategies for coordinated multirobot exploration with recurrent connectivity constraints," *Autonomous Robots*, vol. 42, no. 4, pp. 875-894, 2018.

<figure><img src="image_21.png" /></figure>

Marco Aggravi (S'11, M'16) received his M.Sc. degree in Computer Science Engineering in 2011 and his Ph.D. degree in Information Engineering and Science in 2016, both from the University of Siena, Italy. In 2011, he was a Visiting Researcher at the Institute of Robotics and Mechatronics of the German Aerospace Center (DLR), Germany. In 2016, he was a Visiting Researcher at the Georgia Institute of Technology, USA. He is currently a Post-doctoral Researcher with the Rainbow group at Irisa and Inria in Rennes, France.

<figure><img src="image_22.png" /></figure>

Giusepper Sirignano received his B.S. and M.S. degrees in Computer Engineering from the University of Salerno, Italy, in 2015 and 2018, respectively. He was a Visiting student at National Institute for Research in Digital Science and Technology (Inria), Rennes, France in 2017. After completing his graduate studies, he has worked as IT Consultant for two companies (Technology Reply, Pirelli) in Milan, Italy. He is currently working as Software Engineer at Touring Club Italiano, Milan, Italy. His job activities include software design and implementation, database management and network configuration.
<figure><img src="image_23.png" /><figcaption>Inria in Rennes, France.</figcaption></figure>

**Paolo Robuffo Giordano** (M'08, SM'16) received his M.Sc. degree in Computer Science Engineering in 2001, and his Ph.D. degree in Systems Engineering in 2008, both from the University of Rome "La Sapienza". In 2007 and 2008 he spent one year as a PostDoc at the Institute of Robotics and Mechatronics of the German Aerospace Center (DLR), and from 2008 to 2012 he was Senior Research Scientist at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany. He is currently a senior CNRS researcher head of the Rainbow group at Irisa and

<figure><img src="image_24.png" /><figcaption>of the Eurohaptics Society.</figcaption></figure>

**Claudio Pacchierotti** (S'12, M'15, SM'20) is a researcher of the CNRS at IRISA and Inria RBA, Rennes, France. He earned his Ph.D. degree at the Univ. Siena in 2014; then he was a PostDoc at the Italian Institute of Technology, Genova. He visited the Penn Haptics Group at the University of Pennsylvania in 2014, the DIMEG of the University of Padua in 2013, and the MIRA of the University of Twente in 2014. He received the 2014 EuroHaptics Best PhD Thesis Award. Pacchierotti is Senior Chair of the IEEE Technical Committee on Haptics and Secretary