<figure><img src="image_1.png" /></figure>

HAL
open science

# Interaction-aware Predictive Collision Detector for Human-aware Collision Avoidance

Thomas Genevois, Anne Spalanzani, Christian Laugier

► To cite this version:

Thomas Genevois, Anne Spalanzani, Christian Laugier. Interaction-aware Predictive Collision Detector for Human-aware Collision Avoidance. IV 2023 - IEEE Intelligent Vehicles Symposium, Jun 2023, Anchorage, United States. pp.1-7, 10.1109/IV55152.2023.10186778. hal-04173625

HAL Id: hal-04173625

https://inria.hal.science/hal-04173625

Submitted on 29 Jul 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Interaction-aware Predictive Collision Detector for Human-aware Collision Avoidance

Thomas Genevois<fnref n="1" />, Anne Spalanzani<fnref n="1" /> and Christian Laugier<fnref n="1" />

**Abstract**— With their progressive deployment in increasingly complex environments, autonomous vehicles will more often interact with humans in shared spaces. However proactive planners, the most effective for human-aware navigation, are rarely applicable with real-world constraints because of their inherent complexity. Meanwhile classical approaches fail to navigate in cooperation with humans in complex or crowded scenarios. Therefore we propose to extend a global kinodynamic predictive collision avoidance approach with an interaction-aware behavioral prediction model for human-vehicle interactions. Thanks to a grid based Bayesian perception, our approach is versatile in modeling uncertainty and complex scenes. We deploy this solution on a robotic car and show that it can be used in real-world applications. With a qualitative and quantitative validation, we show that this interaction-aware collision avoidance solution is safe and performs well in crowded scenarios. Less computationally demanding and more versatile than proactive planners but still able to benefit from cooperation with humans, this interaction-aware approach offers a compromise between predictive and proactive planners.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Experimental navigation of a car in a crowd with our human-aware collision avoidance system. The system is deployed on an actual vehicle but, for experimental safety matters, it operates with virtual pedestrians that are generated in Augmented Reality on LiDAR sensors [7]. The top left image displays the predicted occupancy grid and the trajectory candidates that were generated during this test. A video of this experiment is available at https://inria.hal.science/hal-04073269</figcaption></figure>

## I. INTRODUCTION

Autonomous vehicles have been deployed on public roads and are already used for transportation of goods and people. They are currently restricted to simple ODD<fnref n="1" /> but they should be progressively released in new ODD of increasing complexity [1], such as city centers and shared spaces. In order to face these incoming challenges, autonomous driving software should evolve to operate under a wide range of circumstances and be robust to unexpected events.

To contribute to this development, we propose a collision avoidance solution that is versatile in the sense that it can operate in various different ODD. It relies on a PCD<fnref n="2" /> that interfaces a Bayesian grid-based perception and a sampling based model predictive planner. The originality of this solution is that it operates at a low semantic level, without considering obstacles as objects but only as elementary probability of occupancy on a Dynamic Occupancy Grid. Thanks to this, our collision avoidance approach is able to deal with a wide range of scenarios and the complexity of its computations does not depend on the complexity of the scenario. But this first collision avoidance system applies the same prediction model to all agents and considers the worst case in the kinodynamic capabilites of the agents. It is then generally too cautious and in some cases suffers from the "Freezing Robot Problem" [2]. So, to solve this issue, we

augment the PCD with a specific prediction model for each class of agent of a urban scene. In this paper, we explain how a new prediction model can be added to the PCD and we propose a pedestrian specific prediction model.

Although pedestrians are very common agents that most vehicles encounter, they remain difficult to handle since they are quite unpredictable and vulnerable road users. For this, human motion and their motivations have been studied [3] [4] and expert models have been proposed for pedestrian-vehicle interactions such as [5]. Various solutions [6] have been proposed for robotic navigation in human environments but they are often dedicated to restricted ODD (kinematic constraints, speed limitations, number and category of agents, ...). Only a few of them consider interaction-aware navigation and there are very few automotive applications.

In this context, our contributions are :

* the development of an interaction-aware occupancy prediction model specifically for pedestrians

* the Predictive Collision Detector, a versatile collision avoidance framework that allows to combine several occupancy prediction models

* the qualitative and quantitative validation of the method We show in this paper that adding the pedestrian specific prediction model enables cooperation with humans for a more effective navigation in shared environments, without affecting safety. Also, our approach is more versatile than most human-aware navigation solutions since it handles more constraints and complex scenarios. Finally, We present the implementation on an experimental vehicle (fig. 1).

<footnote n="1">¹Univ. Grenoble Alpes, INRIA, 38000 Grenoble, France.
e-mail: firstname.lastname@inria.fr</footnote>

<footnote n="1">¹ODD : Operational Design Domain</footnote>

<footnote n="2">²PCD : Predictive Collision Detector</footnote>
### A. Related works

Human aware navigation, also called social navigation, covers decision making, global planning and local planning. In the present work, we consider only the task of local planning which is to ensure to reach a target at a local level while dynamically avoiding collisions. [6] presents a survey of human aware planners. [8] compares the performance of such planners. Human aware planners can be classified in 4 main categories. *Reactive planners* that only consider the current state of the environment for avoidance. This is the case of classical planners such as the Dynamic Window Approach [9] or the Potential Fields method [10]. Reactive planners often do not suffice in human populated environments that are dynamic. *Kinodynamic predictive planners* estimate the future state of the environment to achieve a more accurate avoidance of moving obstacles. In this category, the common Velocity-Obstacle [11] approach is used in human aware navigation, it is included as a fallback mode in [12]. As predictive planners model humans as simple moving obstacles, they do not capture the possible cooperation with humans and then they encounter the FRP<fnref n="3" /> [2] when in too crowded environments. *Learning based planners* have been proposed, generally relying on deep Reinforcement Learning [13] [14]. These approach are mainly limited by the availability of relevant and large enough data sets. Also they do not generalize as well as model based approaches for scenarios that differ from the training data [6]. *Proactive planners* estimate the future state of the environment and how human agents would react to the possible actions of the robot. It provides optimal navigation solutions for the cooperation of robots and humans. Yet the complexity of proactive planning makes it require strong simplifications of the planning problem in order to operate in real-time. [15] achieves proactive navigation of an autonomous car in a crowd but had to uncouple the longitudinal and steering planning. [12] navigates proactively a robot in an indoor environment that can be densely populated however the cooperative planning is restricted to the 2 nearest humans only. Also [12] identifies the "entanglement issue" that arises when humans do not behave as expected in the proactive planning. [16] presents a proactive planner that plans local trajectories and uses an extended Social Force Model to estimate and minimize its impact on nearby humans. Although, as most other proactive planners, its complexity increases exponentially with the number of agents. Despite the benefits of proactive planning, it is not always desirable to expect an active cooperation from the humans. Many robotic applications require the robot to assume the responsibility of collision avoidance and leave the initiative to humans. This is the case in automotive applications where it is generally considered that right-of-way is given, not taken [17]. For these reasons, the new category of *Behavioral predictive planners* is emerging. These planners predict the future state of the environment

and embed human motion prediction systems that estimate the intentions of interacting humans. [18] achieves navigation of multiple robots in an indoor human-shared environment thanks to a prediction of human motion which is based on a SFM<fnref n="4" /> [4]. [19] introduces a probabilistic approach with a variable confidence in a utility-based human motion prediction model and used it for quadcopter navigation in human vicinity. These 2 works considered that human trajectories are agnostic of the presence of robots. This conservative assumption is effective for small robots interacting with few persons but would lead to the FRP in automotive navigation in a crowd. [20] presents PORCA, a behavioral predictive planner that is aware of interaction between ego-vehicle and pedestrians so, unlike most predictive planners, it is free from the FRP. Experimental results are presented on a robot scooter. However PORCA is limited because it can not model non-holonomic constraints of vehicles. Moreover it uncouples longitudinal and steering planning and applies social navigation only on longitudinal planning.

In this context, our motivation is to propose first a global approach with a kinodynamic PCD-based planner and then to extend it for interaction-aware behavioral predictive planning with pedestrians, considering together longitudinal and steering control with potential non-holonomic constraints.

### B. Approach

The proposed approach relies on a perception system that provides a Bayesian Dynamic Occupancy Grid [21]. Since [22] defines as Collision Detectors the software which interfaces sampling-based planners with occupancy grid perception, we name Predictive Collision Detector our approach that estimates and investigates future occupancy grids in order to provide collision probabilities to be used in a sampling-based planner. For each cell of the grid that has a probability to be occupied by an agent, we propagate this probability of occupancy over time, with a prediction model and considering the possible variations from the expected motion. In the global case, the agent prediction model is a kinodynamic unicycle model considering worst cases of acceleration, deceleration and steering. As this approach considers elementary occupancy probability, without using objects or other features, it provides a unique format to represent any scenario, no matter its complexity. It also considers uncertainty. Moreover the complexity of the computation does not depend on the complexity of the scenario. Even though the prediction requires heavy computations, it is actually highly parallelizable and provide satisfying real time perfomance when it is implemented in GP-GPU [23]. The prediction is then used to compute the probability of collision over a list of potential positions of the ego vehicle in time. A sampling based Model Predictive Control [24] defines the local trajectory minimizing the collision risk and an objective function. The sampling based approach allows us to consider complex realistic motion models.

<footnote n="3">³FRP: Freezing Robot Problem [2]</footnote>

<footnote n="4">⁴SFM: Social Force Model [4]</footnote>
In order to define specific prediction models for classes of agents, our approach relies on a deep learning based approach to extract semantic classification of agents and construct a semantic occupancy grid as in [25]. We then use [21] to filter the occupancy probability, classification probability, extract speed estimation and generate a Dynamic and Semantic Occupancy Grid. Then, instead of propagating the occupancy probability in time with only one global model, we propagate it according to the semantic classification. Thanks to this, we can simultaneously model accurately several specific classes of agents while preserving the global prediction model for other agents. This allows our prediction to operate in various ODD. Concerning pedestrians, we adapted for prediction the SFM-based pedestrian-vehicle interaction model that has been proposed for simulation in [5]. In a validation with real data, this model has been shown to represent accurately various actual human behaviors. Since the original model requires the knowledge of pedestrians' intentions, it needs to be adapted for its use in prediction, relying only on observable data. Thanks to this prediction that is aware of humans and interaction with them, our planners performs smooth maneuvers around humans and does not suffer from the FRP. Also, since the human motion prediction model is explicit, we can verify that the beliefs of the planner are reasonable in order to produce a safe and human-friendly behavior. Namely we specified that the robot should not expect humans to react to its actions and that it should comply with human intentions that are observable. So the robot does not take the initiative and respects human decisions to force the right-of-way, yield or make a detour.

### III. METHODOLOGY

#### A. Structure of the collision avoidance system

The proposed collision avoidance system consists of the following modules:

* A Bayesian Occupancy Filter [21] fuses and filters sensor data. It produces a Dynamic Occupancy Grid i.e. a discrete 2D representation of the likelihood of space to be occupied together with estimated speed of the observed motions.

* A Prediction module that projects the current occupancy grid along the motion particles and estimates the evolution of the occupancy grid over the next seconds. The main contribution of this paper is an evolution of this module, from a simple kinodynamic prediction to a behavioral and class-specific prediction.

* A Collision Detector that investigates the predicted occupancy grid and returns the probability of collision in each position that the planner queries.

* A Sampling-based Planner. Considering the collision probability in sampled positions, a Sampling-based Model Predictive Control planner [24] estimates the collision risk over trajectory candidates. For our application, a cost function helps to find the best trajectory, among the safe ones, to follow a predefined path. Our solution is theoretically compatible with other sampling-based planners.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Image 1 presents a simulated scene, the ego vehicle is moving at about 10km/h. Image 2 displays the predicted occupancy with the kinodynamic unicycle model. Image 3 shows the prediction with the interaction-aware model. Predictions are similar for pedestrian A who is not interacting with the car anymore. Predictions differ slightly regarding pedestrians of group B who are currently moving away from the car's trajectories. Predictions are very different for group C. The SFM model expects a cooperation from them but it is still unclear in which direction the person on the right will go.</figcaption></figure>

#### B. Combining several models in the prediction module

In a Dynamic Occupancy Grid, the motion of agents is represented by a list of particles, with a speed estimation, that convey an occupancy probability in the cell where they lie. Let $\mathcal{P}$ be a particle with a speed $V_{\mathcal{P}}(t_0)$ and an occupancy probability $p$. Its position is $X_{\mathcal{P}}(t_0)$. $t_0$ refers to the current time. A prediction model is defined by $\mathcal{U}$, a set of actions that the agent could do, $P(u)$, the likelihood of each action $u \in \mathcal{U}$ and $X_{\mathcal{P}}(t) | u$ the future position of the particle if action $u$ is realized. Implementing a prediction model in the PCD consists in sampling $\mathcal{U}$ and generating a sub-particle $\mathcal{P}_u$ for each $u$. Then each $\mathcal{P}_u$ propagates its occupancy probability that is reported on the prediction grid for the next time steps at the corresponding $X_{\mathcal{P}}(t) | u$. We first propose a kinodynamic PCD using a unicycle motion model for agents where $\mathcal{U}$ represents tuples of acceleration/deceleration and steering speed with a uniform distribution between the worst cases of acceleration and steering. Fig. 2 shows an example of a prediction grid that has been generated with this model. We propose now to extend this by considering a specific prediction model for each class of agent.

The input of our system is a Semantic Occupancy grid that also provides $P(C_k), k \in [0, n]$, the probability of $\mathcal{P}$ to belong to an agent of class $k$ among $n$ classes. A prediction model is defined for each class. Then $\mathcal{P}$ is split into a set of sub-particles, one for each action of each model. The sub-particles are then propagated according to the models and they carry an occupancy probability given by (1).

$$ \forall k \in [0, n], \forall u \in \mathcal{U}_k, P(u) = 1 - (1 - p)^{P(u|C_k) \times P(C_k)} \quad (1) $$

This preserves the total occupancy probability given that $\sum_{k \in [0, n]} P(C_k) = 1$ and $\sum_{u \in \mathcal{U}_k} P(u | C_k) = 1$. $C_0$ corresponds to the class of unidentified agents. The kinodynamic prediction model is maintained for this class. So we preserve the global and defensive approach to collision avoidance of our previous work as a default behavior for unidentified agents.
This makes our solution robust to uncertain classification. Even though the division in many sub-particles increases the complexity of the computations, this approach leads to an efficient implementation in GP-GPU [23] since the computations are individually simple and parallelizable.

### C. Pedestrian dedicated prediction model

[5] presents a model to simulate the motion of pedestrians in interaction with a vehicle. It relies on the SFM [4] that models the motion intention and interactions with other agents as forces acting on a physical system. It also uses a state machine to represent decisions like crossing before a car, stopping or stepping back. This simulation model requires adaptations to be used as a prediction model. First, we decide to ignore interactions between the pedestrian and other agents and obstacles, the model is simplified by considering only the interaction between the pedestrian and the ego-vehicle. Second, as the decisions are not known to the prediction, we consider a reduced set of states that can be inferred from the observations. In the adapted model, we use the states "stopping"(S), "walking"(W), "running"(R) and we define a new state which is "unidentified motion pattern"(U). We assume that the observed current speed results from a Gaussian Mixture Model which is composed by the speed distributions of the 4 states. Then the state distribution for each particle is estimated by (2).

$$ \forall s \in \{S, W, R, U\}, P(s \mid \|V\|) = \frac{\mathcal{N}(\|V\| \mid \{\mu_s, \sigma_s\})}{\sum_{x \in \{S, W, R, U\}} \mathcal{N}(\|V\| \mid \{\mu_x, \sigma_x\})} \quad (2) $$

Then the prediction model is equal to the simulation one for states S, W and R. The human is modeled as a physical body reacting to 2 forces, its internal motivation and the interaction with the vehicle. [4] defines the model (3).

$$ \begin{aligned} E &= X_{\text{vehicle}} - X_{\mathcal{P}}, \quad D = \lambda(V_{\mathcal{P}} - V_{\text{vehicle}}) + \frac{E}{\|E\|} \\ \theta &= \langle E, D \rangle, \quad T = \frac{D}{\|D\|}, \quad N = \perp T, \quad B = \gamma \|D\| \\ F_{\text{motivation}} &= \frac{V_{\text{desired}} - V_{\mathcal{P}}}{\tau} \\ F_{\text{interaction}} &= -Ae^{-\frac{\|E\|}{B}}(e^{-(n'B\theta)^2}T + e^{-(n'B\theta)^2}N) + \varepsilon \\ \frac{dV_{\mathcal{P}}}{dt} &= F_{\text{motivation}} + F_{\text{interaction}} \end{aligned} \quad (3) $$

As in [5], $F_{\text{interaction}}$ is assumed to be null in states S and R. $X_{\mathcal{P}}(t_0)$, $V_{\mathcal{P}}(t_0)$ are known from the occupancy grid and $X_{\text{vehicle}}$, $V_{\text{vehicle}}$ from the odometry. The parameters for pedestrian-vehicle interaction are defined in [5]. To consider the uncertainties of human motion, we add $\varepsilon$, the interaction model error which is assumed to follow a Gaussian distribution. $V_{\text{desired}}$, the intended pedestrian speed, is assumed to follow a Gaussian distribution with an average value that depends on the state as in [5]. The distribution of $\varepsilon$ and $V_{\text{desired}}$ forms the action set $\mathcal{U}$ of this prediction model. $\mathcal{P}$ is divided in sub-particles with different values of $\varepsilon$ and $V_{\text{desired}}$. Then $X_{\mathcal{P}}|u$ is obtained by integration of $\frac{dV_{\mathcal{P}}}{dt}|u$ and $V_{\mathcal{P}}|u$. Finally, in state U, the prediction follows the

<figure><img src="image_5.png" /><figcaption>Fig. 3. Image 1 shows the experimental vehicle, driving at low speed near several persons. Image 2 displays the predicted occupancy grid that has been computed with the interaction-aware prediction model in this scene. This displays follows the same color scheme as fig. 2. The persons on the right are currently interacting with the car. Static obstacles are also visible.</figcaption></figure>

global kinodynamic model. This fallback mode ensures a defensive prediction when no motion pattern is recognized. Fig. 2 and 3 show examples of predictions grids that have been generated with this model.

As this model is similar to the expert one [5], we can expect a realistic prediction. Also, as it defines explicitly the beliefs of the planner regarding human motion, we can verify that they are reasonable and will produce a human-friendly behavior. For example, as $F_{\text{interaction}}$ is null for stopping and running humans, the robot is not expecting cooperation from these persons and will carefully avoid them. The interaction is considered only with pedestrians at walking speed.

## IV. IMPLEMENTATION AND VALIDATION

### A. Implementation and testing tools

The proposed method has been deployed on an experimental robotic car which is equipped with 5 LiDAR sensors and 1 camera. An accurate localization is computed from an IMU and a RTK GPS. The system, as described in section III-A, combines path following and obstacle avoidance. The planner jointly controls steering and longitudinal speed, up to 30km/h in these tests.

The simulator SPACiSS [26] is used to simulate pedestrians and their interactions with the car. Simulated pedestrians behave either as individuals or groups, they mostly walk and cooperate with the vehicle but can also decide to cross before it, forcing it to stop, they might run in doing so. SPACiSS intends to reproduce realistic crowd behaviors.

### B. Transferability to real-world applications

In the design, development and deployment of our solution, we considered most of the real-world constraints. We rely only on perception data, without prior knowledge of the scene. We check the collision considering a realistic representation of the shape of the vehicle. The planner
considers a non-holonomic motion model and latency in the command. All the software has been implemented on GP-GPU[23] for an efficient parallelization of the computation. The PCD considers a dense action set that leads to generate 100 sub-particles for each observed motion such that the possible actions of the agents are representatively sampled. Though the PCD is executed in a maximum of 15 ms for a densely crowded area, on a NVIDIA GeForce GTX 1080. This allows to meet the real-time constraints and the whole collision avoidance runs at 15 Hz.

This allowed us to deploy the solution on our prototype vehicle. However we do not have the means to guarantee experimental safety during autonomous navigation of a prototype car near humans. For this reason, we only present prediction results with real pedestrians, as in fig. 3. Then we tested the collision avoidance on the real vehicle with virtual pedestrians that were introduced through Augmented Reality on sensor data [7] (fig. 1). With this, we show that the proposed method can be transferred to real world applications. Also we use a Software-in-the-Loop simulation framework for navigation experiments. In this environment, the virtual ego-vehicle is a twin of the experimental vehicle that provides similar sensor data, responds to the same command messages, has similar motion model, shape and real-time constraints. Therefore, with tests in simulation, we can pretty much infer the result of real-world tests.

### C. Qualitative analysis

The interaction-aware prediction model that we used has been developed and validated with real human motion data in [5]. So we do not consider the validation of the prediction on its own in the current paper but we evaluate the benefits of embedding it in the collision avoidance. For this, we consider realistic constraints, a perception from sensor available data, uncertainty and partial knowledge of the scene, non-holonomic motion, actual shapes of ego-vehicle and obstacles, no restriction on the number and types of agents and obstacles. None of the proactive and interaction-aware predictive planners that have been presented in section II-A operates under these constraints. So we can not directly compare our planner with these ones but we can qualitatively show that embedding an interaction-aware prediction in our PCD has granted our planner with some typical features of proactive planners.

The examples in Fig. 4 show that our collision avoidance solution can avoid to disturb pedestrians when possible, this is a desirable behavior. We also observe that, when it is necessary, it can cooperate with a pedestrian to negotiate a simultaneous avoidance. As mentioned in [6], this cooperation is a typical feature of proactive planners. However, our planner, not proactive, waits for the pedestrian to initiate the cooperation by showing a clear intention. This is less effective that what proactive planners could achieve but much more that what non interaction-aware planners could do.

Fig. 5 provides an example where the proposed method deals with several classes of agents and still considers the specificity of humans, without any ODD transition. In

<figure><img src="image_6.png" /><figcaption>Fig. 4. These graphs presents the successive positions of the ego-vehicle and a pedestrian in a mutual avoidance situation when they are face to face on the same path. In (1), there is no restriction on the lane width. We observe that the car takes all the burden of the avoidance and does not disturb the pedestrian. In (2), we consider a restricted lane width that prevents the car from driving far from the pedestrian. Then the car and the pedestrian cooperate and avoid each other. The avoidance is not symmetric because the car and the human do not have the same kinematics.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. This image presents an autonomous navigation simulation with different types of agents. A view of the scene is in the bottom left corner. The predicted occupancy grid is displayed, with the same color scheme as in fig. 2. The gray squares illustrate the trajectory sampling, they are red when the collision risk is too high. In such a situation, the PCD represents the motion of cars with kinodynamic prediction while the behavior of pedestrians is modeled with interaction-aware prediction.</figcaption></figure>

this, in presence of humans, our solution is more effective than global purpose collision avoidance and it is also more versatile than most human-aware navigation systems that do not consider other agents than humans.

### D. Quantitative analysis

1) Performance: Fig. 6 presents a comparative study over 180 simulations of the time that both the interaction-aware PCD planner and the kinodynamic PCD planner need to cross a shared space with a variable density of crowd. The planners show comparable performance in low crowd density. The interaction-aware performs significantly better in average densities. In high density crowds, the kinodynamic planner faces the FRP and can not cross while the interaction-aware planner is still able to cross in a reasonable time. With these results, we prove than the integration of the interaction-aware
<figure><img src="image_8.png" /><figcaption>Fig. 6. 180 simulations are run in a randomly initialized scenario were the car has to cross a 25m long and 8m wide shared space with an unorganized longitudinal 2 ways flow of people. The top images illustrate this scenario. The graph presents the required time to cross depending on crowd density. Each boxplot displays the median, quartiles, extrema and outliers over 10 simulations. The absence of some boxplots traduces that the car was not able to cross. Results show that the interaction-aware planner performs better and is not subject to the FRP.</figcaption></figure>

prediction in our collision avoidance leads to a significant improvement of the navigation in presence of humans. As proactive planners, our new planner is not subject to the FRP since it takes advantage of cooperation opportunities.

2) Safety: The previous section shows that the interaction-aware planner performs well in crowded environment but it is mandatory to verify that it remains safe. For this, we compute the Responsibility-Sensitive Safety<fnref n="5" /> distance [17] with the recommended parameters for human-vehicle interaction. Then we define a safety metric over a whole test, the RSS satisfaction ratio by $min_{(r,k)} \frac{distance \ to \ human \ k}{RSS \ distance}$. If this metric is above 1, the RSS recommended safety distance has always been respected. Although as RSS considers the worst case of acceleration and late braking, it is not actually unsafe if the RSS satisfaction is below 1. As an example, the RSS recommended distance for a pedestrian walking near a stopped car is about 1.5m. Then we modify the simulation in order to add "distracted pedestrians" that will not avoid the car until it is closer than 1m. In this way, we generate scenarios where the planner must avoid such pedestrians. Fig. 7 presents a comparison of the RSS satisfaction ratio over 160 simulations in a cooperative crowd and a partly cooperative crowd. The latter includes 25% of distracted pedestrians. We chose this proportion because it still allows the car to count on the cooperation of most pedestrians and drive rather fast then the impact of non cooperative ones is more visible. The effect is visible, in average, the crossing time increases by 18% and the minimum distance to pedestrians is reduced by 38%. However, as the interactive-aware planner slows down, stops or avoids, the RSS satisfaction ratio is almost unchanged. The planner is able to guaranty a similar level

<figure><img src="image_9.png" /><figcaption>Fig. 7. 160 simulations are run in a randomly initialized scenario were the car has to cross a 25m long and 8m wide shared space with an unorganized longitudinal opposite flow of people. The top images illustrate this scenario with 23 persons/100m². The graph presents the worst RSS satisfaction ratio depending on crowd density. Each boxplot displays the median, quartiles, extrema and outliers over 10 simulations. Half of the simulations use a non cooperative crowd. The interaction-aware planner is used in all cases. Results show that the non-cooperation of some individuals of the crowd has no visible effect of the safety metric, proving that the planners adapts its drive to guarantee similar safety standards.</figcaption></figure>

of safety despite the non cooperation of some humans. No collision occurred during these tests. This study shows that, even if our planner can benefit from the cooperation of some humans, it actively avoids collisions with humans that do not cooperate. The safety is guaranteed, even in dense crowds.

<figure><img src="image_10.png" /><figcaption>Fig. 8. 160 simulations are run in a randomly initialized scenario were the car has to cross a 25m long and 8m wide shared space with an unorganized longitudinal opposite flow of people. The graph presents the social work which reflects the disturbance that both the interaction-aware PCD and the kinodynamic PCD planners cause to pedestrians. Each boxplot displays the median, quartiles, extrema and outliers over 10 simulations. As expected, results show that the interaction-aware planner causes more disturbances to low density crowds than its kinodynamic counterpart which is highly conservative. But it causes a reduced disturbance to high density crowds.</figcaption></figure>

3) Pedestrian comfort: Even though our approach does not include an explicit minimization of the disturbance it causes to humans, it is an important aspect of human-aware navigation. As proposed in [16], we use the social work metric to measure the impact of our vehicle on humans. We compute the social work according to the SFM of [26]. Fig. 8 shows a comparison of this metric between the

<footnote n="5">⁵RSS : Responsibility-Sensitive Safety</footnote>
kinodynamic and interaction-aware planners. It is reasonable
that the kynodynamic planner causes less disturbances to
pedestrians since it drives very conservatively, considering all
worst cases scenarios. But the fact that the interaction-aware
planner causes less disturbances to humans in dense crowds
demonstrates its ability to drive smoothly within crowds.

V. CONCLUSIONS

First the Predictive Collision Detector, a versatile collision
avoidance framework with agent class specific prediction
models has been presented. Then an interaction-aware be-
havioral prediction model for human-vehicle interactions has
been proposed. This model has been deployed to create an
interaction-aware collision avoidance system. It has been
shown that this solution could be transferred to real-world
applications. Some qualitative results showed that the pro-
posed method is able in some situations to minimize its
impact on nearby humans but also to cooperate with them
if needed. At the same time, it can consider other classes of
agents. Finally, with a quantitative validation, we proved that
the proposed interaction-aware collision avoidance drives
safely and performs better than its kinodynamic counterpart
in crowded environments. With this interaction-aware colli-
sion avoidance solution, we hope to provide a compromise
between proactive planners that are difficult to deploy with
real-world constraints and predictive planners that are limited
in crowded environments.

Several possible improvements and open questions remain.
First, for safety matters, we were unable to evaluate our
method with actual human behaviors but it is an important fu-
ture step. Second, human intention detection could drastically
improve the accuracy of the interaction-aware prediction.
Finally, our solution could also include specific prediction
models for other classes, such as cars or cyclists.

ACKNOWLEDGMENT

This work has been developed within the scope of an
"Institut de Recherche Technologique NanoElec" project,
founded by the French program "Investissement d'Avenirs"
ANR-10-AIRT-05. The authors wish to thank N. Turro,
L. Rummelhard and R. Baruffa for their collaboration and
continuous support.

REFERENCES

[1] B. Berman, "The key to autonomous vehicle safety is odd." [Online].
Available: https://www.sae.org/news/2019/11/odds-for-av-testing

[2] P. Trautman and A. Krause, "Unfreezing the robot: Navigation in dense, interacting crowds," in *IEEE/RSJ International Conference on Intelligent Robots and Systems*, 2010.

[3] E. T. Hall, "The hidden dimension: man's use of space in public and private the bodley head," *London, Sydney, Toronto*, vol. 121, 1969.

[4] M. Moussaïd, D. Helbing, S. Garnier, A. Johansson, M. Combe, and G. Theraulaz, "Experimental study of the behavioural mechanisms underlying self-organization in human crowds," *Proceedings of the Royal Society B: Biological Sciences*, 2009.

[5] M. Prédhumeau, L. Mancheva, J. Dugdale, and A. Spalanzani, "An Agent-Based Model to Predict Pedestrians Trajectories with an Autonomous Vehicle in Shared Spaces," in *AAMAS 2021 - 20th International Conference on Autonomous Agents and Multiagent Systems*. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2021.

[6] J. Cheng, H. Cheng, M. Q.-H. Meng, and H. Zhang, "Autonomous navigation by mobile robots in human environments: A survey," in *IEEE International Conference on Robotics and Biomimetics (RO-BIO)*, 2018.

[7] T. Genevois, J.-B. Horel, A. Renzaglia, and C. Laugier, "Augmented reality on lidar data: Going beyond vehicle-in-the-loop for automotive software validation," in *IEEE Intelligent Vehicles Symposium (IV)*, 2022.

[8] H. Khambaita and R. Alami, "Assessing the social criteria for human-robot collaborative navigation: A comparison of human-aware navigation planners," in *IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)*, 2017.

[9] D. Fox, W. Burgard, and S. Thrun, "The dynamic window approach to collision avoidance," *IEEE Robotics & Automation Magazine*, 1997.

[10] O. Khatib, "Real-time obstacle avoidance for manipulators and mobile robots," in *Autonomous robot vehicles*. Springer, 1986, pp. 396-404.

[11] P. Fiorini and Z. Shiller, "Motion planning in dynamic environments using velocity obstacles," *The International Journal of Robotics Research*, vol. 17, no. 7, pp. 760-772, 1998.

[12] P. Teja Singamaneni, A. Favier, and R. Alami, "Human-aware navigation planner for diverse human-robot interaction contexts," in *IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, 2021.

[13] M. Everett, Y. F. Chen, and J. P. How, "Collision avoidance in pedestrian-rich environments with deep reinforcement learning," *IEEE Access*, vol. 9, pp. 10 357-10 377, 2021.

[14] S. Liu, P. Chang, W. Liang, N. Chakraborty, and K. Driggs-Campbell, "Decentralized structural-rnn for robot crowd navigation with deep reinforcement learning," in *IEEE International Conference on Robotics and Automation (ICRA)*, 2021.

[15] M. Kabtoul, A. Spalanzani, and P. Martinet, "Proactive And Smooth Maneuvering For Navigation Around Pedestrians," in *IEEE International Conference on Robotics and Automation (ICRA)*, 2022.

[16] G. Ferrer and A. Sanfeliu, "Proactive kinodynamic planning using the extended social force model and human motion prediction in urban environments," in *IEEE/RSJ International Conference on Intelligent Robots and Systems*, 2014.

[17] S. Shalev-Shwartz, S. Shammah, and A. Shashua, "On a formal model of safe and scalable self-driving cars," 2017. [Online]. Available: https://arxiv.org/abs/1708.06374

[18] M. Boldrer, A. Antonucci, P. Bevilacqua, L. Palopoli, and D. Fontanelli, "Multi-agent navigation in human-shared environments: A safe and socially-aware approach," *Robotics and Autonomous Systems*, vol. 149, p. 103979, 2022.

[19] J. F. Fisac, A. Bajcsy, S. L. Herbert, D. Fridovich-Keil, S. Wang, C. J. Tomlin, and A. D. Dragan, "Probabilistically safe robot planning with confidence-based human predictions," 2018. [Online]. Available: https://arxiv.org/abs/1806.00109

[20] Y. Luo, P. Cai, A. Bera, D. Hsu, W. S. Lee, and D. Manocha, "Porca: Modeling and planning for autonomous driving among many pedestrians," *IEEE Robotics and Automation Letters*, vol. 3, 2018.

[21] L. Rummelhard, J.-A. David, A. Gonzalez Moreno, and C. Laugier, "A cross-prediction, hidden-state-augmented approach for dynamic occupancy grid filtering," in *IEEE International Conference on Control Automation Robotics & Vision (ICARCV)*, 2022.

[22] B. Siciliano and O. Khatib, "Robotics foundations," in *Springer Handbook of Robotics*. Springer International Publishing, 2016.

[23] D. Luebke, "Cuda: Scalable parallel programming for high-performance scientific computing," in *IEEE International Symposium on Biomedical Imaging: From Nano to Macro*, 2008.

[24] D. D. Dunlap, E. G. Collins Jr, and C. V. Caldwell, "Sampling based model predictive control with application to autonomous vehicle guidance," in *Florida Conference on Recent Advances in Robotics*, 2008.

[25] Ö. Erkent, C. Wolf, and C. Laugier, "End-to-End Learning of Semantic Grid Estimation Deep Neural Network with Occupancy Grids," *Unmanned systems*, vol. 7, no. 3, pp. 171-181, July 2019.

[26] M. Prédhumeau, "Simulating Realistic Pedestrian Behaviors in the Context of Autonomous Vehicles in Shared Spaces," in *20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)*. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2021.