<figure><img src="image_1.png" /></figure>

HAL
open science

# Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps

Rabbia Asghar, Manuel Alejandro Diaz-Zapata, Lukas Rummelhard, Anne

Spalanzani, Anne Spalanzani, Christian Laugier

► To cite this version:

Rabbia Asghar, Manuel Alejandro Diaz-Zapata, Lukas Rummelhard, Anne Spalanzani, Anne Spalanzani, et al.. Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps. NAVER LABS 2023 - Europe International Workshop on AI for Robotics, Nov 2023, Grenoble, France. hal-04416326

HAL Id: hal-04416326

https://hal.science/hal-04416326v1

Submitted on 25 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps

Rabbia Asghar, Manuel Diaz-Zapata, Lukas Rummelhard, Anne Spalanzani, Christian Laugier

Université Grenoble Alpes, Inria

Inria
UGA
Université Grenoble Alpes

## Motivation

**Challenges in motion forecasting:**

* Uncertainty in sensor data

* Multiple feasible futures

* Complex behavior of agents

## What OGMs offer

* Bird's-eye-view grid

* Efficient sensor fusion

* Real-time computation

* Models uncertainty

* No confidence thresholding

<figure><img src="image_4.png" /></figure>

But, conventional Occupancy Grid Maps (OGMs) predictions struggle with retaining dynamic vehicles in the scene and suffer from subjective choice of evaluation (MSE, IS).

## Contribution

* Incorporate contextual information to Dynamic Occupancy Grid Maps that relies on **vehicle semantics** and **map**

* Predict **vehicle semantic grids** that can be evaluated against ground truth annotations.

<figure><img src="image_5.png" /><figcaption>Vehicle Semantic labels</figcaption></figure>

<caption>Grid fixed to world frame</caption>

## Methods

<figure><img src="image_6.png" /></figure>

The prediction architecture takes inspiration from both **video prediction** and **conditional variational approach**.

DOGM with semantic labels and raster map features are fed to spatio-temporal and the probabilistic blocks to capture the evolution of the scene and predict a sequence of future vehicle semantic grids.

## Qualitative Results

<figure><img src="image_7.png" /><figcaption>DOGM with semantic labels
Input (1s)</figcaption></figure>

Experiments are conducted on Nuscenes dataset with grid dimension of 60x60m. For training, the grid images are resized to 256x256 pixels. In total, dataset has 3,499 training and 750 validation sequences.

<figure><img src="image_8.png" /><figcaption>Showcase Video - Youtube link</figcaption></figure>

## Quantitative Results

<figure><img src="image_9.png" /></figure>

**Retention of dynamic vehicles** compared to conventional OGM approaches.

**Ablation study** on adding information to DOGM. We report mean values of IoU and AUC over 2.5s prediction horizon.

<table><thead><tr><td>Input</td><td>IoU(↑)</td><td>AUC(↑)</td></tr></thead><tbody><tr><td>DOGM</td><td>0.336</td><td>0.496</td></tr><tr><td>DOGM + Map</td><td>0.430</td><td>0.604</td></tr><tr><td>DOGM + Semantics</td><td>0.590</td><td>0.771</td></tr><tr><td>DOGM + Map + Semantics</td><td><strong>0.605</strong></td><td><strong>0.781</strong></td></tr></tbody></table>

## Conclusions

* The addition of semantic labels plays a significant role in improving the prediction results.

* Superior ability to predict both static and dynamic vehicles in the scene.

* For future work, we envision fusion of semantic labels in the DOGM grid generation and prediction of other agents.

## References

[1] L. Rummelhard, J.-A. David, A. G. Moreno, and C. Laugier, "A cross-prediction, hidden-state-augmented approach for dynamic occupancy grid filtering," in ICARCV, 2022.

[2] A. Hu, Z. Murez, N. Mohan, S. Dudas, J. Hawke, V. Badrinarayanan, R. Cipolla, and A. Kendall, "FIERY: Future instance segmentation in bird's-eye view from surround monocular cameras," in ICCV, 2021.

[3] S. Lee, H. G. Kim, D. H. Choi, H.-I. Kim, and Y. M. Ro, "Video prediction recalling long-term motion context via memory alignment learning," in CVPR, 2021.