# CG-Net: Urban Trajectory Forecasting with Bipartite Graphs for Agents, Scene Context and Candidate Centerlines

Kaushik Bhowmik<fnref n="1" />, Anne Spalanzani², Philippe Martinet³

**Abstract**— Trajectory forecasting in urban environments is a critical task that needs to be addressed for the safety of autonomous vehicles, particularly in urban road intersection scenarios, where agents exhibit diverse behaviors mainly due to complex interactions between agents and the environment and the diversity of paths available to the agents. Current state-of-the-art methods do not perform well in urban road intersection scenarios. To address this issue, the proposed novel framework CandidateGraph-Net (CG-Net), improves trajectory prediction in urban road intersection scenarios by encoding the available candidate centerlines at the current location of the target agent. The proposed interaction encoder in CG-Net is inspired by human behavior. It is modeled utilizing a bipartite graph attention network to predict the trajectory of the target agent. It estimates the trajectory in the same way as humans anticipate the trajectory of other vehicles and pedestrians in dynamic environments. The agent embeddings in the interaction encoder at each time step pay attention to nearby agents and surrounding scene elements simultaneously. This enables the model to learn how to prioritize interactions between nearby agents and the environment map. Further, CG-Net's performance is evaluated using the Argoverse 2 motion forecasting dataset. The results demonstrate its effectiveness in urban road intersection scenarios, with an overall improvement in key metrics such as minFDE and minADE compared to baseline methods. These improvements highlight CG-Net's ability to perform better motion forecasting in urban road intersection scenarios.

**Index Terms**— Motion Forecasting, Deep Learning, Autonomous Vehicles

## I. INTRODUCTION

Autonomous vehicles must have a proper situational understanding of the surroundings to operate safely on the road. This includes perception, motion forecasting and motion planning capability. This task becomes more challenging with the growing influx of different types of agents in urban environments, including vehicles, pedestrians, motorcyclists, cyclists, and buses. Motion forecasting of these agents in urban scenarios has become more challenging, mainly due to the erratic behavior shown by these agents and the complex interactions happening between agent-agent and agent-scene interaction.

Different approaches for modeling and learning interactions [1] can be classified mainly into different categories:

* **Rationality utility-based models** simulate the most commonly encountered interactive scenarios in urban environments.

For vehicles, it involves modeling the behavior of the car following, merging in/out and lane change in urban environments. Other approaches that fall within these categories involve modeling human driving motion in crowded traffic environments as fluid-like behavior [2], flocking like swarms [3], game theory models with learning-based approach [4].

* **Social field and social force models** ([5], [6]), propose to model human driving interactions by introducing the concept of virtual potential fields or forces that are exerted by scene elements and other agents in the scenario on the target agent, which affects the decision-making capability of the target agent.

* **Computational cognitive models** ([7], [8]), consider an individual's driving behavior instead of the interaction model between agents.

* **Deep learning and graph-based models** ([9], [10]) propose to learn the interaction between agents and the scene using the data-driven learning approach. Various methods for feature representation to understand social interactions have been explored such as occupancy grid maps ([11], [12]), interaction graphs [13]. In recent years, modern deep learning-based approaches have gained huge popularity in the field of motion forecasting, with initial learning-based methods ([9], [14]) operating on rasterized representation for the environment representation. These deep learning-based approaches have shown promising performances. However, these approaches have limitations mainly because of utilizing the dense rasterized representation of the environment, which makes it computationally expensive for multi-agent trajectory prediction.

Among the various scene representations with the introduction of annotated HD maps, the graph-based vectorized representation introduced in VectorNet [10] has gained huge popularity, mainly because it is computationally more efficient and provides superior outcomes in evaluation metrics for motion forecasting tasks. Current state-of-the-art approaches like Wayformer [15] and QCNet [16] have followed same representation approach. These powerful architectures have adopted the transformer-based architecture [17] with factorized attention as their encoders for agent, scene, and interaction encoding.

For generating multi-modal output predictions, earlier approaches such as Sophie [18], Social-GAN [14] generate multi-modal predictions from generative models by sampling

<footnote n="1">¹ Inria at Univ. Grenoble Alpes, 38000 Grenoble, France ² Univ. Grenoble Alpes, Inria, F-38000 Grenoble, France ³ Centre Inria d'Université Côte d'Azur, (INRIA), ACENTAURI, Sophia Antipolis, France email: First Name.Last Name@inria.fr</footnote>
from latent variables which introduce test-time stochastic
behavior, which can be dangerous, given the safety-critical
nature of the autonomous driving application. Current
popular approaches like QCNet [16] and
Multipath [11], mainly generate multimodal predictions in
two stages, utilizing anchor-based and anchor-free solutions
where, the anchor-free module generates adaptive anchors
in a data-driven manner and the anchor-based module is
responsible for the refinement of the generated anchors.

In this work, we address the performance issues of motion
forecasting at challenging urban road intersection scenarios
by incorporating information about the candidate centerlines
of the target agent in the proposed pipeline. Additionally, the
interaction encoder is modeled by drawing inspiration from
human behavior. The main contributions of this work are:

1) New candidate centerline encoder that incorporates information about different candidate centerlines available to the agent at its current location, to perform better trajectory prediction at urban road intersections.

2) New interaction encoder, modeled as a bipartite graph attention network to update target agent embeddings in the same way as humans pay attention to agents and environment map simultaneously at each time step.

3) QCNet [16] decoder architecture adaptation to include candidate centerline encodings corresponding to the target agent, resulting in improved motion forecasting.

4) Evaluation of our proposed method on the real-world Argoverse 2 motion forecasting dataset [19], which demonstrates superior performance compared to state-of-the-art methods on standard performance metrics like minADE, and minFDE. We further conduct an ablation study to demonstrate the impact of the proposed candidate centerline encoder.

## II. APPROACH OVERVIEW

The proposed approach for the trajectory prediction model
is discussed in detail here, with an overview of the pipeline
shown in Figure 1.

<figure><img src="image_1.png" /><figcaption>Fig. 1: Proposed trajectory prediction pipeline</figcaption></figure>

### A. Problem Formulation

Given a scenario with *N* agents surrounding the autonomous vehicle, the trajectory prediction module at each time step receives as input the agent information in the form of agent states, which include the agent's velocity $v_i^t$, position $p_i^t = (p_{i,x}^t; p_{i,y}^t)$, orientation $\theta_i^t$, temporal position *t*, and type (vehicle, pedestrian, bus, etc.), as well as the displacement vector of the agent $p_i^t - p_{i-1}^t$. Additionally, map information in the form of HD maps (map polygons), where each map polygon is annotated with sampled points and semantic attributes, is provided as input.

Given the information on the map and the states of the agents within an observation window of *T* time steps, the objective of the trajectory prediction module is to forecast *K* future trajectories for each target agent over a future horizon of $T_f$ time steps and assign a probability score to each prediction.

### B. Local space-time coordinates

We define the local spacetime state coordinates for the agent at each time step *t* within the observation window as $s_i^t = (p_i^t; \theta_i^t; t)$, where $p_i^t$ and $\theta_i^t$ are the spatial and angular positions of the agent, respectively. For lanes and crosswalks, the position $L_p$ and the orientation $L_\theta$ at the entry point of the polyline of the scene element are chosen as a reference.

### C. Data pre-processing and feature encoding

Accurate forecasting of agent motion requires capturing agent dynamics, scene understanding, and identifying the potential paths available for the agent. This section describes the methodology for calculating the agent embedding using agent states and semantic information, computing positional encodings, and extracting candidate centerlines.

**Calculation of agent embeddings:** At each time step, the agent embedding is created utilizing agent features which comprise the displacement vector $p_i^t - p_{i-1}^t$, agent velocity $v_i^t$, agent orientation $\theta_i^t$ and agent semantic information. Then, this combined agent embedding for each time step is transformed into Fourier features [20], to get the updated agent embeddings, corresponding to each time step in the observation window.

**Calculation of candidate centerlines:** Based on Argoverse 1 motion forecasting API [21], the candidate centerlines for the target agent are determined using the agent's trajectory and the HD map. Initially, lane segments within a 2.5m distance bubble are identified. If no lane is found, the bubble expands until at least one lane is detected. Then, depth-first search is conducted to retrieve all successor and predecessor lane candidates. These lane segments are merged over the lane graph, followed by the removal of overlapping and unnecessary extensions. Next, centerlines for each lane sequence are generated. Finally, the number of candidates is reduced by considering the distance traveled along the centerline. Each candidate centerline is represented by $C_i$, which consists of candidate centerline points corresponding to that centerline. For illustration, see Figure 1.

**Calculation of position embeddings:** Following the sinusoid positional encoding ([17], [22]), we denote $g_i = $
<figure><img src="image_2.png" /><figcaption>Fig. 2: Proposed encoder architecture (detailed) with adapted QCNet decoder</figcaption></figure>

($x_i, y_i, \theta_i$) as the global pose and orientation of the agent $i$. The global positional encoding (GPE) of $g_i$ is computed using sinusoidal positional encoding (PE) and angular encoding (AE):

$$GPE(g_i) = \text{concat}(\text{PE}(x_i), \text{PE}(y_i), \text{AE}(\theta_i)) \quad (1)$$

where,

$$\text{PE}_{2i}(x) = \sin\left(x \cdot \omega^{\frac{2i}{D}}\right), \quad \text{PE}_{2i+1}(x) = \cos\left(x \cdot \omega^{\frac{2i}{D}}\right),$$

$$\text{AE}_{2i}(\theta) = \sin\left(\theta \cdot (i + 1)\right), \quad \text{AE}_{2i+1}(\theta) = \cos\left(\theta \cdot (i + 1)\right)$$
for $i \in \{0, \dots, \frac{D}{2} - 1\}$, and where $\omega$ is the base frequency.

## D. Proposed encoder architecture

The proposed encoder consists of five major blocks: agent history encoder, interaction encoder, scene encoder, candidate centerline encoder and multi-layer perceptron (MLP) as shown in Figure 2.

**Agent history encoder:** The agent embeddings at each time step in this block are updated utilizing temporal attention from the previous time steps, which is responsible for capturing the temporal evolution of the agent's trajectory without taking into account the nearby agents and scene elements.

**Scene encoder:** Following the state-of-art representation ([10], [16]), the scene encoder is modeled as a bipartite graph, where each polyline embedding in the HD map is modeled as destination node. Each destination node is initialized using polyline semantics (type and intersection) along with position and orientation attributes. The source nodes, corresponding to the polyline nodes, are created by combining the polyline node attributes (position and orientation) with their corresponding semantic attributes. These source nodes are then used to perform an attention-based update of the polyline embeddings. Finally, the polyline embeddings are updated using an attention mechanism with a distance threshold of 100 meters.

**Interaction encoder:** The inspiration for the interaction encoder is taken from Vivit [23], and how humans perform

motion forecasting by prioritizing the effect of nearby agents (e.g. vehicles, pedestrians) and scene elements (e.g., road lanes, pedestrian crossings) at each time step, which allows the model to perform more accurate updates to the target agent embedding. The interaction encoder block is modeled as a bipartite attention graph network, where the attention-based update of the embedding of each agent at each time step is performed by utilizing the encoding of nearby agents and scene elements with the distance threshold of 100 meters based on relative position and orientation. In this representation, the destination node of the agent embeddings at each step is initialized utilizing the agent's type, combined with the positional encoding of the agent at each step.

**Candidate centerline encoder:** Candidate centerline encoding is created for the agent types (vehicle, motorcyclist, cyclist, bus). The candidate centerline encoder is modeled as a bipartite graph attention network. The number of destination nodes in the candidate centerline encoder corresponding to each target agent in the bipartite graph is determined by the number of available candidate centerlines at the last observation time step. Each destination node is initialized using the agent's positional encoding as described in Section II-C, and agent type information. The destination nodes corresponding to the target agent of the bipartite graph are updated utilizing the source nodes of the corresponding lane features which comprise lane vectors extracted from the candidate centerlines and then passing these lane vectors through Fourier features [20] to create candidate centerline encodings for the agent.

**Multi-layer perceptron (MLP):** The MLP block generates the final agent embedding by combining the embeddings from the agent history encoder block and the interaction encoder block for each time step.

## E. Proposed decoder architecture

In the second stage of the trajectory prediction pipeline, utilizing the agent encodings, map polyline encodings, and
<figure><img src="image_3.png" /><figcaption>Fig. 3: Adapted QCNet decoder architecture with Mode2Centerline cross attention</figcaption></figure>

candidate centerline encodings corresponding to the target agent, we decode K future trajectories for the target agent. Following QCNet [16] decoder architecture, which performs prediction in two stages (proposal stage and refinement stage) utilizing the blocks Mode2Scene and Mode2Mode. The Mode2Scene block is responsible for updating mode queries with multiple scene contexts, which include the history encodings of the target agent, the map encodings, and the neighboring agents' encodings. The Mode2Mode is responsible for updating the K mode queries with each other using self-attention to improve the diversity of forecasted trajectories. We adapt the QCNet [16] decoder architecture, to also include candidate centerline encoding for the target agent in the first stage (refinement stage).

The proposed Mode2Centerline block updates the mode queries by performing cross-attention and utilizing the candidate centerline encoding corresponding to the target agent. The adapted decoder pipeline is shown in Figure 3.

### III. EXPERIMENTS

#### A. Dataset

We utilized the Argoverse 2 motion forecasting dataset [19], a large-scale motion forecasting dataset, to test the efficiency of our proposed approach. The Argoverse 2 dataset consists of 250000 scenarios spanning six US cities: Austin, Detroit, Miami, Pittsburgh, Palo Alto, and Washington D.C. The total length of the unique roadways in the dataset is 2110 Km. The dataset has 10 unique object types. The dataset was recorded at a sampling rate of 10 Hz. The Argoverse 2 motion forecasting dataset offers a large observation window of 5 seconds, requiring the model to predict the next 6 seconds. Using this dataset, we evaluated the proposed model's performance in motion forecasting.

**Evaluation Metrics:** In line with the standard evaluation protocol, we adhere to the same evaluation criteria, which include: minimum Average Displacement Error $minADE_K$, minimum Final Displacement Error $minFDE_K$, Brier-minimum Final Displacement Error $brier-minFDE_K$, and Miss Rate $MR_K$ for evaluation. The metric $minFDE_K$

evaluates the $l_2$ distance between the endpoint of the best forecasted trajectory and ground truth, where "best" refers to the trajectory having minimum endpoint error. The metric $minADE_K$ evaluates the average $l_2$ distance in meters between the ground-truth trajectory and the best of K predicted trajectories. The metric $MR_K$ measures the proportion of scenarios where none of the predictions are within 2.0 meters of the ground truth, based on endpoint error, relative to the total number of scenarios. The metric $brier-minFDE_K$ is computed similarly to $minFDE_K$ but with the endpoint $l_2$ distance augmented by $(1 - P_{best})^2$, where $P_{best}$ represents the probability of the best-predicted trajectory. Generally, K is selected as 1 and 6. If the model outputs more than K predicted trajectories, then only the predictions with the top-K probability scores are considered for evaluation.

#### B. Training

Following QCNet [16], the future trajectory of each i-th agent is modeled as a mixture of Laplace distributions:

$$f(\{\mathbf{p}_i^t\}_{t=1}^{T_f}) = \sum_{k=1}^{K} \lambda_{i,k} \prod_{t=1}^{T_f} \text{Laplace}(p_i^t | \mu_{i,k}^t, b_{i,k}^t), \quad (2)$$

where $\{\lambda_{i,k}\}_{k=1}^K$ are the mixing coefficients and the k-th mixture component's Laplace density at time step t is parameterized by the location $\mu_{i,k}^t$ and the scale $b_{i,k}^t$.

The classification loss, $\mathcal{L}_{cls}$, is used to optimize the mixing coefficients predicted by the refinement module. This loss minimizes the negative log-likelihood from Eq. (2), while gradients for the locations and scales are stopped, ensuring that only the mixing coefficients are optimized. Subsequently, a winner-take-all strategy is applied to optimize the locations and scales generated by the proposal and refinement modules. This approach performs backpropagation exclusively on the best-predicted proposal and its refinement.

The final loss function integrates the trajectory proposal loss $\mathcal{L}_{traj}$, the trajectory refinement loss $\mathcal{L}_{refine}$, and the classification loss $\mathcal{L}_{cls}$ to enable end-to-end training:

$$\mathcal{L} = \mathcal{L}_{traj} + \mathcal{L}_{refine} + \mathcal{L}_{cls}, \quad (3)$$

The model is trained with an initial learning rate of $1 \times 10^{-4}$ using the AdamW optimizer, and the learning rate is scheduled.
<table><caption>TABLE I: Quantitative results on the Argoverse 2 motion forecasting leaderboard [19], ranked by $minFDE_6$. For each metric, the best result is in **bold**.</caption><thead><tr><th>Method</th><th>b-minFDE<sub>6</sub> ↓</th><th>minADE<sub>6</sub> ↓</th><th>minFDE<sub>6</sub> ↓</th><th>MR<sub>6</sub> ↓</th><th>minADE<sub>1</sub> ↓</th><th>minFDE<sub>1</sub> ↓</th><th>MR<sub>1</sub> ↓</th></tr></thead><tbody><tr><td>THOMAS [24]</td><td>2.16</td><td>0.88</td><td>1.51</td><td>0.20</td><td>1.95</td><td>4.71</td><td>0.64</td></tr><tr><td>ProphNet [25]</td><td>1.88</td><td>0.66</td><td>1.32</td><td>0.18</td><td>1.76</td><td>4.77</td><td>0.61</td></tr><tr><td>QCNet (w/o ensemble) [16]</td><td>1.91</td><td>0.65</td><td>1.29</td><td>0.16</td><td>1.69</td><td>4.30</td><td>0.59</td></tr><tr><td>SmartRefine [26]</td><td>1.86</td><td>0.63</td><td>1.23</td><td>0.15</td><td>1.65</td><td>4.17</td><td>0.58</td></tr><tr><td>DyMap (with ensemble) [27]</td><td><b>1.78</b></td><td>0.66</td><td>1.21</td><td>0.15</td><td><b>1.59</b></td><td><b>3.99</b></td><td>0.57</td></tr><tr><td><b>CG-Net</b></td><td>1.82</td><td><b>0.626</b></td><td><b>1.20</b></td><td><b>0.144</b></td><td>1.61</td><td>4.02</td><td><b>0.56</b></td></tr></tbody></table>

ruled with CosineAnnealingLR for 80 epochs with batch size of 6.

## IV. RESULTS

### A. Qualitative Evaluation

1) **Qualitative comparison between CG-Net and QCNet approach:** We perform qualitative analysis and compare the performance of our proposed approach CG-Net and the QCNet approach, to perform long-term motion forecasting in urban road intersection scenarios. The visualizations of the predicted trajectories for QCNet and CG-Net are shown in Figure 4, which demonstrates the superior performance of CG-Net in urban road intersection scenarios. The top row illustrates a multi-lane road intersection where QCNet fails to generate valid trajectories for the target agent, while CG-Net predictions successfully cover ground truth trajectory. The middle row depicts a multi-lane urban road intersection with multiple vehicles, where QCNet fails to capture the ground truth trajectory, whereas CG-Net provides accurate predictions. The bottom row shows a complex urban scenario where a road intersection is encountered in the later stage of the observation window. Despite this challenge, the CG-Net predictions cover the ground truth trajectory.

2) **Generalization to various scene and agent types:** We evaluate the performance of CG-Net in terms of its generalization across different map environments and agent types. Visualizations of the predicted trajectories are shown in Figure 5. Figures 5a and 5b demonstrate that CG-Net accurately predicts pedestrian trajectories along the roadside, capturing the temporal evolution of motion while adhering to the constraints imposed by the surrounding map. Figure 5c illustrates a roundabout scenario, where a vehicle approaches the roundabout, and CG-Net successfully forecasts its trajectory while respecting lane constraints.

### B. Quantitative Evaluation

1) **Comparison with QCNet and other baseline approaches:** We evaluate the performance of our proposed CG-Net approach against QCNet and other popular baselines on the Argoverse 2 test set. The Argoverse 2 dataset favors the evaluation of motion forecasting frameworks for long prediction horizons of 6 seconds. The results of the standard evaluation metrics are shown in Table I. We show that CG-Net performs better than popular baseline approaches in key significant metrics like minADE, minFDE and MR for performance evaluation.

<figure><img src="image_4.png" /><figcaption>Fig. 4: Visual comparison of the predicted trajectories (QC-Net and CG-Net) for three urban road intersection scenarios. Orange: Ground truth, Green: CG-Net, Blue: QCNet.</figcaption></figure>

2) **Ablations study on encoder block:** We analyze the effect of different components in the proposed encoder architecture within the CG-Net pipeline. Specifically, we consider three configurations: only Agent History Encoder (AHE), only Interaction encoder (IE), and both AHE and IE combined. The performance metrics for the different cases of the ablation study are presented in Table II. We observe that only using IE leads to sub-optimal performance, because the agent encoding at each time step do not attend to each other temporally, which is important for capturing motion dynamics. While using only AHE, enables the temporal update of agent encoding within the encoder module, allowing
<figure><img src="image_5.png" /><figcaption>Fig. 5: Qualitative evaluation of CG-Net predictions for pedestrian trajectories near the roadside and vehicle behavior at a roundabout.</figcaption></figure>

<table><caption>TABLE II: Ablation study on CG-Net encoder modules with Argoverse 2 validation set. For each metric, the best result is in **bold**.</caption><thead><tr><th>Approach</th><th>minADE<sub>6</sub> ↓</th><th>minFDE<sub>6</sub> ↓</th><th>MR<sub>6</sub> ↓</th></tr></thead><tbody><tr><td>IE only</td><td>0.77</td><td>1.27</td><td>0.16</td></tr><tr><td>AHE only</td><td>0.71</td><td>1.20</td><td>0.15</td></tr><tr><td>CCE (AHE and IE)</td><td><strong>0.70</strong></td><td><strong>1.178</strong></td><td><strong>0.14</strong></td></tr></tbody></table>

the model to better capture temporal evolution of agent dynamics. The model can attend to the scene elements and candidate centerlines through decoder blocks, which enable the model to learn motion forecasting behavior. However, combining both AHE and IE in the encoder block results in the best motion forecasting performance. This configuration utilizes both temporal modeling (from AHE) and interaction modeling (from IE), resulting in superior motion forecasting performance.

3) Ablations study on decoder block: We study the effect of the candidate centerline encoding (CCE) in the decoder on the entire CG-Net pipeline: CG-Net without CCE, CCE introduced only to the first stage (proposal) of the decoder, CCE introduced only to the final stage (refinement) of the decoder and CCE incorporated in both stages (proposal and refinement) of the adapted QCNet decoder architecture. The performance metrics for the different cases of the ablation study is shown in Table III. We report that introducing candidate centerline encodings to the initial stage (proposal stage) of the adapted QCNet decoder architecture outperforms all other cases in the evaluation metrics. We believe that candidate centerline encodings tend to guide the mode queries to produce trajectory proposals for the agents. However, once the trajectory proposals are generated, attending to them again using candidate centerline encodings during the refinement stage, which focuses solely on refining the proposals does not improve motion forecasting performance.

4) Agent-wise motion forecasting performance evaluation: We further evaluate the agent-wise motion forecasting performance using the validation split of the Argoverse 2 motion forecasting dataset, with results presented in Table IV. Based on the selected evaluation metrics, we observe that CG-Net achieves the best performance for pedestrian motion forecasting. This is likely because pedestrians in the

<table><caption>TABLE III: Ablation study on CG-Net decoder with Argoverse 2 validation set. For each metric, the best result is in **bold**.</caption><thead><tr><th>Approach</th><th>minADE<sub>6</sub> ↓</th><th>minFDE<sub>6</sub> ↓</th><th>MR<sub>6</sub> ↓</th></tr></thead><tbody><tr><td>No CCE</td><td>0.71</td><td>1.206</td><td>0.15</td></tr><tr><td>CCE (both stages)</td><td>0.71</td><td>1.19</td><td>0.148</td></tr><tr><td>CCE (second stage only)</td><td>0.71</td><td>1.196</td><td>0.145</td></tr><tr><td>CCE (first stage only)</td><td><strong>0.70</strong></td><td><strong>1.178</strong></td><td><strong>0.14</strong></td></tr></tbody></table>

Argoverse 2 dataset tend to move slowly, and the dataset lacks complex trajectories such as group behaviors and more challenging examples of traffic violations commonly observed in real-world scenarios.

We also note that the motion forecasting performance for vehicle and bus agent types requires improvement. This is due to the presence of more complex behaviors in these agents, such as lane changes on empty streets happening in the prediction horizon, a few out-of-distribution scenarios, and abrupt U-turns, all of which contribute to decreased performance for these agent types. This limitation can be addressed by training on a larger and more diverse dataset that includes a wider variety of motion trajectories.

<table><caption>TABLE IV: Agent-wise motion forecasting performance of CG-Net on the Argoverse 2 validation set.</caption><thead><tr><th>Agent type</th><th>Total samples</th><th>minADE<sub>6</sub> ↓</th><th>minFDE<sub>6</sub> ↓</th><th>MR<sub>6</sub> ↓</th></tr></thead><tbody><tr><td>Cyclist</td><td>134</td><td>0.87</td><td>1.63</td><td>0.20</td></tr><tr><td>Pedestrian</td><td>1572</td><td>0.33</td><td>0.60</td><td>0.045</td></tr><tr><td>Motorcyclist</td><td>303</td><td>0.64</td><td>1.20</td><td>0.13</td></tr><tr><td>Vehicle</td><td>22134</td><td>0.72</td><td>1.21</td><td>0.15</td></tr><tr><td>Bus</td><td>845</td><td>0.939</td><td>1.21</td><td>0.14</td></tr></tbody></table>

V. CONCLUSION

In this work, we have introduced CG-Net, which overcomes the challenges faced by current state-of-the-art motion forecasting models to perform better trajectory prediction at urban road intersections. The presented approach utilizes the information about the candidate centerlines available for the agent at its current location. The interaction encoder mimics human-like behavior to predict motion, prioritizing interactions between nearby agents and the surrounding environment to perform better motion forecasting in urban
road intersections where multiple paths are available to the agent. We evaluate our work against several state-of-the-art trajectory prediction methods. Further, we perform an ablation study on the developed framework CG-Net to study the importance of the candidate centerline encoder. In the future, this work will be extended to perform multi-agent motion forecasting.

## VI. ACKNOWLEDGMENT

This project was funded by the ANR project: ANR-21-CE22-0014 ANNAPOLIS and was provided with computing AI and storage resources by GENCI at IDRIS, thanks to the grant 2024-AD011015615 on the supercomputer Jean Zay's A100 partition.

## REFERENCES

[1] W. Wang, L. Wang, C. Zhang, C. Liu, L. Sun et al., "Social interactions for autonomous driving: A review and perspectives," *Foundations and Trends in Robotics*, vol. 10, no. 3-4, pp. 198-376, 2022.

[2] R. Mahjourian, J. Kim, Y. Chai, M. Tan, B. Sapp, and D. Anguelov, "Occupancy flow fields for motion forecasting in autonomous driving," *IEEE Robotics and Automation Letters*, vol. 7, no. 2, pp. 5639-5646, 2022.

[3] C. Bechinger, R. Di Leonardo, H. Löwen, C. Reichhardt, and G. Volpe, "Active particles in complex and crowded environments," *Reviews of Modern Physics*, vol. 88, no. 4, p. 045006, 2016.

[4] W. Schwarting, A. Pierson, J. Alonso-Mora, S. Karaman, and D. Rus, "Social behavior for autonomous vehicles," *Proceedings of the National Academy of Sciences*, vol. 116, no. 50, pp. 24972-24978, 2019.

[5] D. Helbing and P. Molnar, "Social force model for pedestrian dynamics," *Physical review E*, vol. 51, no. 5, p. 4282, 1995.

[6] M. Prédhumeau, J. Dugdale, and A. Spalanzani, "Adapting the social force model for low density crowds in open environments," in *Conference of the European Social Simulation Association*. Springer, 2019, pp. 519-531.

[7] N. Chater, J. Misyak, D. Watson, N. Griffiths, and A. Mouzakitis, "Negotiating the traffic: Can cognitive science help make autonomous vehicles a reality?" *Trends in cognitive sciences*, vol. 22, no. 2, pp. 93-95, 2018.

[8] D. Zhou, H. Liu, H. Ma, X. Wang, X. Zhang, and Y. Dong, "Driving behavior prediction considering cognitive prior and driving context," *IEEE Transactions on Intelligent Transportation Systems*, vol. 22, no. 5, pp. 2669-2678, 2020.

[9] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, "Social lstm: Human trajectory prediction in crowded spaces," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2016, pp. 961-971.

[10] J. Gao, C. Sun, H. Zhao, Y. Shen, D. Anguelov, C. Li, and C. Schmid, "Vectornet: Encoding hd maps and agent dynamics from vectorized representation," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 11525-11533.

[11] Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, "Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction," in *Conference on Robot Learning*. PMLR, 2020, pp. 86-99.

[12] T. Phan-Minh, E. C. Grigore, F. A. Boulton, O. Beijbom, and E. M. Wolff, "Covernet: Multimodal behavior prediction using trajectory sets," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 14074-14083.

[13] Y. Hu, W. Zhan, and M. Tomizuka, "Probabilistic prediction of vehicle semantic intention and motion," in *2018 IEEE Intelligent Vehicles Symposium (IV)*. IEEE, 2018, pp. 307-313.

[14] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi, "Social gan: Socially acceptable trajectories with generative adversarial networks," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2018, pp. 2255-2264.

[15] N. Nayakanti, R. Al-Rfou, A. Zhou, K. Goel, K. S. Refaat, and B. Sapp, "Wayformer: Motion forecasting via simple & efficient attention networks," in *2023 IEEE International Conference on Robotics and Automation (ICRA)*. IEEE, 2023, pp. 2980-2987.

[16] Z. Zhou, J. Wang, Y.-H. Li, and Y.-K. Huang, "Query-centric trajectory prediction," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2023, pp. 17863-17873.

[17] A. Vaswani, "Attention is all you need," *Advances in Neural Information Processing Systems*, 2017.

[18] A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese, "Sophie: An attentive gan for predicting paths compliant to social and physical constraints," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2019, pp. 1349-1358.

[19] B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. K. Pontes et al., "Argoverse 2: Next generation datasets for self-driving perception and forecasting," in *Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)*.

[20] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Ragha-van, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, "Fourier features let networks learn high frequency functions in low dimensional domains," *Advances in neural information processing systems*, vol. 33, pp. 7537-7547, 2020.

[21] M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan et al., "Argoverse: 3d tracking and forecasting with rich maps," in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2019, pp. 8748-8757.

[22] Z. Zhang, A. Liniger, C. Sakaridis, F. Yu, and L. V. Gool, "Real-time motion prediction via heterogeneous polyline transformer with relative pose encoding," *Advances in Neural Information Processing Systems*, vol. 36, 2024.

[23] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, and C. Schmid, "Vivit: A video vision transformer," in *Proceedings of the IEEE/CVF international conference on computer vision*, 2021, pp. 6836-6846.

[24] T. Gilles, S. Sabatini, D. Tsishkou, B. Stanciulescu, and F. Moutarde, "Thomas: Trajectory heatmap output with learned multi-agent sampling," in *International Conference on Learning Representations*, 2022.

[25] X. Wang, T. Su, F. Da, and X. Yang, "Prophetn: Efficient agent-centric motion forecasting with anchor-informed proposals," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2023, pp. 21995-22003.

[26] Y. Zhou, H. Shao, L. Wang, S. L. Waslander, H. Li, and Y. Liu, "Smartrefine: A scenario-adaptive refinement framework for efficient motion prediction," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024, pp. 15281-15290.

[27] B. Fan, H. Yuan, Y. Dong, Z. Zhu, and H. Liu, "Bidirectional agent-map interaction feature learning leveraged by map-related tasks for trajectory prediction in autonomous driving," *IEEE Transactions on Automation Science and Engineering*, 2025.