Motion Forecasting with Unlikelihood Training in Continuous Space
Authors: Deyao Zhu
Mohamed Zahran
Li Erran Li
Mohamed Elhoseiny
Venue: CoRL2021 Oral (2021)

Abstract: Motion forecasting is essential for making safe and intelligent decisions in robotic applications such as autonomous driving. Existing methods often formulate it as a sequence-to-sequence prediction problem, solved in an encoder-decoder framework with a maximum likelihood estimation objective. State-of-the-art models leverage contextual information including the map and states of surrounding agents. However,  we observe that they still assign a high probability to unlikely trajectories resulting in unsafe behaviors including road boundary violations. Orthogonally, we propose a new objective, unlikelihood training, which forces predicted trajectories that conflict with contextual information to be assigned a lower probability. We demonstrate that our method can improve state-of-art models' performance on challenging real-world trajectory forecasting datasets (nuScenes and Argoverse) by avoiding up to 56% context-violated prediction and improving up to 9% prediction accuracy.

Motion Forecasting with Unlikelihood Training in Continuous Space
Deyao Zhu!
Mohamed Zahran
Li Erran Li Mohamed Elhoseiny!
King Abdullah University of Science and Technology. 2 Udacity,
AWS AI, Amazon and Columbia University
deyao. wwwkkaust.kt.... eew.usstteuu..... sa mohammed zaaz zahran@uuacity com Brranlli@gmail. com
mohamed mohamed. wwlossinnykaasst. eew.assstedde.... edu.sa
Abstract: Motion forecasting is essential for making safe and intelligent de- cisions robotic applications such as autonomous driving. Existing methods often formulate it as sequence-fo-sequeecee prediction problem, solved in an encoder-decoder framework with maximum likelihood estimation objective. -fate-of-the-art models leverage contextual information, including the map and states of surrounding agents. However, we observe that they still assign a high probability to unlikely trajectories resulting in unsafe behaviors, including road boundary violations. Orthogonally, we propose a new objective, unlikelihood train- ing, which forces predicted trajectories that conflict with contextual information to be assigned a lower probability. We demonstrate that our method can improve state-of-art models' performance the challenging nuScenes and Argoverse real-world trajectory forecasting datasets by avoiding up to 56% context-violated prediction and improving up to 9% prediction accuracy. Code is avaliable betps://githan www./o/ii/o/CAAAA... otw.iaieelootoooooooooooooeeeee
Keywords: Motion Forecasting, Autonomous Driving
1 Introduction
For robotic applications deployed in the real world, the ability to foresee the future motions of agents in the surrounding environment plays an essential role in safe and intelligent decision- decision-making. In the autonomous driving domain, predict nearby agents' future trajectories accurately, an agent must consider contextual information such as their past trajectories, potential interactions, and map information. State of the art prediction models [1, 2, 3] directly take contextual information as part of their input and use techniques such as graph neural networks to extract high-level features for prediction. They are typically trained with a maximum likelihood estimation (MLE) objective of ground truth trajectories in the predicted distribution. Although MILE loss encourages the prediction to be geometrically close to the ground truth, there is no explicit motivation to learn a distribution that leverages contextual information. These models may predict trajectories that violate the contextual information (e.g., go to opposite driving direction or out the driving area) but still closes to ground truth. In contrast, humans quickly notice that these trajectories unlikely in specific context. This observation suggests that using MLE loss cannot fully leverage contextual information, and hence may predict unsafe trajectories more likely.
To address the problem, existing methods like [4, 5], using contextual information, either introduce new learning parameters or apply reinforcement learning methods requiring further reward engineer- ing. We propose novel and simple method, unlikelihood training, that injects contextual information as learning signal. Our loss penalizes the trajectories that violate the contextual information, dubbed as negative trajectories, by minimizing their likelihood in the predicted distribution. To generate negative or unlikely trajectories, we first draw many candidate trajectories from our model's predicted distribution. Then, we introduce a context checker to identify the trajectories that violate contextual information negative trajectories". The model is then encouraged to use contextual information to
Work done outside Amazon. "h context checker is not required to be differentiable
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

avoid predictions that violate context by minimizing the likelihood of negative trajectories. Hence, the predicted trajectories can be more accurate and, more importantly, safer.
Unlikelihood training [6] has been applied to neural language generation in discrete space. To the best of our knowledge, we are the first to propose unlikelihood training for continuous space of trajectories. For the discrete space of token sequences, repeating tokens or n-grams in the generated sequence are chosen as negative tokens. In contrast, we design context checker to select negative trajectories sampled from the continuous distribution of model predictions as input to an unlikelihood loss. In this case, our learning signals contain not only the information of the ground truth trajectories from the MLE, but also the the context information from our proposed loss. Our method can be viewed as simple add-on to models that estimate the distribution of future trajectories. It improves their performance by encouraging models to focus more on contextual information without increasing the number of model parameters. Our contributions are summarized as follows:
We propose a novel and simple method, continuous unlikelihood training for motion fore- casting in autonomous driving that encourages models to use contextual information by minimizing the likelihood of context-violated trajectories. Our method can be incorporated into state-of-the-art models that predict the future as distributions.
Our large-scale experimental results two challenging real-world trajectory forecasting datasets, nuScenes, and Argoverse, show that continuous unlikelihood training can im- prove the quality of the predicted distribution by avoiding maximally 56% context-violated prediction and improving 9% prediction performance.
2 Related Work
Trajectory Forecasting Trajectory forecasting of dynamic agents, core problem for robotic applications such as autonomous driving and social robots, has been well studied in the literature. ffae--oftthe-ar models solve it
multi-modal prediction problem [7, 8, 9, 3, 10, 2, 11, 1, 12]. MTP [8] and MultiPath [9] predict multiple future trajectories without learning low dimensional latent agent behaviors. Recent works including [7, 10, 3, 12] encode agent behaviors in continuous low dimensional latent space while MFP [2] and Trajectron++ [1] use discrete latent variables. Discrete latent variables succinetly capture semantically meaningful modes such as turn left, turn right. MFP [2] and Trajectron++ [1] learn discrete latent variables without explicit labels. They adopt a maximum likelihood estimation (MLE) objective or its approximations (c.g.. VAE). In this paper, we show that MLE loss can ignore contextual information such as maps and states of surrounding agents. As result, models with MLE loss only may assign a very high probability to unlikely trajectories. To alleviate this issue, R2P2 [13] trains surrogate ground truth distribution first and use it to regularize the training of the predictor. However, the surrogate still trained by MLE loss only. In contrast, we propose an unlikelihood training objective that is based directly on the context information to reduce unlikely prediction, and we show that models with the maximum likelihood estimation objective can benefit from it.
geguencc-to-squueee
Unlikelihood Training. In the context of language generation, recent works including [6. 14] propose unlikelihood training as a new method to utilize negative language data, which includes common degenerate cases like token repetition and distribution mismatch. The distribution mismatches occur as an artifact of beam search, which is biased to high-frequency tokens compared to fow-frequency tokens which appear rarely. The method minimizes the likelihood of negative tokens to improve text generation quality in addition to maximizing the likelihood of the ground truth tokens. However, their approach operates in the discrete space. In contrast, our proposed method works in the continuous space of trajectories. We model negative data by context checker that we propose, which operates on predictions to populate unlikely trajectories. Predicted trajectories that violate the checker are selected to leverage contextual information by minimizing their likelihood objective. We show the generality of our approach on two state-of-the-ar distribution-ban based approaches: Trajectron++ [1] and distribution-batt variant of LaneGCN [15].

---

(a) Trajectron++[1]
(b) Ours
Figure 1: Examples of predicted distribution in a complex scenario from models trained with MLE only (a) and with our unlikelihood loss additionally (b). Some of the Trajectron++'s predicted distributions out of the drivable region cover the lane in the wrong direction (Highlighted by ellipses). Our method helps alleviate this issue. Predicted distribution is plotted as the colored region. White points denote the ground truth trajectories. More qualitative results are in appendix.
3 Method
3.1 Problem Definition
We aim at predicting the future trajectory Y, more accurately vehicle given input X, X; can include related information like rasterized maps or past positions of vehicle and surrounding agents. Because the generality of our training mechanism, we skip the specific choice of this additional information and denote it as X; for conciseness. Due to different driving strategies, driving intents, and the complex traffic environment, there are usually multiple possible future trajectories given an input X₁; ;; despite that there is only one ground truth future trajectory Yg in dataset recorded in the real world. To facilitate predicting multiple modalities, most state of the art methods [1, 2] model distribution of possible future trajectories po (Y; X;) cover all the possibilities given the input X; instead of predicting one trajectory. 8 denotes the learning parameters of the model. Training such models can be done by Maximum Likelihood Estimation (MLE) that directly maximizes the likelihood of ground truth trajectory Y. or its lower bound in the predicted distribution.
Limitation of MLE on Motion Forecasting MLE encourages the model predict a distribution that allocates reasonable probability mass to the region where Yi is located by minimizing the KL- divergence between the ground truth distribution and the predicted distribution. Because the trajectory distribution domain is over the geometric locations, MLE makes these two distributions "close" to each other geometrically. However, we argue that maintaining the geometrical nearness only not good enough for motion forecasting tasks in autonomous driving. In complex traffic scenarios, there can be many potential trajectories close enough to the ground truth geometrically but are very unlikely to happen. For example, if the ground truth trajectory Ygt is the outermost lane, trajectory that close to Yi.g but outside the drivable region is unlikely to happen in the real world. MLE loss will not impose significant enough penalty to avoid such prediction. Fig. (a) demonstrates prediction example from an MLE-based method Trajectron++ [1]. Part of the distribution this example is the lane in the wrong direction. The
outside of the derivable region learning signals that contain the geometric location information the ground truth trajectories. All the other contextual information like the drivable region is missing in this learning signal. Therefore, the model is not encouraged during training to utilize the rich contextual information in the input to avoid predictions that geometrically close ground truth but violates the context. In contrast, this is quite a simple task for humans.
loss only offers
3.2 Unlikelihood Loss
To mitigate this problem, we design a new loss term that encourages the model to consider contextual information more. Inspired by the positive and negative data pairs in contrastive learning and unlikelihood training, we additionally train our model minimize the likelihood of trajectories that violate the contextual information given input X,. We denote them negative trajectories Ys,eg-

---

Let's first assume that we already have distribution of negative trajectories Paeg Y X One intuitive way is to directly minimize the log likelihood of Y, Yengg in our predicted distribution, similar to MLE but in an opposite manner, as shown in Eq.1. We call it unlikelihood loss. We use coefficient y to balance Trunliee Our loss can be combined with models that predict trajectory distribution as output. Eq.2 shows the final training objective.
Trubiik Ex,, Viney Pres (Y Yi xx) llog Yineg
X:)
L Loig + Lmllike
Tonig indicates the original training objective of the method we combine with. Lunlike helps leverage the context into the learning objective, force the model to better extract and use contextual information X, and generate a more reasonable distribution to avoid high Lunlike-
3.3 Negative Trajectories
Our proposed loss term is highly dependent on the negative samples Ysn from the distribution Phog(2 YY X2). However, these are not given in the dataset. To solve this issue, we approximate the samples by directly drawing a set of trajectories from the predicted distribution and select the trajectories that violate the contextual information out by context checker. Note that this checker does not need to be differentiable and it can be as complex and advanced as necessary. The type of unlikely predictions the model learns avoid by our method depends on the type of unlikely trajectories the checker can identify.
Context Checker Design. We provide and implement a map-based checker design here to judge whether given trajectory is compliant with context and road rules. In detail, the checker examines whether the trajectory goes into the lane in the opposite direction or out of the road. We create a map that stores the lane direction at every location of lanes and the drivable region. Two examples are shown Fig.2. We first check whether all the locations region. If so, we further calculate angles between velocity and the lane direction at each time step to see whether they are all within 90 degree. The velocities are approximated by differentiating trajectory. The trajectories that fail to pass the test are identified as negative trajectories Y, Y,neg- Note that the lane direction information might be incomplete in some examples in the dataset, and in the case, only the drivable region information. In addition, human drivers also go against the context occasionally. We disable Lunlike if the ground truth trajectory violates the context to allow a similar prediction. Although this checker is not perfect, it is able select out meaningful negative trajectories to support the training. Note that recent works like Beelines [16] provide other methods to detect the unlikely trajectories and it is possible to adopt them additional rules in checker. We leave it for future research.
given trajectory are in the drivable
(a) negative
(b) positive case
Figure 2: Examples of maps used in the checker in nuScences [17]. The green and blue regions together denote the drivable region. The lane direction is available in blue region. The lane directions plotted as red arrows at random location. The blue line denotes the trajectory to check. Its velocity directions are represented as yellow arrows. (a) shows a negative trajectory that goes out of the road. (b) shows positive trajectory that passes the checker.
3.4 Algorithm
The final algorithm shown in Alg. 1. At each iteration, we first run forward pass the model to get the output distribution po X;) given the input X; Then, K negative candidate trajectories are drawn from this distribution and we select the negative trajectories out Yineg via our checker. After that, ground truth trajectory and negative trajectories are used to calculate the loss function, and

---

the model is updated. Finally, we proceed to the next iteration. Note that if there are no Ynng in the K negative candidates judged by our checker for data i, we don't apply Lunlike on i.
3.5 Gradient Analysis
Assume a single-mode prediction case that the future position Vi.gt. at time step t of agent i is modeled by simple Gaussian distribution (y Ps.t, oz, f and og are calculated by the model. With a single negative position Vinngg, we define simple loss for step t I, = - Mog.WWygte fe; ō,T) + lygen!y fhs, T;T) and omit the subseript for brevity. The gradient of L with respect to fa, and o; in this case is (derivation in supplementary)::
aLe
aj::
77
(yyrtt (4x) + (ite ynegge))
(3)
aL
1
(Ilygr.t fa.ll Iyyggg -Ar
(4)
06,
Eq.3 shows that the center of the predicted distribution fu is pushed towards yyt, and pushed away from Yneg.t by this learning objective. In Eq.4, when Vgt.t is closer the center than Yaeg,tt 8e, positive and oe is decreased. Note that Yneg.t is selected out from samples of A (pr, o,J)). this means when M(p, T)J covers context-violated region and this region is farther than ground truth region, (se o){) will shrink to exclude the negative region and become better estimation the true data distribution.
0L
Algorithm 1: Training process
Soft y. At the beginning of training, the model per- forms poorly and may generate bad predictions far away from the ground truth ygt.z- In this case, the standard deviation o will increase according to Eq.4 and make the prediction more uncertain. To alleviate this issue, we turn on Lunlise smoothly after the first few training epochs by making y in Eq.2 as Sigmoid function of the training epoch centered at specified epoch as a warm-up.
Initialize the model parameters 0,
learning rate and coefficient ? while not converge do
X,, ig ~
Predict distribution po (Y, X,)
Draw K trajectories
YiA Po(YY X,)
Select Yineg from Y via checker
Compute original loss Lorig Compute Luntike using Eq.
Experimental Results
L Lorig + Lunlike
In this section, present the experimental results of our method. Our method can be applied to the state of the art models that generate a future trajectory distribution and further improve the predicted distribution quality. We evaluated our method on nuScenes dataset [18] applied to Trajectron++ [I] and Argoverse dataset [19] applied a distribution- based variant of LaneGCN [15].
0
(o
end
Evaluation Metries We use average le displacement error (ADE) and final l2 displacement error (FDE) to evaluate the prediction. We target improving the quality of the predicted distribution. To quantify the distribution quality, we randomly sample 200 trajectories from the predicted distribution and average ADE and FDE over these samples as the metric ADE-Fus ADE-Full and FDE-Full. In addition, we report the context-vion violattin rate of these 200 trajectories as measured by the context checker.
4.1 Experiments on nuScenes Dataset
nuScenes dataset [18] contains 1000 city driving scenes from both left-hand (Singapore) and right- hand (Boston) traffic regions. Each scene 20s long and recorded 2Hz. nuScenes is one of the most prominent open-source motion forecasting datasets with detailed semantic maps.
Test Model Trajectron++ [1] is CVAE-based [20] model. Its input X, contains positions, veloci- ties, headings of the predicted and surrounding vehicles, and map patch. The output distribution (Y: X,) = Σx P61 X2) (Y; X,, 2) a Gaussian mixture model with 25 components and modeled by an encoder net pei (z X,) and decoder net рed pei (Y; x) In addition, it has
5

---

Table 1: nuScenes: Experimental results nuScenes dataset [18] with Trajectron++ [1]. Our pro- posed loss avoids 16% context-violated prediction and improves the prediction accuracy performance of Trajectron++ by more than 8%, which indicates better-predicted distribution. Mean and standard deviation are calculated over 5 runs.
Model
FDE-Full
ADE-Full
Context- Violation-Rate minFDE10
minADE10
Ground Truth
5.44% 10.59% 8,85% + 0.32%
Trajectron++
2.74=0.10 2.51=0.06
104=0005
1.6800005
0.68=0.02 0.61=00022
Trajectron++ with Luntike
0.95=0.03
1,5200.07
Relative Inprovement
+8%
+9%
16%
+10%
+10%
Table 2: nuScenes:
Experimental results on the nuScenes dataset for single prediction. FDE and ADE for all methods here are computed by only one predicted trajectory. For both Trajectron++ and our method, this trajectory is sampled by greedy search. Our method helps to improve the predicted accuracy. Mean and standard deviation calculated over 5 runs.
Model
FDE-1
Const. Velocity [1]
0.32
0.89
1.70
2.73
S-LSTM [21]
0.47
1.61
CSP [22]
0.46
2,35
1,50
CAR-Net [23]
0.38
1.35
SpAGNN [24]
0.36
1,23
Trajectron++ [1]
0.06=0.01
0.43:00.01
1.08:0.04
2.05:1:008
Trajectron++ with Luntike
0.05::0.00
0.42:0001
1.05::0.02
1.99:10.05
another encoder net [= X,, Yi (gt)) used only in training. The original learning objective is shown in supplementary materials. In denotes the mutual information.
Experiments The batch size is set to 1024. Models are trained for 35 epochs and we test the weights from the best epoch measured by average ADE on the validation set. The coefficient y in Alg. I increases gradually from 0 to as a sigmoid function centered at the 24th epoch. The initial learning rate is 3e-3 and it decays exponentially by 0.9995 per iteration. These hyperparameters except are optimized for Trajectron++ and lead to better performance than that in the original paper. To increase the numerical stability of the training. we add small constant le Eq. to avoid the infinite gradient region of log function. In this case, Eq.1 becomes:
Lunlike Ex,, Pen ( [log(po (Yiney X.) +c))]
(5)
Mathematically, term scales the original gradient by PolYy meng (x))
(.. X Proof is shown in supplemen-
tary materials. In addition, rotate the scenes randomly from 15a 345 in the training set for data augmentation following the setting of the original Trajectron++. For each model, we run five experiments and report the mean and standard deviation of the measured metrics. The models trained to predict 3 seconds into the future. To evaluate generalization beyond the training horizon, test on both second and 4-second prediction horizons.
Tab. shows the quantitative results of the 4s prediction measured by the FDE/ADE-Ful metrics and the context-violation rate. Our unlikelihood loss encourages the prediction to respect context information more measured by the reduction of the context violation rate from 10.59 to 8.85%. Besides, method increases the prediction accuracy both FDE and ADE are reduced by about 8%. Results indicate that our method helps improve the quality of the predicted distribution from both comtext-compliant and accuracy perspectives. This is also demonstrated in the qualitative comparison in 1ig.1. The predicted distribution from our methods covers the not-drivable and wrong direction region less. In contrast, original Trajectron++ tends violate the contextual information when the prediction horizon is long. This shows that our method encourages the model to be more sensitive to the road boundary and the lane direction. Comparison with other methods shown in Tab.2. The FDE for the single predicted trajectory is also improved by our method.
4.2 Experiments on Argoverse Dataset
Argoverse dataset [19] contains 300,000 5-second tracked scenarios in 2 American cities Miami and Pittsburgh with semantic maps. The data is recorded at 10 Hz. The first 2 seconds are used as input to predict the next 3 seconds' future.

---

Table 3: Argoverse: Experimental results on Argoverse dataset. Compared to our base model Gaussian LaneGCN, our proposed method reduces the context violation by a large margin and avoid 56% of prediction that violates context. ADE-Full and FDE-Full are reduced by 8% and 6%, respectively. Results indicate better-predicted distribution with our unlikelihood loss L unlike.
Model
ADE-Full
FDE-Full
Context-Vio.
ADE-1
FDE-1
Ground Truth
0.85%
Gaussian LameGCN
1811
3.49
9.0%
1.38
3.03
Gaussian LaneGCN with Luniibbe
1.67
3.29
4.0%
1.38
3,01
Relative Improvement
+8%
+6%
+ 556
0%
+1%
Test Model LaneGCN [15] is one of the state of the art models on Argoverse dataset. It takes the historical trajectories and a graph representation of lanes as input and directly generates deterministic trajectories the prediction of each vehicle. The predicted trajectories are trained by regression using smooth L1 loss. To investigate the effectiveness of our method on distribution- based models, we applied our method to distribution- baasdd variant of LaneGCN that predicts 6 trajectory distributions instead. The predicted position at each timestep is modeled by a simple isotropic Gaussian distribution with predicted variance. This is done by extending the dimension of the final output layer to include a log variance prediction additionally for each timestep. In addition, the model is modified to predict velocity distribution. The trajectory distribution is calculated by the integration over the velocity distribution. The smooth LI regression loss is replaced by the minus log likelihood of the ground truth future (MLE training). The original training objective shown in supplementary materials. This variant is denoted as Gaussian LaneGCN.
Experiments Argoverse does not release the ground truth future trajectories for the test dataset. Therefore, we use the original validation set as our test set in this experiment. For both Gaussian LaneGCN and our method, we set the batch size to 128. The initial learning rate is 1e-3 and it decays to le-4 after 32 epochs. The model is trained for 36 epochs. The coefficient of the unlikelihood loss Alg. increases gradually from to as a sigmoid function centered the 12th epoch. The results are listed in Tab.3. With our unlikelihood training loss, we significantly reduce the context violation rate from 9.0% to 4.0% compared to the original Gaussian LaneGCN. 56% of predictions that violate the context are avoided with our method. In addition, ADE-Full and FDE-Full are also reduced by 8% and 6%, respectively. Results indicate that our method successfully improves the predicted distribution quality by encouraging the model to focus context information more.
4.3 Ablation Study
The Influence of Our Loss Without Map Input Here we remove the map input from the model and demonstrate the influence of our unlikelihood loss in nuScenes in Tab.4. Interestingly, our loss helps improve the performance from 2.89 to 2.67 measured by the FDE-Full although the model does not see the context during inference. We think the reason is that compared to the Trajectron++ without map input, our unlikelihood loss still offers context information to support the training since this loss is calculated using the context. Therefore, our model receives more information during training and performs better. Besides, our method without map input even achieves a comparable result (FDE-Full 2.67) compared to Trajectron++ with map input (FDE-Full 2.74). This experiment shows that our loss can inject context information into learning signals. In addition, using map input only as the context information source improves FDE-Full of Trajectron++ by 0.15 (from 2.89 to 2.74). In contrast, if we use both map input and our unlikelihood loss the performance is improved by 0.38 (from 2.89 to 2.51). Our loss doubles the contribution of context information.
Remove Context- Vivolttd Prediction Directly An intuitive way to avoid vinlaxt-violatd predic- tions is simply removing these predictions as post-processing step. Even though this is straight- forward, the prediction model in this case isn't 't trained to obtain the context violation knowledge. Our method encourages the model to understand the context better and avoid the context violation by itself. Therefore, our model is more 'intelligent' compared to the method directly removing the context-violated prediction and leads to a better prediction. In addition, context violation is sometimes needed. A small part of the ground truth trajectories also violates the context. The model trained by our method can still give us context-violated prediction as our context violation rate is not zero in Tabl. We list the ADE/FDE- Full of Trajectron++, Trajectron++ removing the context- context-violated prediction
7

---

Table 4:
The effect of Lunlike without map imput to the model on nuScenes. Compared to Trajectron++ without map input, our loss performs better, which indicates that Luntike offers context information for the training.
Table 5: The performance when simply remov- ing the context-violated prediction. Although this straightforward method improves the original base- line, method achieves the best scores. This indicates that our unlikelihood loss helps models
understand the context better.
Map Input
Lundike
FDE-Full
ADE-Full
2.89:10.11
1.08:10.03
Method
ADE-Full
FDE-Full
2.67100.8
1.01:0.02
Trajectontt+
1.04 0.05
2.74 = 0.10
2.74:10.10
1.04::0.05
Trajecron+++ with Removing Violation
0.98 0.04
2.55 = 0.10
2.51:10.06
0.95:10.03
Trajectron++ with Luntike
0.95 0.03
Table 6: Ablation study on extended check-horizon of negative candidates nuScenes dataset. When the horizon is properly extended from 3s to 4s, the checker can recognize more negative trajectories and improves the prediction accuracy. Evaluation horizon is 4s. Results are averaged over 2 runs.
Model
Check-Horizon
ADE-Full
FDE-Full
Cotext-Vio.
ADE-1
FDE-1
Trajectron*+
1.04
2.74
10.6%
0.77
2.05
Trajectron+++ with Luntike 3s
0.95
2.51
8.9%
0.74
1.99
Trajectrou++ with Luntike
0,92
2.43
8,5%
0.75
1,98
Trajectron+++ with Luntike
5s
0.96
2.58
9.3%
0.77
2.03
Trajectron++ with Lonttie
68
0.98
2.59
8.9%
0.77
2.03
directly, and our method in Tab.5. Although simply removing the context-violated prediction helps improve the distribution accuracy, our method still performs better.
Hyperparameter In our previous experiments, 7 is simply set to 1 and turned on smoothly during training. We further performed ablations on 7 with values including 0.1, 0.3, 3, 10. We observed performance degradation for smaller and similar performance to for bigger y; see detailed ablations in the Section 'Hyperparameter 7 of the Appendix.
Check-horizon for negative candidates Assume we have a 3-second predicted trajectory that obeys our checker but tends to violate it in the near future (e.g., in 1 second). Such trajectory is able to pass our checker's examination but still unlikely happen in real world. To classify negative trajectory, check the future of this prediction by extending the prediction horizon of the candidate trajectories and checking this extended version. Tab.6 shows ablation studies with extended check-horizon from 3 seconds to 4, 5, 6 seconds to verify whether this mechanism improves our method. Note that the extended horizon only used for selecting out the negative trajectories. Trajectories are truncated back to 3 seconds when computing the unlikelihood loss Lunlike- Our method benefits from an adequately extended 4 seconds improves the FDE Full by about 0.08. Note that when further extend the check-horizon to 5 seconds and seconds, we do not observe better performance. This might be because the future of the prediction itself is too inaccurate when extend the horizon too much and make the judgment of the original prediction unreliable. Numbers are averaged over two training instances.
as extending the negative trajectories to
5 Conclusion
We present an unlikelihood guided trajectory prediction method that leverages context information into learning signals by minimizing the probability of unlikely trajectories. During training, our context checker detects predicted unlikely trajectories and their probabilities are minimized using an unlikelihood loss. Our method can be seamlessly incorporated into state-of-the-art distribution-basee models with a maximum likelihood estimation objective. Our experimental results demonstrate that our method significantly improves the predicted distribution quality of state-of-the-art trajectory prediction models. We hope that our work may encourage future work on exploring better unlikelihood methods for trajectory prediction and improved context checker models.

---

Acknowledgement
This work funded by a KAUST BAS/1/1685-01-0.
References
[1] Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone. Trajectron++: Multi-agent generative trajectory forecasting with heterogeneous data for control. arXiv preprint arXiv:2001.0533 03093, 2020.
[2] C. Tang and R. R. Salakhutdinov. Multiple futures prediction. In Advances in Neural Information Processing Systems, pages 15398-1555 15408, 2019.
[3] N. Rhinehart, R. McAllister, K. Kitani, and S. Levine. Precog: Prediction conditioned on goals in visual multi-agent settings. In The IEEE International Conference Computer Vision (ICCV), October 2019.
[4] S. Casas, C. Gulino, S. Suo, and R. Urtasun. The importance of prior knowledge in precise multimodal prediction. arXiv preprint arXiv:2006.0266 2020.
[5] S. H. Park, G. Lee, M. Bhat, J. Seo, M. Kang. J. Francis, A. R. Jadhav, P. P. Liang, and L..P. Morency. Diverse and admissible trajectory forecasting through multimodal context understanding. European Conference on Computer Vision, 2020.
[6] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston. Neural text generation with unlikelihood training. arXiv preprint arXiv; 04339, 2019.
[7] N. Lee, W. Choi, P. Vernaza, C. B. Choy, P. H. S. Tomr, and M. Chandraker. DESIRE: distant future prediction in dynamic scenes with interacting agents. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 2165-2174, IEEE Computer Society, 2017. 10..1...119//CC111 (0.1100//CPP.20177 org/0.. 1091/00R./2000000 233.
URL https:
[8] H. Cui, Radosavljevic, F. Chou, T. Lin, Nguyen, T. Huang, Schneider, and N. Djuric. Multimodal trajectory predictions for autonomous driving using deep convolutional networks. CoRR, abs/ 111.1/809.11773. 2018. URL hatt:://arxta org/abs/ aaaaaa//.....///////////// 10732.
[9] Y. Chai, B. Sapp. M. Bansal, and D. Anguelov. Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction. arXiv preprint arXiv: 1910.0549,, 2019.
[10] V. Kosaraju, A. Sadeghian, R. Martin-Martin, I. Reid, S. H. Rezatofighi, and S. Savarese. Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks. arXiv preprint arXiv: 1907.03395 2019.
[11] D. Ridel, N. Deo. D. Wolf, and M. Trivedi. Scene compliant trajectory forecast with agent- centric spatio-tempont grids. IEEE Robotics and Automation Letters, 5(2):2816-25) 2823,, 2020.
[12] X. Huang, S. G. McGill, J. A. DeCastro, B. C. Williams, L. Fletcher, J. J. Leonard, and G. Rosman. Diversity-awe vehicle motion prediction via latent semantic sampling. ArXiv, abs/1911.12736, 2019.
[13] N. Rhinehart, K. M. Kitani, and Vernaza. R2p2: A reparameterized pushforward policy for diverse, precise generative path forecasting. In Proceedings of the European Conference on Computer Vision (ECCV), pages 772-788, 2018.
[14] M. Li, S. Roller, I. Kulikov, S. Welleck, Y. Boureau, K. Cho, and J. Weston. Don't say that! making inconsistent dialogue unlikely with unlikelihood training. CoRR, Abs//191..0386. 2019. /ttettt:://xxiii
[15] M. Liang, B. Yang, R. Hu, Y. Chen, R. Liao, S. Feng, and R. Urtasun. Learning lane graph representations for motion forecasting. In European Conference on Computer Vision, pages 541-556. Springer, 2020.

---

[16] S. Shridhar, Y. Ma, T. Stentz, Z. Shen, G. C. Haynes, and N. Traft, Beelines: Motion prediction metries for self- self-driving safety and comfort, 2021.
[17] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint 190iv 1027, 2019.
[18] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv: 1027, 2019.
[19] M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan, al. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8748 8757, 2019.
[20] K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, pages 3483-3491, 2015.
[21] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 961-971, 2016.
[22] N. Deo and M. M. Trivedi. Multi-modal trajectory prediction of surrounding vehicles with maneuver based Istms. In 2018 IEEE Intelligent Vehicles Symposium (IV), pages 1179-1184. IEEE, 2018.
[23] A. Sadeghian, Legros, M. Voisin, R. Vesel, A. Alahi, and S. Savarese. Car-net: Clairvoyant attentive recurrent network. In Proceedings of the European Conference on Computer Vision (ECCV), pages 151-167, 2018
[24] Casas, C. Gulino, R. Liao, and R. Urtasun. Spatially-aware graph neural networks for relational behavior forecasting from sensor data. arXiv preprint arXiv:1910008233, 2019.
10


Reviews:
[{"review_id": "kkQ-tap-f2u", "reviewer": "Reviewer_XGcM", "summary": "One of the problems in trajectory prediction is that the predicted trajectories do not abide by its context, i.e., they can go out of drivable areas. To solve this issue, the authors proposed an additional \"unlikelihood\" loss function which discourages the likelihood term to put lower density near negative trajectories. The authors show the improvement on nuScenes dataset where the loss is added to Trajectron++, and Argoverse dataset where the loss is added to Gaussian LaneGCN. The ablation suggests that this approach performs better than an adhoc post-processing approach that removes violation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. I like the key idea is based on the fact that validating trajectories is easier than generating them. So, we can first classify good and bad trajectories and use those to shape the output distribution. \n2. The idea of using \"unlikelihood\" seems to be novel, and this paper can inspire future avenues in exploiting negative trajectories.\n3. The ablation and the results are convincing.\n\nWeaknesses\n1. The loss function used in the experiment is not clear to me. For example in eq 5, what does p_theta(Y|X,z) looks like? Is it a gaussian with learned mean and variant? \n2. How does metrics reflect the multimodal nature of the trajectories? L2 measurement alone does not capture how accurate we model Y|X when they are multimodal. \n3. Figure 1 is not very clear in demonstrating the issue the paper is trying address. The uncertainty is only slightly off the track. Also, it seems like the high uncertainty is more of a problem in this figure. Also it would be nice to compare the result of this method before and after in in figure 1.\n4. How sensitive is the result to the hyper-parameter? Is the result overfit to the validation data -- as the paper mentioned that the validation data are used for testing?\n\nOthers:\n1. There might be similar approaches in ML+motion planning to avoid constraint violation. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper provides a simple trick that can be added generative models for motion forecasting. The paper provides convincing results that it can improve over the state-of-the-arts. However, I still have slight concern over the hyper-parameter tuning and test settings.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "I'd like to see the authors address the 3 points mentioned above in the weakness section. \nSome word choices: geometrical nearness --> geometrical proximity", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {}}, {"review_id": "fGeyaHhXKR", "reviewer": "Reviewer_nXVp", "summary": "The work introduces the application of 'unlikelihood' loss to augmenting standard training losses for trajectory learning. The work supports the notion of applying (soft) constraint to the training objective by analysing contextual information to inform what should or should not be penalized.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The idea is very simple but seems to be effective\n- It addresses a seemingly obvious but in my mind absolutely necessary encoding of violations in the application domain\n- The paper is well written and the ideas come across clearly", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall I think the paper presents a simple but very practically useful application of the unlikelihood loss for trajectory learning.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "Comments and Questions\n- Connection with MAP: In the abstract the contribution is stated as \"Orthogonally, we propose a new objective, unlikelihood training, which forces predicted trajectories that conflict with contextual information to be assigned a lower probability\". While the term \"unlikelihood\" in this context makes perfect sense (and is adopted from prior work - welleck 2019), in my mind what it *really* seems to be in the Bayesian sense is that it is simply a _hyperprior_ in precisely the same sense as in maximum a posteriori (MAP) estimation.  I see that the hyperprior is, in the unliklihood sense, simply another distribution imbued by the \"contextual information\". Could the authors provide any commentary on a link between the two?\n\n- Unlikelihood data\nThe authors state \"However, these are not given in the dataset. To solve this issue, we approximate the samples by directly drawing a set of trajectories from the predicted distribution and select the trajectories that violate the contextual information out by a context checker.\" This brings to mind the idea of an adversarial game being played between the model and the checker. Although here the checker is assumed to be a kind of oracle. Would there be any possiblity for encoding uncertainty over the checker itself? Furthermore, it would seem the method is highly dependent on sample complexity. I.e. if the complete set of \"invalid\" trajectories is sampled uninformatively, then it would seem that the loss would practically contribute not much (since there are no negative samples to learn from). This problem would be exacerbated (potentialy exponentially in higher dimensions). Have the authors considered how such a problem may manifest in the given problems and also how to alleviate such problems (perhaps some kind of informative sampling regime).\n\n- Figure 3. I find the diagram somewhat visually uninformative. This could be improved in many ways: highlighting areas of major differences as given in the caption. At the very least the diagram could be helped with a legend that identifies the various components of interest to leave the reader wondering less (e.g. are there any signficant differences between the blue, green and red distributions? - this is unclear from the photo)"}}, {"review_id": "AEodx6u3R96", "reviewer": "Reviewer_Fbgh", "summary": "This paper's contrition is to use map information to mitigate false positives in motions forecasting tasks, by minimizing the predictive likelihood of motions that are known to violate known lane directions in a HD map.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n - Motivation\n - Clarity of writing\n - Experiments and multiple real datasets used\n\nWeaknesses:\n - Context checker is not learned, and some of the closest related works were not cited. While the learned part of this work is the original predictive model, and the context checker provides a superior loss function, as the authors demonstrate, the rules are handcrafted (e.g. binary 90 degree rule on lines 156-157), which makes this difficult to immediately extend to anything else, like cyclists, pedestrians, etc. While the point of this paper is not necessarily this, but to open research up to unlikelihood training in motion forecasting, other methods have used learning based approaches to mitigate false positives for autonomous vehicle motion in ways that are less hand-crafted and more widely applicable (see R2P2 algorithm, cited below [A]). In addition, the Beelines paper (cited below [B]) is very similar by using map information to alter the loss function for motion forecasting, and should be discussed in the context of this work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend weak-accept in light of the \"strengths and weaknesses\" discussion above. However, willing to consider strong-accept based on author responses. This is an important area that the community should be looking into more, and comparing against related works would help strengthen this paper.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "Line 42 \"we are the first to propose unlikelihood training for continuous space of trajectories\": See R2P2 [A], e.g. fig 6, noting H(q,p) term to penalize bad samples. Might be worth referencing this work and discussing the differences.\n\nLines 44-46: \"inject contextual information as a learning signal\" seemed a little vague - could this be clarified?\n\nGrammar line 64: \"while Tang and Salakhutdinov [2], Salzmann et al. [1] uses\"\n\nLine 63 and 66: small preference against using reference numbers as nouns to begin a sentence.\n\nLine 78-79, could probability delete \"in contrast to continuous predictions that represent the future trajectories in our setting\" since next sentence says this too.\n\nLine 92: perhaps clarify different between x and X? X is random and x is not?\n\nLine 101: should \"Y_{i}\" be \"y_{i,gt}\" (line 92) or \"Y_{i,gt}\" (line 113) or \"\\bold{Y}_{i,gt}\" (Fig1 caption) or \"\\bold{y}_{i,gt}\" (line 171). Should some of these mean the same thing and should be following consistent notation? \n\nLine 102-103: change \"between the predicted distribution and ground truth distribution\" --> \"between the ground truth distribution and the predicted distribution\"\n\nLeft quotations on lines 105, 137, 138, 283\n\nI think the connection to REINFORECE in lines 138-142 is tenuous, consider removing.\n\nLine 151: \"map-based checker\": this is related to the Beelines [B] paper, worth citing and comparing to in the related work, since Beelines also contributes map information to alter the typical ML loss for forecasting in a way that avoids the more obvious-to-humans out-of-bounds errors.\n\nLine 157-158: \"The trajectories that fail to pass the test are identified as negative trajectories Y i,neg\": I wonder if there's a future work here based on a continuum of being more and more negative, the more the vehicle heading differs from the lane's heading, in addition to binary pass/fail tests.\n\nEquation 4: it's interesting the log_sigma cancelled out on line 174 and now doesn't feature in the sigma gradient term, effectively turning this into an L2 learning task with an initially-set hyperparameter of sigma. I.e. the two Gaussians have in effect lost their \"complexity terms\" that regularize sigma. I assume one can expect non-equal number of positives and negatives sample in general, so this affect would disappear, unless the positive and negative batches are *averaged* and not *summed*? Possibly worth commenting on this somewhere lines 177-186, if space limitation allow for (not big priority).\n\nLines 187-194: interesting observation! Nice explanation.\n\nTable 1: what is the \"Context-Violation-Rate\" of the ground truth trajectories for reference? (presumably something non-zero, but close to zero? Unless the \"disable rule\" in lines 160-161 ensure it'd be zero exactly?)\n\nLines 235-237: nice explanation.\n\nLine 284: \"and leads to a better prediction\" - interesting! Can the standard error be added into Table 5 that shows this, analogous to Table 4 etc?\n\nI like the combination of likely and unlikely trajectories in equation 2, especially since even if there's an error in the context checker, significant likelihood of an other-wise valid motion can override such errors. But this really depends on how is gamma set: how should gamma be set? Edit I see on lines 259 it's scheduled 0 to 1, is there anything special about the number 1 in this context, or could it be anything? (do positive and negative sample matter equally for the purpose of accurate prediction? Could you have got better scores if this was schedule up to a maximum of 0.1 or 10?). Additional experiments not necessarily required, more after an explanation and clarification in the text.\n\n[A]\nR2P2: A reparameterized pushforward policy for diverse, precise generative path forecasting\nby Nicholas Rhinehart, Kris Kitani, and Paul Vernaza\nhttps://openaccess.thecvf.com/content_ECCV_2018/papers/Nicholas_Rhinehart_R2P2_A_ReparameteRized_ECCV_2018_paper.pdf\n\n[B]\nBeelines: Motion Prediction Metrics for Self-Driving Safety and Comfort\nby Skanda Shridhar, Yuhang Ma, Tara Stentz, Zhengdi Shen, Galen Clark Haynes, Neil Traft\nat ICRA 2021, https://arxiv.org/abs/2011.00393"}}]

Author rebuttal:

Meta-review: Summary: 
This paper presents a method to improve the quality of agent predictions by incorporating a contextual loss term that motivates unrealistic predictions (e.g. violate lane direction) to be assigned lower likelihoods. The method is validated on two benchmark scene sets. 

Clarity: All reviewers commend the paper for its clear presentation. The paper flowed well and presented a well-motivated, well-defined piece of technical work. 

Significance: Reviewers strongly agree that this paper pushes an area of work that could be extremely impactful to the community. 

Quality: The experimental results are overall convincing. In fact, reviewers nXVp and XGcM both note that the proposed method seems simple, however, provides convincing improvements to algorithm performance. This gives the method a high likelihood of being adopted by the research community. 

Originality: There was mixed feedback about the novelty of the proposed approach, and the paper could improve in articulating its contribution. (See “Opportunities for improvement” for more details).

Pros: 
This paper presents a method that is philosophically compelling in principle and whose utility is backed up by comprehensive simulation experiments. The paper is also very well-written; readers are able to grasp the main ideas right away and see those main ideas come through in the experimental section. 

Main opportunities for improvement: 
There are two main areas that can be improved to further solidify this paper. 

The first is orienting the proposed work better in current literature. Reviewers mention a number of related papers and methods; this paper should explicitly describes similarities and differences between itself and these existing techniques. Reviewers Fbgh and nXVo provide specific examples of missing related works.  

The second is to refine the experimental results section. The revision should expand on 1. implementation details the should be clarified and 2. further analyses and explanations of the existing results. All reviewers, and in particular reviewer Fbgh, make specific suggestions for what needs to be addressed.

Thank you for considering our feedback, and we look forward to seeing the updated paper. 

================ Final Decision

All reviewers recommended accept. I decided to recommend oral presentation because after revisions, reviewer Fbgh updated their score to strong accept as well, the paper has strong scores for impact (4s across the board), extremely strong scores for technical quality and clarity across the board, and positive originality scores. In general, the paper seems to be very well received. 
Decision: Accept (Oral)