Motion Forecasting with Unlikelihood Training
Authors: Deyao Zhu
Mohamed Zahran
Li Erran Li
Mohamed Elhoseiny
Venue:  (2021)

Abstract: Motion forecasting is essential for making safe and intelligent decisions in robotic applications such as autonomous driving. State-of-the-art methods formulate it as a sequence-to-sequence prediction problem, which is solved in an encoder-decoder framework with a maximum likelihood estimation objective. In this paper, we show that the likelihood objective itself results in a model assigning too much probability to trajectories that are unlikely given the contextual information such as maps and states of surrounding agents. This is despite the fact that many state-of-the-art models do take contextual information as part of their input. We propose a new objective, unlikelihood training, which forces generated trajectories that conflicts with contextual information to be assigned a lower probability by our model. We demonstrate that our method can significantly improve state-of-art models’ performance on challenging real-world trajectory forecasting datasets (nuScenes and Argoverse) by 8% and reduce the standard deviation by up to 50%. The code will be made available.

Under review as a conference paper at ICLR 2021
MOTION FORECASTING WITH UNLIKELIHOOD TRAIN- ING
Anonymous authors
Paper under double-bling review
ABSTRACT
Motion forecasting is essential for making safe and intelligent decisions in robotic applications such as autonomous driving. State-of-thh-aan methods formulate it as a suuience-to-seqenne prediction problem, which is solved in an encoder- decoder framework with maximum likelihood estimation objective. In this pa- per, we show that the likelihood objective itself results in model assigning too much probability to trajectories that are unlikely given the contextual information such as maps and states of surrounding agents. This is despite the fact that many state-of-the-art models do take contextual information as part their input. We propose a new objective, unlikelihood training, which forces generated trajecto- ries that conflict with contextual information to be assigned lower probability by our model. We demonstrate that our method can improve state-of-art models' performance on challenging real-world trajectory forecasting datasets (nuScenes and Argoverse) by 8% and reduce the standard deviation by up 50%. Code will be made available.
INTRODUCTION
For robotic applications deployed in the real world, the ability to foresee the future motions of agents in the surrounding environment plays essential role for safe and intelligent decision making. This is a very challenging task. For example. in the autonomous driving domain, to predict nearby agents' future trajectories, an agent needs to consider contextual information such as their past trajectories, potential interactions, and maps. State of the art prediction models (Salzmann et al. 2020 Tang & Salakhutdinov 2019, Rhinehart et al. 2019) directly take contextual information as part of their input and use techniques such as graph neural networks to extract high-level features for prediction. They are typically trained with a maximum likelihood estimation (MLE) objective that maximizes the likelihood of ground truth trajectories in the predicted distribution. Although MLE loss encour- ages the prediction to be close to the ground truth geometrically. does not focus on learning a good distribution that is plausible with respect the contextual information. These models predict trajectories that violate the contextual information (e.g., go to opposite driving direction or out of the driving area) but still closes ground truth. In contrast, humans can easily notice that these trajectories are unlikely in specific context. This phenomenon suggests that simply applying MLE loss cannot fully exploit contextual information.
To address the problem, we propose a novel and simple method, unlikelihood training, that injects contextual information into the learning signal. Our loss penalizes the trajectories that violate the contextual information, called negative trajectories, by minimizing their likelihood in the predicted distribution. To generate negative trajectories, we first draw number of candidate trajectories from our model's predicted distribution. Then, context checker is used to cut out the trajectories that violate contextual information as negative trajectories. This context checker does not need to be differentiable. By minimizing the likelihood of negative trajectories, the model is forced to the contextual information to avoid predictions that violate context. Therefore, the prediction quality is improved.
Existing methods (Casas et al. 2020 Park et al., 2020) using contextual information as learning signals either introduce learning parameters using high-variance learning methods such as the REINFORCE algorithm (Casas et al. 2020). In contrast, our method injects rich contextual information into the training objective and keeps the training process simple.

---

Under review as a conference paper at ICLR 2021
Unlikelihood training (Welleck et al. 2019) has been applied to neural text generation. We the first to propose unlikelihood training for continuous space of trajectories. For the discrete space of token sequences, repeating tokens n-grams in the generated sequence are chosen negative tokens. In contrast, we design a context checker to select negative trajectories sampled from the continuous distribution of model predictions.
Our method can be viewed as a simple add-on to any models that estimate the distribution of future trajectories. It improves their performance by encouraging models to focus more on contextual information without increasing the complexity of its original training process.
Our contributions are summarized follows:
We propose novel and simple method, unlikelihood training for motion forecasting in autonomous driving that encourages models use contextual information by minimizing the likelihood of trajectories that violate contextual information. Our method can be casily incorporated into state-of-the--rt models.
Our experimental results on challenging real-world trajectory forecasting datasets, nuScenes and Argoverse. shows that unlikelihood training can improve prediction perfor- mance by 8% and reduce the standard deviation by up to 50%.
RELATED WORK
In this section, we briefly review the two most related topics.
Trajectory Forecasting Trajectory forecasting of dynamic agents, a core problem for robotic ap- plications such autonomous driving and social robots, has been well studied in the literature. State-of-thee-rt models solves it as a
multi-modal prediction problem (Lee
let al. [2017] Cui et al., 2018 Chai et al. 2019 Rhinehart al.] (209 Kosaraju al.
2019 Tang
& Salakhutdinov 2019: Ridel et al.] 2020 Salzmann et al. [2020] Huang et al. 2019). (Cui et al.
2018 Chai al. 2019 Ridel et al.] (2020) predicts multiple future trajectories without learning
low dimensional latent agent behaviors. (Lee et al. 2017] Kosaraju et al., 2019; Rhinchart et al,
2019 Huang et al. 2019) encodes agent behaviors in continuous low dimensional latent space while
Tang & Salakhutdinov 2019: Salzmann et al.) 2020) uses discrete latent variables. Discrete latent variables succinctly capture semantically meaningful modes such as turn left, turn right. (Tang & Salakhutdinov 2019) (allzaann et al. (020)) learns discrete latent variables without explicit la- bels. All them use maximum likelihood estimation (MLE) objective or its approximations (e.g., VAE). In this paper, we show that MLE loss can ignore contextual information such maps and states of surrounding agents. As a result, models with such a loss can assign too much probability to unlikely trajectories. We propose unlikelihood training objective to avoid such cases. All models with the maximum likelihood estimation objective can potentially benefit from our methods.
Contrastive learning and unlikelihood training To date, several studies have investigated the possibilities to benefit from negative data. One of the popular direction is contrastive learning. Con- trastive learning has achieved significant success in many fields (Oord et al. 20188] Kipf et al. 2019
Ma & Collins, 2018 Abid & Zou| 2019 Welleck et al. 2019). NCE (Ma Collins 2018 CPC
(Oord et al.) 2018) maximizes the mutual information between data and latent representation by novel contrastive loss to extract useful representation from data. C-SWMs (Kipf al. 2019) utilizes
contrastive learning to learn better world model for reinforcement learning tasks. Recently, unlike- lihood training (Welleck et al. 2019) proposes a new method to utilize negative data. addition, to maximize the likelihood of the ground truth token, it minimizes the likelihood of negative tokens for better text generation. Their method is on the discrete space of token sequences. Repeating tokens n-grams in the generated sequence is chosen negative tokens. In contrast, proposed method works in the continuous space of trajectories. We design a novel method, context checker, to select negative trajectories.
2

---

Under review as a conference paper at ICLR 2021
3 METHOD
3.1 PROBLEM FORMULATION
We targeting at better predicting the future trajectory Y, of vehicle given input X, X, can include any related information like rasterized maps or past positions of vehicle and surrounding agents, depends on the design of the method. Here we skip the detailed choice of imput and denote it as X; for conciseness. Due to different driving strategies, driving intents, and the complex traffic environment, there usually multiple possible future trajectories given input I; (although there is only one ground truth future trajectory Vt.gt in dataset recorded in the real world). To handle this situation, most state of the methods (Salzmann al. 2020) model distribution of possible future trajectories po Y; X;) to cover all the possibilities given the input X; instead of predicting one trajectory. 0 denotes the learning parameters of the model. To train such methods, most state- of-the-art models usually use maximum likelihood estimation (MLE) maximize the likelihood of ground truth trajectory ot in the predicted distribution. For example, the loss of CVAE-based model Trajectron++ (Salzmann et al. 2020) is Eq T This loss is used to maximize the lower bound of ground truth's 's likelihood when the coefficient
Luijtt - -E (eggoss== (,, (log poz Ygt [,,
+ &D (gos(z X, Vpgg X;)) Ig(X;;;
(1)
2 - loggp((Y.ng
X,, gt (x( 2), when
Limitation of MLE on Motion Forecasting MLE encourages the model to predict a dis- tribution that allocates reasonable probability mass to the region where Y; is located by min- imizing the KL-divergence of predicted distri- bution and ground truth distribution. Because the domain of trajectory distribution is the geometric locations, MLE makes these two dis- tributions "close" each other geometrically. However, we argue that maintaining the geo- metrical nearness only is not good enough for motion forecasting task in autonomous driv- ing. In complex traffic scenarios, there can be many potential trajectories close enough to the ground truth geometrically but are very unlikely to happen. For example, if the ground truth tra- jectory Yi,tt is on the outermost lane, a trajec- tory that is close to gi but outside the driv- able region is unlike to happen in the real world. However, MIE loss will not impose a signif- icant enough penalty on such a case to avoid such prediction. [ demonstrates a pre- diction example from Trajectron++ (Salzmann al. 2020) where part of the distribution is outside of the derivable region or on the lane with the wrong direction. The MLE-based loss
Figure I: Examples of predicted distribution from Trajectron++ (Salzmann et al. 2020). White points denote the ground truth trajectory Yr.gr and the color region indicates the predicted distribu- tion. Some of the prediction go outside of the drivable region or go to the lane in opposite di- rection.
only offers learning signals that contain the geometric location information of the ground truth tra- jectories. All the other contextual information, like the drivable region and the lane direction, are missing in the learning signals. While the imputs model cannot use it to avoid the prediction that is geometrically close to ground truth but violates context. In contrast, this quite simple task for humans.
model contain rich contextual information, the
3.2 UNLIKELIHHOD LOSS
To mitigate this problem, we design a new loss term that encourages the model to consider the contextual information. Inspired by contrastive learning and unlikelihood training, additionally
3

---

Under review as a conference paper at ICLR 2021
train model to minimize the likelihood of trajectories that violate the contextual information given input X₁. We denote them as negative trajectories Yi.egg Let's first assume that we already have a distribution of negative trajectories Pneg (Y: X2). One intuitive way is to directly minimize the log likelihood of Yeg in our predicted distribution, similar to MLE but in an opposite manner
Lunlike Ex.,,D Yi,ne P (Y. ((xx.) Po Po
PeOPPo Yi,ee
x
However, the gradient of log function tends to infinity when the input tends to O, which leads to unstable training since the model are optimized to minimize pe( poneg X;) and po i,neg
X.) 2 O. To avoid the infinity gradient region of log function, we add a small constant e to the
likelihood. The final loss term we propose
Lunlike Ex,.DD,Y Pay (X|x)]log(p (Yineg X:)) + ++)]
(3)
We call it unlikelihood loss. We use a coefficient y to balance Luntike. The final training objective in case we combine our method with Trajectron++ is
L == Lunjt Tunnikk
Eq is also easily adapted to combine with any other models that predict trajectory distribution as output. With the help of Lunlike- we inject the contextual information into the learning signal, force the model to better extract and use contextual information in X;, and generate more reasonable predicted distribution to avoid high Lunlike-
3.3 NEGATIVE TRAJECTORIES
Our proposed loss term is highly dependent on the negative samples Y.neg from the distribution Pneg( (Y, I.). However, these not given in the dataset. To solve this issue, approximate the samples by directly drawing a set of trajectories from the predicted distribution and select the trajectories that violate the contextual information out by context checker. Note that this checker does not need to be differentiable and it can be as complex and advanced as necessary. The type of unlike predictions the model Iearns to avoid by our method depends on the type of unlike trajectories the checker can detect.
Design of Our Checker We implement a map-based checker to judge whether a given trajectory suits the context not. In detail, the checker examines whether the trajectory goes into the lane in the opposite direction or out of the road. We create a map that stores the lane direction at every location of lanes and the drivable region. Two examples are shown in Fig We first check whether all the locations of a given trajectory are in the drivable region. If so, we further calculate angles be- tween velocity and the lane direction each time step see whether they are all inside 90 degree. The velocities are approximated by differentiating the trajectory. The trajectories that fail to pass the exam are judged as negative trajectories Y.ne. Note that the lane direction information originally comes from the dataset and is usually incomplete or invalid like in the intersection. In this case, we only use the drivable region information. In addition, there are also small part of ground truth trajectories in the dataset that violates the lane direction or drivable region. In this case, our checker skips the checking and we train it without Lunlike to allow similar prediction. Note that this checker is not perfect due to the incomplete information and simple checking mechanism. A trajectory that passes the exam of the checker doesn't 't mean it is 100% compatible with the context. However, method can still work properly, because our method only depends the negative trajectories that do not pass the exam and has nothing to do with the passing ones. Our checker design offers reasonable negative trajectories support approach. But of course, a more advanced checker helps the model to avoid more complex unlikely prediction. We leave it open for future research.
3.4 ALGORITHM
The finial algorithm is shown in Alg I At each iteration, we first run the forward pass of the model to get the output distribution po Y, X,) given the input X,. Then, K negative candidate
4

---

Under review as a conference paper at ICLR 2021
20
(a) negative case
(b) positive case
Figure 2: Examples of maps used in the checker in nuScences (Caesar et al., 2019a). Green and blue region together denote the drivable region and the blue means that we have lane direction information here. Random locations are sampled and their lane directions are plotted red arrows to show the concrete directions. Blue line denotes the trajectories to check and the velocity directions are represented as yellow arrows. (a) shows a negative trajectory that goes out the road. (b) is passing case.
trajectories are drawn from this distribution and select the negative trajectories out Y Yneg via our checker. After that, ground truth trajectory and negative trajectories are used to calculate the loss function, update the model, and go to next iteration. Note that if there are Yiney in the K negative candidates judged by our checker for data we don't 't apply Tuuiike on i.
3.5 GRADIENT ANALYSIS
Let's assume single-mode prediction case that future position yiggt, at time step of agent i is modeled by simple Gaussian distribution ((ys, Mr I). and og, calcu- lated by the model. With a single negative position Vinee,t we define simple loss for step L, llog.NN(p PogN(Wytt.t ftc o,I) + gradient of La with respect to far and o; in this case is (Derivation in App
fe, o,I) and omit the subscript for brevity. The
aL.
(Wyttt (ax) + (fee- Bnegg)))
(5)
0p::
07
0L 1 (llUge, fuell 11 12)
(6)
06:
09
Eq shows that the center of the predicted distribution Aa is pushed towards Bou, and pushed away from Uneg.t by this learning objective. In Eqlo when ygt, is closer to the center than Uneg.ts de, is positive and o, is decreased. Note that Yneg.t is selected out from samples of M o, f)), this means when ((pg T))) covers context-violated region and this region is farther than ground truth
0L
region, M(r, T)) will shrink to exclude the negative region and become better estimation to the true data distribution.
When the prediction is not accurate (e.g- at the beginning of training), our ground truth Ugt. may be farther than the negative location Yneg.t- In this case, ((t,, will expand to better cover the ground truth and make the prediction more uncertain. A simple approach to alleviate this issue turning off our unlikelihood loss Luntike the first few training epochs. We implement this by making 7 in Eq 4 as a sigmoid function centered at specified epoch. By this way, smoothly turn on Lunlike during training.
EXPERIMENTAL RESULTS
5

---

Under review as a conference paper at ICLR 2021
In this section, present the experimental re- sults of our method to demonstrate our perfor- mance. Our method is easily to applied state of the art models that generate future trajectory distribution and further improve their performance. In our experiments, we se- lect Trajectron++ (Salzmann et al., 2020), one of the state-of-ther the- art methods on NuScenes dataset (Caesar et al. 2019b) with open-source implementation, as our base model. We ex- tend the implementation to work on Argov- erse dataset (Chang et al., 2019). We evaluate our approach on these two motion forecasting datasets.
Algorithm 1: Training process (Use Trajec-
tron++ as base model)
Initialize the model parameters 0;
Initialize learning rate a and coeifficient while not converge do
X,, Yi ~ B
forward pass to compute po (Y, X.)) draw K trajectotries Yi,: ~ po Y; X;)
select Yineg via checker
Compute Lirae+ using Eq Compute Luntike using Eq
L = Luyjt + LLmnlike θ = 0 (Vo(L)
end
Test Model Trajectron++ (Salzmann et al.
2020) is CVAE-based (Sohn et al.) 2015) model. Its input X; contains positions, velocities, heading the predicted and surrounding vehicles, and a map patch. The output distribution pa (Y: X;)) = Σ> Pai (z Xi) (Y; x) is a Gaussian mixture model with 25 compo- nents and modeled by an encoder net poi X;) and a decoder net pri (Y; x) ). In addition, it has another encoder (øs(= X,, Yr.gt) used only in training. The original learning objective is shown in Eq 1g denotes the mutual information.
Evaluation Metries we use average l2 displacement error (ADE) and final l2 displacement error (FDE) to evaluate the prediction performance. Each of them contains some sub-versions. FDE-1 the FDE calculated using only 1 predicted trajectory. both original Trajectron++ and our ap- proach, this single trajectory is drawn from predicted distribution by greedy search step by step. ADE-Ful//DE-Full represents the quality of the whole output distribution. To compute ADE- Full/FDE-Full, we randomly sample 200 trajectories and calculate the average performance as the reported scores. In addition, we use our context checker to measure the context-violation rate in these 200 trajectories as metric to show the context-related performance.
4.1 NUSCENES DATASET
nuScenes dataset (Caesar et al. 2019b) contains 1000 city driving scenes from both left-hand (Sin- gapore) and right-hand (Boston) traffic regions. Each scene is 20s long and recorded 2Hz. is one of the biggest open-source motion forecasting datasets with detailed semantic maps.
Experiments The batch size is set to 1024. Models are trained for 35 epochs and we test the weights from the best epoch measured by average ADE on validation set. The coefficient y in Alg increases gradually from to as a sigmoid function centered at 24th epoch. Initial learning rate is 3e-3 and it decays exponentially by 0.9995 per iteration. These hyperparameters except y optimized for Trajectron++ and lead to better performance than that in original paper. In addition, we rotate the scenes randomly from 15t 345" in the training set for data augmentation following the setting of original Trajectron++, For each model, we run 5 experiments and report the mean and standard deviation of the measured metrics. The models are trained to predict seconds into the future. To evaluate on generalization beyond the training horizon, we test on both 3 second and 4 second prediction horizons.
Tab shows the quantitative results 3s and 4s prediction measured by the FDE/ADE-Full metrics and the context-violation rate. Our unlikelihood loss improves Trajectron++ by about 8% for both metrics in both 3 and 4 second prediction horizons. Results indicate that our method helps improve the accuracy of the predicted distribution. This is also demonstrated in the qualitative comparison in Fig 3 The predicted distribution from our methods covers the not-drivable region less compared to original Trajectron++ without our proposed loss. In contrast, original Trajectron++ tends to violate the contextual information when prediction horizon is long. This shows that method encourages the model to be more sensitive to the road boundary and the lane direction. addition, we observe reduction of the performance variance measured by standard deviation in Tat indicates that our

---

Under review as a conference paper at ICLR 2021
Experiment results on nuScenes dataset (Caesar et al., 2019b) with Trajectron++ (Salzmann et al., 2020). FDE and ADE are averaged over 200 trajectories drawn from the predicted distribution. Our proposed loss improves the performance of Trajectron++ by about 8%, and avoid 16% context-violated prediction compared to Trajectron++, which indicates a better predicted dis- tribution. Mean and standard deviation are calculated over 5 runs.
Table nuScenes:
Model
FDE-Full
ADE-Full
Context- Violation-Ratee 4s
3s
4s
3s
4s
Trajectron++
1.46:10.07
2.7410.10
0.59
1.04.0.05
7.29%:10.22%%
10.59% 8.85 + 0.32%
Ours 1.34+00.0
2.51+0.06
0.54+0002
0.95+
6.57% +..55%
Experimental results on the nuScenes dataset for single prediction. FDE and ADE are computed by only one predicted trajectory. For both Trajectron++ our method, this trajectory is sampled by greedy search. Our method helps to improve the predicted accuracy. Mean and standard deviation are calculated over 5 runs.
Table 2: nuScenes:
Model
FDE-1
3s
4s
Const. Velocity (Salzmann al. 2020
0.32
0.89
1.70
2.73
S-LSTM Alahi
2016
0.47
1.61
CSP DDo & Inved]
0.46
2.35
1.50
CAR-Net Sadeghuan et al.
2018
0.38
1.35
SpAGNN C Casas ci al. 2019
0.36
1.23
Trajectron=+ (Salzmann
2020
0.07
0.45
1.14
2.20
Trajectron++ (our hyperparameters)
0.06110.01
0.43:10.01
1.08100004
2.05-10.00
Ours
0.05++0.00
0.42++0.01
1.05++0.02
1.99:+0.00
method helps to stable the training process. Comparison with other methods is shown in Tab The FDE for single predicted trajectory also improved by our method.
4.2 ARGOVERSE DATASET
Argoverse dataset Chang et al. 2019) contains 300,000 5-second tracked scenarios in 2 American cities Miami and Pittsburgh. The data is recorded in 10 Hz. The first 2 seconds are used input to predict the next seconds future. It is also one of the biggest open-source motion forecasting datasets that offer semantic maps.
Experiments We downsample the data from 10Hz to 2Hz make the setting similar to nuScenes following the setting of (Park et al. (020). Argoverse does not release the ground truth future trajectories for the test dataset. Therefore, we use the original validation set as our test set this experiments and randomly split the original training set into our training set with 95% data and our validation set with 5% data. Batch size 256. Initial learning rate is 3e-3 and it decays exponentially by 0.9995 per iteration. The model is trained for maximal 60 epochs and we select the weights from the best epoch measured by average ADE in validation set. The coeffient y in Alg [ increases gradually from 0 to Argoverse. To make the experiments runnable, we made some small modifications that are explained in Appx D We execute training instances for both original Trajectron++ and our method, report the average performance and the standard deviation. The results are listed in Tab Compared to Trajectron++ without our method, our model improves the accuracy of the prediction by about 8% with our simple loss and reduces the performance variance by about 50% measured by standard deviation.
sigmoid function centered at 1 88th epoch. Trajectron++ is not designed for
4.3 ABLATION STUDY
Prediction horizon for negative candidates Assume we have predicted trajectory with a length six, and it is the drivable region and obeys the lane direction. However, the trajectory tends to

---

Under review as a conference paper at ICLR 2021
(a) Ours
(b) Trajectron++
Figure 3: Qualitative results of method and Trajectron++ in complex scenario. Some of the predicted distribution of Trajectron++ are out of the road or cover the lane with wrong direction. Our method helps alleviate this issue. Predicted distribution plotted colored region and white points denotes the ground truth trajectories. More results are in Appx
Table 3: Argoverse: Experimental results on Argoverse dataset. Compared to our base model Trajectron++, our proposed method helps increase the accuracy and stable the performance.
Model
ADE-1
FDE-1
ADB-Full
FDE-Full
Trajectron++
1.15
2.73=0.24
1.43:0.15
3,40
Ours 1.06:10.08
2.58
1.34
3.25
Table 4: nuScenes:
Ablation study on prediction horizon of negative candidates on nuScenes
dataset.
FDE Full
FDE
Violations
Model
PH.
1s
2s
35
1s
2s
36
4s
1s
2s
3s
46
Trajectront+
0.107
0.560
9.169'2
9.710%
13.048%
21.369%
Ours
3s
0.105
0.538
1.320
2.494
0.054
0.414
1.029
1.960
9.188%
9.619%
11.818%
17.460%
11.872%
17.551%
Ours
5s
0.087
0.497
1.259
2.406
0.056
0,397
1.002
1,889
9.174%
9.667%
12.106%
17.992%
0.099
0.521
1.297
2,474
0.059
0.405
1.004
9.170%
9.636%
12.046%
17.761%
hit the road boundary in the near future (e.g., I second). Such a trajectory can pass our checker's exam, but it is still unlikely to happen in the real world. We can easily select out such trajectory by extending the prediction horizon the candidate trajectories and examining our checker's extended version. To verify whether this helps us build better checker, we extend the prediction horizon for negative candidates from seconds to 4, 5, 6 seconds, respectively, and examine them by our original checker. The selected negative trajectories are truncated back to 3 seconds for computing Luntike- We can see in table 4 that the model benefits from an adequately extended prediction horizon. The prediction horizon for ground truth trajectory is 3s. By extending the negative trajectories 1 second more, we improve the prediction accuracy. Numbers are averaged over two training instances.
CONCLUSION
We present unlikelihood guided trajectory prediction method, that minimizes the probability of un- likely trajectories. During training, our context checker detects predicted unlikely trajectories and their probabilities reduced through an unlikelihood loss. Our method can be incorporated into state-of-the -het--rrt models with a maximum likelihood estimation objective. Our experimental results demonstrate that method significantly improves state-of-the-art trajectory prediction models.
8

---

Under review as a conference paper at ICLR 2021
We hope that work may encourage future work exploring better unlikelihood methods for trajectory prediction and improved context checker models.
REFERENCES
Abubakar Abid and James Zou. Contrastive variational autoencoder enhances salient features. arXiv preprint wtt.i-9022.0611,, 2019.
Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pallern recognition, pp. 961-971, 2016.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv: 1903. 1027, 2019a.
Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for 2019b.
autonomous driving. arXiv preprint arXiv:
Sergio Casas, Cole Gulino, Renjie Liao, and Raquel Urtasun. Spatially-aware graph neural networks for relational behavior forecasting from sensor data, arXiv preprint arXiv: 19X00009/000233: 2019.
Sergio Casas, Cole Gulino, Simon Suo, and Raquel Urtasun. The importance of prior knowledge in precise multimodal prediction. arXiv preprint arXiv:2006. 02636, 2020.
Yuning Chai, Benjamin Sapp. Mayank Bansal, and Dragomir Anguelov. Multipath: Multiple prob- abilistic anchor trajectory hypotheses for behavior prediction. arXiv preprint arXiv:1910.05449, 2019.
Ming-Fang Chang. John Lambert, Patsorn Sangkloy, Jagject Singh, Slawomir Bak, Andrew Hart- nett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, et al. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 8748-8757 2019.
Henggang Cui, Vladan Radosavljevic, Fang-Chiel -high Chou, Tsung-Han Lin, Thi Nguyen, Tzu- Kuo Huang, Jeff Schneider, and Nemanja Djuric. Multimodal trajectory predictions for tonomous driving using deep convolutional networks. CoRR, abs/1809.10732. 2018. URL httt:://axiv..
abs /1009.000...0 10732
Nachiket Deo and Mohan M Trivedi. Multi-modal trajectory prediction of surrounding vehicles with maneuver based Istms. In 2018 IEEE Intelligent Vehicles Symposium (/V), pp. 1799-1144.. IEEE, 2018.
Xin Huang, Stephen G. McGill, Jonathan A. DeCastro, Brian Charles Williams, Luke Fletcher, John J. Leonard, and Guy Rosman. Diversity-aware vehicle motion prediction via latent semantic sampling. ArXiv, abs/191 1.1263, 2019.
Thomas Kipf, Elise van der Pol, and Max Welling. Contrastive learning of structured world models. arXiv preprint arXiv: /9t1::/991122222 2019.
Vineet Kosaraju, Amir Sadeghian, Roberto Martin-Martin, Ian Reid, Hamid Rezatofighi, and Silvio Savarese. Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks. arXiv preprint arXiv: 2019.
Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher Choy, Philip H. S. Torr, and Manmohan Chandraker. DESIRE: distant future prediction in dynamic scenes with interacting agents. In 2017 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26. 2017, pp. 2165-21744 IEEE Computer Society, 2017. doi: 10. N..1109/CPR..201..000 URL htt /oo.
Zhuang Ma and Michael Collins. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. arXiv preprint 180.00/800001//2 2018.

---

Under review as a conference paper at ICLR 2021
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.0374 2018.
Seong Hyeon Park, Gyubok Lee, Manoj Bhat, Jimin Seo, Minseok Kang, Jonathan Francis, Ash- win R Jadhav, Paul Pu Liang, and Louis-Philippe Moreney. Diverse and admissible trajectory forecasting through multimodal context understanding. European Conference on Computer Vi- sion, 2020.
Nicholas Rhinehart, Rowan McAllister, Kris Kitani, and Sergey Levine. Precog: Prediction condi- tioned on goals in visual multi-agent settings. In The IEEE International Conference Computer Vision (ICCV), October 2019.
Daniela Ridel, Nachiket Deo, Denis Wolf, and Mohan Trivedi. Scene compliant trajectory forecast with agent-centric spatio-temporal grids. IEEE Robotics and Automation Letters, 5(2):2816 2823, 2020.
Amir Sadeghian, Ferdinand Legros, Maxime Voisin, Ricky Vesel, Alexandre Alahi, and Silvio Savarese. Car-net: Clairvoyant attentive recurrent network. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 151-167, 2018.
Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. Trajectron++: Multi- agent generative trajectory forecasting with heterogeneous data for control. arXiv preprint arXiv:2001.03093, 2020.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In Advances in neural information processing systems, pp. 3483-3491, 2015.
Charlie Tang and Russ Salakhutdinov. Multiple futures prediction. Advances in Neural Infor- mation Processing Systems, pp. 15398-15408, 2019.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. arXiv preprint wrXiv:1908.04311 2019.
Tianyang Zhao, Yifei Xu, Mathew Monfort, Wongun Choi, Chris Baker, Yibiao Zhao, Yizhou Wang, and Ying Nian Wu. Multi-agent tensor fusion for contextual trajectory prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11126-12134 2019.
10

---

Under review as a conference paper at ICLR 2021
A
DERIVATION OF GRADIENT
Here we show how to obtain Eq 5 and Eq
T = - log Mogg((u)) (Ugt,t' Pr, Tτ log No (yneg,
1
(log 2m log oi 1|ugr,t pall?
(log2n 2 + log o7 Vneget -ie11
2
0
1
(llygt, - Aull? Vnnegtt pall?)
20?
aL,
0
1
|| - frel2 1|yneg,t f
pp
ap. 20,
(yygt, (ae)) + (ft: yreggt)
o0
AL 0
(1lyge,t fiel I| - 12al12)
08, 0ā, 207
(1y,tt fsel12 I1eegt (2=
B GRADIENT ANALYSIS IN GAUSSIAN MIXTURE MODEL
In case we model the output distribution as Gaussian mixture model PGMM( = Σx (NNi fii d), the gradient of w.r.t. the mean of component i is
Oh.
(- log pCMM
Op.it
Opitt
PGMM (yneg.))
1
(N i,, Ō,,
ON ON Pii 0s,c2)
-
PUMM (ygt.t)
PUMM (Yneg,))
Opi,e (ynagi (aa)))
di N(
N ((ynega Pii ō2,
(ygt, paa.))
1 (CMMMyygh.t
PGMM (yneg,t)
This gradient shows that the center P..t component i will be pushed towards the ground truth location Itt.t and way from the negative location Ynog,t- For oia we have
0L
ON (Nigtt (yg.,
ON (yneg.ti Piies āz.
dil
0ō
(y
PCMM PCMM(ynegg.)
0o,
di
(11 - 22 07,0)
N (ynega Pii, Gr,eI)
(1egee. -
PGMM (gneg.t)
C THE INFLUENCE OF OUR LOSS WITHOUT MAP INPUT
In this experiment, we remove the map input to the model and demonstrate the influence of our unlikelihood loss in this case. Results shown in Tab Interestingly, our loss helps improve the performance from 1.52 to 1.42 measured by the FDE-Full in 3 seconds case although the model doesn't see the context during inference. We think the reason is that compared the Trajectron++

---

Under review as a conference paper at ICLR 2021
Table 5: The influence of our loss without map input to the model on nuScenes. Compared to Trajectron++ without map input, our loss performs better, which indicates that our loss can inject context information into learning signals
Model
FDE-Full 3s
ADE-Full 3s
Trajectron++ without map input Trajectron++
1.52+4000
0.61
1.46.10.07
0.59-10.04
Ours without map input Ours
1.42:10.03
0.57.10.01
1.34+0.04
0.54=00.22
without map input, our unlikelihood loss still offers context information to support the training since this loss is calculated using the context. Therefore, our model receives more information during training and performs better. Besides, our method without map input even achieves comparable result (FDE-FUII 1. 1.42) compared Trajectron++ with map input (FDE-Full 1.46). This experiment shows that our loss can inject context information into learning signals. In addition, map input im- proves FDE-Full of Trajectron++ in 3s prediction from 1.52 to 1.46, which about cm. However, when we further add our unlikelihood loss, performance improved from 1,46 to 1.34, which is 12 cm. Our loss triple the contribution of context information, which is significant.
D MODIFICATION ON TRAJECTRON++ FOR ARGOVERSE EVALUATION
The original Trajectron++ (Salzmann et al. 2020 is evaluated on nuScenes dataset (Caesar et al, 2019b) but not on Argoverse (Chang al. 2019). To make it runnable Argoverse, make some modifications on Trajectron++. The input states of the full version of Trajectron++ consist of the positions, velocities, accelerations, heading angles and heading angular velocities. However, Argoverse doesn't t offer the data for heading angles and heading angular velocities. Therefore, remove them from the input states. In addition, Trajectron++ has unicycle physics model to convert the predicted velocity from neural networks to locations. The unicycle physics model requires also the angular velocities information. Therefore, replace the unicycle model by the single integrator model that requires only velocities and initial positions. The maps offered by nuScens and Argoverse are different, too. We stack the drivable region and region of interest offered by Argoverse and upsample them to the same resolution as nuScenes as the input map. Please refer Argoverse for the details. Lastly, Trajectron++ use separate modules to process surrounding vehicles and surrounding pedestrians in nuScenes. However, Argoverse doesn't I offer the labels for surrouding agents and just simply denote them as "others". Therefore, we remove pedestrian modules in Trajectron++ and use the vehicle modules process both surrounding pedestrians and vehicles.
12

---

Under review as a conference paper at ICLR 2021
E COMPARISON WITH OTHER METHODS ON ARGOVERSE
Our experiment setting on Argoverse follows AtGlobal-CAM-NN (Park et al. 2020). Here we list the detailed comparison with their numbers using their evaluation metries min minADE-12 and minFDE- 12, which are the minimal ADE and FDE over 12 prediction candidates, in T Both Trajectron++ and our method outperform their numbers. addition, our unlikelihood loss helps improve the performance of Trajectron++. Our scores and the standard deviation are calculated over training instances.
Table 6: Experimental results on Argoverse dataset
Model
minADE-12
minFDF-12
CSP Deo & Trivedi 2018 Park al. 2020
1.39
2.57
DESIRE (. al. 2017 Park el
2040
0.90
1.45
MATF-GAN Zhap
2019: Park
2020
1.26
2.31
R2P2-MA [Rhinchart et al. 2019
Park ctal. 2020
1.11
1.77
MuGGloal-CAAM-1 (Park cral. 2020
0.73
1.12
Trajectron++ (Salzmann
2120
0.65++0.09
1.08
Ours
0.63=#0004
1.08=0.06
QUALITATIVE RESULTS
Here, we demonstrate our method's qualitative results compared with Trajectron++ for seconds prediction. We randomly sample 50 trajectories from the predicted prediction, use kernel density estimation (KDE) to approximate the total output distribution from the samples, and print it out in Fig44 White points represent the ground truth trajectories. Compared to Trajectron++, method suits the contextual information more and therefore more accurate and plausible.
13

---

Under review as a conference paper at ICLR 2021
(a) Ours
(b) Trajectron++
(c) Ours
(d) Trajectron++
Do
(e) Ours
( Trajectron++
Figure 4: Qualitative results of our method and Trajectron++.
14


Reviews:
[{"review_id": "ZjuXst0YXQW", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an approach to improving the accuracy of trajectory prediction by adding a loss that minimizes the likelihood of trajectories that violate the contextual constraints. The negative trajectories themselves are sampled from the current version of the prediction model and are limited to those that violate the contextual constraints, such as the lane direction and driving area.\n\nAuthors show that their method improves state-of-the-art Trajectron++ on the widely used NuScenes dataset. Additionally, they compare to their baseline method on the Agroverse dataset, but due to absence of other results on the Agroverse dataset, this serves more as an ablation study. \n\nMy biggest concern is the absence of larger evaluation on popular benchmarks such as ETH and UCY. Many of the methods that authors compare with, including Trajectron++ and Social LSTM, provide results on those benchmarks and evaluation on those could strengthen the paper. \n\nMy preliminary evaluation is 5 because while authors show improvements for one particular method:\n- The evaluation on ETH / UCY is lacking\n- The evaluation showing improvements when combining with other trajectory prediction methods is lacking\n- Novelty is limited since the idea itself has already been applied a number of times, both referenced by the authors for the NLP domain (+[Sequence level training with recurrent neural networks, ICLR'16]) and in the object tracking / trajectory prediction domain ([Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking, CVPR'19])\n\nTypos: \nabstract: (that conflicts --> that conflict)\npage 1: (checker is used to cull out --> to cuT out)\npage 4, sec. 3.3 (there are not given -> theSe are not given)\npage 4, sec. 3,3 (In detail, The checker --> In detail, the checker)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "H4hFGYpoWVB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors focus on vehicular motion forecasting on roadways. To this end, they propose an interesting tweak to existing approaches. In addition to maximizing the likelihood of ground truth trajectories, the authors consider an \"unlikelihood\" weighted subloss which penalizes sections of the event space that shouldn't happen with context (such a driving on the wrong side of the road). They do this by sampling trajectories and labeling them with a context checker. They evaluate their approach qualitatively and quantitatively on the Argoverse and nuScenes datasets. They show improved quantitative performance over baselines.\n\nPros:\n\n1. The authors focus on a problem with immediate societal applications. Motion forecasting is important for autonomous vehicles and robotics in general.\n\n2. The paper is well written and easy to understand.\n\n3. The proposed approach (unlikelihood training) makes intuitive sense and is useful for any situation with negative examples or known prior boundaries on the event space.\n\nCons:\n\n1. Quantitative improvement over baselines is fairly small on an absolute level.\n\n2. It would be useful to have more mathematical/theoretical justification for unlikelihood loss. I am certain that estimating a probability distribution subject to boundaries on the event space has been encountered many times before in machine learning and statistics. Could this problem be considered an optimization problem with constaints?\n\nOverall, the paper focuses on a timely and useful problem of vehicular motion prediction, but stronger quantitative results and more theoretical grounding would be ideal.", "extra_scores": {}, "extra_text": {}}, {"review_id": "B0YRYqpsA1N", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**SUMMARY**\n\nThe present paper considers the problem of context integration in probabilistic agent trajectory predictors, particularly Trajectron++. It starts with the observation that these predictors often do a bad job at considering non-drivable areas in their predictions even if context information is injected as part of the input. The paper proposes adjusting the loss function by adding an unlikelihood loss term. This term modifies existing MLE-type losses in a way that discourages context violations (in the present case, it discourages prediction of trajectories that are on non-drivable space). \nThe unlikelihood loss is integrated into Trajectron++ and it is demonstrated that the resulting variant of Trajectron++ not only improves upon the prediction error but also reduces the likelihood mass in regions violating the context cues. \n\n**STRENGTHS**\n- The paper addresses a very important problem for probabilistic trajectory prediction. Predictors regularly violating context cues may lead to very dangerous downstream behavior in a driving stack. This makes the general problem of investigating different means of context integration highly relevant.\n- The general idea of incorporating context via a combination of a checker and a modified loss term is cool and seems to provide a very strong prior. \n\n**WEAKNESSES**\n- The evaluation does not sufficiently support the author's claim that the proposed method is generally useful (and to what extent) beyond Trajectron++.\n- With the field moving fast, most of the baseline models are not state-of-the-art anymore. This is not necessarily a major problem as the idea has value for itself. It's just that the current formulation of the abstract might suggest that the proposed work is the best performing predictor. I am not sure if this is still true.\n- The unlikelihood loss term itself and adding the $\\varepsilon$ term within the log seem both to have no theoretical justification (if there is one, please provide a reference. I looked only briefly but Welleck et al.'s original work seem not to have one either). It is important to keep in mind that changing the loss also changes the underlying distribution assumption. In the present case, this has potential implications for how consistent the network's prediction of the variance is.\n\n**CLARITY**\n\nOverall, the paper is clearly written. I have only some smaller questions/remarks:\n- given that several new prediction papers are coming out each month, it seems to make almost no sense to have a \"trajectory prediction\" section in related work. Maybe refer to a recent survey and then discuss a subset of methods that incorporate context in different ways. E.g. top-down images, vectors, latent biases, ...\n- The authors write \"When the prediction is not so accurate (e.g. at the beginning of training), our ground truth $y_{gt,t}$ may be farther than the negative location $y_{neg,t}$. In this case, $N(\\hat{\\mu}_t; \\hat{\\sigma}_t$) will expand to better cover the ground truth and make the prediction more uncertain\". I am not sure I fully understand why this is happening. Could the authors elaborate on that?\n- When describing FDE-1, the authors write \"In both original Trajectron++ and\nour approach, this single trajectory is drawn from predicted distribution by greedy search step by step\". What does the search look for? The closest sampled trajectory? If yes, why not simply use the mean?\n\n**REPRODUCIBILITY**\n\nI believe this work to be reproducible. I also applaud the authors for planning to make their code available.\n\n**CORRECTNESS**\n- The claim \"a distribution that allocates reasonable probability mass to the region where Yi is located by minimizing the KL-divergence of predicted distribution and ground truth distribution.\" is in its generality not true. MLE does not require the use of KL-Divergences. This only happens in some models, particularly those that use (ammortized) variational inference such as VAEs.\n- In eq. 3, simply adding the small constant seems like a hack. I usually don't mind those but in the present case, the hack has a strong impact on what it is that the model is ultimately optimizing.\n- I am not sure if the final loss (4) proposed in this work gives rise to any meaningful probabilistic interpretation. \n- The authors claim that \"Eq.4 is also easily adapted to combine with any other models\".  This is not necessarily true. While the proposed loss can be integrated in most other models, the adaptation is not necessarily easy and obvious. For example, how would one adapt it to a GAN based predictor? I can think of several options (e.g. as a loss on the generator, by providing additional information to the discriminator, ...) neither of which is the obvious right choice. Also, the work assumes the availability of an output distribution to draw K negative candidate trajectories from. However, some predictors merely output a single trajectory (or a predefined small number of trajectories) and not an entire distribution. They may not contain K negative examples.\n- The new loss term might induce an undesired bias for scenarios where violating context would be the right thing to do and the reasonable thing to predict. Thus, it would be interesting to see how the approach performs in such cases in comparison to the baseline.\n\n**EVALUATIONS**\n- The claim that the \"method can be viewed as a simple add-on to any models that estimate the distribution of future trajectories.\" should be justified by experiments on multiple predictors and not only Trajectron++. In general, I would recommend focusing evaluations more on comparing different methods of context integration in trajectory prediction.\n-  It is also claimed that the \"coefficient gamma in Alg.1 increases gradually from 0 to 1 as a sigmoid function centered at 24th epoch.\" Is this rescaled in some way to ensure the graduality?\n- \"To evaluate on generalization beyond the training horizon, we test on both 3 second and 4 second prediction horizons.\" This testing protocol seems interesting to me but I wonder if it is meaningful. I would argue that in practice the prediction horizon can be known at training time and predictors do not need to generalize to other horizons but I am curious about the authors' thoughts on this.\n- In the same spirit, shouldn't the split be between different locations? I.e., are all vehicles in NuScene's test set from different locations as in the training set? Same question for the split you used in Argoverse? I couldn't find this information on their websites. For the present work, this seems important to judge whether the network learned to actually deal with different context or if unlikelihood training supported overfitting to existing locations.\n- Could you mention how many iterations there are per epoch. This would simplify understanding the extent of learning rate decay.\n- How was $\\epsilon$ from eq (3) chosen? Did different choices have an effect on the training outcome?\n- It would be interesting to see the tradeoff between using this type of training and injecting a top-down image. Would it be possible to train Trajectron in a way that does not involve context-input as part of the ablation studies?\n- In App B. the authors write that \"The input states of the full version of Trajectron++ consist of the positions, velocities, accelerations, heading angles and heading angular velocities. However, Argoverse doesn’t offer the data for heading angles and heading angular velocities. Therefore, we remove them from the input states. [...] we replace the unicycle model by the single integrator model that requires only velocities and initial positions\" I am not sure if this is the fair thing to do. If understood correctly, NuScenes does not contain some of this information either. Instead, it is computed via some of NuScenes' devkit helper function (see e.g. [here](https://github.com/nutonomy/nuscenes-devkit/blob/d8403d35a49f9a5f2b8707129c8af1eff6a8906c/python-sdk/nuscenes/prediction/helper.py#L342)). It would probably be fairer to Trajectron++ to adapt these helpers (where possible) to Argoverse instead of changing Trajectron's input and dynamic model.\n\n**REVIEW CONCLUSION**\n\nOverall, I believe the work analyzes an important problem (context integration) and takes an important direction (loss-based penalizing of context violations). At this stage, the work makes a much stronger contribution claim than its evaluations would allow substantiating. They also seem to not focus on comparing different means of context integration, which is the main area of contribution of this work. Finally, it is not clear which negative side-effects are induced by the unlikelihood loss (does the predicted variance become inconsistent? is the resulting predictor too strongly biased against \"rightful\" context violations?).  That being said, I think the general approach is worthy of further research and I want to encourage the authors to follow this direction.\n\n\n**POST-DISCUSSION UPDATE**\n\nI want to thank the authors for responding to my questions, correcting my misunderstandings, and addressing some of the raised points. Overall, I still believe that the work is not yet ready to be published. One of my main concerns is that a method for context integration should be evaluated in comparison to multiple other methods for context integration (ideally on multiple predictors) in order to see which approach is particularly meaningful and why.  As mentioned in the initial review, the general idea is interesting and worthy of resubmission after the authors address the issues raised in the reviews.", "extra_scores": {}, "extra_text": {}}, {"review_id": "2zIor_Sjs2t", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I work extensively and publish regularly in autonomous vehicle trajectory prediction and am well-placed to provide a review for this paper.\n\n*Summary:* The paper addresses the problem of trajectory prediction, which is to predict the future trajectory of a vehicle given its current trajectory, taking into account the nearby vehicle data. The main contribution of the paper is to investigate the possibility of improving performance of trajectory prediction using negative trajectories, that are generated heuristically.\n\nThis paper suffers from some major flaws in three key areas -- 1) Insignificance of results, 2) Insufficient Evaluation and 3) Low technical novelty. I will be talking about all of these separately below:\n\n**Insignificance of results:** The main contribution of the paper is the uncertainty loss term. Therefore, in order to evaluate the contribution of this loss term, it would need to be evaluated against methods that use the same hyperparameters in a loss equation that excludes the uncertainty loss term. That would be the trajectron++ (using the current paper's hyperparameters) (Tab.2, second last row). The maximum average improvement is 6 cm. To put this in perspective, this improvement is less than a *quarter* of the width of a standard vehicle tire. The advantages and usefulness of improvements on such a small scale are therefore questionable.\n\nThis insignificance of results is not just contained in Tab.2. In Tables 1 and 3, the maximum average improvement is 9cm. This is likewise not useful. I have worked with several codebases that do trajectory prediction and I can say from experience that improvements on such small scales can easily be achieved simply by tweaking hyperparameters. Therefore, I have serious doubts whether the results are significant with respect to the proposed contribution or are just a consequence of model tuning.\n\n*Conclusion*: The current results have not convinced me even slightly if the proposed uncertainty loss makes any difference to the current SOTA in trajectory prediction\n\n**Insufficient Evaluation:**  Trajectory prediction using deep neural nets is a very well studied field with dozens of benchmark approaches now that report results using the ADE/FDE metrics. So any contribution would need to be evaluated against multiple methods. In this paper, the proposed contribution has only been evaluated against one method.\n\nIt should be clearly noted that to specifically measure the value of the contribution, other than trajectron++ (\"our hyperparameters\"), any of the other baselines are not relevant as the evaluation needs to be performed keeping all the parameters same, which is not the case with any of the other methods. (If the parameters are different, then it cannot be ascertained whether the gains come from the uncertainty loss term, or from somewhere else)\n\n*Conclusion:* The validity of the method is questionable since it is compared to only one baseline, when there are more than a dozen SOTA methods available.\n\n**Low Technical Novelty:** The novelty of the approach consists of the so-called \"Context Checker\" which identifies negative trajectories. However, the basis on which these negative trajectories are identified, as well as the definition of negative trajectories, is deeply heuristic and contrived.\n\nTo begin with, trajectory prediction consists of a distribution of future possibilities. The entire distribution contributes to the uncertainty. So the proposed approach wherein the authors manually select a only a part of this distribution, give it a fancy name like \"negative trajectory\", and re-train it without a formal explanation seems heuristic and unsound.\n \nAnother concern that I immediately registered was the number of such negative trajectories, which would be negligible compared to the entire distribution of future trajectories. In fact, this is even explicitly agreed upon by the authors themselves in Section 3.3 (\"Design of our Checker\"). The low number of such \"negative\" examples would do little to impact the performance, which corroborates my earlier point about the insignificance of results.\n\nFinally, the entire approach is a simple and heuristic extension of trajectron++. The extension involves simply re-labeling a part of the predictions of trajectron++ and re-training them. This is not enough to meet the standards of ICLR.\n\n*Conclusion:* The incremental extension over Trajectron++ and heurisitc and unsound nature of the novelty.\n___\n**Overall:** My overall recommendation for paper is a **strong reject** since according to my review, it fails on all of the crucial parameters used to judge a paper.\n\n=========== Updated Score============\n\nOf the three concerns above, the authors satisfactorily addressed the point related to technical novelty, hence I am increasing my score.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper makes use of the unlikelihood objective from Welleck et al (2019) which was shown in NLP to the problem of forecasting motion trajectories on roads. The unlikelihood term is meant to lower the probability mass in non-driveable areas. The paper makes use of Trajectron++, and existing trajectory forecasting model to demonstrate the idea. While the idea is interesting, the notion of using negative examples to lower the likelihood outside a valid domain has been used in multiple occasions. The paper mentions contrastive learning, but I did not see a meaningful discussion on the difference between unlikelihood training and contrastive learning, beyond what exists in the related works section. Also, due to the unlikelihood term having appeared in Welleck et al, reviewers are hesitant to acknowledge novelty of the method. One of the reviewers also questions the significance of the results, which the authors countered by saying that their method reduces the violation rate from 10.6% to 8.9% in their predictions. This is good, but combined with the former issue implies that the paper needs more work before publication.    
Decision: Reject