Original: Robust Trajectory Prediction against Adversarial Attacks
Revised: Robust Trajectory Prediction against Adversarial Attacks

Original: Abstract: Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems.
Revised: Abstract: Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems.

Original: However, these methodsare vulnerable to adversarial attacks, leading to serious consequences such as collisions.
Revised: However, these methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions.

Original: In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks including (1) designing effective adversarial training methods and (2) adding domain-specific data augmentation to mitigatethe performance degradation on clean data.
Revised: In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks including (1) designing effective adversarial training methods and (2) adding domain-speciﬁc data augmentation to mitigate the performance degradation on clean data.

Original: We demonstrate that our method isable to improve the performance by 46% on adversarial data and at the cost ofonly 3% performance degradation on clean data, compared to the model trainedwith clean data.
Revised: We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.

Original: Additionally, compared to existing robust methods, our methodcan improve performance by 21% on adversarial examples and 9% on clean data.
Revised: Additionally, compared to existing robust methods, our method can improve performance by 21% on adversarial examples and 9% on clean data.

Original: Our robust model is evaluated with a planner to study its downstream impacts.
Revised: Our robust model is evaluated with a planner to study its downstream impacts.

Original: Wedemonstrate that our model can significantly reduce the severe accident rates (e.g.,collisions and off-road driving).
Revised: We demonstrate that our model can signiﬁcantly reduce the severe accident rates (e.g., collisions and off-road driving) 1 .

Original:  Introduction
Revised:  Introduction

Original: Trajectory prediction is a critical component of modern autonomous driving (AD) systems.
Revised: Trajectory prediction is a critical component of modern autonomous driving (AD) systems.

Original: It allowsan AD system to anticipate the future behaviors of other nearby road participants and plan its actions accordingly.
Revised: It allows an AD system to anticipate the future behaviors of other nearby road participants and plan its actions accordingly.

Original: Recent trajectory prediction models built on Deep Neural Networks (DNN) havedemonstrated state-of-the-art performance on large-scale benchmarks [1–7], showing a promisingpath towards learning-based trajectory prediction for AD systems.
Revised: Recent trajectory prediction models built on Deep Neural Networks (DNN) have demonstrated state-of-the-art performance on large-scale benchmarks [1–7], showing a promising path towards learning-based trajectory prediction for AD systems.

Original: As trajectory prediction plays animportant role in AD systems, accurate predictions are required for making safe driving decisions.
Revised: As trajectory prediction plays an important role in AD systems, accurate predictions are required for making safe driving decisions.

Original: It is crucial to understand how unknown scenarios will affect trajectory predictions and then bolsterthe robustness of such trajectory predictions in return.
Revised: It is crucial to understand how unknown scenarios will affect trajectory predictions and then bolster the robustness of such trajectory predictions in return.

Original: To achieve this goal adversarial attacks [8–10] are often used as a proxy to measure the worst-caseperformance of the model when facing unseen scenarios.
Revised: To achieve this goal, adversarial attacks [8–10] are often used as a proxy to measure the worst-case performance of the model when facing unseen scenarios.

Original: Similarly, we use a standard adversarialattack setup [11] for trajectory predictions.
Revised: Similarly, we use a standard adversarial attack setup [11] for trajectory predictions.

Original: As illustrated in Fig.
Revised: As illustrated in Fig.

Original: 1, an adversarial agent (red vehicle)aims to cause a traffic accident.
Revised: 1, an adversarial agent (red vehicle) aims to cause a trafﬁc accident.

Original: It drives along a carefully designed trajectory (i.e., adv history) toinfluence the trajectory prediction model of the Autonomous Vehicle (green vehicle).
Revised: It drives along a carefully designed trajectory (i.e., adv history) to inﬂuence the trajectory prediction model of the Autonomous Vehicle (green vehicle).

Original: Such an adversary can critically compromise the predicted trajectories of all other agents by altering its route ininconspicuous ways.
Revised: Such an adversary can critically compromise the predicted trajectories of all other agents by altering its route in inconspicuous ways.

Original: By fooling the trajectory prediction models, it can further affect downstreamplanning of the AV systems and cause serious consequences.
Revised: By fooling the trajectory prediction models, it can further affect downstream planning of the AV systems and cause serious consequences.

Original: Using the adversarial attack as theproxy, this work aims to develop effective techniques to bolster the robustness of trajectory prediction models against adversarial attacks and improve the AD’s safety under uncertain scenarios.
Revised: Using the adversarial attack as the proxy, this work aims to develop effective techniques to bolster the robustness of trajectory prediction models against adversarial attacks and improve the AD’s safety under uncertain scenarios.

Original:  At the same time, adversarial robustness for machine learning is a widely-studied area, but mostworks focus on classification tasks [12–23].
Revised: [Figure] At the same time, adversarial robustness for machine learning is a widely-studied area, but most works focus on classiﬁcation tasks [12–23].

Original: Among the proposed techniques, adversarial training [8]remains the most effective and widely used method to defend classifiers against adversarial attacks.
Revised: Among the proposed techniques, adversarial training [8] remains the most effective and widely used method to defend classiﬁers against adversarial attacks.

Original: The general strategy of adversarial training is to solve a min-max game by generating adversarialexamples for a model at each training step and then optimizing the model to make correct predictions[Figure] Safe for these samples.
Revised: The general strategy of adversarial training is to solve a min-max game by generating adversarial examples for a model at each training step and then optimizing the model to make correct predictions for these samples.

Original: However, directly applying adversarial training to trajectory prediction presentsa number of critical technical challenges.
Revised: However, directly applying adversarial training to trajectory prediction presents a number of critical technical challenges.

Original: First, most trajectory prediction methods employ probabilistic generative models to cope with theuncertainty in motion forecasting [2–7].
Revised: First, most trajectory prediction methods employ probabilistic generative models to cope with the uncertainty in motion forecasting [2–7].

Original: As we will show in this paper, the stochastic components ofthese models (e.g., posterior sampling in VAEs) can obfuscate the gradients that guide the adversarial generation, making na ¨ ıve adversarial training methods ineffective.
Revised: As we will show in this paper, the stochastic components of these models (e.g., posterior sampling in VAEs) can obfuscate the gradients that guide the adversarial generation, making na ¨ ıve adversarial training methods ineffective.

Original: Second, adversarial trainingon trajectory prediction task aims to model joint data distribution of future trajectories and adversarial past trajectories.
Revised: Second, adversarial training on trajectory prediction task aims to model joint data distribution of future trajectories and adversarial past trajectories.

Original: However, the co-evolution of the adversarial sample distribution and theprediction model during the training process makes the joint distribution hard to model and oftendestabilizes the adversarial training.
Revised: However, the co-evolution of the adversarial sample distribution and the prediction model during the training process makes the joint distribution hard to model and often destabilizes the adversarial training.

Original: Finally, prior work [23] shows that adversarial training oftenleads to degraded performance on clean (unperturbed) data, while retaining good performance inbenign cases is crucial due to the critical role of trajectory prediction for AVs.
Revised: Finally, prior work [23] shows that adversarial training often leads to degraded performance on clean (unperturbed) data, while retaining good performance in benign cases is crucial due to the critical role of trajectory prediction for AVs.

Original: Hence, an effectiveadversarial training method must carefully balance the benign and the adversarial performance of amodel.
Revised: Hence, an effective adversarial training method must carefully balance the benign and the adversarial performance of a model.

Original: Our approach.
Revised: Our approach.

Original: We propose an adversarial training framework for trajectory predictions named RobustTraj , by addressing the aforementioned challenges.
Revised: We propose an adversarial training framework for trajectory predictions named RobustTraj , by addressing the aforementioned challenges.

Original: First, to address the issue of an obfuscatedgradient in adversarial generation due to stochastic components, we devise a deterministic attack thatcreates a deterministic gradient path within a probabilistic model to generate adversarial samples.
Revised: First, to address the issue of an obfuscated gradient in adversarial generation due to stochastic components, we devise a deterministic attack that creates a deterministic gradient path within a probabilistic model to generate adversarial samples.

Original: Second, to address the challenge of an unstable training process due to shift in adversarial distributions, we introduce a hybrid objective that interleaves the adversarial training and learning fromclean data to anchor the model output on stable clean data distribution.
Revised: Second, to address the challenge of an unstable training process due to shift in adversarial distributions, we introduce a hybrid objective that interleaves the adversarial training and learning from clean data to anchor the model output on stable clean data distribution.

Original: Finally, to achieve balancedperformances on both adversarial and clean data, we introduce a domain-specific data augmentationtechnique for trajectory prediction via a dynamic model.
Revised: Finally, to achieve balanced performances on both adversarial and clean data, we introduce a domain-speciﬁc data augmentation technique for trajectory prediction via a dynamic model.

Original: This data augmentation technique generatesdiverse, realistic, and dynamically-feasible samples for training and achieves a better performancetrade-off on clean and adversarial data.
Revised: This data augmentation technique generates diverse, realistic, and dynamically-feasible samples for training and achieves a better performance trade-off on clean and adversarial data.

Original: We empirically show that RobustTraj can effectively defend two different types of probabilistic trajectory prediction models [4, 7] against adversarial attacks, while incurring minimal performancedegradation on clean data.
Revised: We empirically show that RobustTraj can effectively defend two different types of probabilistic trajectory prediction models [4, 7] against adversarial attacks, while incurring minimal performance degradation on clean data.

Original: For instance, RobustTraj can increase the adversarial performance of
Revised: For instance, RobustTraj can increase the adversarial performance of

Original: AgentFormer [4], a state-of-the-art trajectory prediction model, by 46% at the cost of 3% performance drop on clean data.
Revised: AgentFormer [4], a state-of-the-art trajectory prediction model, by 46% at the cost of 3% performance drop on clean data.

Original: To further show impacts of our method on the AD stack, we plug ourrobust trajectory prediction model into a planner and demonstrate that our model reduces serious accidents rates (e.g., collisions and off-road driving) under attacks by 100%, compared to the standardnon-robust model trained using only clean data.
Revised: To further show impacts of our method on the AD stack, we plug our robust trajectory prediction model into a planner and demonstrate that our model reduces serious accidents rates (e.g., collisions and off-road driving) under attacks by 100%, compared to the standard non-robust model trained using only clean data.

Original:  Related Work
Revised:  Related Work

Original: Adversarial attacks and defenses on trajectory prediction.
Revised: Adversarial attacks and defenses on trajectory prediction.

Original: A recent work began to study theadversarial robustness of trajectory prediction models [11].
Revised: A recent work began to study the adversarial robustness of trajectory prediction models [11].

Original: Zhang et al.
Revised: Zhang et al.

Original: [11] demonstrated thatperturbing agents’ observed trajectory can adversarially impact the prediction accuracy of a DNNbased trajectory forecasting model.
Revised: [11] demonstrated that perturbing agents’ observed trajectory can adversarially impact the prediction accuracy of a DNNbased trajectory forecasting model.

Original: To mitigate the issue, Zhang et al.
Revised: To mitigate the issue, Zhang et al.

Original: [11] proposed several defensemethods such as data augmentation and trajectory smoothing.
Revised: [11] proposed several defense methods such as data augmentation and trajectory smoothing.

Original: However, these methods are lesseffective when facing adaptive attacks [24].
Revised: However, these methods are less effective when facing adaptive attacks [24].

Original: In our work, we propose to use adversarial trainingwhich provides the general adversarial robustness that can resist adaptive attacks. 
Revised: In our work, we propose to use adversarial training which provides the general adversarial robustness that can resist adaptive attacks. Adversarial scenario generation.

Revised: A few recent studies work on generating adversarial trafﬁc scenarios such that the autonomous driving systems fail to make safe driving decisions [25, 26].

Original: Adversarial training.
Revised: However, generating realistic trafﬁc scenarios is challenging and the generated adversarial scenarios can be unrealistic and violate trafﬁc rules by directly optimizing the latent vectors of the trafﬁc model Rempe et al. In this work, we consider defending against realistic adversarial scenarios grounded on the scenarios from a dataset. Wang et al.

Revised: [25] perturb the raw input data to mislead the full stack AV system.

Revised: However, in this work, our primary goal is to study and improve the robustness of trajectory prediction models.

Revised: To obtain salient and unambiguous insights, we minimize the conﬂating factors in our analysis without considering the perception model.

Original:  A variety of adversarial training methods have been proposed to defend
Revised: Adversarial training. A variety of adversarial training methods have been proposed to defend

Original: DNN-based models against adversarial attacks [8, 19, 12–23].
Revised: DNN-based models against adversarial attacks [8, 19, 12–23].

Original: The most common strategy is to design a min-max game with the inner maximization process and outer minimization process.
Revised: The most common strategy is to design a min-max game with the inner maximization process and outer minimization process.

Original: Theinner maximization process generates adversarial examples that maximize an adversarial objective(e.g. , make wrong prediction).
Revised: The inner maximization process generates adversarial examples that maximize an adversarial objective (e.g., make wrong prediction).

Original: The outer minimization process then updates the model parametersto minimize the error on the adversarial examples.
Revised: The outer minimization process then updates the model parameters to minimize the error on the adversarial examples.

Revised: Several recent works also propose to mix clean data and adversarial examples for improving robustness [27, 28] and performance on clean data [18].

Original: Although there exists a large body of literaturein studying adversarial robustness for machine learning, most focus on the problem of discriminative model (e.g., object recognition), leaving other problem domains (e.g., conditional generativemodels) largely unexplored.
Revised: Although there exists a large body of literature in studying adversarial robustness for machine learning, most focus on the problem of discriminative model (e.g., object recognition), leaving other problem domains (e.g., conditional generative models) largely unexplored.

Original: In this work, we develop a novel adversarial training method for trajectory prediction models, where most state-of-the-art trajectory prediction models are generative andprobabilistic, by addressing a number of critical technical challenges.
Revised: In this work, we develop a novel adversarial training method for trajectory prediction models, where most state-of-the-art trajectory prediction models are generative and probabilistic, by addressing a number of critical technical challenges.

Original:  Preliminaries and Formulation
Revised:  Preliminaries and Formulation

Original: Trajectory prediction.
Revised: Trajectory prediction.

Original: The goal is to predict future trajectory distribution P θ ( Y | X ) of N agentsconditioned on their H history time states X = (cid:0) X − H +1 , . . .
Revised: The goal is to predict future trajectory distribution P θ ( Y | X ) of N agents conditioned on their H history time states X = (cid:0) X − H +1 , . . .

Original: , X 0 (cid:1) , and other environment contextsuch as maps 1 to predict T time-step future trajectories
Revised: , X 0 (cid:1) , and other environment context such as maps 2 to predict T time-step future trajectories

Original: Y =
Revised: Y =

Original: (cid:0) Y 1 , . .
Revised: (cid:0) Y 1 , . .

Original: , Y T (cid:1) .
Revised: , Y T (cid:1) .

Original: For observed timesteps t ≤ 0 , we denote the agent states as X t =
Revised: For observed time steps t ≤ 0 , we denote the agent states as X t =

Original: ( x t 1 , . . .
Revised: ( x t 1 , . . .

Original: , x ti , . . .
Revised: , x ti , . . .

Original: , x tN ) , where x ti is the state of agenti at the time step t .
Revised: , x tN ) , where x ti is the state of agent i at the time step t .

Original: Similarly, Y t =
Revised: Similarly, Y t =

Original: ( y t 1 , . . .
Revised: ( y t 1 , . . .

Original: , y tN ) denotes the states of N agents at a future time stept ( t > 0 ).
Revised: , y tN ) denotes the states of N agents at a future time step t ( t > 0 ).

Original: We denote the ground truth and the predicted future trajectories as Y and ˆ Y , respectively.
Revised: We denote the ground truth and the predicted future trajectories as Y and ˆ Y , respectively.

Original: We denote the history information encoded by a function f as the decision context C = f ( X ) .
Revised: We denote the history information encoded by a function f as the decision context C = f ( X ) .

Original: Probabilistic trajectory prediction models.
Revised: Probabilistic trajectory prediction models.

Original: In this work, we focus on defending generative, probabilistic trajectory prediction models, as they have demonstrated superior performance in modeling uncertainty in predicting future motions [2–7].
Revised: In this work, we focus on defending generative, probabilistic trajectory prediction models, as they have demonstrated superior performance in modeling uncertainty in predicting future motions [2–7].

Original: We consider  two most popular types of generative models: conditional variational encoders (CVAEs) and conditional GANs (cGANs), bothcan be viewed as latent variable models.
Revised: We consider the two most popular types of generative models: conditional variational encoders (CVAEs) and conditional GANs (cGANs), both can be viewed as latent variable models.

Original: We define latent variables
Revised: We deﬁne latent variables

Original: Z = { z 1 , ..., z i , ..., z N }where z i represents the latent variable of the agent i .
Revised: Z = { z 1 , ..., z i , ..., z N } where z i represents the latent variable of the agent i .

Original: CVAE formulates the generative problemas: p θ ( Y | X ) =
Revised: CVAE formulates the generative problem as: p θ ( Y | X ) =

Original: (cid:82) p θ ( Y | X , Z ) · p θ ( Z | X )
Revised: (cid:82) p θ ( Y | X , Z ) · p θ ( Z | X )

Original: d Z , where p θ ( Z | X ) is a conditional Gaussian prior( N ( p µθ ( Z | X ) , p σθ ( Z | X )) ) with mean p µθ ( Z | X ) and standard deviation p σθ ( Z | X ) ; p θ ( Y | X , Z ) is aconditional likelihood model.
Revised: d Z , where p θ ( Z | X ) is a conditional Gaussian prior ( N ( p µθ ( Z | X ) , p σθ ( Z | X )) ) with mean p µθ ( Z | X ) and standard deviation p σθ ( Z | X ) ; p θ ( Y | X , Z ) is a conditional likelihood model.

Original: The model is usually trained through optimizing a negative evidencelower objective [4]:
Revised: The model is usually trained through optimizing a negative evidence lower objective [4]:

Original: [Equation]
Revised: [Equation]

Original: ∥

Original: [Equation]
Revised: [Equation]

Original: (1) where q ϕ ( Z | Y , X ) is an approximate posterior parameterized by ϕ , p θ ( Z | X ) is a conditional Gaussian prior parameterized by θ , and p θ ( Y | Z , X ) is a conditional likelihood modeling future trajectory
Revised: where q φ ( Z | Y , X ) is an approximate posterior parameterized by φ , p θ ( Z | X ) is a conditional Gaussian prior parameterized by θ , and p θ ( Y | Z , X ) is a conditional likelihood modeling future trajectory

Original: Y via the latent codes Z and past trajectory X .
Revised: Y via the latent codes Z and past trajectory X .

Original: Additionally, L diversity =
Revised: Additionally, L diversity =

Original: min k ∥ ˆ Y ( k ) − Y ∥ 2 is adiversity loss, which encourages the network to produce diverse samples.
Revised: min k (cid:107) ˆ Y ( k ) − Y (cid:107) 2 is a diversity loss, which encourages the network to produce diverse samples.

Original: Given each past trajectory
Revised: Given each past trajectory

Original: X , the model generates K sets of latent codes { Z (1) , · · · , Z ( k ) , · · · , Z ( K ) } from the conditional1 For simplify, we ignore contextual information.
Revised: X , the model generates K sets of latent codes { Z (1) , · · · , Z ( k ) , · · · , Z ( K ) } from the conditional

Original: Gaussian prior N ( p µθ ( Z | X ) , p σθ ( Z | X )) , where Z ( k ) =
Revised: Gaussian prior N ( p µθ ( Z | X ) , p σθ ( Z | X )) , where Z ( k ) =

Original: { z k 1 , · · · , z kn } , resulting in K future trajectories ˆ Y ( k ) .
Revised: { z k 1 , · · · , z kn } , resulting in K future trajectories ˆ Y ( k ) .

Original: Similarly, in a conditional Generative Adversarial Net (cGAN)-based model (e.g., Social-GAN [1]),it uses a loss function as follows:
Revised: Similarly, in a conditional Generative Adversarial Net (cGAN)-based model (e.g., Social-GAN [1]), it uses a loss function as follows:

Original: [Equation]
Revised: [Equation]

Original: where G represents the generator and D represents the discriminator.
Revised: where G represents the generator and D represents the discriminator.

Original: ˆ Y ( k ) =
Revised: ˆ Y [Equation]

Original: G ( Y | X , Z ( k ) ) is oneof the predicted trajectories in which Z ( k ) is randomly sampled from N (0 , 1) .
Revised: G [Equation] Y | X , Z ( k ) ) is one of the predicted trajectories in which Z ( k ) is randomly sampled from N (0 , 1) .

Revised: During the training,

Revised: L gan is maximized to train D and L total is minimized to train G .

Original: Threat model.
Revised: Threat model.

Original: We follow the setup in prior work [11] and adopt an idealized threat model, wherethe adversary alters its observed history X by adding a perturbation δ bounded by the adversarialset S ϵp = { δ | ∥ δ ∥ p ≤ ϵ } , where ϵ is the perturbation bound.
Revised: We follow the setup in prior work [11] and adopt an idealized threat model, where the adversary alters its observed history X by adding a perturbation δ bounded by the adversarial set S (cid:15)p = { δ | (cid:107) δ (cid:107) p ≤ (cid:15) } , where (cid:15) is the perturbation bound.

Original: The perturbation aims to mislead theprediction ˆ Y .
Revised: The perturbation aims to mislead the prediction ˆ Y .

Original: A na¨ıve adversarial attack is to find the perturbation through an adversarial objectiveδ = arg max δ ∈ S { min k ∈{ 1 , · ·· ,K } ∥ p θ ( Y | X + δ, Z ( k ) )
Revised: A na ¨ ıve adversarial attack is to ﬁnd the perturbation through an adversarial objective δ = arg max δ ∈ S { min k ∈{ 1 , ··· ,K } (cid:107) p θ ( Y | X + δ, Z ( k ) )

Original: − Y ∥ 2 } , where p θ ( Y | X + δ, Z ( k ) ) is thepredicted trajectory conditioned on the random variable Z ( k ) and adversarial history trajectory X + δ .
Revised: − Y (cid:107) 2 } , where p θ ( Y | X + δ, Z ( k ) ) is the predicted trajectory conditioned on the random variable Z ( k ) and adversarial history trajectory X + δ .

Original: Na¨ıve adversarial training.
Revised: Na¨ıve adversarial training.

Original: Adversarial training formulates a min-max game with an inner maximization process that optimizes the perturbation δ to generate adversarial examples for misleadingthe model at each training iteration, and an outer minimization process that optimizes the model parameters to make correct predictions for these examples.
Revised: Adversarial training formulates a min-max game with an inner maximization process that optimizes the perturbation δ to generate adversarial examples for misleading the model at each training iteration, and an outer minimization process that optimizes the model parameters to make correct predictions for these examples.

Original: We follow the standard adversarial trainingformulation [8]:
Revised: We follow the standard adversarial training formulation [8]:

Original: [Equation]
Revised: [Equation]

Original:  RobustTraj : Robust Trajectory Prediction
Revised:  RobustTraj : Robust Trajectory Prediction

Original: As stated earlier, applying adversarial training for trajectory prediction presents three critical challenges: (1) gradient obfuscation due to model stochasticity, (2) unstable learning due to changing adversarial distribution, and (3) performance loss in the benign situation.
Revised: As stated earlier, applying adversarial training for trajectory prediction presents three critical challenges: (1) gradient obfuscation due to model stochasticity, (2) unstable learning due to changing adversarial distribution, and (3) performance loss in the benign situation.

Original: In this section, wedescribe each challenge in more detail and present the corresponding solutions in our RobustTrajmethod.
Revised: In this section, we describe each challenge in more detail and present the corresponding solutions in our RobustTraj method.

Original: Improve adversarial generation with Deterministic Attack .
Revised: Improve adversarial generation with Deterministic Attack .

Original: Since trajectory predictionis inherently uncertain and there is no single correct answer, probabilistic generative models areusually used to cope with the stochastic nature of the trajectory prediction task.
Revised: Since trajectory prediction is inherently uncertain and there is no single correct answer, probabilistic generative models are usually used to cope with the stochastic nature of the trajectory prediction task.

Original: Such stochasticity will obfuscate the gradients that are used to generate effective adversarial examples in the innermaximization process of adversarial training.
Revised: Such stochasticity will obfuscate the gradients that are used to generate effective adversarial examples in the inner maximization process of adversarial training.

Original: The na¨ıve attack mentioned in section 3 is a straightforward way to achieve this goal.
Revised: The na¨ıve attack mentioned in section 3 is a straightforward way to achieve this goal.

Original: However, this optimization involves a stochastic sampling process
Revised: However, this optimization involves a stochastic sampling process

Original: Z ( k ) ∼ N ( p µθ ( Z | X ) , p σθ ( Z | X )) .
Revised: Z [Equation] k ) ∼ N ( p µθ ( Z | X ) , p σθ ( Z | X )) .

Original: Such a stochastic process will obfuscate the gradients for findingthe optimal adversarial perturbation δ , making the outer minimization (robust training) less effective.
Revised: Such a stochastic process will obfuscate the gradients for ﬁnding the optimal adversarial perturbation δ , making the outer minimization (robust training) less effective.

Original: In order to sidestep such stochasticity, we propose the deterministic attack that creates a deterministic gradient path within the model to generate the adversarial perturbation.
Revised: In order to sidestep such stochasticity, we propose the deterministic attack that creates a deterministic gradient path within the model to generate the adversarial perturbation.

Original: ˆ Z .
Revised: ˆ Z .

Original: Specifically, weuse a deterministic latent code by replacing the sampling process Z ( k ) ∼ N ( p µθ ( Z | X ) , p σθ ( Z | X )) ,with the maximum-likelihood sample (here, i.e ˆ Z = p µθ ( Z | X ) ).
Revised: Speciﬁcally, we use a deterministic latent code by replacing the sampling process Z ( k ) ∼ N ( p µθ ( Z | X ) , p σθ ( Z | X )) , with the maximum-likelihood sample (here, i.e ˆ Z = p µθ ( Z | X ) ).

Original: The objective for generating theadversarial perturbation is thus:
Revised: The objective for generating the adversarial perturbation is thus:

Original: [Equation]
Revised: [Equation]

Original: We empirically show that gradients from this deterministic gradient path can effectively guide the generation of adversarial examples.
Revised: We empirically show that gradients from this deterministic gradient path can effectively guide the generation of adversarial examples.

Original: We name our attack as Deterministic Attack 
Revised: We name our attack as Deterministic Attack .

Revised: [Figure]

Original: Stabilize adversarial training with bounded noise and hybrid objective.
Revised: Stabilize adversarial training with bounded noise and hybrid objective.

Original: During the adversarialtraining process, the distribution of the perturbed input X + δ coevolves with the training process asδ is calculated via an inner maximization process at each training iteration.
Revised: During the adversarial training process, the distribution of the perturbed input X + δ coevolves with the training process as δ is calculated via an inner maximization process at each training iteration.

Original: Although δ is bounded[Figure] by the adversarial set S ϵp , the resulting latent condition variable C = f
Revised: Although δ is bounded by the adversarial set S (cid:15)p , the resulting latent condition variable C = f

Original: ( X + δ ) can be arbitrarilynoisy since the Lipschitz constant of neural network layers ( f ) is not bounded during training (See
Revised: ( X + δ ) can be arbitrarily noisy since the Lipschitz constant of neural network layers ( f ) is not bounded during training (See

Original: Lemma 1. in Appendix A).
Revised: Lemma 1. in Appendix A).

Original: Since C = f ( X + δ ) is noisy, it is a less informative signal comparedto the deterministic signal X .
Revised: Since C = f ( X + δ ) is noisy, it is a less informative signal compared to the deterministic signal X .

Original: Thus, modeling p θ ( Y | X + δ, Z ) becomes substantially harder.
Revised: Thus, modeling p θ ( Y | X + δ, Z ) becomes substantially harder.

Original: In anextreme case that C = f ( X + δ ) is super noisy and contains no information, the training processcan degenerate to model p θ ( Y | Z ) , resulting in the undesirable worse performance on the clean data.
Revised: In an extreme case that C = f ( X + δ ) is super noisy and contains no information, the training process can degenerate to model p θ ( Y | Z ) , resulting in the undesirable worse performance on the clean data.

Original: To further validate the above hypothesis that it is hard to model p θ ( Y | X + δ, Z ) with a changingdata distribution of X + δ , we conduct an additional experiment.
Revised: To further validate the above hypothesis that it is hard to model p θ ( Y | X + δ, Z ) with a changing data distribution of X + δ , we conduct an additional experiment.

Original: For simplicity, we use MNIST [25]as the dataset.
Revised: For simplicity, we use MNIST [29] as the dataset.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, we divide each digit image into four quadrants.
Revised: 2, we divide each digit image into four quadrants.

Original: We take thetop-left quadrant as the condition X and the remaining quadrants as the output Y .
Revised: We take the top-left quadrant as the condition X and the remaining quadrants as the output Y .

Original: We train a CVAE( p θ ( Y | X + δ, Z ) ) to model Y by using clean data ( X ) or noisy data ( X + δ ), where δ representssalt and pepper noise [26] or adversarial noise [8], resulting in Fig.
Revised: We train a CVAE ( p θ ( Y | X + δ, Z ) ) to model Y by using clean data ( X ) or noisy data ( X + δ ), where δ represents salt and pepper noise [30] or adversarial noise [8], resulting in Fig.

Original: 2 (a), (b), (c) respectively.
Revised: 2 (a), (b), (c) respectively.

Original: Thetop-left region of each image in the first row is the conditional variables X .
Revised: The top-left region of each image in the ﬁrst row is the conditional variables X .

Original: The rest of rows are thegenerated images with different Z .
Revised: The rest of rows are the generated images with different Z .

Original: Each column in the same row uses the same Z .
Revised: Each column in the same row uses the same Z .

Original: We observe thatthe model trained on clean data successfully captures the conditional distribution (i.e., the generatedimage highly depends on X ) while the model trained with adversarial noise degenerates and ignoresthe condition (i.e., each row generates images of the same digit).
Revised: We observe that the model trained on clean data successfully captures the conditional distribution (i.e., the generated image highly depends on X ) while the model trained with adversarial noise degenerates and ignores the condition (i.e., each row generates images of the same digit).

Original: This result shows that the conditional generative model fails to learn from X .
Revised: This result shows that the conditional generative model fails to learn from X .

Original: To provide a quantitative analysis, we measure thecorrelation between the label of the generated images and the label of their conditioned image quadrants, resulting in Fig 2 (d).
Revised: To provide a quantitative analysis, we measure the correlation between the label of the generated images and the label of their conditioned image quadrants, resulting in Fig 2 (d).

Original: More details on how to calculate the correlation are in the Appendix
Revised: More details on how to calculate the correlation are in the Appendix

Original: A. We observe that the correlation drops as the noise level increases for both adversarial nose andsalt and pepper noise.
Revised: A. We observe that the correlation drops as the noise level increases for both adversarial nose and salt and pepper noise.

Original: Adversarial noise is more effective to degenerate the conditional generativemodel.
Revised: Adversarial noise is more effective to degenerate the conditional generative model.

Original: Therefore, we conclude that (1) the noises in the conditional data lead to degenerated conditional generative model (i.e., from CVAE to VAE); (2) the level of degeneration depends on thenoise levels.
Revised: Therefore, we conclude that (1) the noises in the conditional data lead to degenerated conditional generative model (i.e., from CVAE to VAE); (2) the level of degeneration depends on the noise levels.

Original: Based on the analysis result, to better learn a robust trajectory prediction model, we need to bound| f ( X + δ ) − f ( X ) | to reduce the noise level.
Revised: Based on the analysis result, to better learn a robust trajectory prediction model, we need to bound | f ( X + δ ) − f ( X ) | to reduce the noise level.

Original: Hence, we propose the following regularization loss
Revised: Hence, we propose the following regularization loss

Original: L reg :
Revised: L reg :

Original: [Equation]
Revised: [Equation]

Original: where d is a distance function (e.g., we use L 2 norm as the distance metric).
Revised: where d is a distance function (e.g., we use L 2 norm as the distance metric).

Original: In addition, because the clean data has a fixed distribution, simultaneously learning from the cleandata during the adversarial training process anchors the conditional distribution on a stable cleandata distribution.
Revised: In addition, because the clean data has a ﬁxed distribution, simultaneously learning from the clean data during the adversarial training process anchors the conditional distribution on a stable clean data distribution.

Original: Specifically, we propose the following hybrid objective:
Revised: Speciﬁcally, we propose the following hybrid objective:

Original: [Equation]
Revised: [Equation]

Original: where L total could be the loss in Eq.
Revised: where L total could be the loss in Eq.

Original: 1 for CVAE-based model or Eq.
Revised: 1 for CVAE-based model or Eq.

Original: 2 for cGAN-based model.
Revised: 2 for cGAN-based model.

Original: Protect benign performance using data augmentation.
Revised: Protect benign performance using data augmentation.

Original: Adversarial training often leads to performance degradation on clean data [23].
Revised: Adversarial training often leads to performance degradation on clean data [23].

Original: However, trajectory prediction is a critical component forsafety-critical AD systems and its performance degradation can result in severe consequences (e.g.,collisions).
Revised: However, trajectory prediction is a critical component for safety-critical AD systems and its performance degradation can result in severe consequences (e.g., collisions).

Original: Thus, it is important to balance the 
Revised: Thus, it is important to balance the model performance on the clean and adversarial data when designing adversarial training algorithms.

Original: To further improve the performance on clean and adversarial data, we need to address the overfitting problem of the min-max adversarial training [27].
Revised: To further improve the performance on clean and adversarial data, we need to address the overﬁtting problem of the min-max adversarial training [31].

Original: Data augmentation is shown to be effectivein addressing the problem in the image classification domain [28].
Revised: Data augmentation is shown to be effective in addressing the problem in the image classiﬁcation domain [32].

Original: However, data augmentation intrajectory prediction is rarely studied and non-trivial.
Revised: However, data augmentation in trajectory prediction is rarely studied and non-trivial.

Original: To design an effective augmentation algorithm, Rebuffi et al.
Revised: To design an effective augmentation algorithm, Rebufﬁ et al.

Original: [28] argues that the most important criterion is that the augmented data shouldbe realistic and diverse.
Revised: [32] argues that the most important criterion is that the augmented data should be realistic and diverse.

Original: Thus, we design a dynamic-model based data augmentation strategy Ashown in the Appendix A. By using the augmentation, we can generate diverse, realistic multi-agenttrajectories for each scene and construct D aug .
Revised: Thus, we design a dynamic-model based data augmentation strategy A shown in the Appendix A. By using the augmentation, we can generate diverse, realistic multi-agent trajectories for each scene and construct D aug .

Original: RobustTraj .
Revised: RobustTraj .

Original: In summary, our adversarial training strategy for trajectory prediction models is formulated as follows:[Equation] { X , [Equation]
Revised: In summary, our adversarial training strategy for trajectory prediction models is formulated as follows:

Original: [Equation]

Original: D

Original: [Equation]
Revised: [Equation]

Original: D [Equation] where D , D aug are the training data and augmented data; L adv is adversarial loss to generate effective202 adversarial examples in Eq. 4; L total is the loss in Eq.
Revised: where D , D aug are the training data and augmented data; L adv is adversarial loss to generate effective adversarial examples in Eq. 4; L total is the loss in Eq.

Original: 1 or Eq.
Revised: 1 or Eq.

Original: 2 to train a robust model againstadversarial examples; L reg and L clean are loss shown in Eq.
Revised: 2 to train a robust model against adversarial examples; L reg and L clean are loss shown in Eq.

Original: 5 and Eq.
Revised: 5 and Eq.

Original: 6 to provide a stable signal fortraining.
Revised: 6 to provide a stable signal for training.

Original: β is a hyper-parameter for adjusting the regularization.
Revised: β is a hyper-parameter for adjusting the regularization.

Original:  Experiments and Results5.1 Experimental setup
Revised:  Experiments and Results 5.1 Experimental setup

Original: Dataset and models.
Revised: Dataset and models.

Original: We follow the setting in prior work [4, 3] and use the nuScenes dataset [29] forevaluation.
Revised: We follow the setting in prior work [4, 3] and use the nuScenes dataset [33] for evaluation.

Original: For the trajectory prediction models, we select the representative conditional generativemodels based on CVAE (AgentFormer [4]) and cGAN
Revised: For the trajectory prediction models, we select the representative conditional generative models based on CVAE (AgentFormer [4]) and cGAN

Original: (Social-GAN [1]).
Revised: (Social-GAN [1]).

Original: AgentFormer is a stateof-the-art model based on CVAE and Social-GAN is a classic model based on cGAN.
Revised: AgentFormer is a stateof-the-art model based on CVAE and Social-GAN is a classic model based on cGAN.

Original: We reportthe final results for all three models: AgentFormer (AF), mini-AgentFormer (mini-AF) and SocialGAN.
Revised: We report the ﬁnal results for all three models: AgentFormer (AF), mini-AgentFormer (mini-AF) and SocialGAN.

Original: More details are shown in the Appendix B.
Revised: More details are shown in the Appendix B.

Original: Training details and hyperparameter choices.
Revised: Training details and hyperparameter choices.

Original: For the adversarial training, we choose a 2-step
Revised: For the adversarial training, we choose a 2-step

Original: Projected Gradient Descent (PGD) attack for the inner maximization and choose β = 0 .
Revised: Projected Gradient Descent (PGD) attack for the inner maximization and choose β = 0 .

Original: We train50 epochs and 100 epochs for AgentFormer and Social-GAN respectively.
Revised: We train 50 epochs and 100 epochs for AgentFormer and Social-GAN respectively.

Original: For other hyperparameters during training, we follow the original settings for AgentFormer and Social-GAN.
Revised: For other hyperparameters during training, we follow the original settings for AgentFormer and Social-GAN.

Original: The detailsfor choosing these hyperparameters can be found in the Appendix B. All experiments are done onthe NVIDIA V100 GPU [30].
Revised: The details for choosing these hyperparameters can be found in the Appendix B. All experiments are done on the NVIDIA V100 GPU [34].

Original: We consider various baselines, including na ¨ ıve adversarial training(na ¨ ıve AT) and four defenses proposed by Zhang et al.
Revised: We consider various baselines, including na¨ıve adversarial training (na¨ıve AT) and four defenses proposed by Zhang et al.

Original: [11]: data augmentation with adversarial examples ( DA ), train-time smoothing , test-time smoothing , DA + train-time smoothing and detection+ test-time smoothing .
Revised: [11]: data augmentation with adversarial examples ( DA ), train-time smoothing , test-time smoothing , DA + train-time smoothing and detection + test-time smoothing .

Original: Attack and evaluation metrics.
Revised: Attack and evaluation metrics.

Original: For the adversarial attack, we choose a 20-step PGD attack (anablation study on step convergence can be found in the Appendix B).
Revised: For the adversarial attack, we choose a 20-step PGD attack (an ablation study on step convergence can be found in the Appendix B).

Original: Without loss of generality, weuse L ∞ as the attack threat model so that S=
Revised: Without loss of generality, we use L ∞ as the attack threat model so that S

Original:  { δ | ∥ δ ∥ ∞ ≤ ϵ } .
Revised: = { δ | (cid:107) δ (cid:107) ∞ ≤ (cid:15) } .

Original: We select ϵ = { 0 . 5 , 1 . 0 } -meter,where the 1-meter deviation is the maximum change for a standard car without shifting to anotherlanes [11].
Revised: We select (cid:15) = { 0 . 5 , 1 . 0 } -meter, where the 1-meter deviation is the maximum change for a standard car without shifting to another lane [11].

Original: We use four standard evaluation metrics for the nuscenes prediction challenge [29]: average displacement error (ADE), final displacement error (FDE), off road rates (ORR), and miss rate(MR).
Revised: We use four standard evaluation metrics for the nuscenes prediction challenge [33]: average displacement error (ADE), ﬁnal displacement error (FDE), off road rates (ORR), and miss rate (MR).

Original: We evaluate the model’s performance on both clean and adversarial data.
Revised: We evaluate the model’s performance on both clean and adversarial data.

Original: For convenience,we use ADE , FDE , ORR , MR to represent the performance on the clean data and Robust ADE , Robust FDE , Robust ORR , Robust MR to represent the performance under attacks.
Revised: For convenience, we use ADE , FDE , ORR , MR to represent the performance on the clean data and Robust ADE , Robust FDE , Robust ORR , Robust MR to represent the performance under attacks.

Original: We compute thesemetrics with the best of five predicted trajectory samples, i.e., K = 5 .
Revised: We compute these metrics with the best of ﬁve predicted trajectory samples, i.e., K = 5 .

Original: 5.2 Main results
Revised: 5.2 Main results

Original: Here, we present our main results of RobustTraj .
Revised: Here, we present our main results of RobustTraj .

Original: We compare it with the baselines including modeltrained with clean data ( Clean ) and na¨ıve adversarial training ( Na¨ıve AT ), and existing defensemethods for trajectory prediction [11].
Revised: We compare it with the baselines including model trained with clean data ( Clean ) and na¨ıve adversarial training ( Na¨ıve AT ), and existing defense methods for trajectory prediction [11].

Original: The results have been shown in Table 1.
Revised: The results have been shown in Table 1.

Original: [Table]

Original: [Figure]

Original: We observe that our method achieves the best robustness and maintains good clean performance formost cases.
Revised: We observe that our method achieves the best robustness and maintains good clean performance for most cases.

Original: For instance, with ϵ = 0 .
Revised: For instance, with (cid:15) = 0 .

Original: 5 attack on AgentFormer model, our method is able to reduce46% prediction errors ( 5 . 09 − 2 .
Revised: 5 attack on AgentFormer model, our method is able to reduce 46% prediction errors ( 5 . 09 − 2 .

Original: under the attack at a cost of 2.6% ( 1 . 91 − 1 .
Revised: under the attack at a cost of 2.6% ( 1 . 91 − 1 .

Original: clean performancedegradation on ADE, compared to the model trained with clean data at ϵ = 0 .
Revised: clean performance degradation on ADE, compared to the model trained with clean data at (cid:15) = 0 .

Original: Compared to theexisting methods, our method also significantly outperforms in terms of the robustness.
Revised: Compared to the [Table] [Figure] existing methods, our method also signiﬁcantly outperforms in terms of the robustness.

Original: For instance,with ϵ = 1 . 0 attack on AgentFormer model, our method achieves 45% better robustness with 9%better clean performance on ADE compared to the best results from existing methods [11].
Revised: For instance, with (cid:15) = 1 . 0 attack on AgentFormer model, our method achieves 45% better robustness with 9% better clean performance on ADE compared to the best results from existing methods [11].

Original: Impacts to downstream planners.
Revised: Impacts to downstream planners.

Original: To further study the downstream impact of our robust trajectorymodel in the AD stack, we plug it into a planner.
Revised: To further study the downstream impact of our robust trajectory model in the AD stack, we plug it into a planner.

Original: We select a MPC-based planner and evaluate thecollision rates under the attack.
Revised: We select a MPC-based planner and evaluate the collision rates under the attack.

Original: To perform the attack on a closed-loop planner, we conduct attackson a sequence of frames with the expectation over transformation (EOT) [31] method.
Revised: To perform the attack on a closed-loop planner, we conduct attacks on a sequence of frames with the expectation over transformation (EOT) [35] method.

Original: We followthe setting from Zhang et al.
Revised: We follow the setting from Zhang et al.

Original: [11] and choose ϵ = 1 .
Revised: [11] and choose (cid:15) = 1 .

Original: We choose AgentFormer model since it hasthe most competitive performance.
Revised: We choose AgentFormer model since it has the most competitive performance.

Original: As a result, we observe that, while AgentFormer model trainedon clean data leads to 10 collision cases under attack, the robust trained model with the proposed
Revised: As a result, we observe that, while AgentFormer model trained on clean data leads to 10 collision cases under attack, the robust trained model with the proposed

Original: RobustTraj is able to avoid all the collisions.
Revised: RobustTraj is able to avoid all the collisions.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, we demonstrate that the proposed
Revised: 3, we demonstrate that the proposed

Original: RobustTraj is able to avoid the collisions (Fig.
Revised: RobustTraj is able to avoid the collisions (Fig.

Original: 3 (d)) while the DA + Train-time Smoothing methodproposed by Zhang et al.
Revised: 3 (d)) while the DA + Train-time Smoothing method proposed by Zhang et al.

Original: [11] is not (Fig.
Revised: [11] is not (Fig.

Original: 3 (c)).
Revised: 3 (c)).

Original: 5.3 Component analysis
Revised: 5.3 Component analysis

Original: In this section, we analyze the effectiveness of the three components.
Revised: In this section, we analyze the effectiveness of the three components.

Original: We use the mini-AgentFormermodel since it has competitive performance and is lightweight for a fast adversarial training process.
Revised: We use the mini-AgentFormer model since it has competitive performance and is lightweight for a fast adversarial training process.

Original: Effectiveness of the Deterministic Attack .
Revised: Effectiveness of the Deterministic Attack .

Original: To demonstrate the importance of the Deterministic Attack , we compare it with competitive alternatives, Latent Attack and Context Attack , which also construct the deterministic path.
Revised: To demonstrate the importance of the Deterministic Attack , we compare it with competitive alternatives, Latent Attack and Context Attack , which also construct the deterministic path.

Original: However, they only attack a partial model as opposed to our end-to-endfull model attack.
Revised: However, they only attack a partial model as opposed to our end-to-end full model attack.

Original: More details about these attacks are in theAppendix A.
Revised: More details about these attacks are in the

Original:  We evaluate their attackeffectiveness by attacking a normally trained trajectory prediction model (without robust training).
Revised: Appendix A. We evaluate their attack effectiveness by attacking a normally trained trajectory prediction model (without robust training).

Original: In Fig.
Revised: In Fig.

Original: 4, we demonstrate that Deterministic Attackis the most effective attack among all.
Revised: 4, we demon- strate that Deterministic Attack is the most effective attack among all.

Original: Additionally,we embed them into the whole adversarial trainingpipeline and evaluate the adversarial robustness.
Revised: Additionally, we embed them into the whole adversarial training pipeline and evaluate the adversarial robustness.

Original: Theresults are shown in Table 2.
Revised: The results are shown in Table 2.

Original: We observe that themodel trained with the Deterministic Attack achievesthe best robustness in terms of ADE.
Revised: We observe that the model trained with the Deterministic Attack achieves the best robustness in terms of ADE.

Original: More resultswith other metrics and the other ϵ are in the Appendix B. 
Revised: More results with other metrics and the other (cid:15) are in the Appendix B. [Table]

Original: Effect of additional loss functions.
Revised: Effect of additional loss functions.

Original: We evaluate theperformance of the models trained with additionalloss terms: L clean and L reg .
Revised: We evaluate the performance of the models trained with additional loss terms: L clean and L reg .

Original: In Table 2, we can seethat the regularization term L reg improves robustnessof the models and achieves better clean performance.
Revised: In Table 2, we can see that the regularization term L reg improves robustness of the models and achieves better clean performance.

Original: It shows that the regularization of the introduced noises on conditional variables help the model to stabilize the training procedure.
Revised: It shows that the regularization of the introduced noises on conditional variables help the model to stabilize the training procedure.

Original: By addingthe clean loss L clean , we observe that both the robustness and clean performance are improved further, which means the benign data indeed anchors the model output on clean data distribution andprovides a stable signal for the better robust training for generative models.[Table]
Revised: By adding the clean loss L clean , we observe that both the robustness and clean performance are improved further, which means the benign data indeed anchors the model output on clean data distribution and provides a stable signal for the better robust training for generative models.

Original: Effect of domain-specific augmentation.
Revised: Effect of domain-speciﬁc augmentation.

Original: To demonstrate the effectiveness of the domain-specific augmentation, We also combine it with all ofthe above components to validate itseffect.
Revised: To demonstrate the effectiveness of the domain-speciﬁc augmentation, We also combine it with all of the above components to validate its effect.

Original: The results are shown in Table 2.
Revised: The results are shown in Table 2.

Original: We observe that it achieves abetter performance on clean and adversarial data.
Revised: We observe that it achieves a better performance on clean and adversarial data.

Original:  Limitations
Revised:  Limitations

Original: In this work, we identified the challenges of applying adversarial training on trajectory prediction models based on conditional generative models.
Revised: In this work, we identiﬁed the challenges of applying adversarial training on trajectory prediction models based on probabilistic generative models since they could cope with the natural uncertainty of motion forecasting.

Original: Though the conditionalgenerative model is the main-stream architecture for the trajectory prediction task, there are otherarchitectures (e.g., flow-based method [5, 6] and RL-based method [32]) for generating multi-modalpredictions.
Revised: Though the probabilistic generative model is the main-stream architecture for the trajectory prediction task, there are other architectures (e.g., LSTM [1, 36], ﬂow-based method [5, 6] and RL-based method [37]) for generating multi-modal predictions.

Revised: Additionally, we only study the adversarial set with the threat model of L ∞ perturbation on trajectories instead of other types of threat models (e.g., optimization on the latent space [26], perturbation on raw sensor data [25]).

Revised: Moreover, the primary goal of this paper is to study and improve the robustness of trajectory prediction models.

Revised: To obtain salient and unambiguous insights, we minimize the conﬂating factors in our analysis without considering the perception model in our pipeline.

Original: We leave these as future works for building robust trajectory prediction models.
Revised: We leave these as future work for building robust trajectory prediction models.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we aim to study how to train robust generative trajectory prediction models againstadversarial attacks, which is seldom explored in the literature.
Revised: In this paper, we aim to study how to train robust generative trajectory prediction models against adversarial attacks, which is seldom explored in the literature.

Original: To achieve this goal, we first identify three key challenges in designing an adversarial training framework to train robust trajectoryprediction models.
Revised: To achieve this goal, we ﬁrst identify three key challenges in designing an adversarial training framework to train robust trajectory prediction models.

Original: To address them, we propose an adversarial training framework with three maincomponents, including (1) a deterministic attack for the inner maximization process of the adversarial training, (2) additional regularization terms for stable outer minimization of adversarial training,and (3) a domain-specific augmentation strategy to achieve a better performance trade-off on cleanand adversarial data.
Revised: To address them, we propose an adversarial training framework with three main components, including (1) a deterministic attack for the inner maximization process of the adversarial training, (2) additional regularization terms for stable outer minimization of adversarial training, and (3) a domain-speciﬁc augmentation strategy to achieve a better performance trade-off on clean and adversarial data.

Original: To show the generality of our method, we apply our approach to two trajectoryprediction models, including (1) a CVAE-based model, AgentFormer, and (2) a cGAN-based model,
Revised: To show the generality of our method, we apply our approach to two trajectory prediction models, including (1) a CVAE-based model, AgentFormer, and (2) a cGAN-based model,

Original: Social-GAN.
Revised: Social-GAN.

Original: Our extensive experiments show our method could significantly improve the robustness with a slight performance degradation on the clean data, compared to the existing techniquesand dramatically reduce the severe collision rates when plugged into the AD stack with a planner.
Revised: Our extensive experiments show our method could signiﬁcantly improve the robustness with a slight performance degradation on the clean data, compared to the existing techniques and dramatically reduce the severe collision rates when plugged into the AD stack with a planner.

Original: We hope our work can shed light on developing robust trajectory prediction systems for AD. 
Revised: We hope our work can shed light on developing robust trajectory prediction systems for AD. Acknowledgement

Revised: This work was supported by NSF under the National AI Institute for Edge Computing Leveraging

Revised: Next Generation Wireless Networks, Grant # 2112562, as well as NSF grant CNS-1930041 and

Revised: CMMI-2038215.