Original: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION
Revised: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Understanding the interaction between multiple agents is crucial for realistic and plausible vehicle trajectory prediction.
Revised: Understanding the interaction between multiple agents is crucial for realistic and plausible vehicle trajectory prediction.

Original: Accordingly, existing methods tried to model and predict the interaction using observed past trajectories of agents with pooling, attention, or graph-based methods.
Revised: Accordingly, existing methods tried to model and predict the interaction using observed past trajectories of agents with pooling, attention, or graph-based methods.

Original: However, we observed that they easily fail under complex road structures.
Revised: However, we observed that they easily fail under complex road structures.

Original: It is becausethat they do not explicitly utilize the map information for predicting the relationship, and they only model the relationship between vehicles in a deterministic manner, not in a stochastic manner.
Revised: It is because they do not explicitly utilize the map information for predicting the relationship, and they only model the relationship between vehicles in a deterministic manner, not in a stochastic manner.

Original: In this paper, we propose a new method to model a stochastic future relationship among agents using the lane information contained in the map.
Revised: In this paper, we propose a new method to model a stochastic future relationship among agents using the lane information contained in the map.

Original: Our method first predicts a probability of lane-level waypoint occupancy of vehicles.
Revised: Our method first predicts a probability of lane-level waypoint occupancy of vehicles.

Original: Then the proposed method utilizes the temporal probability of passing the same lanes to learn the interaction between agents.
Revised: Then the proposed method utilizes the temporal probability of passing the adjacent lanes to learn the interaction between agents.

Original: In addition, we model the interaction using probabilistic distribution.
Revised: In addition, we model the interaction using probabilistic distribution.

Original: This distribution is learned by the posterior distribution of interaction from GT future trajectory.
Revised: This distribution is learned by the posterior distribution of interaction from GT future trajectory.

Original: We validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.
Revised: We validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.

Original: The code will be available in public upon acceptance.
Revised: The code will be available in public upon acceptance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: For safe autonomous driving, predicting the future trajectory of a vehicle is very important.
Revised: For safe autonomous driving, predicting the future trajectory of a vehicle is very important.

Original: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Zeng et al.
Revised: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Zeng et al.

Original: Rhinehart et al.
Revised: Rhinehart et al.

Original: Recently, thanks to deep learning, more accurate prediction can be made by considering not only its own state but also the relationship with the High-Definition (HD) map ( Liang et al.
Revised: Recently, thanks to deep learning, more accurate prediction can be made by considering not only its own state but also the relationship with the High-Definition (HD) map ( Liang et al.

Original: ; Zeng et al.
Revised: ; Zeng et al.

Original: or surrounding agents ( Lee et al.
Revised: or surrounding agents ( Lee et al.

Original: ; Chandra et al.
Revised: ; Chandra et al.

Original: Since surrounding vehicles are not stationary in current position, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.
Revised: Since surrounding vehicles are not stationary, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.

Original: In addition, sinceeach vehicle is controlled by an individual driver, its interaction has stochastic nature.
Revised: In addition, since an individual driver controls each vehicle, its interaction has stochastic nature.

Original: Previous works utilized past trajectories of the surrounding vehicles to model interaction by pooling global features, operating multi-head attention, or adapting spatio-temporal graph methods.
Revised: Previous works utilized past trajectories of the surrounding vehicles to model interaction by pooling, operating multi-head attention, or adapting spatio-temporal graph methods.

Original: However, we observed that they easily fail under complex road structures.
Revised: However, we observed that they easily fail under complex road structures.

Original: On the left of Fig. 1, the past trajectories of agents are shown, and on the right is a plot showing the attention weight among agents obtained by the previous method (Mercat et al.
Revised: On the left of Fig. 1, the past trajectories of agents are shown, and on the right is a plot showing the attention weight among agents obtained by the previous method (Mercat et al.

Original: that learned the interaction among agents using multi-head attention (MHA).
Revised: that learned the interaction among agents using multi-head attention (MHA).

Original: Since agent 0 and agent 4 are expected to join in the future, the attention weight between the two is expected to be high.
Revised: Since agent 0 and agent 4 are expected to join in the future, the attention weight between the two is expected to be high.

Original: However,it can be seen that the model predicts low attention weight between them.
Revised: However, the model predicts low attention weight between them.

Original: This shows the difficulty of reasoning future relationships between agents based only on past trajectories.
Revised: It shows the difficulty of reasoning future relationships between agents based only on past trajectories.

Original: The reasoning should have been much easier if it leveraged the road structure.
Revised: The reasoning should have been much easier if it leveraged the road structure.

Original: Looking at the decision-making process of human drivers, they first set their goal and roughly infer how the surrounding vehicles will behave in the future while they head to their own goal.
Revised: First, let us recall the decision-making process of human drivers. They first set their goal and roughly infer how the surrounding vehicles will behave in the future while they head to their own goal.

Original: After that, theyprobabilistically infer the interaction with other agents by inferring how much the future path of other vehicles will overlap the path set by themselves.
Revised: After that, they infer the interaction with other agents by inferring how likely the future path of other vehicles will overlap the path set by themselves.

Original: We define  interaction from this process as a "Future Relationship" to mimic the decision-making process of human drivers.
Revised: We define the interaction from this process as a " Future Relationship " to mimic the decision-making process of human drivers.

Original: To model and train the Future Relationship, as shown in Fig. 3, we use the following approaches.
Revised: We use the following approaches to model and train the Future Relationship, as shown in Fig. 3.

Original: First, because  vehicles on adjacent lanes will have much influence on each other, a future location of each vehicle should be obtained to leverage Future Relationship. From the fact that vehicles mainly 
Revised: First, because two vehicles in adjacent locations and headings will have much influence on each other, a future motion of each vehicle should be obtained. Since vehicles mainly move along lanes, we utilize lane information as strong prior for representing the rough future motion of vehicles. Because

Original: [Figure]
Revised: [Figure]

Original: the probability of a vehicle passing a specific lane segment at every intermediate

Original: We represent the waypoint occupancy using categorical distribution and sample a future location feature from waypoint occupancy to infer Future Relationship.

Original: Fig. 3 shows the waypoint occupancy of each agent aggregated across time. The probability that a vehicle passesthat lane at each timestamp is represented using the tone of redcolor. Each agent chooses multiple lane segments as their future location with probability.
Revised: shows the waypoint the passes lane during the prediction horizon is drawn using the tone of red

Original: In conclusion, waypoint occupancy enables the model to learn how to obtain Future Relationship among vehicles on adjacent lanes in a stochastic manner.

Original: Second, based on sampled future location feature, we infer the Future Relationship in probabilistic distribution.
Revised: Second, based on the waypoint occupancy we infer the Future Relationship in probabilistic distribution.

Original: In most vehicle trajectory prediction tasks, interaction between agents is still made in a deterministic manner.
Revised: In most vehicle trajectory prediction tasks the interaction between agents is still made in a deterministic manner.

Original: Because interaction between agents is highly stochastic and there can be multiple possible interactions, the deterministic relation inferenceactually averages out interactions that interrupts socially-aware trajectory prediction.
Revised: However, we tackle that interaction between vehicles is highly stochastic, and there can be multiple possible interactions. The deterministic relation inference averages out diverse interactions, interrupting socially-aware trajectory prediction.

Original: Therefore, we model FutureRelationship in
Revised: Therefore, we define Future

Original:  Gaussian Mixture (GM) distribution.
Revised: Relationship in Gaussian Mixture (GM) distribution.

Original: Motivated by Neural Relational Inference (NRI) ( Kipf et al.
Revised: Motivated by Neural Relational Inference (NRI) ( Kipf et al.

Original: (2018)), we propose a method to train the distribution with the posterior distribution explicitly.
Revised: we propose a method to train the diverse interaction distribution explicitly.

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: 1) We propose to inference Future Relationship using lane information, assumming that vehicles generally follow  lane.
Revised: 1) We propose to infer Future Relationship using lane information, assuming that vehicles generally follow the lane.

Original: We propose the model to infer the distribution of interaction as Gaussian Mixture, and propose the new learning method to make the model learn the distribution from the posterior. 3)
Revised: We propose the model to infer the distribution of interaction as Gaussian Mixture and propose a new learning method to make the model learn the distribution from the posterior. 3)

Original: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.
Revised: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.

Original: In both datasets, there was a remarkable improvement in prediction performance and achieved state-of-the-art performance in nuScenes dataset.
Revised: In both datasets, there was a remarkable improvement in prediction performance and achieved state-of-the-art performance in nuScenes dataset.

Original: R ELATED W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION
Revised: [Equation] W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION

Original: Predicting future trajectories at once is a challenging task.
Revised: Predicting future trajectories at once is a challenging task.

Original: Instead, the first goal-conditional prediction, which oversamples goal candidates and selects a final set of goals, isknown to be helpful and has shown state-of-the-art performance ( Zhao et al.
Revised: Instead, the first goal-conditional prediction, which oversamples goal candidates and selects a final set of goals, is helpful and has shown stateof-the-art performance ( Zhao et al.

Original: ; Gu et al.
Revised: ; Gu et al.

Original: (2021); Phan-Minh et al.
Revised: (2021); Phan-Minh et al.

Original: Chai et al.
Revised: Chai et al.

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: (2021)), especially in long-time prediction tasks.
Revised: (2021)), especially in long-range prediction tasks.

Original: CoverNet ( PhanMinh et al.
Revised: CoverNet ( Phan-Minh et al.

Original:  and MultiPath ( Chai et al.
Revised: (2020)) and MultiPath ( Chai et al.

Original: (2019)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is neglected.
Revised: (2019)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is not considered.

Original: Recent studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.
Revised: Recent studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.

Original: TNT ( Zhao et al.
Revised: TNT ( Zhao et al.

Original: (2021)) uses goal points sampled from a lane centerline; GoalNet ( Zhang et al.
Revised: (2021)) uses goal points sampled from a lane centerline; GoalNet ( Zhang et al.

Original: (2021)) uses lane segments as anchors and predicts multiple trajectories based on each anchor.
Revised: (2021)) uses lane segments as anchors and predicts multiple trajectories based on each anchor.

Original: Similarly, we set lane segments with their connectivity with nearby segments as goal candidates to model the inherent uncertainty of trajectory prediction.
Revised: Similarly, we set lane segments with their connectivity with nearby segments as goal candidates to model the inherent uncertainty of trajectory prediction.

Original: The relationship between lane segments, such as predecessor and successor, provides global connectivity, enabling the model to predict realistic and plausible trajectories.
Revised: The relationship between lane segments, such as predecessor and successor, provides global connectivity, enabling the model to predict realistic and plausible trajectories.

Original: 2. I NTERACTION MODELING
Revised: 2. I NTERACTION MODELING

Original: Trajectory prediction considering the interaction between agents helps to predict socially awarefuture trajectory.
Revised: Trajectory prediction considering the interaction between agents helps to predict socially aware trajectory.

Original: In the very early stage, modeling interaction using a global pooling layer ( Deo & Trivedi (2018); Gupta et al.
Revised: In the very early stage, modeling interaction using a global pooling layer ( Deo & Trivedi (2018); Gupta et al.

Original: (2018)) were proposed.
Revised: (2018)) were proposed.

Original: In another way, researchers attempt to obtain interaction throughan attention-basedmethod ( Ngiam et al.
Revised: In another way, researchers attempt to obtain interaction through attention-based ( Ngiam et al.

Original: ; Mercat et al.
Revised: ; Mercat et al.

Original: Vemula et al.
Revised: Vemula et al.

Original: and GNN-based method ( Carrasco et al.
Revised: and GNNbased method ( Carrasco et al.

Original: ; Cao et al.
Revised: ; Cao et al.

Original: Zeng et al.
Revised: Zeng et al.

Original: ; Casas et al.
Revised: ; Casas et al.

Original: ; Liang et al.
Revised: ; Liang et al.

Original: ; Gao et al.
Revised: ; Gao et al.

Original: In the previous method, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.
Revised: In the previous method, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.

Original: Besides, a line of works using Neural Relational Inference (NRI) that explicitly predicts and learn interaction using a latent interaction graph was proposed ( Kipf et al.
Revised: Besides, a line of works using Neural Relational Inference (NRI) that explicitly predicts and learn interaction using a latent interaction graph was proposed ( Kipf et al.

Original: EvolveGraph ( Li et al.
Revised: EvolveGraph ( Li et al.

Original: (2020)) utilizes two interaction graphs, static and dynamic,  ( Chen et al.
Revised: (2020)) utilizes two interaction graphs, static and dynamic, and NRI-MPM ( Chen et al.

Original: (2021)) uses  relation interaction mechanism and spatio-temporal message passing mechanism.
Revised: (2021)) uses a relation interaction mechanism and spatio-temporal message passing mechanism.

Original: Similarly, we apply  NRI-based method to predict future trajectory explicitly in complex road structures.
Revised: Similarly, we apply the NRI-based method to predict future trajectory explicitly in complex road structures.

Original: 2. M ULTI - MODAL TRAJECTORY PREDICTION
Revised: 2. M ULTI - MODAL TRAJECTORY PREDICTION

Original: The trajectory prediction task is the problem of predicting the future that has not occurred.

Original: Due to the nature of the problem; stochasticity, there is not a unique answer, but several futures are possible.
Revised: Due to the nature of the trajectory prediction; stochasticity, there is no unique answer, but several futures are possible.

Original: Therefore, recent works use deep generative models such as GAN (Goodfellow et al.
Revised: Therefore, recent works use deep generative models such as GAN (Goodfellow et al.

Original: and VAE (Kingma & Welling (2013)).
Revised: and VAE (Kingma & Welling (2013)).

Original:  (Gupta et al.
Revised: GAN-based (Gupta et al.

Original: ; Kosaraju et al.
Revised: ; Kosaraju et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2021b))use the GAN and  (Ivanovic & Pavone (2019); Salzmann et al.
Revised: (2021b)) and CVAE-based models (Ivanovic & Pavone (2019); Salzmann et al.

Original: ; Tang & Salakhutdinov (2019))use the Conditional VAE (Sohn et al. (2015)) to predict multiple futures by sampling multiple latent vectors.
Revised: ; Tang & Salakhutdinov (2019)) predict multiple futures by sampling multiple latent vectors.

Original: As it is important to sample a meaningful latent vector for predicting diverse and plausible future trajectories, trying to define a well-organized latent space became a natural choice ( Ma et al.
Revised: As it is important to sample a meaningful latent vector for predicting diverse and plausible future trajectories, trying to define a well-organized latent space became a natural choice ( Ma et al.

Original: ; Bae et al. (2022)).
Revised: ; Bae et al. (2022)).

Original: The most closely related work to ours is GRIN (Li et al.
Revised: The most closely related work to ours is GRIN (Li et al.

Original: (2021a)), whichalso argues that multi-modality in trajectory prediction comes from two sources: i) personal intention and ii) social relations with other agents.
Revised: (2021a)), which argues that multi-modality in trajectory prediction comes from two sources: i) personal intention and ii) social relations with other agents.

Original: GRIN only uses the past trajectories of agents to model the latent space of personal intention and social relations.
Revised: GRIN only uses the past trajectories of agents to model the latent space of personal intention and social relations.

Original: However, our model first predicts the lanes to be occupied by agents and then models the latent space of social relations based on the occupancy.
Revised: However, our model propose to utilize lane information for modeling social relations based on the occupancy.

Original: Leveraging the future occupancyinformation to model the interaction makes the model interpretable and easy to be trained since the pseudo labels of occupancy could be generated from ground-truth trajectories.
Revised: Leveraging the future occupancy makes the model interpretable and easy to be trained since the pseudo labels of occupancy can be obtained from ground-truth trajectories.

Original: Also, we sample the personal intention of a driver from a set of lane segments feature by following the standard goal-conditioned method (Zhao et al.
Revised: Also, we sample the personal intention of a driver from a set of lane segments feature by following the standard goal-conditioned method (Zhao et al.

Original:  F ORMULATION
Revised:  F ORMULATION

Original: In each scene,setting the reference time step to 0, the trajectory during the  past t p + 1 time steps of N agents is denoted by x − = { x i − t p :0} 1: N and x + = { xi 1: t f } 1: N for ground-truth (GT) future trajectory wherethe location of the i -th agent at the t -th time step is x it = ( x it , y it ) .
Revised: In each scene, the observed past trajectory and GT future trajectory of N vehicles are denoted as x − = { x } 1: N − t p :0 and x + = { x } 1: N 1: t f where t p + 1 and t f are past/future timesteps.

Original: We use lane information from the HD map, expressed as M segmented lane polylines ( Zhao et al.
Revised: We use lane information from the HD map, expressed as M segmented lane polylines ( Zhao et al.

Original: ; Varadarajan et al.
Revised: Varadarajan et al.

Original: The segmented lane polylines are shown in Fig.

Original: 2, where different colors denote different individual segments.

Original: The lane information is represented as a graph GGG =
Revised: The lane information is represented as a graph GGG =

Original: ( ℓℓℓ, e ) where the node ℓℓℓ = { x m 1: s , y m 1: s } 1: M is the location of M lane segments of sequence length
Revised: ( ℓℓℓ, e ) where the node ℓℓℓ = { ℓ } 1: M is the set of directed sequences of xy coordinates.

Original: s .
Revised: The segmented lane nodes are shown in Fig. 2, where different colors denote different individual segments ( ℓ m ).

Original: We define edge e as five types:predecessor e mpre ,successor e msuc ,left neighbor em left ,right neighbor em right ,and in-the-same-intersection e minter .
Revised: The edge is defined as five types: e = { e pre , e suc , e left , e right , e inter } 1: M which are predecessor, successor, left/right neighbor and in-the-same-intersection.

Original: The overall input observation is denoted by X = { x − , GGG} .
Revised: We denote overall input as X = { x − , G G G} .

Original:  We also define the future lane occupancy (waypoint occupancy) denoted as τττ1: tf = (cid:81) N,Mi,m p iℓ, 1: t f ( ℓ m ) , where p iℓ, 1: t f ( ℓ m ) is the probability of the m -th lane segment being occupied by the i -th vehicle during future time 
Revised: The output is denoted as Y , which is the set of F samples of xy coordinates of the target vehicles. We also define the future lane occupancy (waypoint occupancy) denoted as τττ t = (cid:81) N,Mn,m p nt ( ℓ m ) , where p nt ( ℓ m ) is the probability of the m -th lane segment being occupied by the n -th vehicle at future time step t . The waypoint occupancy during future timestemps ( t f ) predicted from X is denoted by τττ − 1: t f and GT waypoint occupancy obtained from GT future trajectory is denoted by τττ +1:

Original: 1 : t f . 
Revised: t f . [Figure]

Original: Predicted waypoint occupancy from X is denoted by τττ − t f and ( waypoint obtained from GT
Revised: 𝐱 [Figure] ( 𝐡 % )

Original: future trajectory is denoted by
Revised: F Future Trajectories ( 𝐘 ) IntentionFeature ( 𝐡 & ) x !,# x !,$ ··· [Figure] ( 𝐱

Revised: [Figure]

Revised: x

Revised: [Figure]

Revised: Waypoint Occupancy ( 𝛕 ’:) ! )

Revised: 𝑎𝑔𝑒𝑛𝑡 𝑖 0 𝑡 " ··· i j 𝑧 #$% 𝑧 &’( c x %

Revised: [Figure]

Revised: $ ··· 𝑎𝑔𝑒𝑛𝑡 𝑗 0 𝑡 " ···

Revised: [Figure]

Revised: 𝑗 𝑎𝑔𝑒𝑛𝑡 𝑖

Revised: Decoder

Revised: [Figure]

Original:  M ETHOD
Revised:  M ETHOD

Original: As mentioned above, we focus on modeling  "Future Relationship" between agents.
Revised: We focus on modeling the "Future Relationship" between agents.

Original: The naive method to infer the future relationship is to predict all future  trajectoriesof the vehicles and then use the similarity among the trajectories.
Revised: The naive method to infer the Future Relationship is predicting all future vehicle trajectories and then using the similarity among the trajectories.

Original: However, this method needs to perform prediction twice, which is inefficient and redundant.
Revised: However, this method needs to perform prediction twice, which is inefficient and redundant.

Original: Also, the criteria for calculating the similarity between predicted paths are not clear.
Revised: Also, the criteria for calculating the similarity between trajectories are unclear.

Original: Inspired by the idea that the vehicles mainly follow lanes, we predict the interaction between vehicles using theprobability of two agents pass the same lane segment.
Revised: Inspired by the idea that the vehicles mainly follow lanes, we predict the interaction between vehicles using the lane information as a strong prior of interaction modeling.

Original: The key idea is that if two agents are expected to pass the same lane, they will have a high chance to interact in the future .
Revised: The key idea is that if two agents are expected to pass an adjacent lane, they will have a high chance of interacting in the future .

Original: The overall structure is shown in Fig.
Revised: The overall structure is shown in Fig.

Original: Our model first predicts the waypoint occupancy,that the probability of  lane segment where the vehicle is expected to pass during the future time step 1: t f(Sec. 4.1). The waypoint occupancy of vehicles is fed to a Future Relationship Module (FRM)
Revised: First, we predict the waypoint occupancy, the probability of the lane segment where the vehicle will pass during the future time step 1: t f

Original: (Sec. 4.2).
Revised: (Sec. 4.1).

Original: From each vehicle’s waypoint occupancy, the FRM infers interaction feature between agents as GaussianMixture.
Revised: Then, from each vehicle’s waypoint occupancy, the FRM infers interaction features between agents as Gaussian

Revised: Mixture (Sec. 4.2).

Original: Message passing through inferred interaction features is performed to update the agent feature.
Revised: Next, a message passing through inferred interaction features is performed to obtain interaction feature.

Original: In the decoding stage (Sec. 4.4), updated agent features and goal lane feature are concatenated and fed to the decoder. In addition, the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature. 
Revised: Finally, in the decoding stage (Sec. 4.3), the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature. Here, the intention feature is the concatenation of past motion and goal lane features.

Original: The interaction is learned in two manners: waypoint occupancy is trained by minimizing thecross entropy, and distribution of interaction feature is trained by CVAE  (Sec. 4.3).
Revised: Our method follows the CVAE scheme (Sec. 4.2.1, 4.2.2), where the condition corresponds to the intention feature, and the latent code corresponds to the interaction feature.

Original: Details of the proposed methods are described in the following sections.
Revised: We describe the details in the following sections.

Original: 4. W AYPOINT PREDICTION
Revised: 4. W AYPOINT O CCUPANCY

Original: Past trajectories x − and lane graph GGG are encoded into agent and lane feature: hhh − x and hhh ℓ .
Revised: Past trajectories x − and lane graph GGG are encoded into past motion and lane features: hhh − x and hhh ℓ .

Revised: The same motion encoder is used to obtain future motion features hhh + x .

Original: Then following TNT ( Zhao et al.
Revised: Similar to TNT ( Zhao et al.

Original: , wefirst predict goal lane probability ( τ τ τ − t f ) as categorical distribution, then sample goal lane feature ( ˆ hhh − ℓ ) for all agents. Here, we additionally predict waypoint occupancy during 1: t f − 1 timesteps ( τττ − 1: t f − 1 ). Because waypoint occupancy varies according to the sampled goal, it is predicted through Cross Attention (CA) as:
Revised: , we predict waypoint occupancy in categorical distribution from past motion and lane features.

Original: [Equation]
Revised: [Equation]

Original: where [ · , · ] means concatenation.
Revised: where [ · , · ] is concatenation.

Original: As waypoint occupancy is a probability,  a softmaxlayer is used so that it becomes (cid:80) 
Revised: As waypoint occupancy is a probability, we apply a softmax so that it becomes (cid:80) M

Original: Mm τ − ,i,m 1: t f = 1 .
Revised: τ mt = 1 .

Original: Sampled goal lane features (ˆ hhh − ℓ ) is concatenated to agent feature and this is interpreted as the agents’ intentions  feature ( hhh − I ) to reach on the map, so it is utilized as agents’ personal intention.
Revised: Then goal feature ( hhh − g ) is sampled from hhh ℓ following waypoint occupancy at t f ( τττ − t f ). The concatenation of hhh − g and hhh − x is interpreted as the agents’ intentions to reach, representing the intention feature ( h h h − I ).

Original: Besides, τττ − 1: t f − 1 can be thought as a temporal feature containing  probabilistic future of the vehicle so it is passed to FRM. 
Revised: Besides, τ τ τ − 1: t f − 1 is a temporal feature containing the probabilistic future motion, so it is passed to FRM. For GT waypoint occupancy ( τττ +1: t f ), we can obtain it directly from GT future trajectory. We can get binary lane occupancy of vehicles as we know the position and heading of vehicles at future timesteps. The details about getting GT waypoint occupancy are described in the supplementary material.

Revised: Similarly, the unique GT goal feature( hhh + g ) also can be obtained by indexing from hhh ℓ .

Original: 4. F UTURE R ELATIONSHIP M ODULE (FRM)
Revised: 4. F UTURE R ELATIONSHIP M ODULE (FRM)

Original: The procedure for the prior distribution of Future Relationship is depicted in Fig.
Revised: Attaining Future Relationship is depicted in Fig.

Original: At first, waypoint occupancy is smoothed by Graph Convolutional Network (GCN) ( Welling & Kipf (2016)).
Revised: At first, waypoint occupancy is smoothed by Graph Convolutional Network (GCN) ( Welling & Kipf (2016)).

Original: It is because when a vehicle passes a specific lane, the vehicle will affect other vehicles that pass the adjacent lane, not only exactly the same lane.
Revised: It is because when a vehicle passes a specific lane, the vehicle will affect other vehicles that pass the adjacent lane, not necessarily the same lane.

Original: Different smoothingis applied for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employingdifferent 2-hop GCN layers where one layer is expressed as:
Revised: Therefore, we apply different smoothing for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employing 2-hop GCN layers. Each layer is expressed as:

Original: [Equation]
Revised: [Equation]

Original: Here, σ is softmax followed by ReLU activation, D e ∈ R M × M is a degree matrix, A e ∈ R M × M is an adjacency matrix, and W e ∈ R t f − 1 ×
Revised: Here, σ is softmax followed by ReLU activation, D e ∈ R M × M is a degree matrix, A e ∈ R M × M is an adjacency matrix, and W e ∈ R t f − 1 ×

Original: d is a weight matrix for each edge type.
Revised: d is a weight matrix for each edge type.

Original: Next, with the obtained waypoint occupancy, we compute the inter-agent proximity (PR) between vehicles i and j ( pr ij  )  which is the temporal probability that two agents would occupy adjacent lanes.
Revised: Next, with the obtained waypoint occupancy, we compute the inter-agent proximity (PR = { pr ij } 1: N, 1: N ) between vehicle pairs which is the temporal probability that two agents would occupy adjacent lanes.

Original: The PR can be obtained with a cross product of  τττ− ′ 1: t f − 1 across  lane axis.
Revised: The PR can be obtained with a cross product of ˜ τττ 1: t f − 1 across the lane axis.

Revised: It is visually drawn in the middle of Fig.

Original: [Equation]
Revised: [Equation]

Original: The PR is then utilized as an initial bi-directional interaction edge between agents.
Revised: Our method follows CVAE scheme as mentioned above, the prior distribution of relation should be computed. We utilize past motion features ( hhh − x ) and predicted inter-agent proximity (PR − ) to obtain the prior. In this subsection, we omit superscript - for simplification.

Original: Unlike the physical system where we can categorize interaction clearly, the interaction between chaotic systems cannot be clearly categorized.
Revised: There are two design factors for our interaction modeling: i) interaction should reflect diverse and stochastic properties, and ii) interaction occurs in every pair of vehicles.

Original: Therefore, its distribution is assumed as a Gaussian Mixture (GM) distribution.
Revised: As a result, the prior of relation is assumed to be Gaussian Mixture (GM) per agent pairs.

Original: In addition, interactions among multiple agents are independent, so we predict interaction distribution for every pair of agents.

Original: Then, we define interaction edge e ij between agent i and agent j as a d-dimentional feature which has GM distribution ( p θ ( eee | X ) ∼ (cid:81)
Revised: Then, we define interaction edge e ij between agent i and agent j as a d-dimensional feature that has GM distribution ( p θ ( eee | X ) ∼ (cid:81)

Original: Kk =1 πππ k N ( µµµ k , III σσσ 2 k ) )
Revised: Kk =1 πππ k N [Equation] µµµ k , [Equation]

Original:  following GMVAE ( Dilokthanakul et al.
Revised: I I σσσ 2 k ) ) following GMVAE ( Dilokthanakul et al.

Original: After that, the distribution parameters ( µµµ− K ,σσσ− K ,πππ − K= { µµµ ijK ,σσσ ijK ,πππ ijK } 1: N, 1: N ) are given by neural network F θ :
Revised: The distribution parameters ( µµµ K ,σσσ K ,πππ [Equation] [Equation] N, 1: N ) are obtained from the neural network F θ :

Original: [Equation]
Revised: [Equation]

Original: F θ is composed of MLP  layerfollowed by 1d-convolution along the time dimension ( Deo & Trivedi (2018)) to capture interdependency across time.
Revised: F θ is composed of MLP layers and 1-d conv layer to capture temporal dependency (Deo & Trivedi (2018)).

Original: Then,  the interaction edge is sampled over πππ− K  and ϵϵϵ .
Revised: Then, we perform sampling twice: one for the interaction mode k (from πππ K ) and one for ϵ (from Gaussian noise).

Original: This allows to have K distinct interactions, each interaction following Gaussian distribution:
Revised: It allows having K distinct interactions where each interaction follows Gaussian distribution:

Original: [Equation]
Revised: [Equation]

Original: Message passing function between the agent feature is then performed based on the interaction edge.
Revised: We then compute interaction feature ( hhh − R = { h iR } 1: N ) by message passing from sampled interaction edge.

Original: As we defined interaction as a d -dimensionalfeature that has the same dimension as an agent feature, the message function is composed of summation followed by Hadamard product ( ⊗ ) of interaction edge and the other agent features. Interaction feature ( hhh iR ) can be obtained by:
Revised: Because we define interaction as a d -dimensional feature, the message function is composed of summation followed by Hadamard product ( ⊗ ) of interaction edge and the other agent features:

Original: [Equation]
Revised: [Equation]

Original: Stochasticity is obtained by giving flexibility within a limited range by predicted mean and standard deviation.
Revised: Stochasticity is obtained by giving flexibility within a limited range by predicted mean and standard deviation.

Original: It is because, unlike a symmetric and deterministic interaction between particles, such as pull or push ( Kuramoto (1975)), each driver can asymmetrically consider stochastic interaction.
Revised: It is because, unlike a symmetric and deterministic interaction between particles, such as pull or push ( Kuramoto (1975)), each driver can asymmetrically consider stochastic interaction.

Original: 4. CVAE POSTERIOR
Revised: 4.2. CVAE POSTERIOR

Original: To learn the  distribution of  theinteraction, the GT future trajectory ( x +  ) and GT(binary) waypoint occupancy ( τττ + ) are utilized.
Revised: To obtain the posterior distribution of relation, the future motion feature ( hhh + x ) and GT waypoint occupancy ( τττ + ) are used.

Original: The future trajectory is encoded with the same agent encoder to get the agent feature hhh + x . With GT waypoint occupancy ( τττ + ), the same procedureas described in Eqs.(2)-(3) are performed to obtain posterior spatial proximity (PR + ).
Revised: Similarly, we omit superscript + in this subsection for simplification. Inter-agent proximity is obtained by following same procedure in Eqs.

Original: The difference between the prior is that the posteriordistribution of interaction is modeled in a single Gaussian distribution.
Revised: The difference from the prior is that the posterior is modeled in a single Gaussian distribution.

Original: Thus, F θ is replaced with F ϕ which outputs parameters of Gaussian distribution (µµµ + ,σσσ + =
Revised: Thus, F θ is replaced with F ϕ which outputs parameters of Gaussian distribution (

Original: [Equation]
Revised: [Equation]

Original: ,σσσ

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We only sample ϵϵϵ ∼ N (000 , I I I ) , then interaction edge is obtained as z e z e z e+ = µµµ + + σσσ + ϵϵϵ .
Revised: Then we sample ϵϵϵ ∼ N (000 , III ) , and interaction edge is obtained: z e z e z e = µµµ + σσσϵϵϵ .

Original: Message passing is performed the same as Eq. 7, then agent features are updated.
Revised: Following the same message passing, interaction features ( hhh + R ) is obtained.

Original: 4. D ECODER
Revised: 4. D ECODER

Original: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).
Revised: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).

Original: During training, GT goal feature (ˆ hhh + ℓ ) is concatenated to agent feature ( hhh − x ) to get a unique GT intention feature.
Revised: GT goal feature ( hhh + g ) is concatenated to the past motion feature ( hhh − x ) to get a unique GT intention feature.

Original: The GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times.
Revised: The GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times from the posterior distribution.

Original: During inference, the goal feature (ˆ hhh − ℓ ) is sampled F times from inferred waypoint occupancy.
Revised: During inference, the goal feature ( hhh − g ) is sampled F times.

Original: Intention feature is obtained by concatenation of sampled goal feature and agent feature ( hhh − x ).
Revised: The intention feature is obtained by concatenation of sampled goal features and the past motion feature ( hhh − x ).

Original: Interaction feature ( hhh − R ) isalso sampled F times from prior distribution and inferred goal feature ( ˆ hhh − ℓ ) are used.
Revised: The interaction feature ( hhh − R ) is sampled F times from prior distribution.

Original: The decoder is modeled by 2-layer MLP and predicts the sequence of x,y coordinates from the concatenation of the intentionfeature and interaction feature.
Revised: The decoder is modeled by 2-layer MLP and predicts the sequence of x,y coordinates from the concatenation of the intention and interaction feature.

Original: 4. T RAINING
Revised: 4. T RAINING

Original: For intention, because the ground truth waypoint occupancy τττ +  is observable, we train τττ − with the ground truth.
Revised: Because the ground ( τττ + ) is available for waypoint occupancy, we train τττ − with supervision.

Original: Since the waypoint occupancy is modeled as a classification per agent, we learn it using negative log-likelihood (NLL).
Revised: Since the waypoint occupancy is modeled as a classification per agent, we learn it using negative log-likelihood (NLL): L nll = − τττ

Original: Therefore, the NLL loss is computed on lanes that the GT trajectory actually occupies: L nll = − τττ +1:{++
+} t f log ( τττ − 1: t f ) .
Revised: {++
+} t f log ( τττ − 1: t f ) .

Original: In contrast, because interaction e e e is unobservable, we optimize the negative evidence lower bound (ELBO) to learn the distribution of interaction utilizing the CVAE framework.
Revised: In contrast, because interaction zzz e is unobservable, we optimize the negative evidence lower bound (ELBO) to learn the distribution of interaction following the CVAE scheme.

Original: [Equation]
Revised: [Equation]

Original:  q  is an approximate posterior distribution of interaction, and p  is  prior.
Revised: where q π is an approximate posterior distribution of relation, and p θ is the prior.

Original: Because our model only allows interaction posterior to be Gaussian distribution, the Kullback–Leibler (KL) divergence term[Table] [Table] can be simplifiedas below.
Revised: Because our model only allows interaction posterior to be Gaussian distribution, the Kullback–Leibler (KL) divergence term can be simplified below.

Original: The detailed derivation with the reparameterization trick is included in the supplementary material.
Revised: The detailed derivation with the reparameterization trick is included in the supplementary material.

Original: [Equation]
Revised: [Equation]

Original: The common drawback of NRI-methods is the "degenerate" problem which leads the decoder to ignore the relation edge during training.
Revised: The common drawback of NRI method is the "degenerate" problem which leads the decoder to ignore the relation edge during training.

Original: So we train the network to that two random characteristics; intention and interaction to have different role.
Revised: So we train the network to that two features, intention and interaction, to have different roles.

Original: Because the GT trajectory is conditioned on the GT goal feature, wedirectly use the GT goal feature to compute the reconstruction term.
Revised: Because the GT trajectory is conditioned on the GT goal feature, we use the GT goal feature to compute the reconstruction term.

Original: Thus, training in this way restricts the role of interaction edge to be narrowed down to momentary motion: L recon =
Revised: Therefore, training in that way restricts the role of interaction edge to be narrowed down to momentary motion: L recon =

Original: min eee { E [ log ( p ( Y | X ,eee,τττ + )] } .
Revised: min eee { E [ log ( p ( Y | X ,eee,τττ + )] } .

Original: The final loss is the sum of the three losses, and they are trained jointly:
Revised: The final loss is the sum of the three losses, and they are trained jointly:

Original: L =
Revised: L =

Original: L nll + L KL + L recon .
Revised: L nll + L KL + L recon .

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original:  E XPERIMENT AND R ESULT
Revised:  E XPERIMENT AND R ESULT

Original: We train and evaluate on nuScenes ( Caesar et al.
Revised: We train and evaluate on nuScenes ( Caesar et al.

Original: and Argoverse ( Chang et al.
Revised: and Argoverse ( Chang et al.

Original: datasets.
Revised: datasets.

Original: The nuScenes/Argoverse datasets provide thepast 2/2 seconds of observation, and 6/ seconds of future trajectory is predicted with 0.5/0.1 sampling time.
Revised: The nuScenes/Argoverse datasets provide the 2/2 seconds of past, and 6/3 seconds of future trajectory is predicted with 0.5/0.1 sampling time.

Original: Training/validation/test sets consist of real-world driving scenes of 32186/8560/9041 in nuScenes and 205942/39472/78143 in Argoverse.
Revised: Training/validation/test sets consist of real-world driving scenes of 32186/8560/9041 in nuScenes and 205942/39472/78143 in Argoverse.

Original: For a baseline model, we follow TNT for goal conditioned model and MHA is used to encode interaction from past trajectories.
Revised: For the baseline model, we follow TNT for goal conditioned model, and MHA encodes interaction from past trajectories.

Original: For implementation and training details, refer to the supplementary material.
Revised: For implementation and training details, refer to the supplementary material.

Original: 5. Q UANTITATIVE RESULT
Revised: 5. Q UANTITATIVE RESULT

Original: Our method outperforms the SoTA models in all metrics used in  nuScenes benchmark, as shown in Tab. 1.
Revised: Our method outperforms the SoTA models in all metrics used in the nuScenes benchmark, as shown in Tab. 1.

Original: The proposed model outperforms the runner-up method, PGP ( Deo et al.
Revised: The proposed model outperforms the runner-up method, PGP ( Deo et al.

Original: (2022)), by a large margin.
Revised: (2022)), by a large margin.

Original: The superiorityin validity implies that explicit interaction modeling via inferring waypoint occupancy helps scene understanding of our model compared to the implicit interaction modeling of PGP.
Revised: The superiority implies that explicit interaction modeling via inferring waypoint occupancy helps scene understanding of our model compared to the implicit interaction modeling of PGP.

Original: When more samples were predicted, mADE 10 and MR 10 showed improvements of 5.3% and 8.8%.
Revised: When more samples are predicted, mADE 10 and MR 10 showed improvements of 5.3% and 8.8%.

Original: THOMAS ( Gilles et al.
Revised: THOMAS ( Gilles et al.

Original: (2022b)) had first placed in mFDE 1 by proposing a recombination module that post-processes the marginal predictions into the joint predictions aware of other agents.
Revised: (2022b)) had first placed in mFDE 1 by proposing a recombination module that post-processes the marginal predictions into the joint predictions aware of other agents.

Original: Our model shows higher performance  thanthat of THOMAS, which indicates better interaction modeling ability withoutthe post-processing.
Revised: Our model shows higher performance in mFDE 1 than THOMAS, indicating better interaction modeling ability without post-processing.

Original: This is possible because inferring future relationship helps to understand the future interaction with other agents; details are provided in the ablation study.
Revised: This is possible because inferring future relationships helps to understand the future interaction with other agents; details are provided in the ablation study.

Original: We also validated our method on  Argoverse dataset.
Revised: We also validated our method on the Argoverse dataset.

Original: The proposed Future Relationship Module improved prediction performance compared to baseline in mADE 6 /mADE 1 by 4.2%/5.7%. Our method made competitive performance, but it did not achieve SoTA in Argoverse dataset.
Revised: Our model does not achieve SotA performance but still makes remarkable performance improvement in both valid/test sets. Moreover, except for the HiVT, our method make competitive performance in mADE.

Original: We [Table]
Revised: Please note that our model (0.82) is still comparable to SceneTransformer (0.80) and Multipath++ (0.79) in the test set results. In the case of

Original: [Table] [Table] [Table]
Revised: HiVT, it uses the surrounding vehicles’ trajectories for training. It results in increasing the training data, so a direct comparion to HiVT is rather unfair.

Original: [Table]
Revised: We do not achieve SotA in Argoverse because the proposed method is less effective than in nuScenes.

Original:  conclude that the relatively small performance gain on Argoverse compared to nuScenes comes from the different configurations of datasets.
Revised: We conclude that the relatively small performance gain on Argoverse compared to nuScenes comes from the different configurations of datasets.

Original: A model should predict the 6-second future in nuScenes while it only needs to predict the 3-second future in Argoverse.
Revised: A model should predict the 6-second future in nuScenes while it only needs to predict the 3-second future in Argoverse.

Original: Intuitively, the impact of interaction gets larger in long-time prediction task.
Revised: Intuitively, the impact of interaction gets more significant in long-range prediction task.

Original: We thus measure the performance gain on nuScenes when the prediction horizon is the same as that of Argoverse and report the results in Tab. 
Revised: We thus measure the performance gain on nuScenes when the prediction horizon is the same as that of Argoverse and report the results in Tab. 3.

Original: In 6-second prediction, the proposed interaction inference method improved mADE 1 by more than 10%, but its effect halves in a 3-second prediction which is a similar effect on Argoverse.
Revised: In 6-second prediction, the proposed interaction inference method improved mADE 1 by more than 10%, but its effect halves in a 3-second prediction which is a similar effect on Argoverse.

Original: This result shows that our interaction inference method is more effective in longer prediction tasks.
Revised: This result shows that our interaction inference method is more effective in longer-range prediction tasks.

Original: 5. Q UALITATIVE RESULT
Revised: 5. Q UALITATIVE RESULT

Original: In Fig. 6, prediction samples (F=6) of  baseline and our method are depicted.
Revised: In Fig. 6, prediction samples (F=6) of the baseline and our method are depicted.

Original: To simply check the effect of our method, we took the samples where two agents are in andthe plotted the prediction of a single agent per scene.
Revised: To simply check the effect of our method, we took the samples where two agents are in and plotted the prediction of a single agent per scene.

Original: Green, blue and red solid lines represent past trajectories, future trajectories of other agents and prediction samples of target agents.
Revised: Green, blue and red solid lines represent past trajectories, future trajectories of other agents, and prediction samples of target agents.

Original: In two scenes, each target agent sets its intention to where the other agent is likely to pass in the future.
Revised: In two scenes, each target agent sets its intention to where the other agent is likely to pass in the future.

Original: The prediction samples from our model (right) generate samples that understand and leverage the interaction with other agent.
Revised: The prediction samples from our model (right) generate samples that understand and leverage the interaction with other agents.

Original: Unlike the baseline method that ignores other agent and generates spatially uniform trajectories, our model considers other agents to surpass or wait  them.
Revised: Unlike the baseline method that ignores other agents and generates spatially uniform trajectories, our model considers other agents to surpass or wait for them.

Original: Moreover, not only considering multi-modal interaction, we allow stochasticity within a single mode of interaction.
Revised: Moreover, not only considering multi-modal interaction, we allow stochasticity within a single mode of interaction.

Original: As a result, it can be seen that our method generates diverse yet interaction aware samples.
Revised: As a result, it can be seen that our method generates diverse yet interaction-aware samples.

Original: In addition, our method can consider stochastic interaction when multiple agents are present.
Revised: In addition, our method can consider stochastic interaction when multiple agents are present.

Original: Similarly, we predicted  trajectories of a single agent per scene, and we fixed the intention feature of the target vehicle.
Revised: Similarly, we predicted the trajectories of a single agent per scene, and we fixed the intention feature of the target vehicle.

Original: With fixed intention, two relation edges are sampled, and corresponding predicted trajectories and degree of relation ( ∥ zzz e ∥ ) are plotted.
Revised: Two relation edges are sampled with fixed intention, and corresponding predicted trajectories and degree of relation ( ∥ zzz e ∥ ) are plotted.

Original: In the first row, target agent (0) infers large interaction with agent 2 in sample 1.
Revised: In the first row, target agent (0) infers large interaction with agent 2 in sample 1.

Original: Because agent 2 is moving in the same direction and  predicted to go ahead, the model generates an accelerating trajectory to follow agent 2.
Revised: Because agent 2 is moving in the same direction and is predicted to go ahead, the model generates an accelerating trajectory to follow agent 2.

Original: In sample 2, in contrast, the interaction between agent 1 is sampled large because they are expected to be in the same intersection.
Revised: In sample 2, in contrast, the interaction between agent 1 is sampled large because they are expected to be in the same intersection.

Original: In this case, the model generates decelerating trajectory considering  future motion of agent 1.
Revised: In this case, the model generates decelerating trajectory considering the future motion of agent 1.

Original: In addition, with fixed intention, it shows that all generated trajectories are properly on goal lane segments.
Revised: In addition, with fixed intention, it shows that all generated trajectories are properly on goal lane segments.

Original: It means that the proposed training method effectively restricts the role of interaction feature affecting momentary motion as we defined.
Revised: It means that the proposed training method effectively restricts the role of interaction features affecting momentary motion, as we defined.

Original: 5. A BLATION STUDY
Revised: 5. A BLATION STUDY

Original: Ablation on the impact of model design is shown in the upper part in Tab. 4.
Revised: Ablation on the impact of model design is shown in the upper part in Tab. 4.

Original: Ours w/o FR does not consider Future Relationship in interaction modeling.
Revised: Ours w/o FR does not consider Future Relationship in interaction modeling.

Original: It only utilizes past trajectories to infer the interaction and does not predict waypoint occupancy.
Revised: This model only utilizes past trajectories to infer the relation and does not predict waypoint occupancy.

Original: Its performance is almost  the same as the Baseline which uses MHA of past trajectories to model interaction.
Revised: After all, its performance is almost identical to the baseline, which uses MHA of past trajectories to model interaction.

Original: This result shows that leveraging Future Relationship indeed helps to infer interaction.
Revised: This result shows that leveraging Future Relationship indeed helps to infer interaction.

Original: Ours w/o GCN does not smooth predicted/GT waypoint occupancy, which deteriorates performance as it leads to insufficient lane features and causes inaccurate inter-agent proximity (PR) estimation.
Revised: Ours w/o GCN does not smooth predicted/GT waypoint occupancy. It causes inaccurate inter-agent proximity (PR) estimation especially in the posterior distribution. Because GT waypoint occupancy is a binary value, computing PR from it could cause inaccurate proximity and brings performance decline.

Original: In the proposed model, we allow asymmetric interaction between two agents.
Revised: In the proposed model, we allow asymmetric relation between two agents.

Original: Ours w/
Revised: Ours w/

Original: Sym applies hard symmetric interaction ( Li et al.
Revised: Sym applies hard symmetric relation ( Li et al.

Original: , and it shows that our asymmetric design is more suitable to model the driver interaction.
Revised: , and it shows that our asymmetric design is more suitable for modeling the driver relation.

Original: The importance of multimodal stochastic interaction modeling is shown in the lower part of Tab. 4.
Revised: The importance of multi-modal stochastic relation modeling is shown in the lower part of Tab. 4.

Original: Ours w/ GP models the prior distribution of interaction as Gaussian distribution instead of GM.
Revised: Ours w/ GP models the prior distribution of relation as Gaussian distribution instead of GM.

Original: As it only considers a single modality of agent interaction, the performance drops compared to when considering multiple modalities.
Revised: As it only considers a single modality of agent relation, the performance drops compared to when considering multiple modalities.

Original: Ours w/ Deterministic only predict mean of interaction feature in Eq.4.
Revised: Ours w/ Deterministic only predicts the mean of relation feature in Eq.

Original: Although it is able to model multi-modal interaction, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.
Revised: Although it can model multi-modal relation, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.

Original: The result also shows  deterministic interaction modeling severely degrades prediction performance when F is large.
Revised: The result also shows that deterministic relation modeling severely degrades prediction performance when F is large.

Original: On the other hand, Ours w/ GP shows relatively less drop in performance as it does not lose stochasticity even after removing the GM prior.
Revised: On the other hand, Ours w/ GP shows relatively less drop in performance as it does not lose stochasticity even after removing the GM prior.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose Future Relationship to learn the interaction between vehicles for trajectory prediction effectively.
Revised: In this paper, we propose Future Relationship to learn the interaction between vehicles for trajectory prediction effectively.

Original: Beyond utilizing only past trajectory to infer interaction, Future Relationship has advantages in explicitly using lane information.
Revised: Beyond utilizing only past trajectory to infer interaction, Future Relationship has advantages in explicitly using lane information.

Original: We show thatusing lane information enables proper interaction inference even in complex road structures.
Revised: We show that lane information enables proper interaction inference even in complex road structures.

Original: In addition, we propose the model that obtains interaction in a probabilistic manner, and generates diverse yet socially plausible trajectory samples.
Revised: In addition, we propose the model that obtains interaction probabilistically, generating diverse yet socially plausible trajectory samples.

Original:  The proposed method  brings remarkable performance improvement in popular real-world trajectory prediction datasets: nuScenes and Argoverse. 
Revised: We trained our model with the CVAE scheme and validated in popular real-world trajectory prediction datasets. The proposed method achieves SotA performance in long-range prediction task, nuScnes, and brings remarkable performance improvement in short-range prediction task, Argoverse. Because considering Future Relationship is novel, we expect that using more sophisticated training methods (Ye et al.

Revised: ; Zhou et al.

Revised: or better baseline model (such as GANet (Wang et al.

Revised: (2022)) will also lead to performance improvements in our model.