Original: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION
Revised: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Understanding the interaction between multiple agents is crucial for realistic and plausible vehicle trajectory prediction.
Revised: Understanding the interaction between multiple agents is crucial for realistic and plausible vehicle trajectory prediction.

Original: Accordingly, existing methods tried to model and predict the interaction using observed past trajectories of agents with pooling, attention, or graph-based methods.
Revised: Accordingly, existing methods tried to model and predict the interaction using observed past trajectories of agents with pooling, attention, or graph-based methods.

Original: However, we observed that they easily fail under complex road structures.
Revised: However, we observed that they easily fail under complex road structures.

Original: It is because they do not explicitly utilize the map information for predicting the relationship, and they only model the relationship between vehicles in a deterministic manner, not in a stochastic manner.
Revised: It is because they do not explicitly utilize the map information for predicting the relationship, and they only model the relationship between vehicles in a deterministic manner, not in a stochastic manner.

Original: In this paper, we propose a new method to model a stochastic future relationship among agents using the lane information contained in the map.
Revised: In this paper, we propose a new method to model a stochastic future relationship among agents using the lane information contained in the map.

Original: Our method first predicts a probability of lane-level waypoint occupancy of vehicles.
Revised: Our method first predicts a probability of lane-level waypoint occupancy of vehicles.

Original: Then the proposed method utilizes the temporal probability of passing the adjacent lanes to learn the interaction between agents.
Revised: Then the proposed method utilizes the temporal probability of passing the adjacent lanes to learn the interaction between agents.

Original: In addition, we model the interaction using probabilistic distribution.
Revised: In addition, we model the interaction employing probabilistic distribution.

Original: This distribution is learned by the posterior distribution of interaction from GT future trajectory.
Revised: This distribution is learned by the posterior distribution of interaction from GT future trajectory.

Original: We validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.
Revised: As a result, we validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.

Original: The code will be available in public upon acceptance.
Revised: The code will be available in public upon acceptance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: For safe autonomous driving, predicting the future trajectory of a vehicle is very important.
Revised: For safe autonomous driving, predicting the future trajectory of a vehicle is very important.

Original: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Zeng et al.
Revised: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Zeng et al.

Original: Rhinehart et al.
Revised: Rhinehart et al.

Original: Recently, thanks to deep learning, more accurate prediction can be made by considering not only its own state but also the relationship with the High-Definition (HD) map ( Liang et al.
Revised: Recently, thanks to deep learning, more accurate prediction can be made by considering not only its own state but also the relationship with the High-Definition (HD) map ( Liang et al.

Original: ; Zeng et al.
Revised: ; Zeng et al.

Original: or surrounding agents ( Lee et al.
Revised: or surrounding agents ( Lee et al.

Original: ; Chandra et al.
Revised: ; Chandra et al.

Original: Since surrounding vehicles are not stationary, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.
Revised: Since surrounding vehicles are not stationary, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.

Original: In addition, since an individual driver controls each vehicle, its interaction has stochastic nature.
Revised: In addition, since an individual driver controls each vehicle, its interaction has stochastic nature.

Original: Previous works utilized past trajectories of the surrounding vehicles to model interaction by pooling, operating multi-head attention, or adapting spatio-temporal graph methods.
Revised: Previous works utilized past trajectories of the surrounding vehicles to model interaction by pooling, operating multi-head attention, or adapting spatio-temporal graph methods.

Original: However, we observed that they easily fail under complex road structures.
Revised: However, we observed that they easily fail under complex road structures.

Original: On the left of Fig. 1, the past trajectories of agents are shown, and on the right is a plot showing the attention weight among agents obtained by the previous method (Mercat et al.
Revised: On the left of Fig. 1, the past trajectories of agents are shown. The plot on the right shows the attention weight among agents obtained by the previous method (Mercat et al.

Original: that learned the interaction among agents using multi-head attention (MHA).
Revised: that learned the interaction among agents using multi-head attention (MHA).

Original: Since agent 0 and agent 4 are expected to join in the future, the attention weight between the two is expected to be high.
Revised: Since agent 0 and agent 4 are expected to join in the future, the attention weight between the two is expected to be high.

Original: However, the model predicts low attention weight between them.
Revised: However, the model predicts low attention weight between them.

Original: It shows the difficulty of reasoning future relationships between agents based only on past trajectories.
Revised: It shows the difficulty of reasoning future relationships between agents based only on past trajectories.

Original: The reasoning should have been much easier if it leveraged the road structure.
Revised: The reasoning should have been much easier if it leveraged the road structure.

Original: First, let us recall the decision-making process of human drivers.
Revised: Let us recall the decision-making process of human drivers.

Original: They first set their goal and roughly infer how the surrounding vehicles will behave in the future while they head to their own goal.
Revised: They first set their goal and roughly infer how the surrounding vehicles will behave in the future .

Original: After that, they infer the interaction with other agents by inferring how likely the future path of other vehicles will overlap the path set by themselves.
Revised: After that, they infer the interaction with other agents by inferring how likely the future path of other vehicles will overlap the path set by themselves.

Original:  We define the interaction from this process as a " Future Relationship " to mimic the decision-making process of human drivers.
Revised: The drivers consider interaction more significant the more the future paths of other vehicles overlap with their own. We define the interaction from this process as a " Future Relationship ".

Original: We use the following approaches to model and train the Future Relationship, as shown in Fig. 3.
Revised: We use the following approaches to model and train the Future Relationship, as shown in Fig. 3.

Original: First, because two vehicles in adjacent locations and headings will have much influence on each other, a future motion of each vehicle should be obtained.
Revised: First, we obtain the rough future motion of each vehicle.

Original: Since vehicles mainly move along lanes, we utilize lane information as strong prior for representing the rough future motion of vehicles.
Revised: Since vehicles mainly move along lanes, we utilize lane information as strong prior for representing the rough future motion of vehicles.

Original: Because 
Revised: Because lane centerlines contain both positional and directional information, rough future motion can be

Original: [Figure]
Revised: [Figure]

Revised: represented as waypoint occupancy.

Revised: The waypoint occupancy is the probability of a vehicle passing a specific lane segment at every timestep.

Revised: In the middle of Fig. 3, each agent’s waypoint occupancy is shown.

Revised: Here, we aggregated the temporal axis for simplification.

Original: shows the waypoint the  passes  lane during the prediction horizon is drawn using the tone of red 
Revised: The probability that the vehicle passes that lane during the prediction horizon is drawn using the tone of red color.

Original: Second, based on the waypoint occupancy we infer the Future Relationship in probabilistic distribution.
Revised: Second, based on the waypoint occupancy, we infer the Future Relationship in probabilistic distribution.

Original: In most vehicle trajectory prediction tasks the interaction between agents is still made in a deterministic manner.
Revised: In most vehicle trajectory prediction methods, the interaction between agents is still made in a deterministic manner.

Original: However, we tackle that interaction between vehicles is highly stochastic, and there can be multiple possible interactions.
Revised: However, we tackle that interaction between vehicles is highly stochastic, and there can be multiple possible interactions.

Original: The deterministic relation inference averages out diverse interactions, interrupting socially-aware trajectory prediction.
Revised: The deterministic relation inference averages out diverse interactions, interrupting socially-aware trajectory prediction.

Original: Therefore, we define Future
Revised: Therefore, we define Future

Original: Relationship in Gaussian Mixture (GM) distribution.
Revised: Relationship in Gaussian Mixture (GM) distribution.

Original: Motivated by Neural Relational Inference (NRI) ( Kipf et al.
Revised: Motivated by Neural Relational Inference (NRI) ( Kipf et al.

Original:  we propose a method to train the diverse interaction distribution explicitly.
Revised: , we propose a method to train the diverse interaction distribution explicitly.

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: 1) We propose to infer Future Relationship using lane information, assuming that vehicles generally follow the lane.
Revised: 1) We propose to infer Future Relationship using lane information, assuming that vehicles generally follow the lane.

Original: We propose the model to infer the distribution of interaction as  Gaussian Mixture and propose a new learning method to make the model learn the distribution from the posterior. 3)
Revised: We propose the model to infer the distribution of interaction as a Gaussian Mixture and propose a new learning method to make the model learn the distribution from the posterior. 3)

Original: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.
Revised: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.

Original: In both datasets, there was a remarkable improvement in prediction performance and achieved state-of-the-art performance in nuScenes dataset.
Revised: In both datasets, there was a remarkable improvement in prediction performance and achieved state-of-the -art performance in nuScenes dataset.

Original:  [Equation] W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION
Revised:  R ELATED W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION

Original: Predicting future trajectories at once is a challenging task.
Revised: Predicting future trajectories at once is a challenging task.

Original: Instead, thefirst goal-conditional prediction, which oversamples goal candidates and selects a final set of goals, is helpful and has shown stateof-the-art performance ( Zhao et al.
Revised: Instead, the goal-conditional prediction, which oversamples goal candidates and selects a final set of goals, is helpful and has shown state-ofthe-art performance, especially in long-range prediction tasks ( Zhao et al.

Original: ; Gu et al.
Revised: ; Gu et al.

Original: (2021); Phan-Minh et al.  Chai et al.
Revised: Phan-Minh et al. (2020); Chai et al.

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: (2021)), especially in long-range prediction tasks.

Original: CoverNet ( Phan-Minh et al.
Revised: CoverNet ( Phan-Minh et al.

Original: (2020)) and MultiPath ( Chai et al.
Revised: (2020)) and MultiPath ( Chai et al.

Original: (2019)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is not considered.
Revised: (2019)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is not considered.

Original: Recent studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.
Revised: Recent studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.

Original: TNT ( Zhao et al.
Revised: TNT ( Zhao et al.

Original: (2021)) uses goal points sampled from a lane centerline; GoalNet ( Zhang et al.
Revised: (2021)) uses goal points sampled from a lane centerline; GoalNet ( Zhang et al.

Original: (2021)) uses lane segments as anchors and predicts multiple trajectories based on each anchor.
Revised: (2021)) uses lane segments as anchors and predicts multiple trajectories based on each anchor.

Original: Similarly, we set lane segmentswith their connectivity with nearby segments as goal candidates to model the inherent uncertainty of trajectory prediction.
Revised: Similarly, we set lane segments as goal candidates to model the inherent uncertainty of trajectory prediction.

Original: The relationship between lane segments, such as predecessor and successor, provides global connectivity, enabling the modelto predict realistic and plausible trajectories.
Revised: The relationship between lane segments, such as predecessor and successor, provides global connectivity and helps the model predict realistic and plausible goal candidates.

Original: 2. I NTERACTION MODELING
Revised: 2. I NTERACTION MODELING

Original: Trajectory prediction considering the interaction between agents helps to predict  socially aware trajectory.
Revised: Considering the interaction between agents helps to predict a socially aware trajectory.

Original: In the very early stage,modeling interaction using a global pooling layer ( Deo & Trivedi (2018); Gupta et al.(2018)) were proposed.
Revised: In the very early stage, interaction is obtained by pooling interaction features in the local region ( Deo & Trivedi (2018); Gupta et al.

Original: In another way, researchers attempt to obtain interaction through attention-based ( Ngiam et al.
Revised: In other way, researchers attempted to obtain interaction through attention-based ( Ngiam et al.

Original: ; Mercat et al.
Revised: ; Mercat et al.

Original: Vemula et al.
Revised: Vemula et al.

Original: and GNNbased method ( Carrasco et al.
Revised: or GNN-based method ( Carrasco et al.

Original: ; Cao et al.
Revised: ; Cao et al.

Original: Zeng et al.
Revised: Zeng et al.

Original: ; Casas et al.
Revised: ; Casas et al.

Original: ; Liang et al.
Revised: ; Liang et al.

Original: ; Gao et al.
Revised: ; Gao et al.

Original: In the previous method, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.
Revised: However, in most previous methods, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.

Original: Besides, a line of works using Neural Relational Inference (NRI)  that explicitly predicts and learn interaction using a latent interaction graph was proposed ( Kipf et al.
Revised: Besides, a line of works employs Neural Relational Inference (NRI) ( Kipf et al. that explicitly predicts and learns interaction using a latent interaction graph.

Original: EvolveGraph ( Li et al.
Revised: EvolveGraph ( Li et al.

Original: (2020)) utilizes two interaction graphs, static and dynamic, and
Revised: (2020)) utilizes two interaction graphs, static and dynamic, and

Original: NRI-MPM ( Chen et al.
Revised: NRI-MPM ( Chen et al.

Original: (2021)) uses a relation interaction mechanism and spatio-temporal message passing mechanism.
Revised: (2021)) uses a relation interaction mechanism and spatio-temporal message passing mechanism.

Original: Similarly, we apply the NRI-based method to predict future trajectory explicitly in complex road structures.
Revised: Similarly, we apply the NRI-based method to predict future trajectory explicitly in complex road structures.

Original: 2. M ULTI - MODAL TRAJECTORY PREDICTION
Revised: 2. M ULTI - MODAL TRAJECTORY PREDICTION

Original: Due to the nature of the trajectory prediction; stochasticity, there is no unique answer, but several futures are possible.
Revised: Due to the nature of the trajectory prediction; stochasticity, there is no unique answer, but several futures are possible.

Original: Therefore, recent works use deep generative models such as GAN (Goodfellow et al.
Revised: Therefore, recent works use deep generative models such as GAN (Goodfellow et al.

Original: and VAE (Kingma & Welling (2013)).
Revised: and VAE (Kingma & Welling (2013)).

Original: GAN-based (Gupta et al.
Revised: GAN-based (Gupta et al.

Original: ; Kosaraju et al.
Revised: ; Kosaraju et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2021b))
Revised: (2021b))

Original: and CVAE-based models (Ivanovic & Pavone (2019); Salzmann et al.
Revised: and CVAE-based models (Ivanovic & Pavone (2019); Salzmann et al.

Original: ; Tang & Salakhutdinov (2019)) predict multiple futures by sampling multiple latent vectors.
Revised: ; Tang & Salakhutdinov (2019)) predict multiple futures by sampling multiple latent vectors.

Original: As it is important to sample a meaningful latent vector for predicting diverse and plausible future trajectories, trying to define a well-organized latent space became a natural choice ( Ma et al.
Revised: As it is important to sample a meaningful latent vector for predicting diverse and plausible future trajectories, trying to define a well-organized latent space became a natural choice ( Ma et al.

Original: ; Bae et al. (2022)).
Revised: ; Bae et al. (2022)).

Original: The most closely related work to ours is GRIN (Li et al.
Revised: The most closely related work to ours is GRIN (Li et al.

Original: (2021a)), which argues that multi-modality in trajectory prediction comes from two sources: i) personal intention and ii) social relations with other agents.
Revised: (2021a)), which argues that multi-modality in trajectory prediction comes from two sources: i) personal intention and ii) social relations with other agents.

Original: GRIN only uses the past trajectories of agents to model the latent space of personal intention and social relations.
Revised: GRIN only uses the past trajectories of agents to model the latent space of personal intention and social relations.

Original: However, our model propose to utilize lane information for modeling social relations based on the occupancy.
Revised: However, our model proposes to utilize lane information for modeling social relations based on the occupancy.

Original: Leveraging the future occupancy makes the model interpretable and easy to be trained since the pseudo labels of occupancy can be obtained from ground-truth trajectories.
Revised: Leveraging the future occupancy makes the model interpretable and easy to be trained since the pseudo labels of occupancy can be obtained from ground-truth trajectories.

Original: Also, we sample the personal intention of a driver from a set of lane segments feature by following the standard goal-conditioned method (Zhao et al.
Revised: Also, we sample the personal intention of a driver from a set of lane segments feature by following the standard goal-conditioned method (Zhao et al.

Original:  F ORMULATION
Revised:  F ORMULATION

Original: In each scene, the observed past trajectory and GT future trajectory of N vehicles are denoted as x − = { x } 1: N − t p :0 and x + = { x } 1: N 1: t f  where t p + 1 and t f are past/future timesteps.
Revised: In each scene, the observed past trajectory and GT future trajectory of N vehicles are denoted as x − = { x } 1: N − t p :0 and x + = { x } 1: N 1: t f , where t p + 1 and t f are past/future timesteps.

Original: We use lane information from the HD map, expressed as M segmented lane polylines ( Zhao et al.
Revised: We use lane information from the HD map, expressed as M segmented lane polylines ( Zhao et al.

Original: Varadarajan et al.
Revised: Varadarajan et al.

Original: The lane information is represented as a graph GGG =
Revised: The lane information is represented as a graph GGG =

Original: ( ℓℓℓ, e ) where the node ℓℓℓ = { ℓ } 1: M is the set of directed sequences of xy coordinates.
Revised: ( ℓℓℓ, e ) where the node ℓℓℓ = { ℓ } 1: M is the set of directed sequences of xy coordinates.

Original: The segmented lane nodes are shown in Fig.

Original: 2, where different colors denote different individual segments ( ℓ m ).

Original: The edge is defined as five types:
Revised: The edge is defined as five types:

Original: e = { e pre , e suc , e left , e right , e inter } 1: M which are predecessor, successor, left/right neighbor and in-the-same-intersection.
Revised: e = { e pre , e suc , e left , e right , e inter } 1: M which are predecessor, successor, left/right neighbor and in-the-same-intersection.

Revised: The segmented lane nodes are shown in Fig.

Revised: 2, where different colors denote different individual segments ( ℓ m ).

Original: We denote overall input as X = { x − , G G G} .
Revised: We denote overall input as X = { x − , G G G} .

Original: The output is denoted as Y , which is the set of F samples of xy coordinates of the target vehicles.
Revised: The output is denoted as Y , which is the set of F samples of xy coordinates of the target vehicles.

Original: We also define the future lane occupancy (waypoint occupancy) denoted as τττ t =
Revised: We also define the future lane occupancy (waypoint occupancy) denoted as τττ t =

Original: (cid:81) N,Mn,m p nt ( ℓ m ) , where p nt ( ℓ m ) is the probability of the m -th lane segment being occupied by the n -th vehicle at future time step t .
Revised: (cid:81) N,Mn,m p nt ( ℓ m ) , where p nt ( ℓ m ) is the probability of the m -th lane segment being occupied by the n -th vehicle at future time step t .

Original: The waypoint occupancy during future timestemps ( t f ) predicted from X is denoted by τττ − 1: t f  and GT waypoint occupancy obtained from GT future trajectory is denoted by τττ +1:
Revised: The waypoint occupancy during future timestamps ( t f ) predicted from X is denoted by τττ − 1: t f , and GT waypoint occupancy obtained from GT future trajectory is denoted by τττ +1:

Original: t f .
Revised: t f .

Original: [Figure]
Revised: [Figure]

Original:  𝐱 
Revised: ( 𝐱 [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: ( 𝐡 % )
Revised: ( 𝐡 % )

Original: F Future Trajectories ( 𝐘 )
Revised: F Future Trajectories ( 𝐘 )

Original: IntentionFeature ( 𝐡 & ) x !,# x !,$ ···
Revised: IntentionFeature ( 𝐡 & ) x !,# x !,$ ···

Original: [Figure]
Revised: [Figure]

Original: ( 𝐱
Revised: ( 𝐱

Original: [Figure]
Revised: [Figure]

Original: x
Revised: x

Original: [Figure]
Revised: [Figure]

Original: Waypoint Occupancy ( 𝛕 ’:) ! )
Revised: Waypoint Occupancy ( 𝛕 ’:) ! )

Original: 𝑎𝑔𝑒𝑛𝑡 𝑖 0 𝑡 " ··· i j 𝑧 #$% 𝑧 &’( c x %
Revised: 𝑎𝑔𝑒𝑛𝑡 𝑖 0 𝑡 " ··· i j 𝑧 #$% 𝑧 &’( c x %

Original: [Figure]
Revised: [Figure]

Original: $ ··· 𝑎𝑔𝑒𝑛𝑡 𝑗 0 𝑡 " ···
Revised: $ ··· 𝑎𝑔𝑒𝑛𝑡 𝑗 0 𝑡 " ···

Original: [Figure]
Revised: [Figure]

Original: 𝑗 𝑎𝑔𝑒𝑛𝑡 𝑖
Revised: 𝑗 𝑎𝑔𝑒𝑛𝑡 𝑖

Original: Decoder
Revised: Decoder

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original:  M ETHOD
Revised:  M ETHOD

Original: We focus on modeling the "Future Relationship" between agents.
Revised: We focus on modeling the "Future Relationship" between agents.

Original: The naive method to infer the
Revised: The naive method to infer the

Original: Future Relationship is predicting all future vehicle trajectories and then using the similarity among the trajectories.
Revised: Future Relationship is predicting all future vehicle trajectories and then using the similarity among the trajectories.

Original: However, this method needs to perform prediction twice, which is inefficient and redundant.
Revised: However, this method needs to perform prediction twice, which is inefficient and redundant.

Original: Also, the criteria for calculating the similarity between trajectories are unclear.
Revised: Also, the criteria for calculating the similarity between trajectories could be unclear.

Original: Inspired by the idea that the vehicles mainly follow lanes, we predict the interaction between vehicles using the lane information as a strong prior of interaction modeling.
Revised: Inspired by the idea that the vehicles mainly follow lanes, we employ lane information for modeling the Future Relationship.

Original: The key idea is that if two agents are expected to pass an adjacent lane, they will have a high chance of interacting in the future .
Revised: The key idea is that if two vehicles are expected to pass an adjacent lane, they will have a high chance of interacting in the future .

Original: The overall structure is shown in Fig.
Revised: The overall structure is shown in Fig.

Original: First, we predict the waypoint occupancy, the probability of the lane segmentwhere the vehicle will pass during the future time step 1: t f (Sec. 4.1).
Revised: First, we predict the waypoint occupancy, the probability of a vehicle passing a specific lane segment during the future time steps 1: t f (Sec. 4.1).

Original: Then, from each vehicle’s waypoint occupancy, the FRM infers interaction features between agents as Gaussian
Revised: Then, from each vehicle’s waypoint occupancy, the FRM infers interaction edges between agents as Gaussian

Original: Mixture (Sec. 4.2).
Revised: Mixture (Sec. 4.2).

Original: Next, a message passing through inferred interaction features is performed to obtain interaction feature.
Revised: Next, a message passing through inferred interaction edges is performed to obtain interaction feature.

Original: Finally, in the decoding stage (Sec. 4.3), the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature.
Revised: Finally, in the decoding stage (Sec. 4.3), the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature.

Original: Here, the intention feature is the concatenation of past motion and goal lane features.
Revised: Here, the intention feature is the concatenation of past motion and goal lane features.

Original: Our method follows the CVAE scheme (Sec. 4.2.1, 4.2.2), where the condition corresponds to the intention feature, and the latent code corresponds to the interaction feature.
Revised: Our method follows the CVAE scheme (Sec. 4.2.1, 4.2.2), where the condition corresponds to the intention feature, and the latent code corresponds to the interaction feature.

Original: We describe the details in the following sections.
Revised: We describe the details in the following sections.

Original: 4. W AYPOINT O CCUPANCY
Revised: 4. W AYPOINT O CCUPANCY

Original: Past trajectories x − and lane graph GGG are encoded into past motion and lane features: hhh − x and hhh ℓ .
Revised: Past trajectories x − and lane graph GGG are encoded into past motion and lane features: hhh − x and hhh ℓ .

Original: The same motion encoder is used to obtain future motion features hhh + x .
Revised: The same motion encoder is used to obtain future motion features hhh + x .

Original: Similar to TNT ( Zhao et al.
Revised: Similar to TNT ( Zhao et al.

Original: , we predict waypoint occupancy in categorical distribution from past motion and lane features.
Revised: , we predict waypoint occupancy in categorical distribution from past motion and lane features.

Original: [Equation]
Revised: [Equation]

Original: where [ · , · ] is concatenation.
Revised: where [ · , · ] is concatenation.

Original: As waypoint occupancy is a probability, we apply a softmax so that it becomes (cid:80) M
Revised: As waypoint occupancy is a probability, we apply a softmax so that it becomes (cid:80) M

Original: τ mt
Revised: τ mt

Original: = 1 .
Revised: = 1 .

Original: Then goal feature ( hhh − g ) is sampled from hhh ℓ following waypoint occupancy at t f ( τττ − t f ).
Revised: Then, goal feature ( hhh − g ) is sampled from hhh ℓ following waypoint occupancy at t f ( τττ − t f ).

Original: The concatenation of hhh − g and hhh − x is interpreted as the agents’ intentions to reach, representing the intention feature ( h h h − I ).
Revised: The concatenation of hhh − g and hhh − x is interpreted as the agents’ intentions to reach, representing the intention feature ( h h h − I ).

Original: Besides, τ τ τ − 1: t f − 1 is a temporal feature containing the probabilistic future motion, so it is passed to FRM.
Revised: Besides, τ τ τ − 1: t f − 1 is a temporal feature containing the probabilistic future motion, so it is passed to FRM.

Original: For GT waypoint occupancy ( τττ +1: t f ), we can obtain it directly from GT future trajectory.
Revised: For GT waypoint occupancy ( τττ +1: t f ), we can obtain it directly from GT future trajectory.

Original: We can get binary lane occupancy of vehicles as we know the position and heading of vehicles at future timesteps.
Revised: We can get binary lane occupancy of vehicles as we know the position and heading of vehicles at future timesteps.

Original: The details about getting GT waypoint occupancy are described in the supplementary material.
Revised: The details about getting GT waypoint occupancy are described in the supplementary material.

Original: Similarly, the unique GT goal feature( hhh + g ) also can be obtained by indexing from hhh ℓ .
Revised: Similarly, the unique GT goal feature ( hhh + g ) also can be obtained by indexing from hhh ℓ .

Original: 4. F UTURE R ELATIONSHIP M ODULE (FRM)
Revised: 4. F UTURE R ELATIONSHIP M ODULE (FRM)

Original: Attaining Future Relationship is depicted in Fig.
Revised: Attaining Future Relationship is depicted in Fig.

Original: At first, waypoint occupancy is smoothed by
Revised: At first, waypoint occupancy is smoothed by

Original: Graph Convolutional Network (GCN) ( Welling & Kipf (2016)).
Revised: Graph Convolutional Network (GCN) ( Welling & Kipf (2016)).

Original: It is because when a vehicle passes a specific lane, the vehicle will affect other vehicles that pass the adjacent lane, not necessarily the same lane.
Revised: It is because when a vehicle passes a specific lane, the vehicle will affect other vehicles that pass the adjacent lane, not necessarily the same lane.

Original: Therefore, we apply different smoothing for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employing 2-hop GCN layers.
Revised: Therefore, we apply different smoothing for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employing 2-hop GCN layers.

Original: Each layer is expressed as:
Revised: Each layer is expressed as:

Original: [Equation]
Revised: [Equation]

Original: Here, σ is softmax followed by ReLU activation, D e ∈ R M × M is a degree matrix, A e ∈ R M × M is an adjacency matrix, and W e ∈ R t f − 1× d is a weight matrix for each edge type.
Revised: Here, σ is softmax followed by ReLU activation, D e ∈ R M × M is a degree matrix, A e ∈ R M × M is an adjacency matrix, and W e ∈ R t f − 1 is a weight matrix for each edge type.

Original: Next, with the obtained waypoint occupancy, we compute the inter-agent proximity (PR = { pr ij } 1: N, 1: N ) between vehicle pairs which is the temporal probability that two agents would occupy adjacent lanes.
Revised: Next, with the obtained waypoint occupancy, we compute the inter-agent proximity (PR = { pr ij } 1: N, 1: N ) between vehicle pairs which is the temporal probability that two agents would occupy adjacent lanes.

Original: The PR can be obtained with a cross product of ˜ τττ 1: t f − 1 across the lane axis.
Revised: The PR can be obtained with a dot product of ˜ τττ 1: t f − 1 across the lane axis.

Original: It is visually drawn in the middle of Fig.
Revised: It is visually drawn in the middle of Fig.

Original: [Equation]
Revised: [Equation]

Original: 4.2. P RIOR OF THE R ELATION
Revised: 4.2. P RIOR OF THE R ELATION

Original: Our method follows  CVAE scheme as mentioned above, the prior distribution of relation should be computed.
Revised: Our method follows the CVAE scheme, so the prior distribution of interaction edge should be computed.

Original: We utilize past motion features ( hhh − x ) and predicted inter-agent proximity (PR − ) to obtain the prior.
Revised: We utilize past motion features ( hhh − x ) and predicted inter-agent proximity (PR − ) to obtain the prior.

Original: In this subsection, we omit superscript - for simplification.
Revised: In this subsection, we omit superscript - for simplification.

Original: There are two design factors for our interaction modeling: i) interaction should reflect diverse and stochastic properties, and ii) interaction occurs in every pair of vehicles.
Revised: There are two design factors for our interaction modeling: i) interaction should reflect diverse and stochastic properties, and ii) interaction occurs in every pair of vehicles.

Original: As a result, the prior of relation is assumed to be Gaussian
Revised: Consequently, the prior of relation is defined as Gaussian

Original: Mixture (GM) per agent pairs.
Revised: Mixture (GM) per agent pair.

Original: Then, we define interaction edge e ij between agent i and agent j as a d-dimensional feature that has GM distribution ( p θ ( eee | X ) ∼ (cid:81)Kk =1 πππ k N [Equation]
Revised: Then, we define interaction edge e ij between agent i and agent j as a d-dimensional feature that has GM distribution ( p θ ( eee | X ) ∼ (cid:81)

Original:  µµµ k , [Equation] I I σσσ 2 k ) ) following GMVAE ( Dilokthanakul et al.
Revised: Kk =1 πππ k N ( µµµ k , III σσσ 2 k ) ) following GMVAE ( Dilokthanakul et al.

Original: The distribution parameters ( µµµ K ,σσσ K ,πππ
Revised: The distribution parameters ( µµµ K ,σσσ K ,πππ

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  N, 1: N ) are obtained from the neural network F θ :
Revised: : N, 1: N ) are obtained from the neural network F θ :

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: F θ is composed of MLP layers and 1-d conv layer to capture temporal dependency (Deo & Trivedi (2018)).
Revised: F θ is composed of MLP layers and 1-d conv layer to capture temporal dependency (Deo & Trivedi (2018)).

Original: Then, we perform sampling twice: one for the interaction mode k (from πππ K ) and one for ϵ (from Gaussian noise).
Revised: Then, we perform sampling twice: one for the interaction mode k (from πππ K ) and one for ϵ (from Gaussian noise).

Original: It allows having K distinct interactions where each interaction follows
Revised: It allows having K distinct interactions where each interaction follows

Original: Gaussian distribution:
Revised: Gaussian distribution:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We then compute interaction feature ( hhh − R
Revised: We then compute interaction feature ( hhh − R

Original: = { h iR } 1: N ) by message passing from sampled interaction edge.
Revised: = { h iR } 1: N ) by message passing from sampled interaction edge.

Original: Because we define interaction as a d -dimensional feature, the message function is composed of summation followed by Hadamard product ( ⊗ ) of interaction edge and the other agent features:
Revised: Because we define interaction as a d -dimensional feature, the message function is composed of summation followed by Hadamard product ( ⊗ ) of interaction edge and the other agent features:

Original: [Equation]
Revised: [Equation]

Original: Stochasticity is obtained by giving flexibility within a limited range by predicted mean and standard deviation.
Revised: Stochasticity is obtained by giving flexibility within a limited range by predicted mean and standard deviation.

Original: It is because, unlike a symmetric and deterministic interaction between particles, such as pull or push ( Kuramoto (1975)), each driver can asymmetrically consider stochastic interaction.
Revised: It is because, unlike a symmetric and deterministic interaction between particles, such as pull or push ( Kuramoto (1975)), each driver can asymmetrically consider stochastic interaction.

Original: 4.2. CVAE POSTERIOR
Revised: 4.2. CVAE POSTERIOR

Original: To obtain the posterior distribution of relation, the future motion feature ( hhh + x ) and GT waypoint occupancy ( τττ + ) are used.
Revised: To obtain the posterior distribution of relation, the future motion feature ( hhh + x ) and GT waypoint occupancy ( τττ + ) are used.

Original: Similarly, we omit superscript + in this subsection for simplification.
Revised: Similarly, we omit superscript + in this subsection for simplification.

Original: Inter-agent proximity is obtained by following same procedure in Eqs.
Revised: Inter-agent proximity is obtained by following same procedure in Eqs.

Original: The difference from the prior is that the posterior is modeled in a single Gaussian distribution.
Revised: The difference from the prior is that the posterior is modeled in a single Gaussian distribution.

Original: Thus, F θ is replaced with
Revised: Thus, F θ is replaced with

Original: F ϕ which outputs parameters of Gaussian distribution ( 
Revised: F ϕ which outputs parameters of Gaussian distribution ( µ µ µ,σ σ σ = { µ ij , σ ij } 1: N, 1: N ):

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then we sample ϵϵϵ ∼ N (000 , III ) , and interaction edge is obtained:
Revised: Then we sample ϵϵϵ ∼ N (000 , I I I ) , and interaction edge is obtained:

Original: z e z e z e = µµµ + σσσϵϵϵ .
Revised: z e z e z e = µµµ + σσσϵϵϵ .

Original: Following the same message passing, interaction features ( hhh + R ) is obtained.
Revised: Following the same message passing, interaction features ( hhh + R ) is obtained.

Original: 4. D ECODER
Revised: 4. D ECODER

Original: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).
Revised: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).

Original: GT goal feature ( hhh + g ) is concatenated to the past motion feature ( hhh − x ) to get a unique GT intention feature.
Revised: GT goal feature ( hhh + g ) is concatenated to the past motion feature ( hhh − x ) to get a unique GT intention feature.

Original: The GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times from the posterior distribution.
Revised: The GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times from the posterior distribution.

Original: During inference, the goal feature ( hhh − g ) is sampled F times.
Revised: During inference, the goal feature ( hhh − g ) is sampled F times.

Original: The intention feature is obtained by concatenation of sampled goal features and the past motion feature ( hhh − x ).
Revised: The intention feature is obtained by concatenation of sampled goal features and the past motion feature ( hhh − x ).

Original: The interaction feature ( hhh − R ) is sampled F times from prior distribution.
Revised: The interaction feature ( hhh − R ) is sampled F times from prior distribution.

Original: The decoder is modeled by 2-layer MLP and predicts the sequence of x,y coordinates from the concatenation of the intention and interaction feature.
Revised: The decoder is modeled by 2-layer MLP and predicts the sequence of x,y coordinates from the concatenation of the intention and interaction feature.

Original: 4. T RAINING
Revised: 4. T RAINING

Original: Because the ground ( τττ + ) is available for waypoint occupancy, we train τττ − with supervision.
Revised: Because the GT ( τττ + ) is available for waypoint occupancy, we train τττ − with supervision.

Original: Since the waypoint occupancy is modeled as a classification per agent, we learn it using negative log-likelihood (NLL): L nll = − τττ
Revised: Since the waypoint occupancy is modeled as a classification per agent, we learn it using negative log-likelihood (NLL [Equation]

Original: [-+
 t f log ( τττ − 1: t f ) .-]
Revised: [-+
 t f log ( τττ − 1: t f ) .-][Equation]

Original: In contrast, because interaction  zzz e is unobservable, we optimize the negative evidence lower bound (ELBO) to learn the distribution of interaction following the CVAE scheme.
Revised: In contrast, because interaction edge zzz e is unobservable, we optimize the negative evidence lower bound (ELBO) to learn the distribution of interaction following the CVAE scheme.

Original: [Equation]
Revised: [Equation]

Original: where q π is an approximate posterior distribution of relation, and p θ is the prior.
Revised: where q π is an approximate posterior distribution of relation, and p θ is the prior.

Original: Because our model only allows interaction posterior to be Gaussian distribution, the Kullback–Leibler (KL) divergence term can be simplified below.
Revised: Because our model only allows interaction posterior to be Gaussian distribution, the Kullback–Leibler (KL) divergence term can be simplified below.

Original: The detailed derivation with the reparameterization trick is included in the supplementary material.
Revised: The detailed derivation with the reparameterization trick is included in the supplementary material.

Original: [Equation]
Revised: [Equation]

Original: The common drawback of NRI method is the "degenerate" problem which leads the decoder to ignore the relation edge during training.
Revised: The common drawback of NRI method is the "degenerate" problem which leads the decoder to ignore the relation edge during training.

Original: So we train the network to that two features, intention and interaction, to have different roles.
Revised: So we train the network to that two features, intention and interaction, to have different roles.

Original: Because the GT trajectory is conditioned on the GT goal feature, we use the GT goal feature to compute the reconstruction term.
Revised: Because the GT trajectory is conditioned on the GT goal feature, we use the GT goal feature to compute the reconstruction term.

Original: Therefore, training in that way restricts the role of interaction edge to be narrowed down to momentary motion: L recon =
Revised: Therefore, training in that way restricts the role of interaction edge to be narrowed down to momentary motion: L recon =

Original: min eee { E [ log ( p ( Y | X ,eee,τττ + )] } .
Revised: min eee { E [ log ( p ( Y | X ,eee,τττ + )] } .

Original: The final loss is the sum of the three losses, and they are trained jointly:
Revised: The overall loss is the sum of the three losses, and they are trained jointly:

Original: L =
Revised: L =

Original: L nll + L KL + L recon .
Revised: L nll + L KL + L recon .

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original:  E XPERIMENT AND R ESULT
Revised:  E XPERIMENT AND R ESULT

Original: We train and evaluate  on  nuScenes ( Caesar et al.
Revised: We train and evaluate our method on two popular real-world trajectory dataset: nuScenes ( Caesar et al.

Original: and Argoverse ( Chang et al.
Revised: and Argoverse ( Chang et al.

Original: datasets. The nuScenes/Argoverse datasets provide the 2/2 seconds of past, and 6/3 seconds of future trajectory is predicted with 0.5/0.1 sampling time.
Revised: In nuScenes/Argoverse datasets, the 2/2 seconds of past is provided, and 6/3 seconds of future trajectory is predicted with 0.5/0.1 sampling time.

Original: Training/validation/test sets consist of real-world driving scenes of 32186/8560/9041 in nuScenes and 205942/39472/78143 in Argoverse.
Revised: Training/validation/test sets consist of real-world driving scenes of 32186/8560/9041 in nuScenes and 205942/39472/78143 in Argoverse.

Original: For the baseline model, we follow TNT for goal conditioned model, and MHA encodes interaction from past trajectories.
Revised: For the baseline model, we follow TNT for goal conditioned model, and MHA encodes interaction from past trajectories.

Original: For implementation and training details, refer to the supplementary material.
Revised: For implementation and training details, refer to the supplementary material.

Original: 5. Q UANTITATIVE RESULT
Revised: 5. Q UANTITATIVE RESULT

Original: Our method outperforms the SoTA models in all metrics used in the nuScenes benchmark, as shown in Tab. 1.
Revised: Our method outperforms the SoTA models in all metrics used in the nuScenes benchmark, as shown in Tab. 1.

Original: The proposed model outperforms the runner-up method, PGP ( Deo et al.
Revised: The proposed model outperforms the runner-up method, PGP ( Deo et al.

Original: (2022)), by a large margin.
Revised: (2022)), by a large margin.

Original: The superiority implies that explicit interaction modeling via inferring waypoint occupancy helps scene understanding of our model compared to the implicit interaction modeling of PGP.
Revised: The superiority implies that explicit interaction modeling via inferring waypoint occupancy helps scene understanding of our model compared to the implicit interaction modeling of PGP.

Original: When more samples are predicted, mADE10 and MR10 showed improvements of 5.3% and 8.8%. 
Revised: When 10 samples are predicted, mADE and MR showed improvements of 5.3% and 8.8%. Previously,

Original: THOMAS ( Gilles et al.
Revised: THOMAS ( Gilles et al.

Original: (2022b)) had first placed in mFDE 1 by proposing a recombination module that post-processes the marginal predictions into the joint predictions aware of other agents.
Revised: (2022b)) had first placed in mFDE 1 by proposing a recombination module that post-processes the marginal predictions into the joint predictions aware of other agents.

Original: Our model shows higher performance in mFDE 1 than THOMAS, indicating better interaction modeling ability without post-processing.
Revised: Our model shows higher performance in mFDE 1 than THOMAS, indicating better interaction modeling ability without post-processing.

Original: This is possible because inferring future relationships helps to understand the future interaction with other agents; details are provided in the ablation study.
Revised: This is possible because inferring future relationships helps to understand the future interaction with other agents; details are provided in the ablation study.

Original: We also validated our method on the Argoverse dataset.
Revised: We also validated our method on the Argoverse dataset.

Original: Our model does not achieve SotA performance but still makes remarkable performance improvement in both valid/test sets.
Revised: Our model does not achieve SotA performance but still makes remarkable performance improvement in both valid/test sets.

Original: Moreover, except for the
Revised: Moreover, except for the

Original: HiVT, our method make competitive performance in mADE.
Revised: HiVT, our method make competitive performance in mADE.

Original: Please note that our model (0.82) is still comparable to SceneTransformer (0.80) and Multipath++ (0.79) in the test set results.
Revised: Please note that our model (0.82) is still comparable to SceneTransformer (0.80) and Multipath++ (0.79) in the test set results.

Original: In the case of
Revised: In the case of

Original: HiVT, it uses the surrounding vehicles’ trajectories for training.
Revised: HiVT, it uses the surrounding vehicles’ trajectories for training.

Original: It results in increasing the training data, so a direct comparion to HiVT is rather unfair.
Revised: It results in increasing the training data, so a direct comparion to HiVT is rather unfair.

Original: We do not achieve SotA in Argoverse because the proposed method is less effective than in nuScenes.
Revised: We do not achieve SotA in Argoverse because the proposed method is less effective than in nuScenes.

Original: We conclude that the relatively small performance gain on Argoverse compared to nuScenes comes from the different configurations of datasets.
Revised: We conclude that the relatively small performance gain on Argoverse compared to nuScenes comes from the different configuration of datasets.

Original: A model should predict the 6-second future in nuScenes while it only needs to predict the 3-second future in Argoverse.
Revised: A model should predict the 6-second future in nuScenes while it only needs to predict the 3-second future in Argoverse.

Original: Intuitively, the impact of interaction gets more significant in long-range prediction task.
Revised: Intuitively, the impact of interaction gets more significant in long-range prediction task.

Original: We thus measure the performance gain on nuScenes when the prediction horizon is the same as that of Argoverse and report the results in Tab. 3.
Revised: We thus measure the performance gain on nuScenes when the prediction horizon is the same as that of Argoverse and report the results in Tab. 3.

Original: In 6-second prediction, the proposed interaction inference method improved mADE 1 by more than 10%, but its effect halves in a 3-second prediction which is a similar effect on Argoverse.
Revised: In 6-second prediction, the proposed interaction inference method improved mADE 1 by more than 10%, but its effect halves in a 3-second prediction which is a similar effect on Argoverse.

Original: This result shows that our interaction inference method is more effective in longer-range prediction tasks. 
Revised: This result shows that our interaction inference method is more effective in longer-range prediction tasks. [Table] [Table]

Original: 5. Q UALITATIVE RESULT
Revised: 5. Q UALITATIVE RESULT

Original: In Fig.6, prediction samples (F=6) of the baseline and our method are depicted.
Revised: In Fig. prediction samples (F=6) of the baseline and our method are depicted.

Original: To simply check the effect of our method, we took the samples wheretwo agents are in and plotted the prediction of a single agent per scene. Green, blue and red solid lines represent past trajectories, future trajectories of other agents, and prediction samples of target agents. In two scenes, each target agent sets its intention to where the other agent is likely to pass in the future. The prediction samples from our model (right) generate samples that understand and leverage the interaction with other agents.
Revised: To simply check the effect of our method, we took the samples where

Original: Unlike the baseline method that ignores other agents and generates spatially uniform trajectories, our model considers other agents to surpass or wait for them.
Revised: Unlike the baseline method that ignores other agents and generates spatially uniform trajectories, our model considers other agents to surpass or wait for them.

Original: Moreover, not only considering multi-modal interaction, we allow stochasticity within a single mode of interaction.
Revised: Moreover, not only considering multi-modal interaction, we allow stochasticity within a single mode of interaction.

Original: As a result, it can be seen that our method generates diverse yet interaction-aware samples.
Revised: As a result, it can be seen that our method generates diverse yet interaction-aware samples.

Original: In addition, our method can consider stochastic interaction when multiple agents are present.
Revised: In addition, our method can consider stochastic interaction when multiple agents are present.

Original: Similarly, we predicted the trajectories ofa single agent per scene, and we fixed the  intentionfeature of the target vehicle.
Revised: In the experiment of Fig. 7, we predict the trajectories of the target agents (denoted as 0) with multiple interacting vehicles. In each scene, intention of the target vehicle is fixed (denoted in green) and two relation edges are sampled.

Original: Two relation edges are sampled with fixed intention, and corresponding predicted trajectories and degree of relation ( ∥ zzz e ∥ ) are plotted.
Revised: The corresponding predicted trajectory samples and degree of relation ( ∥ zzz e ∥ ) are plotted on the right.

Original: In the first row, target agent (0) infers large interaction with agent 2 in sample 1.
Revised: In the first row, target agent (0) infers large interaction with agent 2 in sample 1.

Original: Because agent 2 is moving in the same direction and is predicted to go ahead, the model generates an accelerating trajectory to follow agent 2.
Revised: Because agent 2 is moving in the same direction and is predicted to go ahead, the model generates an accelerating trajectory to follow agent 2.

Original: In sample 2, in contrast, the interaction between agent 1 is sampled large because they are expected to be in the same intersection.
Revised: In sample 2, in contrast, the interaction between agent 1 is sampled large because they are expected to be in the same intersection.

Original: In this case, the model generates decelerating trajectory considering the future motion of agent 1.
Revised: In this case, the model generates decelerating trajectory considering the future motion of agent 1.

Original: In addition, with  fixed intention, it shows that all generated trajectories are properly on goal lane segments.
Revised: Meanwhile, with the fixed intention, it shows that all predicted trajectories are properly on goal lane segments.

Original: It means that the proposed training method effectively restricts the role of interaction features affecting momentary motion, as we defined.
Revised: It means that the proposed training method effectively restricts the role of interaction features affecting momentary motion, as we defined.

Original: 5. A BLATION STUDY
Revised: 5. A BLATION STUDY

Original: Ablation on the impact of model design is shown in the upper part in Tab. 4.
Revised: Ablation on the impact of model design is shown in the upper part in Tab. 4.

Original: Ours w/o FR does not consider Future Relationship in interaction modeling.
Revised: Ours w/o FR does not consider Future Relationship in interaction modeling.

Original: This model only utilizes past trajectories to infer the relation and does not predict waypoint occupancy.
Revised: This model only utilizes past trajectories to infer the relation, thus similar to the NRI.

Original: After all, its performance is almost identical to the baseline, which uses MHA of past trajectories to model interaction.
Revised: After all, its performance is almost identical to the baseline, which uses MHA of past trajectories to model interaction.

Original: This result shows that leveraging Future Relationship indeed helps to infer interaction.
Revised: This result shows that leveraging Future Relationship indeed helps to infer interaction.

Original: Ours w/o GCN does not smooth predicted/GT waypoint occupancy.
Revised: Ours w/o GCN does not smooth predicted/GT waypoint occupancy.

Original: It causes inaccurate inter-agent proximity (PR) estimation especially in the posterior distribution.
Revised: It causes inaccurate inter-agent proximity (PR) estimation especially in the posterior distribution.

Original: Because GT waypoint occupancy is a binary value, computing PR from it could cause inaccurate proximity and brings performance decline.
Revised: Because GT waypoint occupancy is a binary value, computing PR from it could cause inaccurate proximity and brings performance decline.

Original: In the proposed model, we allow asymmetric relation between two agents.
Revised: In the proposed model, we allow asymmetric relation between two agents.

Original: Ours w/
Revised: Ours w/

Original: Sym applies hard symmetric relation ( Li et al.
Revised: Sym applies hard symmetric relation ( Li et al.

Original: , and it shows that our asymmetric design is more suitable for modeling the driver relation.
Revised: , and it shows that our asymmetric design is more suitable for modeling the driver relation.

Original: The importance of multi-modal stochastic relation modeling is shown in the lower part of Tab. 4.
Revised: The importance of multi-modal stochastic relation modeling is shown in the lower part of Tab. 4.

Original: Ours w/ GP models the prior distribution of relation as Gaussian distribution instead of GM.
Revised: Ours w/ GP models the prior distribution of relation as Gaussian distribution instead of GM.

Original: As it only considers a single modality of agent relation, the performance drops compared to when considering multiple modalities.
Revised: As it only considers a single modality of agent relation, the performance drops compared to when considering multiple modalities.

Original: Ours w/ Deterministic only predicts the mean of relation feature in Eq.
Revised: Ours w/ Deterministic only predicts the mean of relation feature in Eq.

Original: Although it can model multi-modal relation, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.
Revised: Although it can model multi-modal relation, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.

Original: The result also shows that deterministic relation modeling severely degrades prediction performance when F is large.
Revised: The result also shows that deterministic relation modeling severely degrades prediction performance when F is large.

Original: On the other hand, Ours w/ GP shows relatively less drop in performance as it does not lose stochasticity even after removing the GM prior.
Revised: On the other hand, Ours w/ GP shows relatively less drop in performance as it does not lose stochasticity even after removing the GM prior.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose Future Relationship to learn the interaction between vehicles for trajectory prediction effectively.
Revised: In this paper, we propose Future Relationship to learn the interaction between vehicles for trajectory prediction effectively.

Original: Beyond utilizing only past trajectory to infer interaction, Future Relationship has advantages in explicitly using lane information.
Revised: Beyond utilizing only past trajectory to infer interaction, Future Relationship has advantages in explicitly using lane information.

Original: We show that lane information enables proper interaction inference even in complex road structures.
Revised: We show that lane information enables proper interaction inference even in complex road structures.

Original: In addition, we propose the model that obtains interaction probabilistically, generating diverse yet socially plausible trajectory samples.
Revised: In addition, we propose the model that obtains interaction probabilistically, generating diverse yet socially plausible trajectory samples.

Revised: The proposed method also provides explainable medium (waypoint occupancy or inter-agent proximity) compared to other prediction models use totally random latent vectors.

Original: We trained our model with the CVAE scheme and validated in popular real-world trajectory prediction datasets.
Revised: We trained our model with the CVAE scheme and validated in popular real-world trajectory prediction datasets.

Original: The proposed method achieves SotA performance in long-range prediction task, nuScnes, and brings remarkable performance improvement in short-range prediction task, Argoverse.
Revised: The proposed method achieves SotA performance in long-range prediction task, nuScnes, and brings remarkable performance improvement in short-range prediction task, Argoverse.

Original: Because considering 
Revised: Because considering Future

Original: Future Relationship is novel, we expect that using more sophisticated training methods (Ye et al.
Revised: Relationship is novel, we expect that using more sophisticated training methods (Ye et al.

Original: ; Zhou et al.
Revised: Zhou et al.

Original: or better baseline model (such as GANet (Wang et al.
Revised: or better baseline model (such as GANet (Wang et al.

Original: (2022)) will also lead to performance improvements in our model.
Revised: (2022)) will also lead to performance improvements in our model.