Original: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION
Revised: L EVERAGING F UTURE R ELATIONSHIP R EASONING FOR V EHICLE T RAJECTORY P REDICTION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Understanding the interaction between multiple agents is crucial for realisticand plausible vehicle trajectory prediction.
Revised: Understanding the interaction between multiple agents is crucial for realistic vehicle trajectory prediction.

Original: Accordingly, existing methods tried to model and predict the interaction using observed past trajectories of agents with pooling, attention, or graph-based methods.
Revised: Existing methods have attempted to infer the interaction from the observed past trajectories of agents using pooling, attention, or graph-based methods, which rely on a deterministic approach.

Original: However, we observed that they easily fail under complex road structures. It is because they do not explicitly utilize the map information for predicting the relationship, and they only model the relationship between vehicles in a deterministic manner, not in a stochastic manner.
Revised: However, these methods can fail under complex road structures, as they cannot predict various interactions that may occur in the future.

Original: In this paper, we propose a new method to model a stochastic future relationship among agents using the lane information contained in the map.
Revised: In this paper, we propose a novel approach that uses lane information to predict a stochastic future relationship among agents.

Original: Our method first predicts a probability of lane-level waypoint occupancy of vehicles.
Revised: To obtain a coarse future motion of agents, our method first predicts the probability of lanelevel waypoint occupancy of vehicles.

Original: Then the proposed method utilizes the temporal probability of passingthe adjacent lanes to learn the interaction between agents.
Revised: We then utilize the temporal probability of passing adjacent lanes for each agent pair, assuming that agents passing adjacent lanes will highly interact.

Original: In addition, we model the interaction employing probabilistic distribution.
Revised: We also model the interaction using a probabilistic distribution, which allows for multiple possible future interactions.

Original: This distribution is learned by the posterior distribution of interaction  from GT future trajectory.
Revised: The distribution is learned from the posterior distribution of interaction obtained from ground truth future trajectories.

Original: As a result, we validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.
Revised: We validate our method on popular trajectory prediction datasets: nuScenes and Argoverse.

Original: The code will be available in public upon acceptance.
Revised: The results show that the proposed method brings remarkable performance gain in prediction accuracy, and achieves state-ofthe-art performance in long-term prediction benchmark dataset.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: For safe autonomous driving, predicting the future trajectoryof a vehicle is very important.
Revised: For safe autonomous driving, predicting a vehicle’s future trajectory is crucial.

Original: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Zeng et al.
Revised: Early heuristic prediction models utilized only the past trajectory of the target vehicle ( Lin et al.

Original: Rhinehart et al.
Revised: ; Barth & Franke (2008)).

Original: Recently, thanks to deep learning, more accurate prediction can be made byconsidering not only its own state but also  the  relationship with the High-Definition (HD) map ( Liang et al.
Revised: However, with the advent of deep learning, more accurate predictions can be made by also considering the vehicle’s relationship with the High-Definition (HD) map ( Liang et al.

Original: ; Zeng et al.
Revised: ; Zeng et al.

Original: or surrounding agents ( Lee et al.
Revised: or surrounding agents ( Lee et al.

Original: ; Chandra et al.
Revised: ; Chandra et al.

Original: Since surrounding vehicles are not stationary, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.
Revised: Since surrounding vehicles are not stationary, predicting relationships with them is much more complicated and has become essential for realistic trajectory prediction.

Original: In addition, sincean individual driver controls each vehicle, its interaction has  stochastic nature.
Revised: Furthermore, since individual drivers control each vehicle, their interaction has a stochastic nature.

Original: Previous works utilized past trajectories of the surrounding vehiclesto model interaction by  pooling,operating multi-head attention, oradapting spatio-temporal graph methods.
Revised: Previous works modeled interaction from past trajectories of the surrounding vehicles by employing pooling, multi-head attention, or spatio-temporal graph methods.

Original: However, we observed that they easily fail under complex road structures.
Revised: However, we observed that these methods easily fail under complex road structures.

Original: On the left of Fig. 1, the past trajectories of agents are shown. The plot on the right shows the attention weight among agents  obtained by the previous method (Mercat et al.
Revised: For example, Fig. 1 shows the past trajectories of agents (left) and the attention weights among agents (right) obtained by a previous method (Mercat et al.

Original: that learned the interaction among agents using multi-head attention (MHA).
Revised: that learned the interaction among agents using multi-head attention (MHA).

Original: Since agent 0 andagent 4 are expected to join in the future, the attention weight between the two is expected to be high.
Revised: Since agents 0 and 4 are expected to join in the future, the attention weight between them should be high.

Original: However, the model predicts  low attention weight between them. It shows the difficulty of reasoning future relationships between agents based only on past trajectories.
Revised: However, the model predicts a low attention weight between them, highlighting the difficulty of reasoning future relationships between agents based solely on past trajectories.

Original: The reasoning should have been much easier if it leveraged the road structure.
Revised: Incorporating the road structure should make the reasoning process much easier.

Original: Let us recall the decision-making process of human drivers.
Revised: The decision-making process of human drivers can provide insights on how to model interaction.

Original: They first set their goal and roughly infer how the surrounding vehicles will behave in the future .
Revised: They first set their goal where they are trying to reach on the map. Next, to infer the interaction with surrounding agents, they roughly infer how the others will behave in the future .

Original: After that, they infer the interaction with other agents by inferring how likely the future path of other vehicles will overlap the path set by themselves.
Revised: After that, they infer the interaction with others by inferring how likely the future path of other vehicles will overlap the path set by themselves.

Original: The drivers consider interaction more significant the more the future paths of other vehicles overlap with their own.
Revised: The drivers consider interaction more significant the more the future paths of other vehicles overlap with their own.

Original: We define the interaction from this process as a " Future Relationship ".
Revised: We define the interaction from this process as a " Future Relationship ".

Original: We use the following approaches to modeland train the Future Relationship, as shown in Fig. 3. 
Revised: We use the following approaches to model Future Relationship, as shown in Fig. 3. [Figure]

Original: First, we obtain the rough future motion of each vehicle.
Revised: First, we obtain the rough future motion of all vehicles in the scene.

Original: Since vehicles mainly move along lanes, we utilize lane information as strong prior for representing the rough future motion of vehicles.
Revised: Since vehicles mainly move along lanes, we utilize lane information as strong prior for representing the rough future motion of vehicles.

Original: Because lane centerlines contain both positional and directional information, rough future motion can be[Figure] represented as waypoint occupancy.
Revised: Because lane centerlines contain both positional and directional information, rough future motion can be represented as waypoint occupancy.

Original: The waypoint occupancy is  the probability of a vehicle passing a specific lane segment at every  timestep.
Revised: The waypoint occupancy is defined as the probability of a vehicle passing a specific lane segment at every intermediate timestep.

Original: In the middle of Fig. 3, each agent’s waypoint occupancy is shown.
Revised: In the middle of Fig. 3, each agent’s waypoint occupancy is shown.

Original: Here, we aggregated the temporal axis for simplification.
Revised: Here, we aggregated the temporal axis for simplification.

Original: The probability that the vehicle passes that lane during the prediction horizon is drawn using the tone of red color.
Revised: The probability that the vehicle passes that lane during the prediction horizon is drawn using the tone of red color.

Original: Second, based on the waypoint occupancy, we infer the Future Relationship in probabilistic distribution.
Revised: Second, based on the waypoint occupancy we infer the Future Relationship in probabilistic distribution.

Original: In most vehicle trajectory prediction methods, the interaction between agents is still made in a deterministic manner.
Revised: In most vehicle trajectory prediction methods the interaction between agents is still made in a deterministic manner.

Original: However, we tackle that interaction between vehicles is highly stochastic, and there can be multiple possible interactions.
Revised: However, we take note that interaction between vehicles is highly stochastic, and there can be multiple possible interactions.

Original: The deterministic relation inference averages out diverse interactions, interrupting socially-aware trajectory prediction.
Revised: The deterministic relation inference averages out diverse interactions, interrupting socially-aware trajectory prediction.

Original: Therefore, we define Future
Revised: Therefore, we define Future

Original: Relationship in Gaussian Mixture (GM) distribution.
Revised: Relationship in Gaussian Mixture (GM) distribution.

Original: Motivated by Neural Relational Inference (NRI) ( Kipf et al.
Revised: Motivated by Neural Relational Inference (NRI) ( Kipf et al.

Original: , we propose a method to train the diverse interaction distribution explicitly.
Revised: , we propose a method to train the diverse interaction distribution explicitly.

Original: In summary, our contributions are:
Revised: In summary, our contributions are:

Original: 1) We propose to infer Future Relationship using lane information, assuming that vehicles generally follow the lane. We proposethe model to infer the  distribution of interaction as a Gaussian Mixture and propose a new learning method to make the model learn the distribution from the posterior. 3)
Revised: 1) We propose a new approach for modeling the interaction between vehicles by incorporating the road structure and defining it as Future Relationship . We propose to infer the Future Relationship in probabilistic distribution using Gaussian Mixture (GM) distribution to capture diverse interaction.

Original: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.
Revised: The proposed method is validated on popular real-world vehicle trajectory datasets: nuScenes and Argoverse.

Original: In both datasets, there was a remarkable improvement in prediction performance and achieved state-of-the -art performance  in nuScenes dataset.
Revised: In both datasets, there is a remarkable improvement in prediction performance and state-of-the-art performance is achieved in the long-range prediction dataset, nuScenes.

Original:  R ELATED W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION
Revised:  R ELATED W ORK 2. G OAL - CONDITIONED TRAJECTORY PREDICTION

Original: Predicting future trajectories at once is a challenging task.
Revised: Predicting future trajectories at once is a challenging task.

Original: Instead,the goal-conditional prediction, which oversamples goal candidates and selects a final set of goals, is helpful and has shown state-ofthe-art performance, especially in long-range prediction tasks ( Zhao et al.
Revised: Instead, goal-conditional prediction, which samples goal candidates and then predicts trajectory conditioned on them, is helpful and has shown state-of-the-art performance, especially in long-range prediction tasks ( Zhao et al.

Original: ; Gu et al.
Revised: ; Gu et al.

Original:  Phan-Minh et al.
Revised: (2021); Phan-Minh et al.

Original: (2020); Chai et al.
Revised: (2020); Chai et al.

Original: ; Zhang et al.
Revised: ; Zhang et al.

Original: CoverNet ( Phan-Minh et al.
Revised: CoverNet ( Phan-Minh et al.

Original: (2020)) and MultiPath ( Chai et al.
Revised: (2020)) and MultiPath ( Chai et al.

Original: (2019)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is not considered.
Revised: (2020)), which quantize the trajectory space to a set of anchors, often generate map-agnostic trajectories that cross non-drivable areas because the surrounding map is not considered.

Original: Recent studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.
Revised: Recently, several studies have exploited the map information to obtain more performant goal candidates based on the assumption that vehicles follow lanes.

Original: TNT ( Zhao et al.
Revised: TNT ( Zhao et al.

Original: (2021)) uses goal points sampled from a lane centerline; GoalNet ( Zhang et al.
Revised: (2021)) uses goal points sampled from a lane centerline, and GoalNet ( Zhang et al.

Original: (2021)) uses lane segments as anchors and predicts multiple trajectories based on each anchor.
Revised: (2021)) uses lane segments as trajectory anchors.

Original: Similarly, we set lane segments as goal candidates to model the  inherent uncertainty of trajectory prediction. The relationship between lane segments, such as predecessor and successor, provides global connectivity and helps the model predict realistic and plausible goal candidates.
Revised: However, while previous methods assume that the likelihood of arriving at a final destination is random, they assume that trajectories are unimodal in order to reach a specific goal area. In this paper, we assume inherent uncertainty in which trajectories can vary due to the interactions with surrounding vehicles in order to reach a specific goal area.

Original: 2. I NTERACTION MODELING
Revised: 2. I NTERACTION MODELING

Original: Considering the interaction between agents helps to predict a socially aware trajectory.
Revised: Considering the interaction between agents helps to predict a socially aware trajectory.

Original: In the very early stage, interaction is obtained by pooling interaction features in the local region ( Deo & Trivedi (2018); Gupta et al.
Revised: In the very early stage, interaction is obtained by pooling interaction features in the local region ( Deo & Trivedi (2018); Gupta et al.

Original: In other way, researchers attempted to obtain interaction through attention-based ( Ngiam et al.
Revised: In other works, researchers attempted to obtain interaction through attention-based ( Ngiam et al.

Original: ; Mercat et al.
Revised: ; Mercat et al.

Original: Vemula et al.
Revised: Vemula et al.

Original: or GNN-based method ( Carrasco et al.
Revised: or GNN-based method ( Carrasco et al.

Original: ; Cao et al.
Revised: ; Cao et al.

Original: Zeng et al.
Revised: Zeng et al.

Original: ; Casas et al.
Revised: ; Casas et al.

Original: ; Liang et al.
Revised: ; Liang et al.

Original: ; Gao et al.
Revised: ; Gao et al.

Original: However, in most previous methods, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.
Revised: However, in most previous methods, interactions between agents are learned only with regression loss, which is insufficient to represent dynamic and rapidly changing situations.

Original: Besides, a line of works  employs Neural Relational Inference (NRI) ( Kipf et al.
Revised: There exists a line of works that employs Neural Relational Inference (NRI) ( Kipf et al.

Original:  that explicitly predicts and learns interaction using a latent interaction graph.
Revised: (2018)) that explicitly predicts and learns interaction using a latent interaction graph.

Original: EvolveGraph
Revised: EvolveGraph

Original: ( Li et al.
Revised: ( Li et al.

Original: (2020)) utilizes two interaction graphs, static and dynamic, and
Revised: (2020)) utilizes two interaction graphs, static and dynamic, and

Original: NRI-MPM ( Chen et al.
Revised: NRI-MPM ( Chen et al.

Original: (2021)) uses a relation interaction mechanism and spatio-temporal message passing mechanism.
Revised: (2021)) uses a relation interaction mechanism and spatio-temporal message passing mechanism.

Original: Similarly, we apply the NRI-based method to predict future trajectory explicitly in complex road structures.
Revised: Similarly, we apply the NRI-based method to predict and train the interaction explicitly.

Original: 2. M ULTI - MODAL TRAJECTORY PREDICTION
Revised: 2. M ULTI - MODAL TRAJECTORY PREDICTION

Original: Due to the nature of the trajectory prediction; stochasticity, there is no unique answer, but several futures are possible.
Revised: Trajectory prediction is a stochastic problem, which means that there are multiple possible futures instead of a unique answer.

Original: Therefore, recent works use deep generative models such as GAN (Goodfellow et al. 
Revised: Recently, deep generative models like GAN (Goodfellow et al. (2014)) or

Original: and VAE (Kingma & Welling (2013)).
Revised: VAE (Kingma & Welling (2013)) have been employed to address this issue.

Original: GAN-based (Gupta et al.
Revised: GAN-based (Gupta et al.

Original: ; Kosaraju et al.
Revised: Kosaraju et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2021b))
Revised: (2021b))

Original: and CVAE-based models (Ivanovic & Pavone (2019); Salzmann et al.
Revised: and VAE-based models (Ivanovic & Pavone (2019); Salzmann et al.

Original: ; Tang & Salakhutdinov (2019)) predict multiple futures by sampling multiple latent vectors.
Revised: ; Tang & Salakhutdinov (2019)) predict multiple futures by sampling multiple latent vectors.

Original: As it is important to samplea meaningful latent vector for predicting diverse and plausible future trajectories, trying to define a well-organized latent space became a natural choice  ( Ma et al.
Revised: A well-organized latent space is necessary to sample meaningful latent vectors for predicting diverse, yet plausible future trajectories. This has become a natural choice in recent works ( Ma et al.

Original: ; Bae et al. (2022)).
Revised: ; Bae et al. (2022)).

Original: The  most closely relatedwork to ours is GRIN (Li et al.
Revised: The work most closely related to ours is GRIN (Li et al.

Original: (2021a)), which argues that multi-modality in trajectory prediction comes from two sources:i) personal intention andii) social relations with other agents.
Revised: (2021a)), which argues that multi-modality in trajectory prediction comes from two sources: personal intention and social relations with other agents.

Original:  GRIN only uses the past trajectories of agents to model the latent space of personal intention and social relations. However, our model proposes to utilize lane information for modeling social relations based on the occupancy.
Revised: However, GRIN only considers past interaction, while we propose to consider future interaction by taking into account the characteristics of vehicle motion.

Original: Leveraging the future occupancy makes the model interpretable and easy to be trained since the pseudo labels of occupancy can be obtained from ground-truth trajectories. Also, we sample the personal intention of a driver from a set of lane segments feature by following the standard goal-conditioned method (Zhao et al.
Revised: Since vehicle motion mainly follows lanes, we utilize lane information to infer future interactions.

Original:  F ORMULATION
Revised:  F ORMULATION

Original: In each scene, theobserved pasttrajectory andGT future trajectory of N vehicles are denoted as x − = { x } 1: N − t p :0 and  x += { x } 1: N 1: t f , where t p + 1 and t f are past/future timesteps.
Revised: In each scene, the past and future trajectories of N vehicles are observed. The past trajectory x − t consists of positions for − t p : 0 timesteps before the current timestep, and the future trajectory x + t consists of positions for 1 : t f timesteps after the current timestep ( t = 0 ).

Original: We use lane information  from the HD map, expressed as M segmented lane polylines ( Zhao et al.
Revised: Lane information is obtained from the HD map, which consists of M segmented lane polylines.

Original: Varadarajan et al.

Original: The lane information is represented as a graph GGG = ( ℓℓℓ, e )  where the node ℓℓℓ = { ℓ } 1: M is the set of directed sequences of xy coordinates. The edge is defined as five types:
Revised: The lane information is represented as a graph: GGG = ( ℓℓℓ, e ) , where the nodes ( ℓℓℓ ) correspond to the different lane segments, and the edges ( e ) represent the relationships between the segments.

Original: e = { e pre , e suc , e left , e right ,e inter } 1: M which are predecessor, successor, left/right neighbor and in-the-same-intersection.
Revised: There are five relationship between segments: predecessor , successor , left/right neighbor and in-same-intersection .

Original: Thesegmented lane nodes are shown in Fig. 2, where different colors denote different individual segments ( ℓ m ). We denote overall input  as X= { x − , G G G} .
Revised: The input to the model is denoted as X , which consists of the past and future trajectories of the vehicles and the lane information. Here, the future trajectories is only used in training.

Original: The output  is denoted as Y , which is the set of F samples of xy coordinates of the target vehicles.
Revised: The output of the model is denoted as Y , which consists of F predicted future trajectories for each agent.

Original: We also define the future lane occupancy (waypoint occupancy) denoted as  τττ t = (cid:81) N,Mn,m p nt ( ℓ m ) , where p nt ( ℓ m ) is the probability of the m -th lane segmentbeing occupied by the n -th vehicle at  future time step t  .
Revised: The model also predicts the future lane occupancy (i.e., which vehicles are occupying which lanes) as a medium using a probability distribution τττ t for each vehicle and lane segment at each future timestep: 1 : t f .

Original: Thewaypoint occupancy during future timestamps ( t f ) predicted from X is denoted by τττ −1: tf , and GT waypoint occupancy obtained from GT future trajectory is denoted by τττ +1: tf .
Revised: The predicted future lane occupancy is denoted as τττ − t , and the ground truth future lane occupancy is denoted as τττ + t .

Original: [Figure]
Revised: [Figure]

Original:  ( 𝐱
Revised: Future GT Trajectory ( 𝐱𝐱 + )

Original: [Figure]
Revised: [Figure]

Revised: 𝐳𝐳 e− / +

Original: [Figure] [Figure] ( 𝐡 % )
Revised: Interaction Feature ( 𝐡𝐡 𝑅𝑅 )

Original: F Future Trajectories ( 𝐘 )
Revised: F Trajectory Samples ( 𝐘𝐘 )

Original: IntentionFeature ( 𝐡 & ) x!,# x !,$ ···
Revised: IntentionFeature ( 𝐡𝐡 𝐼𝐼 ) x

Original: [Figure]
Revised: + , i x + , j ···

Original:  ( 𝐱 [Figure]
Revised: Past Trajectory ( 𝐱𝐱 − ) x − , i ···

Original:  x 
Revised: Waypoint Occupancy ( 𝛕𝛕 1 : t f ) 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑖𝑖 0 𝑎𝑎 𝑓𝑓 i j 𝑧𝑧 𝑒𝑒𝑖𝑖𝑖𝑖 𝑧𝑧 𝑒𝑒𝑖𝑖𝑖𝑖 c x − , j ··· 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑗𝑗 0 𝑎𝑎 𝑓𝑓

Original: [Figure]
Revised: [Figure]

Original: Waypoint Occupancy ( 𝛕 ’:) ! ) 𝑎𝑔𝑒𝑛𝑡 𝑖 0 𝑡 " ··· i j 𝑧 #$% 𝑧 &’( c x %
Revised: 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑗𝑗 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑖𝑖 Decoder

Original: [Figure]
Revised: [Figure]

Original: $ ··· 𝑎𝑔𝑒𝑛𝑡 𝑗 0 𝑡 " ···
Revised: 𝛕𝛕 1 : t f −1 − / +

Original: [Figure]
Revised: [Figure]

Original: 𝑗 𝑎𝑔𝑒𝑛𝑡 𝑖

Original: Decoder
Revised: passing

Original: [Figure]
Revised: [Figure]

Revised: t f

Original: [Figure]
Revised: [Figure]

Original: M ETHOD
Revised: 𝐡𝐡 x [Figure] c 𝐡𝐡 x− Message passing is then performed to obtain the interaction feature.

Revised: Finally, the decoder predicts F future trajectories from concatenation of and interaction features

Revised:  M ETHOD

Original: We focus  on modeling the "Future Relationship" between agents.
Revised: Our focus is on modeling the "Future Relationship" between agents.

Original: The naive method to infer the
Revised: A naive method to infer the

Original: Future Relationship is predicting all future vehicle trajectories and then using the similarity among the trajectories.
Revised: Future Relationship is to predict all future vehicle trajectories and then calculate similarity among them.

Original: However, this methodneeds to perform prediction twice, which is inefficient and redundant.
Revised: However, this method is inefficient and redundant, as it requires performing prediction twice.

Original: Also, the criteria for calculatingthe similarity between trajectories could be unclear.
Revised: Moreover, the criteria for calculating similarity between trajectories may not be clear.

Original: Inspired by the idea that the vehicles mainly follow lanes, we employ lane information for modeling the Future Relationship.
Revised: In this paper, we utilize lane information for modeling the Future Relationship, inspired by the idea that the vehicles mainly follow lanes.

Original: The key idea is that if two vehicles are expected to pass an adjacent lane, they will have a high chance of interacting in the future .
Revised: Our key idea is that if two vehicles are expected to pass on adjacent lanes, they will have a high chance of interacting in the future .

Original: The overall structure is shown in Fig.
Revised: We present the overall structure of our method in Fig.

Original: First, we predict the waypoint occupancy,  the probability of a vehicle passing a specific lane segment duringthe future time steps 1: t f (Sec. 4.1).
Revised: First, we predict the waypoint occupancy, which represents the probability of a vehicle passing a specific lane segment during future time steps 1 : t f (Sec. 4.1).

Original: Then, from each vehicle’s waypoint occupancy, the FRM infers interactionedges between agents as Gaussian Mixture (Sec. 4.2).
Revised: Using this information, our Future Relationship Module (FRM) infers interaction as an edge feature connecting agent node pairs (Sec. 4.2).

Original: Next, a message passing through inferred interaction edges is performed to obtain interaction feature.
Revised: These interaction edges are used to transfer information between agent nodes to form the interaction feature through message passing.

Original: Finally, in the decoding stage (Sec. 4.3), the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature. Here, the intention feature is  the concatenation of past motion and goallane features.
Revised: Finally, in the decoding stage (Sec. 4.3), the decoder predicts future trajectories from the aggregation of the interaction feature and intention feature, which is derived from the concatenation of past motion and goal features.

Original: Our method follows the CVAE scheme (Sec. 4.2.1, 4.2.2), where the condition corresponds to the intention feature, and the latent code corresponds to the interaction feature.
Revised: Following AgentFormer (Yuan et al. (2021)), our method is based on CVAEs where the condition corresponds to the intention, and the latent code corresponds to the interaction feature.

Original: We describe the details in the following sections.
Revised: Then, we compute prior and posterior distribution of the interaction feature, as described in Sec.

Original: 4. W AYPOINT O CCUPANCY
Revised: 4. W AYPOINT O CCUPANCY

Revised: In this section, we describe how to obtain the waypoint occupancy.

Revised: We need two waypoint occupancies: one predicted from past trajectory ( τττ − t ) and the ground truth ( τττ + t ) for obtaining prior and posterior distributions of interaction, respectively.

Original: Past trajectories x − and  lane graph GGGare encoded into past motion and lane features: hhh − x and hhh ℓ .The same motion encoder is used to obtain future motion features hhh + x .
Revised: To predict the waypoint occupancy from past trajectory, we first encode the past trajectories x − and the lane graph GGG into past motion and lane features: hhh − x , hhh ℓ .

Original: Similar to TNT ( Zhao et al.
Revised: Then, following TNT (Zhao et al.

Original: , we predict  waypoint occupancy in categorical distribution from past motion and lane features. [Equation]
Revised: (2021)), we predict the waypoint occupancy as Eq.

Original: where [ · , · ] is concatenation. As waypoint occupancy is a probability, we applya softmax so that it becomes (cid:80) M τ mt = 1 .
Revised: Here, [ · , · ] denotes concatenation, and we apply softmax to ensure that the waypoint occupancy sum up to one: (cid:80) M τ mt = 1 .

Original: Then, goal feature ( hhh − g ) is sampled from hhh ℓ following waypoint occupancy at t f ( τττ − t f ). The concatenation of hhh − g and hhh − x is interpreted as the agents’ intentions to reach, representing the intention feature ( h h h − I ). Besides, τ τ τ − 1: t f − 1 is a temporal feature containing the probabilistic future motion, so it is passed to FRM.
Revised: [Equation]

Original: For GT waypoint occupancy ( τττ +1: t f ), we can  obtain itdirectly from GT future trajectory. We can get binary lane occupancy of vehicles as we know the position and heading of vehicles at future timesteps.
Revised: For GT waypoint occupancy, we can directly obtain it from GT future trajectory since we know the position and heading of vehicles. More details can be found in the supplementary material. Lane Connectivity 𝑁 𝑀 𝑡 !

Original: The details about getting GT waypoint occupancy are described in the supplementary material.
Revised: − 𝝉 !/ # 𝑁 𝑀 𝑡 ! − 𝑡 ! − 𝑁 𝑁 𝑀 𝑡 !

Original: Similarly, the unique GT goal feature ( hhh + g ) also can be obtained by indexing from hhh ℓ .
Revised: − 𝑁 PR !/ Dot Product Transpose 𝑁

Revised: SuccessorPredecessorNeighborIn intersection

Revised: Waypoint occupancy 𝝉$ !/

Revised: # Smoothed waypoint occupancy

Revised: Inter-agentproximity

Revised: GCN 𝐹 $

Revised: Prior Distribution {π %! ,𝜇 %! ,𝜎 %! } 𝑁 𝑁 1 𝑑 𝑑 ×𝐾 𝐹 & 𝑑 𝑑

Revised: Posterior Distribution {𝜇 # ,𝜎 # } 𝒉 𝐱!

Revised: 𝒉 𝐱#

Revised: Only train

Original: 4. F UTURE R ELATIONSHIP M ODULE (FRM)
Revised: 4. F UTURE R ELATIONSHIP M ODULE (FRM)

Original: Attaining Future Relationship is depicted in Fig. 
Revised: Fig. 5 shows the FRM, which consists of three parts: computing inter-agent proximity and obtaining posterior and prior distribution. From intermediate waypoint occupancy of vehicle ( τττ 1: t f − 1 ), we compute how each pair of vehicle pass adjacent lanes adjacent to each other at each timestep (inter-agent proximity). Based on that information and agents’ past motion features, we obtain two distribution of interaction.

Original: At first, waypoint occupancy is smoothed by
Revised: In the following sections, we describe the details of each part. 4.2. I NTER - AGENT P ROXIMITY To compute the inter-agent proximity (PR), we first smooth the waypoint occupancy using a Graph

Original: Graph Convolutional Network (GCN) ( Welling & Kipf (2016)).
Revised: Convolutional Network (GCN) ( Welling & Kipf (2016)).

Original: It is because when a vehicle passes a specific lane, the vehicle will affect other vehicles that pass the adjacent lane, not necessarily the same lane.
Revised: The reason for doing so is that when a vehicle passes a specific lane, it affects other vehicles that pass the adjacent lane, not necessarily the same lane.

Original: Therefore, we apply different smoothing for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employing 2-hop GCN layers.
Revised: Therefore, we apply different smoothing for each lane connectivity (predecessor, successor, neighbor, in-the-same-intersection) by employing 2-hop GCN layers.

Revised: Specifically, each layer aggregated information from neighboring lanes and applies a non-linear transformation.

Revised: This allows the model to capture spatial dependencies among agents and improve the accuracy of the inter-agent proximity computation.

Original: Each layer is expressed as:
Revised: Each layer is expressed as Eq.

Revised: ( 2) where σ , D e , A e and W e are softmax followed by ReLU, degree, adjacency and weight matrix for each edge type, respectively.

Original: [Equation]
Revised: [Equation]

Original: Here, σ is softmax followed by ReLU activation, D e ∈ R M × M is a degree matrix, A e ∈ R M × M is an adjacency matrix, and W e ∈ R t f − 1 is a weight matrix for each edge type.

Original: Next, with the obtained waypoint occupancy, we compute the inter-agent proximity (PR = { pr ij } 1: N, 1: N ) between vehicle pairs which is the temporal probability that two agents would occupy adjacent lanes.

Original: The PR can be obtained with a dot product of ˜ τττ 1: t f − 1 across the lane axis.It is visually drawn in the middle of Fig.
Revised: With this smoothed waypoint occupancy, we can compute the inter-agent proximity using the dot product of ˜ τττ 1: t f − 1 across the lane axis.

Original: [Equation]
Revised: [Equation]

Original: 4.2. P RIOR OF THE R ELATION
Revised: 4.2. P RIOR OF THE I NTERACTION

Original: Our method follows the CVAE scheme, so the prior distribution of interaction edge should be computed.

Original: We utilize past motion features ( hhh − x ) andpredicted inter-agent proximity (PR − ) to obtain the prior.
Revised: To obtain the prior distribution, we use the past motion features ( hhh − x ) and inter-agent proximity.

Original: In this subsection, we omit superscript - for simplification.
Revised: In this subsection, we omit superscript - for simplification.

Original: There are two design factors for our interaction modeling: i) interaction should reflect diverse and stochastic properties, and ii) interaction occurs in every pair of vehicles.
Revised: There are two design factors for our interaction modeling: (i) interaction should reflect diverse and stochastic properties, and (ii) it occurs in every pair of vehicles.

Original: Consequently, the prior of relation is defined as Gaussian Mixture (GM) per agent pair.
Revised: Consequently, the prior distribution is defined as Gaussian Mixture (GM) per agent pair.

Original: Then, we define interaction edge e ij between agent i andagent j as a d-dimensional featurethat has GM distribution ( p θ ( eee | X ) ∼ (cid:81)
Revised: Then, we define interaction edge e ij between agent i and j as a d-dimensional feature ( p θ ( eee | X ) ∼ (cid:81)

Original: Kk =1 πππ k N ( µµµ k , III σσσ 2 k ) ) following GMVAE ( Dilokthanakul et al.
Revised: Kk =1 πππ k N ( µµµ k , III σσσ 2 k ) ) following GMVAE ( Dilokthanakul et al.

Original: The distribution parameters ( µµµ K ,σσσ K ,πππ [Equation] [Equation] : N, 1: N ) are obtained from the neural network F θ :
Revised: The distribution parameters ( µµµ K ,σσσ K ,πππ K = { µµµ ijK ,σσσ ijK ,πππ ijK } 1: N, 1: N ) are obtained from the neural network F θ :

Original: [Equation]
Revised: [Equation]

Original: [Equation] F θ is composed of MLP layers and 1-d conv layerto capture temporal dependency (Deo & Trivedi (2018)).
Revised: F θ is composed of MLP layers and 1-d conv layer (Deo & Trivedi (2018)).

Original: Then, we perform  sampling twice: one for the interaction mode k (from πππ K ) and one for ϵ (from Gaussian noise).
Revised: We then perform two sampling steps, one for the interaction mode k (from πππ K ) and one for ϵ (from Gaussian noise).

Original: It allows having K distinct interactions where each interaction follows
Revised: This allows for K distinct interactions modes:

Original: Gaussian distribution:

Original: [Equation]
Revised: [Equation]

Original: [Equation] We then compute  interaction feature ( hhh − R 
Revised: Next, we compute the interaction feature ( hhh − R =

Original: = { h iR } 1: N ) by message passing from sampled interaction edge.
Revised: { h iR } 1: N ) via message passing from sampled interaction edge, as follows:

Original: Because we define interaction as a d -dimensional feature, the message function is composed of summation followed by Hadamard product ( ⊗ ) of interaction edge and the other agent features:

Original: [Equation]
Revised: [Equation]

Original: Stochasticity is obtained by giving flexibility within a limited range by predicted mean and standard deviation.

Original: It is because, unlike a symmetric and deterministic interaction between particles, such as pull or push ( Kuramoto (1975)), each driver can asymmetrically consider stochastic interaction.

Original: 4.2. CVAE POSTERIOR
Revised: 4.2. CVAE POSTERIOR

Original: To obtain the posterior distribution of relation, the future motion feature ( hhh + x ) and GT waypoint occupancy ( τττ + ) are used.
Revised: To obtain the posterior distribution, we use GT waypoint occupancy ( τττ + ) and the future motion feature ( hhh + x ), which is obtained from GT future trajectory and same motion encoder with past trajectory.

Original: Similarly, we omit superscript + in this subsection for simplification.
Revised: Similarly, we omit superscript + in this subsection.

Original: Inter-agent proximity is obtained by following same procedure in Eqs.
Revised: Inter-agent proximity is obtained with same procedure in Eqs.

Original: The difference from the prior is that the posterior is modeled in a single Gaussian distribution.
Revised: The difference from the prior is that the posterior is modeled in a single Gaussian ( µµµ,σσσ = { µ ij , σ ij } 1: N, 1: N ).

Original: Thus, F θ is replaced with 
Revised: Thus, F θ is replaced with F ϕ :

Original: F ϕ which outputs parameters of Gaussian distribution ( µ µ µ,σ σ σ = { µ ij , σ ij } 1: N, 1: N ):

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Then we sample ϵϵϵ ∼ N (000 , I I I ) , and interaction edge is obtained:
Revised: Then we sample ϵϵϵ ∼ N (000 , I I I ) , and interaction edge is obtained:

Original: z e z e z e = µµµ + σσσϵϵϵ .
Revised: zzz + e = µµµ + σσσϵϵϵ .

Original: Following the same message passing, interaction features ( hhh + R ) is obtained.
Revised: Finally, following Eq. (7), interaction features ( hhh + R ) is obtained.

Original: 4. D ECODER
Revised: 4. D ECODER

Original: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).
Revised: The decoder predicts future trajectories from the aggregation of the interaction feature ( hhh R ) and intention feature ( hhh I ).

Original: GT goal feature( hhh + g ) is concatenated to the past motion feature ( hhh − x ) to get a unique GT intention feature.
Revised: Here, the intention feature is obtained from past motion feature and goal feature following TNT.

Original: The GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times from the posterior distribution.
Revised: During training, the unique GT intention feature is repeated F times, and we sample the interaction feature ( hhh + R ) F times from the posterior distribution.

Original: During inference, thegoal feature ( hhh − g ) is sampled F times. The intention feature is obtained by concatenation of sampled goal features and the past motion feature ( hhh − x  ).
Revised: During inference, the intention feature is obtained from the past motion feature ( hhh − x ) and goal feature ( hhh − g ), which is sampled F times from predicted waypoint occupancy at the final timestep ( τττ − t f ).

Original: The interaction feature ( hhh − R ) is sampled F times from  prior distribution.
Revised: The interaction feature ( hhh − R ) is sampled F times from the prior distribution.

Original: The decoder is modeled by 2-layer MLP and predictsthe sequence of x,y coordinates from the concatenation of the intention and interaction feature.
Revised: The decoder is composed of 2-layer MLP and predicts sequence of x,y coordinates. More details can be found in the supplementary material.

Original: 4. T RAINING
Revised: 4. T RAINING

Original: Because the GT  ( τττ + ) is available for waypoint occupancy, we  trainτττ − with supervision. Since the  waypoint occupancy is modeled as a classification per agent, we learn it using negative log-likelihood (NLL [Equation]
Revised: Because the GT waypoint occupancy ( τττ + ) is available, we can train the model to predict waypoint occupancy ( τττ − ) using negative log-likelihood (NLL): L nll = − τττ + log ( τττ − ) .

Original: [Equation] In contrast, because interaction edge zzz e is unobservable, we optimize thenegative evidence lower bound (ELBO) to learn thedistribution of interaction following the CVAE scheme. 
Revised: However, since the interaction edge zzz e is unobservable, we optimize the evidence lower bound (ELBO) to train the interaction distribution using the CVAE scheme. [Equation]

Original: [Equation]
Revised: [Equation]

Original: where q π is an approximate posterior distribution of relation, and p θ is the prior.
Revised: Here, q ϕ is the approximate posterior, and p θ is the prior.

Original: Because our model only allows interaction posterior to be Gaussian distribution,  the Kullback–Leibler (KL) divergence term can be simplified below.
Revised: Since our model only allows the posterior to be Gaussian distribution, we can simplify the Kullback–Leibler (KL) divergence term as follow: [Equation]

Original: The detailed derivation with the reparameterization trick is included in the supplementary material.
Revised: The detailed derivation with the reparameterization trick can be found in the supplementary material.

Original: [Equation] The common drawback of NRI method is the "degenerate" problem which leads the decoder  to ignore the relation edge during training.
Revised: However, a common drawback with the NRI-based method is the "degenerate" issue, where the decoder tends to ignore the relation edge during training.

Original: So we train the network to that two features, intention and interaction, to have different roles.
Revised: To address this issue, we train the network to give different roles to the intention and interaction features.

Original: Because the GT trajectory is conditioned on the GT goal feature, we use the GT goal feature to compute the reconstruction term.
Revised: Since the GT trajectory is conditioned on the GT goal feature, we use the GT goal feature to compute the reconstruction term.

Original: Therefore, training in that way restricts the role of interaction edge tobe narrowed down to momentary motion: L recon =
Revised: This training strategy restricts the role of interaction edge to momentary motion, resulting in the following reconstruction loss: L recon =

Original: min eee { E [ log ( p  ( Y | X ,eee,τττ + )] } .
Revised: min zzz e { E [ log ( p θ ( Y | X ,zzz e ,τττ + )] } .

Original: The overall loss is the sum of the three losses, and they are trained jointly: L  =
Revised: Finally, the overall loss is the sum of the three losses, which are trained jointly: L all =

Original: L nll + L KL + L recon .
Revised: L nll + L KL + L recon .

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  E XPERIMENT AND R ESULT
Revised:  E XPERIMENTS

Original: We train and evaluate our method on two popular real-world trajectory dataset: nuScenes ( Caesar et al.
Revised: We train and evaluate our method on two popular real-world trajectory datasets: nuScenes ( Caesar et al.

Original: and Argoverse ( Chang et al.
Revised: and Argoverse ( Chang et al.

Original: In nuScenes/Argoverse datasets, the 2/2 seconds of pastis provided, and  6/3 seconds of future trajectory is predicted with 0.5/0.1 sampling time.
Revised: nuScenes/Argoverse datasets provide the 2/ seconds of past and require 6/3 seconds of future trajectory at 0.5/0.1 second intervals, respectively.

Original: Training/validation/test sets consist of real-world driving scenes of 32186/8560/9041 in nuScenes and 205942/39472/78143 in Argoverse.
Revised: Training/validation/test sets consist of real-world driving scenes of 32,186/8,560/9,041 in nuScenes and 205,942/39,472/78,143 in Argoverse.

Original: For the baseline model, we follow TNT for goal conditioned model, and MHA encodes interaction from past trajectories.
Revised: For the baseline model in ablation, we follow TNT for goal conditioned model, and MHA encodes interaction from past trajectories.

Original: For implementation and training details,  refer to the supplementary material.
Revised: For implementation and computation details, please refer to the supplementary material.

Original: 5. Q UANTITATIVE RESULT
Revised: 5. Q UANTITATIVE RESULT

Original: Our method outperformsthe SoTA models in allmetrics used in the nuScenes benchmark, as shown in Tab. 1.
Revised: Our method outperforms SoTA models in all nuScenes benchmark metrics, as shown in Tab. 1.

Original: The proposed model outperforms the runner-up method, PGP ( Deo et al.
Revised: Specifically, our model outperforms the runner-up method, PGP ( Deo et al.

Original: (2022)), by a large margin.
Revised: (2022)), by a substantial margin.

Original: The superiority implies that  explicit interaction modeling via inferring waypoint occupancy helps scene understandingof our model compared to the implicit interaction modeling of PGP.
Revised: This result indicates that our explicit interaction modeling via inferring waypoint occupancy helps scene understanding compared to the implicit interaction modeling of PGP.

Original: When 10 samples are predicted, mADE and MR showed improvements of 5.3% and 8.8%.
Revised: When predictingsamples, our model shows improvements of 5.3% and 8.8% in terms of mADE and MR.

Original: Previously,
Revised: Previously,

Original: THOMAS ( Gilles et al.
Revised: THOMAS ( Gilles et al.

Original: (2022b)) had firstplaced in mFDE 1 by proposing a recombination module that post-processesthe marginal predictions into the joint predictions  aware of other agents.
Revised: (2022b)) was ranked first in mFDE 1 by proposing a recombination module that post-processes marginal predictions into the joint predictions that are aware of other agents.

Original: Our model shows higher performance in mFDE 1 than THOMAS, indicating better interaction modeling ability without post-processing.
Revised: However, our model performs better than THOMAS in mFDE 1 , indicating better interaction modeling ability without post-processing.

Original: This is possible because inferring future relationships helps to  understand the future interaction with other agents; details are provided in the ablation study.
Revised: This is possible because inferring future relationships helps to better understand the future interaction with other agents; details are provided in the ablation study.

Original: We also validated our method on the Argoverse dataset.
Revised: We also evaluated our method on the Argoverse dataset.

Original: Our model does not achieve SotA performance but still makes remarkable performance improvement in both valid/test sets.
Revised: While our model does not achieve SotA performance, it still shows remarkable performance improvement in both validation and test sets.

Original: Moreover, except for the HiVT, our method make competitive performance in mADE.
Revised: Moreover, except for the HiVT, our method make competitive performance in mADE.

Original: Please note that our model (0.82) is still comparable to SceneTransformer (0.80) and Multipath++ (0.79) in the test set results.
Revised: Please note that our model (0.82) is still comparable to SceneTransformer (0.80) and Multipath++ (0.79) in the test set results.

Original: In the case of HiVT, it uses the surrounding vehicles’ trajectories for training.
Revised: However, HiVT uses the surrounding vehicles’ trajectories for training, resulting in increased training data.

Original: It results in increasing the training data, so a direct comparion to HiVT is rather unfair.
Revised: Therefore, a direct comparison to HiVT would be rather unfair.

Original: We do not achieve SotA in Argoverse because the proposed method is less effective than in nuScenes.
Revised: We do not achieve SotA in Argoverse because the proposed method is less effective than in nuScenes.

Original: We conclude that the relatively small performance gain on Argoverse compared tonuScenes comes from the different configuration of datasets. A model should predict the 6-second future in  nuScenes while it only needs to predict the 3-second future in Argoverse.
Revised: We attribute this disparity to the differences in dataset configurations, where nuScenes requires predicting a longer future trajectory than Argoverse.

Original: Intuitively, the impact of interaction gets more significant in long-range prediction task.
Revised: As intuition suggests, interaction modeling has a more significant impact on longer-range prediction tasks.

Original: We thus measure the performance gain on nuScenes when the prediction horizon is the same as that of Argoverse and report the results in Tab. 3.
Revised: To validate this assumption, we conducted an ablation study by measuring the performance gain on nuScenes when predicting the same length of future as Argoverse.

Original: In 6-second prediction, the proposed interaction inference method improved mADE 1 by more than 10%, but its effect halves in a 3-second prediction  which isa similar effect on Argoverse.
Revised: The results, presented in Tab. 3, shows that our interaction modeling method improves mADE 1 by over 10% in a 6-second prediction task, but its effect was halved in a 3-second prediction task, which is similar to the results obtained in Argoverse.

Original: This result shows that our interaction inference method is more effective in longer-range prediction tasks.
Revised: This finding suggests that our interaction modeling method is more effective in longer-range prediction tasks.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5. Q UALITATIVE RESULT
Revised: 5. Q UALITATIVE RESULT

Original: In Fig.
Revised: In Fig.

Original:  prediction samples (F=6) of the baseline  and our method are depicted.
Revised: we present prediction samples (F=6) from the baseline (left) and our method (right).

Original: To simply check the effect of our method, we took the samples where
Revised: To assess the efficacy of our method, we brought the samples with two agents and

Original: Unlike the baseline method that ignores other agents and generates spatially uniform trajectories, our model considers other agents to surpass or wait for them.

Original: Moreover, not only considering multi-modal interaction, we  allow stochasticity within a single mode of interaction.
Revised: Moreover, not only considering two modes of interaction; surpass or wait, we also allow stochasticity within a single mode of interaction.

Original: As a result, it can be seen that our method generates diverse yet interaction-aware samples.
Revised: Consequently, our model generates diverse yet interaction-aware samples.

Original: In addition, our method can consider stochastic interaction when multiple agents are present.
Revised: Furthermore, our method can incorporate stochastic interaction when multiple agents are present.

Original: In the experiment of Fig.
Revised: In the experiment shown in Fig.

Original: 7, we predict the trajectories of the target agents (denoted as 0) with multiple interacting vehicles.
Revised: 7, we predict the trajectories of the target agents (denoted as 0) with multiple interacting vehicles.

Original: In each scene,  intention of the target vehicle is fixed (denoted in green) and two relation edges are sampled.
Revised: In each scene, the intention of the target vehicle is fixed (denoted in green) and two interaction edges are sampled.

Original: The corresponding predicted trajectory samples and  degree of relation ( ∥ zzz e ∥ ) are plotted on the right.
Revised: The corresponding predicted trajectory samples and the degree of interaction ( ∥ zzz e ∥ ) are plotted on the right.

Original: In the first row, target agent (0) infers large interaction with agent 2 in sample 1.
Revised: In the first row of the figure, the target agent (0) infers significant interaction with agent 2 in sample 1.

Original: Because agent 2 is moving in the same direction and is predicted to go ahead, the model generates an accelerating trajectory to follow agent 2.
Revised: As agent 2 is moving in the same direction and is predicted to move ahead, our model generates an accelerating trajectory to follow agent 2.

Original: In  sample 2,in contrast, the interaction between agent 1 is sampled large because they are expected to be in the same intersection.
Revised: In contrast, in sample 2, the interaction with agent 1 is sampled as significant because they are expected to be in the same intersection.

Original: In this case, the model generates decelerating trajectory considering the future motion of agent 1.
Revised: In this case, our model generates decelerating trajectory considering the future motion of agent 1.

Original: Meanwhile, with the fixed intention, it shows that all predicted trajectories  are properly on goal lane segments.
Revised: Importantly, all predicted trajectories in these samples are appropriately constrained within the goal lane segments as the intention is set to the green colored lane.

Original: It means that the proposed training method effectively restricts the role of interaction features affecting momentary motion, as we defined.
Revised: This indicates that our training strategy effectively restricts the role of interaction features to momentary motion.

Original: 5. A BLATION STUDY
Revised: 5. A BLATION STUDY

Original: Ablation on the impact of model design is shown in the upper part in Tab. 4.
Revised: Ablation on the impact of model design is shown in the upper part in Tab. 4.

Original:  Ours w/o FR  does not consider Future Relationship in interaction modeling. This model only utilizes past trajectories to infer the relation,thus similar to the NRI.
Revised: The Ours w/o FR variant does not consider Future Relationship in interaction modeling, and only uses past trajectories to infer the relation, similar to the NRI.

Original: After all, its performance is almost identical to the baseline, which uses MHA of past trajectories to model interaction.
Revised: This variant performs almost identically to the baseline, which uses MHA of past trajectories to model interaction.

Original: This result shows that leveraging 
Revised: This result shows the importance of leveraging Future

Original: Future Relationship indeed helps to infer interaction.
Revised: Relationship for plausible interaction inference.

Original:  Ours w/o GCN does not smooth predicted/GT waypoint occupancy.
Revised: The Ours w/o GCN variant omits the smoothing waypoint occupancy.

Original: It causes inaccurate inter-agent proximity (PR) estimation especially in the posterior distribution.
Revised: leading to inaccurate inter-agent proximity (PR) estimation, especially in the posterior distribution.

Original: Because GT waypoint occupancy is a binary value, computing PR from it could cause inaccurate proximity and brings performance decline.
Revised: Since GT waypoint occupancy is a binary value, computing PR from it can result in inaccurate proximity and lower prediction performance.

Original: In the proposed model, we allow asymmetric relation between two agents.
Revised: In the proposed model, we allow asymmetric interaction between two agents.

Original:  Ours w/ Sym applies hard symmetric relation ( Li et al.
Revised: The Ours w/ Sym applies hard symmetric interaction modeling ( Li et al.

Original: , and it shows that our asymmetric design is more suitable for modeling the driver relation.
Revised: , and it shows that our asymmetric design is more suitable for modeling the driver relation.

Original: The importance of multi-modal stochastic relation modeling is shown in the lower part of Tab. 4.
Revised: The importance of multi-modal stochastic interaction modeling is shown in the lower part of Tab. 4.

Original:  Ours w/ GP  models the prior distributionof relation as Gaussian distribution instead of GM. As it onlyconsiders a single modality of agent relation, the performance drops compared to when considering multiple modalities.
Revised: The Ours w/ GP variant models the prior distribution as Gaussian distribution instead of GM, considering only a single modality of interaction, which leads to a performance decline compared to the full model with multi-modal interaction.

Original:  Ours w/ Deterministic only predicts  the mean of relation feature in Eq.
Revised: The Ours w/ Deterministic variant predicts only the mean of interaction edges in Eq.

Original: Although it can model multi-modal relation, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.
Revised: Although it can model multi-modal interaction, the diversity is prone to be limited compared to the stochastic counterpart especially when the sample size F is large.

Original: The resultalso shows that  deterministicrelation modeling severely degrades  prediction performance when F is large.
Revised: The result shows that stochastic modeling is critical for prediction performance, and deterministic modeling significantly degrades the prediction performance when predicting more samples.

Original: On theother hand, Ours w/ GP  shows relatively lessdrop in performance  as it does not lose stochasticity even after removing the GM prior.
Revised: In contrast, the Ours w/ GP variant shows relatively less performance drop as it maintains stochasticity even after removing the GM prior.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose Future Relationship to  learn the interaction between vehicles for trajectory prediction effectively.
Revised: In this paper, we propose Future Relationship to effectively learn the interaction between vehicles for trajectory prediction.

Original: Beyond utilizing only past trajectory to infer interaction, Future Relationship has advantages in explicitly using lane information.
Revised: By explicitly utilizing lane information in addition to past trajectories, our

Original: We show that lane information enables proper interaction inference even in complex road structures.
Revised: FRM can infer proper interactions even in complex road structures.

Original: In addition, we propose the model that obtains interaction probabilistically, generating diverse yet socially plausible trajectory samples. The proposed method also provides explainable medium (waypoint occupancy or inter-agent proximity) compared to other prediction models use totally random latent vectors.
Revised: The proposed model generates diverse yet socially plausible trajectory samples by obtaining interaction probabilistically, which provides explainable medium such as waypoint occupancy or inter-agent proximity.

Original: We trained our model with the CVAE scheme and validated in popular real-world trajectory prediction datasets.
Revised: We trained our model using CVAE scheme and validated it on popular real-world trajectory prediction datasets.

Original: The proposed method achieves SotA performance in  long-range prediction task, nuScnes, and brings remarkable performance improvement in  short-range prediction task, Argoverse.
Revised: Our approach achieved SoTA performance in a long-range prediction task, nuScnes, and brings remarkable performance improvement in a short-range prediction task, Argoverse.

Original: Because considering Future Relationship is novel, we expect that using more sophisticated training methods (Ye et al.
Revised: Modeling Future Relationship is a novel approach, and we anticipate that using more sophisticated training methods (Ye et al.

Original:  Zhou et al.
Revised: ; Zhou et al.

Original: or  better baseline model (such as GANet (Wang et al.
Revised: or a better baseline model (such as GANet (Wang et al.

Original: (2022)) will also lead to performance improvements in our model.
Revised: (2022)) may further improve prediction performance. A CKNOWLEDGMENTS

Revised: This work was supported by Institute of Information & Communications Technology Planning &

Revised: Evaluation(IITP) grant funded by the Korea government(MSIT)

Revised: (No.2014-3-00123, Development of

Revised: High Performance Visual BigData Discovery Platform for Large-Scale Realtime Data Analysis