Causal-based Time Series Domain Generalization for Vehicle Intention Prediction
Abstract
Accurately predicting possible behaviors of trafﬁc participants is an essential capability for autonomous vehicles.
Since autonomous vehicles need to navigate in dynamically changing environments, they are expected to make accurate predictions regardless of where they are and what driving circumstances they encountered.
Therefore, generalization capability to unseen domains is crucial for prediction models when autonomous vehicles are deployed in the real world.
In this paper, we aim to address the domain generalization problem for vehicle intention prediction tasks and a causal-based time series domain generalization (CTSDG) model is proposed.
We construct a structural causal model for vehicle intention prediction tasks to learn an invariant representation of input driving data for domain generalization.
We further integrate a recurrent latent variable model into our structural causal model to better capture temporal latent dependencies from time-series input data.
The effectiveness of our approach is evaluated via real-world driving data.
We demonstrate that our proposed method has consistent improvement on prediction accuracy compared to other state-of-the-art domain generalization and behavior prediction methods.
Introduction
Researchers have developed various machine learning algorithms to enhance the accuracy of intention and motion prediction tasks for autonomous vehicles [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].
Although these machine learning algorithms provide excellent prediction performance, a key assumption underlying the remarkable success is that the training and test data usually follow similar statistics.
Otherwise, when test domains are unseen or Out-of-Distribution (OOD) [11], the resulting train-test domain shift leads to signiﬁcant degradation in prediction performance.
Domain generalization (DG) aims to build models that are designed for increased robustness to domain-shift without requiring access to target domain data.
Many researchers have been working on developing DG algorithms [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], but most of these models are designed for tasks such as object recognition, semantic segmentation, image classiﬁcation, and disease detection.
In this paper, our goal is to design a DG algorithm for vehicle intention prediction task which is different from typical DG tasks in three aspects: (1) the input is numerical time-series data instead of static image; (2) each input data point contains information of multiple agents (e.g. vehicles) with interactions involved, instead of a single agent (e.g. patient) with no interaction; (3) factors that cause domain shift are ambiguous for our task, in contrast to clear domain shift variables for other DG tasks (e.g. image Workshop on rotation angle or style for image classiﬁcation task).
Therefore, directly applying state-of-the-art DG approaches may not be suitable for our problem setting and could result in undesirable performance.
It is thus necessary to develop a domain generalization model that considers the discrepancies in the aforementioned aspects and is applicable for vehicle intention prediction task.
In this paper, a causal-based time series domain generalization (CTSDG) model for vehicle intention prediction is proposed.
To the best of our knowledge, this is the ﬁrst work trying to tackle time series domain generalization problem for vehicle intention prediction task.
The effectiveness of our approach is evaluated on real-world driving data under different scenarios.
Our method also shows consistent improvement on prediction accuracy compared to other state-of-the-art methods.
Method 2.1 Notations
Let X be the input space and Y the label space, a domain is deﬁned as a joint distribution P XY on X × Y .
In the context of domain generalization, we assume to have access to K domains,
D =
{ D k } K k =1 , each associated with a joint distribution P k XY .
In each domain, data-label pairs are sampled from a dataset D k = { ( x ( k ) i , y ( k ) i ) }
N k i =1 with ( x ( k ) i , y ( k ) i ) ∼ P kXY , where N k is the number of labeled data points in the k -th domain and x i = ( x it )
Tt =1 = ([ ξ A,it , ξ B,it ])
Tt =1 denotes a multivariate time series with x it ∈
R H
× 2 .
For a given vehicle, we use ξ to represent historical trajectories.
In this work, given historical trajectories of interacting vehicle pairs (i.e. car A and car B), the domain generalization task is to learn an intention predictor f (i.e. predict interaction outcomes such as pass/yield) that generalizes well to unseen target domains k (cid:48) /
∈ D
k where P ( k (cid:48) )
XY (cid:54) =
P ( k ) XY , ∀ k ∈ { 1 , . .
. , K } .
2.2 Learning Invariant Representation from Causal Model
Domain
X NC
X C
Y x t z t h t xxhhzz
X
[Figure]
Z
In this work, we propose a causal view of data generating process under vehicle interaction settings.
More speciﬁcally, we utilize a structural causal model (SCM) based on domain knowledge to describe how interactive trajectories are generated by any vehicle pair (see Figure 1).
The constructed causal structure provides us with causal relations between input features and the outcome, which is used to learn the intention predictor.
The detailed meaning of each node variable is as follow: Domain ( D ) contains different combinations of map properties such as road topology, speed limit, and trafﬁc rules; Event ( E ) denotes an observable variable relate to the event of two-vehicle interaction, which includes information such as initial interaction states and the length of interaction; Driver ( O ) denotes an unobservable variable relate to each driver’s driving preferences or style, such as the aggressiveness level and whether he/she obeys trafﬁc rules; Causal features ( X C ) represent high-level time-related causal features derived from drivers’ personal behavior O and their interaction status E , which are used by humans to label the intention; Non-causal features ( X NC ) represent domain dependent features which contain not only domainspeciﬁc information D , but also event E . For example, interaction event information such as when two vehicles start noticing each other or when the road negotiation begins, is related to the road geometry and trafﬁc rule; Input Data ( X ) is vehicle interactive trajectories that contain sequential multivariate data, and is constructed from a mixture of causal X C and non-causal X NC features; Latent variable ( Z ) denotes time-related latent representations extracted from time-series vehicle interaction data; Label ( Y ) is the vehicle intention label.
Invariance Condition.
According to Figure 1, X C is the node that causes Y , and by d-separation [22], the intention label is independent of domain conditioned on X C , Y ⊥⊥ D | X C 1 .
In other word, if such a X C can be found, then the distribution of Y conditional on X C is invariant under transferring from the source domains to the target domains.
Therefore, our intention prediction task is to learn y as h ( x c ) where h : C → Y .
However, since X C is unobserved, we need to learn y through observed trajectories X .
Speciﬁcally, we utilize a representation function q : X → Z to map the input space to a latent space, and a hypothesis function φ : Z → C to map the latent space to X C .
Together, h ( φ ( q ( x ))) leads to the desired intention predictor f : X → Y and the corresponding prediction loss can be written as:
[Equation]
where L α is the classiﬁcation loss such as a binary or categorical cross-entropy.
In addition, by d-separation, X C also needs to satisfy an invariance condition: X C ⊥⊥ D |{ E, O } , which means X C does not change with different domains when both event and driver information remain the same.
However, driver information is unobservable and in many dataset there may not be an exact match based on a same interaction event across domains.
Alternatively, we assume that the distance over
X C between same-class inputs from different domains is bounded, which provides an alternative invariance condition that is consistent with the conditional independencies of X C .
Therefore, we would like to minimize the following objective along with the prediction loss:
[Equation]
where L β is the distance metric such as (cid:96) 2 , and Ω : X × X → { 0 , 1 } is a match function such that pairs having Ω( x j , x m )
= 1 have low difference in their causal features.
Contrastive Representation Learning.
To optimize L r , we ﬁrst need to learn a proper match function Ω used in Eq.
Speciﬁcally, we optimize a contrastive representation learning loss that minimizes distance between same-class inputs from different domains.
We regard positive matches as two inputs from the same class but different domains, and negative matches as pairs with different classes.
Then the loss function for every positive match pair ( j, m ) in a sampled mini-batch B is deﬁned as:
[Equation]
where s cos ( x a , x b )
=
φ ( q ( x a ))
T φ ( q ( x b )) / (cid:107) φ ( q ( x a )) (cid:107)(cid:107) φ ( q ( x b )) (cid:107) is the inner product of two (cid:96)
2normalized vectors, τ is a temperature scaling parameter, and |B| is the batch size.
Inspired from [19], for this unsupervised contrastive learning process, we initialize Ω with a random match based on classes and keep updating Ω by minimizing the contrastive loss (3) until convergence.
2.3 Capturing Temporal Latent Dependencies
From our proposed causal model in Fig.1, we also observe that Z ⊥⊥ D | X , meaning the latent representation is independent of domain D conditioned on input data X .
Therefore, by learning the representation function q , we are able to extract a domain-invariant latent variable that represents the input space.
Since the input is time-series data, the learned latent variable is expect to capture temporal latent information in the data.
To explicitly model the dependencies between latent random variable across time steps, we integrate Variational Recurrent Neural Networks (VRNN) [23] into our CTSDG model.
The VRNN contains a VAE at every time step and these VAEs are conditioned on previous auto-encoders via the hidden state variable h t − 1 of an RNN.
In general, the objective function become a timestep-wise variational lower bound:
[Equation]
where q ( z it | x i ≤ t , z i ≤ t ) is the inference model, p ( z it | x i<t , z i<t )) is the prior, p ( x it |
z i ≤ t , x i ≤ t ) is the generative model, and KL ( ·||· ) refers to Kullback-Leibler divergence. [Table]
2.4 Overall Algorithm
Combining the joint optimization problem of equation (1), (2) [Table] and (4) leads to our CTSDG model.
The complete objective function we aim to optimize is:
[Equation]
where γ and λ are hyperparameters that control training balance among three losses for better performance.
In order to prevent the classiﬁcation loss from interfering learning domain-invariant representations, we learn the match function Ω before optimizing the overall objective function (5).
Also, since the causal feature X C is a function of latent variable Z which is extracted by the recurrent latent variable model, the objective function (4) needs to be jointly optimized while learning the matching function using Eq.(3). Input : Source training domain D =
{ D k } Kk =1 ; hyperparameters τ, γ, λ >
Output : Intention Predictor f : X → Y 1 Randomly split source domains D into disjoint train D tr and validation D val ; 2 Initialize Ω as a random match function; 3 while not converged do
Sample a mini-batch from D tr ; if epoch < threshold then
Minimize (3) + γ (4); Update Ω ; else Minimize (5); end 11 end The pseudocode for our proposed CTSDG method is shown in Algorithm 1.
We ﬁrst initialize Ω as a random match function, where given each data point, we randomly select another data point with the same class from another domain.
During each training epoch, we constantly update Ω based on the currently learned hypothesis function φ .
Speciﬁcally, given any data point x , we ﬁnd the nearest neighbour of x among other data points with the same class from another domain by calculating their distance using L β deﬁned in Eq.(2).
After certain number of training epoch is reached, we further optimize the overall objective function (5) until convergence.
Note that when calculating L r in Eq.(2), we apply the updated match function Ω to select match pairs.
Experiments
The experiment is conducted on the INTERACTION dataset [31], where two different scenarios are utilized: USA_Roundabout_FT and CHN_Merging_ZS.
In this work, a domain is deﬁned as a driving region that contains a unique combination of road topology, trafﬁc rules, and speed limit information.
Three domains are extracted from the FT scenario and one domain is extracted from the ZS scenario (see Figure 2).
We compare our method with ten state-of-the-art methods, including seven traditional baseline methods for domain generalization tasks and three methods for behavior prediction tasks.
To evaluate the domain generalization performance, we follow the typically used leave-one-domain-out protocol [32].
Details related to data selection, implementation, and more in-depth evaluations on learned latent representations and temporal latent dependencies can be found in supplementary materials.
Evaluation on USA_Roundabout_FT.
We ﬁrst train all models on different combinations of source domains extracted in FT and report the test results on the remaining target domain in FT. From Table
[Figure] 1, we observe that our method achieves an average prediction accuracy of 93.91% under all three train-test domain pairs and increases the accuracy by 2.54% compared to the best-performed baseline model.
ERM and GNN method slightly outperform CTSDG when FT-2 is the target domain, but their accuracy is 10.24% and 11.26% less than our method when tested on FT-3, respectively.
As seen, our method has consistent prediction accuracy regardless of testing domains.
The standard deviations of
CTSDG are also lower than those of all other baseline methods, which further shows our model’s efﬁcacy as it converges to more stable local optima.
Evaluation on CHN_Merging_ZS.
When we compare the domain in ZS with domains in FT, we notice larger difference in road topology as well as in speed limit.
In other word, the joint distribution shift is more severe between domains from two different scenarios (i.e. ZS and FT) than domains within scenario FT, which makes domain generalization more challenging.
Therefore, to evaluate the generalizability of each model under large domain shift, we select different domain combinations in FT for training and the single domain in ZS for testing.
From Table 1, we observe that CTSDG consistently outperforms all baseline models across all source-target pairs with accuracy 1% ∼ 6% higher.
The average prediction accuracy of our method increases by 5% compared to the bestperformed baseline model.
Also, when the model is trained using all three domains in FT, where the source domains have more diversity, learning based methods that do not speciﬁcally consider domain generalization approaches (i.e. ERM, CVAE, GAN, and GNN) have reasonable performance.
However, the performance of these methods degrade drastically when training data are limited and lack of diversity.
According to the results, we show the diminishing performance of traditional machine learning model to out-of-training distribution regimes and our proposed method is able to mitigate such problem while consistently outperforms other state-of-the-art methods.  Extension to Trajectory Prediction Tasks
In this paper, although we illustrate the performance of our proposed domain generalization method for vehicle intention prediction tasks, our CTSDG approach can also be directly or indirectly utilized for trajectory prediction tasks.
The most straight forward way is to frame the trajectory prediction problem as classiﬁcation over a diverse set of trajectories [33], which enables us to a) ensure a desired level of coverage of the state space, b) eliminate dynamically infeasible trajectories, and c) avoid the issue of mode collapse. Moreover, the accuracy of intention classiﬁcation plays an important role in regression tasks. As stated and shown in [1, 34, 35, 36], goal/intent-conditioned trajectory forecasting can improve joint-agent and per-agent predictions, compared to unconditional forecast. Conditional forecasting can be also used for planning tasks, which effectiveness is demonstrated in [8].
Therefore, having an accurate and domain generalizable intention/goal predictor is the prerequisite of developing state-of-the-art trajectory predictor and motion planner.
Conclusions
In this paper, we present a causal-based time-series domain generalization model (CTSDG) to address domain generalization problem for vehicle intention prediction tasks.
We show that CTSDG has consistent improvements in terms of prediction accuracy compared to other state-of-the-art domain generalization and behavior prediction methods using real-world driving data.
In the future, we expect to extend our work to address trajectory prediction problems and consider generalization of varying number of interacting vehicles. Supplemental Materials A Related Works
Domain generalization aims to generalize models to unseen domains without knowledge about the target distribution during training. Different methods have been proposed for learning generalizable and transferable representations. A commonly-used strategy is to extract task-speciﬁc but domaininvariant features [12, 13, 17, 18, 19, 20]. Ghifary et al.
[12] learn multi-task auto-encoders to extract invariant features which are robust to domain variations.
Li et al.
[13] extend adversarial autoencoders by imposing maximum mean discrepancy measure to align multi-domain distributions.
Li et al.
[17] consider the conditional distribution of label space over input space, and minimize discrepancy of a joint distribution.
Recently, several works [18, 19, 20] utilize causal structures to learn invariance factors, which have shown better model generalizability on unseen data, especially on data from different distributions than the train distribution.
Dataset augmentation is another strategy that focuses on creating more out-of-domain samples by adversarially perturbing samples [37], leveraging self-supervised signals [16], or utilizing convex combinations of input and output data [21].
These data augmentation methods have shown effective for model generalization, but invalid samples could be generated and degrade the model performance if input data is vehicle trajectories that need to satisfy feasibility constraints.
Meta-learning can also be applied in domain generalization, by creating meta-train and meta-test sets, and training a model using the meta-train set in such a way to improve the performance on the meta-test set [38, 14, 15].
While our proposed method can achieve promising results using standard training schemes, combining meta-learning with our approach could be an interesting future direction.
Intention prediction in autonomous driving aims to predict high-level behaviors of road entities, which results are often used in downstream trajectory prediction modules.
For example, works such as [34], [35] and [36] demonstrate that by predicting goal states and assuming that agents navigate toward those goals by following some optimal or learned trajectories, the accuracy of prediction can be improved.
Therefore, accurate intention prediction algorithm is essential for autonomous vehicles to smoothly navigate in dynamic environments.
Zhang et al.
[39] propose to model high level semantics in the form of trafﬁc patterns through a generative model.
Dong et al.
[40] use a probabilistic graphical model to estimate the pass or yield probability under merging scenarios.
Schulz et al.
[41] utilize the dynamic bayesian network method for routing prediction at intersections.
Zyner et al.
[42] obtain a probability distribution over all possible exit branches for a vehicle driving in a roundabout using recurrent neural network.
Works such as [7] and [1], use more complicated
ML models to prediction vehicle’s intention.
Due to the complexity of the real-world and its everchanging dynamics, the deployed autonomous vehicles inevitably face novel situations and should be able to cope with them.
However, it has been repeatedly demonstrated that the reliability of ML models degrades radically when they are exposed to novel settings (i.e., under a shift away from the distribution of observations seen during their training) due to their failure to generalize, leading to catastrophic outcomes [43, 44].
In [11], Filos et al.
also demonstrate that several state-of-the-art behavior prediction models for autonomous driving fail to generalize under arbitrary domain shift.
Therefore, in this paper, we aim to address the domain generalization problem for vehicle intention prediction task and provide an effective solution to enhance reliability of autonomous vehicles.
We hope our work can inspire further research in the ﬁeld of improving generalizability for prediction algorithms in autonomous driving industry.
B Methodology B.1 Illustrate the problem context through a toy example
Consider two vehicles interact with each other under two different domains.
For the ease of understanding, we simplify the problem by temporarily making the following assumptions for the two interaction cases in Figure 3: (1) The same road priorities; (2) The same two drivers interact with each other; (3) The same initial states for both vehicles.
However,there are two major differences between two cases: (1) Different topological road structure of two intersected roads; (2) Two drivers have different driving styles (i.e. we let the driver in red car be more aggressive than the driver in blue car), which information is actually unavailable when making predictions.
[Figure]
Based on previous assumptions and settings, for a given interaction period, a plausible simulated velocity proﬁle of two vehicles is shown in the second row of Fig.
3 for both cases.
The goal is to predict the intention of any selected vehicle, which can be the intent of one car passes or yields the other car.
According to the plots, whether the red car yields or passes the blue car doesn’t rely much on the domain information but depends more on two drivers’ personal information (e.g. driving style) and their initial states.
Speciﬁcally, the geometry of the roads could affect several factors such as when two vehicles will start noticing each other, when the road negotiation will begin, and when drivers will agree on who should go ﬁrst.
All these factors could then inﬂuence the velocity proﬁles of two vehicles but won’t have much effects on the intention of two vehicles when the aforementioned assumptions hold.
In other words, if two vehicles encounter each other twice under different domains, as long as the three assumptions are true, the aggressive driver will always prefer passing than yielding the less aggressive driver.
The intention is determined by features that relate to two drivers’ information and their internal relations (e.g. highlighted in cyan), instead of those that relate to domain itself (e.g. highlighted in orange).
Hence, we construct a model for the data collection process that assumes each input data (i.e. vehicle trajectories) is constructed from a mix of causal and non-causal features.
We consider domain as a main intervention that changes the non-causal features of an input data, and propose that an ideal intention predictor should depends only on the causal features.
Although the aforementioned assumptions will not hold for real-world driving data, we are still able to assume each data is constructed from both causal and non-causal features.
However, it becomes unclear what are the causal and non-causal features.
Therefore, in this work, we construct a structural causal model and propose to learn an invariant representation for domain generalization based on invariant conditions identiﬁed from the causal model.
B.2 Model Insights
According to [45], causality, with its focus on representing structural knowledge about the data generating process that allows interventions and changes, can contribute towards understanding and resolving some limitations of current machine learning methods.
In fact, machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efﬁcient and generalize better [46, 47, 48, 49].
For human drivers, the way they interact with each other under different domains should also processes some invariant structural relations, since they can quickly adapt their driving skills to novel scenarios.
Therefore, we construct a structural causal model (SCM) for vehicle intention prediction task to learn an invariant representation of input driving data for domain generalization.
The domain generalization setting assumes the existence of domain-invariant patterns in the inputs (e.g. semantic features), which can be extracted to learn a predictor that performs well across seen and unseen domains.
Moreover, causal predictive models are shown to have great generalizability since their output depends on stable relationships between input features and the outcome instead of associations between them.
Therefore, in this work, we construct a structural causal model inspired from data generation process and propose to learn an invariant representation from temporal latent dependencies of input data for domain generalization, where the learning scheme is based on an invariant condition identiﬁed from the causal model.
To be more speciﬁc, we assume each input data is constructed from a mix of inherent (causal) and domain-dependent (non-causal) features.
Consequently, the captured temporal latent dependencies from sequential data also contain both causal and non-causal features.
To generate a pair of interactive trajectories from two vehicles, data-generating process ﬁrst samples two drivers’ driving preferences ( O ) and a domain ( D ) that could be correlated to each other shown with dashed arrow in Fig.
1 (e.g. driver behavior may vary from country to country ).
Then, an interaction event ( E ) is sampled, which could correlate to domain information (e.g., the geometry of the roads may affect when two vehicles can notice/observe each other).
The domain information is correspond to non-causal features ( X NC ) and the driver information are correspond to causal features ( X C ).
Part of the event information, such as initial speed or location of two vehicles, are correspond to causal features which will inﬂuence the intention label.
Another part of the event information, such as when the road negotiation begins, are correspond to non-causal features.
Altogether, the causal and non-causal features construct the interactive trajectories ( X ) from where the time-related latent information ( Z ) are captured.
We further utilize D-separation to analyze the formulated structural causal model as it can: (1)connects causal relations and probabilistic independence; (2)computes all and only those independence and conditional independence relations that hold for all values of the parameter for any directed graph.
At the lowest level of the causal hierarchy — association — we have discovered DAGs and d-separation as a powerful tool to reason about conditional (in)dependencies between variables.
C Related Theory C.1 D-separation
Here we provide some background concepts related to d-separation used in Section 2 for analyzing our causal graph.
Deﬁnition 1 (d-connection) :
If G is a directed graph in which X, Y and Z are disjoint subsets of nodes, then X and Y are d-connected by Z in G if and only if there exists an undirected path U between
X and Y such that for every collider C on U, either C or a descendent of C is in Z, and no non-collider on U is in Z.
Deﬁnition 2 (collider) :
For any path between two nodes, a collider is a node where arrows of the path meet head-to-head.
Deﬁnition 3 (d-separation) :
X and Y are d-separated by Z in G if and only if they are not d-connected by Z in G, denoted (X ⊥⊥ Y )
| Z.
[Figure]
C.2 Worse-case Generalization Error
As mentioned in Suppl.
A, utilizing causal structures to learn invariance factors has shown better model generalizability on unseen data and we also demonstrate desired performances of our causalbased model in Section 2.
Theoretically (i.e. Theorem 1), the worse-case generalization error for a causal model is less than or equal to that for the associational model (i.e. model that directly map from X into Y).
Detailed proof can be found in Suppl.
A.2 in [20].
Theorem 1.
Consider a causal model h min c,D : X c → Y and an associational model h min a,D : X → Y trained on a dataset D ∼ P ( X, Y ) with loss L .
Let ( x, y ) ∈ D and ( x (cid:48) , y (cid:48) ) / ∈ S be two input instances such that they share the same true labelling function on the causal features,
y ∈ P ( Y |
X c = x ) and y (cid:48)
∈ P ( Y | X c = x (cid:48) ) .
Then, the worst-case generalization error for the causal model on such x (cid:48) is less than or equal to that for the associational model. [Equation]
[Equation]
C.3 Justiﬁcation of Minimizing Contrastive Loss
According to the invariance condition derived in Section 2.2, the idea function we want to optimize is f ∗ =
arg min h E [ l ( y, h ( x c ))] assuming x c is known.
However, as X C is unobservable, we need to alternatively minimize the objective function in Eq.(5).
Inspired from Theorem 2 in [19], we are able to derive the following theorem under our problem setting.
We can observe that whether X C will be returned by the optimization largely depends on δ c .
Speciﬁcally, if a dataset has low δ c , then the learned representation will likely to be closer to the real X C .
Alternatively, if a dataset has high δ c , the matching condition loses any discriminative power and thus we need to additionally learn a matching function ω to minimize δ c .
Therefore, minimizing the contractive loss (Eq.(3)) is necessary in our problem.
Detailed proof or the original theorem can be found in Suppl.
A.6 in [19].
Theorem 2.
Assume training domains such that for any two same-class inputs x ( k ) j and x ( k (cid:48) )
j from domains k and k (cid:48) , dist ( x ( k ) ac,j , x ( k (cid:48) )
ac,m ) ≤ δ ac where x ac is any non-causal feature.
Further, assume that the distance over X C between same-class inputs from different domains is bounded: dist ( x ( k ) c,j , x ( k (cid:48) ) c,m ) ≤ δ c and δ c < δ ac ( δ c , δ ac ∈ R + ) . Then for some γ and λ , a loss-minimizing classiﬁer for the loss from Eq.(5) is the true function f ∗ , given a P-admissible loss function and a ﬁnite number of domains k with K → ∞ in each domain.
D Experimental Details D.1 Data Information
All data are collected by a drone from bird’s-eye view with 10 Hz sampling frequency.
Information such as reference paths and trafﬁc regulations can be directly extracted from high deﬁnition maps that are in lanelet [50] format.
The detailed information of each domain can be found in Table 2, where we observe that any two domains are different in at least two out of ﬁve aspects.
The intersect angle refers to the angle between two reference paths 2 .
The average length refers to the average length of two reference paths.
Instead of Cartesian coordinate, we utilized the Frenét Frame to represent vehicle’s state.
The vehicle motion in the Frenét Frame can be represented with the longitudinal position along its reference path s ( t ) , and lateral deviation to the reference path d ( t ) .
Therefore the vehicle state at time step t can be deﬁned as ξ t =
( s ( t ) , d ( t )) .
Note that the reference path of a vehicle will change according to the road it is current driving on.
The origin of the reference path can be deﬁned differently according to different objectives and each reference path will have its own Frenét Frame.
Since we are dealing with interaction between two vehicles, we deﬁne the origin as the intersection point (red circles in
Fig. 2) of their reference paths.
We manually select the interaction data from the dataset.
Since we focus on two-vehicle interaction situations, we need to make sure the two vehicles are highly interactive with each other and both vehicles’ behaviors should not be inﬂuenced by any other vehicles.
Also, for ease of interpretation, we balance the data such that there are equal number of interactive trajectory data per class in each domain.
We select a total number of 1354 time-series data from three domains in USA_Roundabout_FT and 138 time-series data from the single domain in CHN_Merging_ZS.
In terms of the labeling process, since we have the entire interaction trajectory of two vehicles, we are able to obtain the ﬁnal outcome directly based on which vehicle passes the conﬂict point ﬁrst.
[Table]
We categorize the intention for two interacting vehicles as either pass or yield and consider only the period that two vehicles have high interaction.
In order to make sure that selected vehicle pairs are highly interactive and their behaviors are not inﬂuenced by other surrounding vehicles, we manually examine and ﬁlter the data.
The are two reasons why we ﬁlter out the data that two vehicle are far from each other.
Firstly, it is difﬁcult to obtain true labels of drivers’ intentions in such situation and their intentions may not simply be pass/yield.
Secondly, if two vehicle are far from each other, there will be a large chance that they either haven’t started interacting with each other or may interacting with other vehicles.
It is worth to mention that our proposed approach can be adapted to multi-class classiﬁcation with more subtle interaction patterns, but it might require human labeling.
D.2 Implementation
For domain generalization tasks, test/target domains are not accessible during training.
Therefore, we randomly sample 20% of data from source domains to construct our validation set which is used for hyperparameter tuning and use the remaining source domain data for training.
We use the Adam optimizer and run all models for 500 epochs with learning rate of 1 e − 3.
During training, we use a mini batch of 16 samples and apply early stopping criteria that the model does not experience a decrease in the validation loss for 20 epochs.
The temperature scaling parameter τ is set to 0.05, and the two control parameters γ and λ are set to 1 e − 4 and 1 , respectively.
For results of each model, we report the average and standard deviation over 3 independent runs.
Input dimension of each data is 10 × 4 , where we consider the past 1s of interactive trajectories with 10Hz sampling frequency and concatenate the states of two vehicles.
Output classifer dimension is 2 corresponding to the possible number of intentions.
Network modules in our model comprises: an encoder network ϕ encτ , which output parameters deﬁne the inference model q
( z it | x i<t , z i<t ) (this is also the representation function q ( x ) ); a prior network ϕ priorτ , which output parameters deﬁne the prior distribution p ( z it | x i<t , z i<t ) ; a decoder network ϕ decτ , which output parameters deﬁne the generative model p ( x it |
z i ≤ t , x i ≤ t ) ; feature extraction networks ϕ xτ and ϕ zτ for x and z , respectively; a recurrent process layer for hidden state update; a hypothesis function φ ( z ) ; and a function h ( x c ) that maps causal features to the output intention.
Speciﬁcally, all ϕ τ consist of two hidden layers of 16 neurons with ReLU activation, where the input dimension is 4 and output dimension is 16.
The hidden size of the recurrent layer is set to 16, where the input dimension is 32 and output dimension is 16.
We consider φ to be the last layer of the network with one hidden layer of 16 neurons and
ReLU activation, where the input dimension is 16 and output dimension is 2.
We take h to be identity function for simplicity.
The dimension of latent representation z is set to 2.
When implementing state-of-the-art methods, since most of the baseline models are not designed for time-series input data, we add a RNN layer in front of each model instead of simply ﬂattening the time-series input.
The total number of parameters of all the baselines and our method are set to about equal.
E Additional Experiments and Results E.1 Ablation Study
We conduct an extensive study using both FT and ZS data to investigate contribution of three components to our method’s performance: loss for extracting temporal latent dependencies, contrastive loss for learning matching function, and metrics to evaluate distance between two representations.
To be more speciﬁc, CTSDG w/o L v is the CTSDG model without adding the loss term L v , where no temporal latent dependencies are modeled while learning invariant representations from the input data; CTSDG w/o (cid:96) con is the CTSDG model without learning the matching function Ω , where the loss L r for satisfying invariance condition of X C is optimized by randomly selected input pairs;
CTSDG w/ s
(cid:96) 1 and CTSDG w/ s (cid:96) 2 are CTSDG models but using (cid:96) 1 and (cid:96) 2 as distance metric between two representations, respectively.
From the ablation study results in Table 3, we observe that by capturing the temporal latent dependencies and learning matching function for input pairs that share causal features, the model is able to have better performance especially when target domain has large domain shift compared to source domains (the fourth row).
Moreover, according to the results from using (cid:96) 1 and (cid:96) 2 as distance metric, the cosine similarity used in our method seems to be the most suitable measure in the causal feature space for this prediction problem. [Table]
E.2 Analysis on Invariant Representation
We utilize t-SNE [51] to analyze the quality of latent representation learned by our proposed model in
Figure 5.
We train our model using two domains (FT-1,2) and evaluate on two unseen domains (FT- & ZS).
It appears from Fig.
5(a) and (b) that CTSDG achieved a higher overlap between train and test domains for FT-3 than ZS, highlighting the difﬁculty of generalizing to the ZS domain.
Such ﬁnding coincides with our prior domain knowledge as well as quantitative results in Table 1.
Moreover, based on Fig. 5(d) and (e), we ﬁnd that the classes are well-separated in the training domains for both models.
We also observe that when the intention label is yield , the data is more evenly dispersed.
The reason could because the input time-series data with yield label are easier for the model to capture underlying domain representations that are invariant across time.
Whereas the time-series data with pass label could have more temporally diversed domain-invariant representations.
In order to illustrate the effectiveness of learning match function Ω , we further utilize t-SNE to compare the latent representation learned by CTSDG with that by CTSDG without the contrastive loss in Eq. 3.
Both models are trained using two domains (FT-1,2) and evaluated on an unseen domain FT-3.
According to Fig.
5 (a) and (c), the model with learned match function has higher overlap between train and test domains than the model with random match function, especially when ground-truth target intentions are yield .
We can also observe from Fig.
5 (d) and (f) that without minimizing contrastive loss (Eq. 3), target data from different classes cannot be well-separated in the latent representation, which generates miss-classiﬁcation cases.
Therefore, we can conclude that by enforcing the difference in individual representations for same-class inputs to be low through (Eq. 3), the model can better capture domain-invariant representations.
[Figure]
E.3 Analysis on Learned Temporal Latent Dependencies.
Figure 6 shows the temporal latent dependencies learned by our CTSDG for a given source-target pair.
We ﬁrst look at the two cases from source domain (i.e. ﬁrst column in Fig.
6(a,b)).
Looking at the neuron activation patterns along the time axis, we can see that neurons are more active when there are changes in either position or velocity proﬁles for two interacting vehicles.
In fact, these changes are related to causal features which play important roles in determining the output intentions based on our proposed causal model.
As we look at the two cases from target domain (i.e. second column in Fig.
6(a,b)), we observe consistent excitatory neuron ﬁring patterns at critical time steps, which indicate that CTSDG successfully transfers knowledge to unseen domains.
Therefore, we can conclude that our CTSDG method is able to capture temporal latent dependencies that are domain invariant and related to causal features, which enables it generalize time-series driving data to unseen domains.
[Figure] According to Table 1, IRM has the best performance among all baseline methods when the source domain is FT-1,2 and target domain is FT-3.
Therefore we compare the temporal latent dependencies learned by IRM with those learned by our method (see Figure 7).
We observe that IRM results in sporadic activation pattern and fails to capture and transfer temporal latent dependencies.
In contrast, our method can extract critical temporal dependencies that are relevant towards two vehicles’ intention and such dependencies can be further transferred from source to target domain.
[Figure]