Online Adaptation of Neural Network Models by Modified Extended Kalman
Filter for Customizable and Transferable Driving Behavior Prediction
Letian Wang1 , Yeping Hu2 , Changliu Liu1
1

Carnegie Mellon University, Pittsburgh, PA, USA 15213
University of California, Berkeley, Berkeley, CA, USA 94720
letianwang0@gmail.com, yeping hu@berkeley.edu, cliu6@andrew.cmu.edu
2

Abstract
High fidelity behavior prediction of human drivers is crucial
for efficient and safe deployment of autonomous vehicles,
which is challenging due to the stochasticity, heterogeneity,
and time-varying nature of human behaviors. On one hand,
the trained prediction model can only capture the motion pattern in an average sense, while the nuances among individuals can hardly be reflected. On the other hand, the prediction
model trained on the training set may not generalize to thetesting set which may be in a different scenario or data distribution, resulting in low transferability and generalizability.
In this paper, we applied a τ -step modified Extended Kalman
Filter parameter adaptation algorithm (MEKFλ ) to the driving behavior prediction task, which has not been studied before in literature. With the feedback of the observed trajectory, the algorithm is applied to neural-network-based models to improve the performance of driving behavior predictions across different human subjects and scenarios. A new
set of metrics is proposed for systematic evaluation of online
adaptation performance in reducing the prediction error fordifferent individuals and scenarios. Empirical studies on thebest layer in the model and steps of observation to adapt arealso provided.

Introduction
As more autonomous vehicles are being deployed on theroad, safety is the most essential aspect to consider especially when interacting with other dynamic road entities. Being able to make high-fidelity future motion predictions forsurrounding entities can greatly improve the safety level ofautonomous vehicles, since actions can be generated accordingly to avoid potentially dangerous situations.
The challenges for building accurate prediction models lie
in following aspects. Firstly, the prediction models are usually trained with a limited pool of real human motion data.
Though able to learn the average behavior pattern in thedata, the model is incapable to capture the nuances among
heterogeneous individuals. Secondly, due to the stochastic
and time-varying nature of human behaviors, the same individual may also exhibit different behaviors in different circumstances, which requires the prediction models to comprise dynamic parameters rather than fixed ones. Thirdly, although a trained model typically performs well on the training set, performance can drop significantly in a slightly dif-ferent test scenario or under a slightly different data distribution.
To address these problems, this paper extends an existinghierarchical prediction model (Wang et al. 2021a) for humandriving behavior to include adaptability in real time. The
pipeline of the framework is as follows: 1) learning the hierarchical prediction model via real human driving data, which
consists of a semantic graph network for intention prediction
and an encoder decoder network for trajectory prediction.
2) executing real-time τ -step online adaptation for the trajectory prediction by the modified Extended Kalman Filter
algorithm (MEKFλ ).
The contribution of the paper is as follows: 1) extendinga hierarchical prediction model to be adaptable to accountfor heterogeneous and stochastic behaviors among different
agents and scenarios. 2) applying MEKFλ online adaptation
algorithm to a more complex problem and demonstrate its
usefulness. 3) proposing a new set of metrics for systematic
evaluations of online adaptation performance. Such metrics
can be widely used to evaluate the performance of online
adaptation in regardless of models and problems. Via these
metrics, we also provided empirical studies on the best layer
and the best steps of observation to adapt.

Related Works
Various methods have been proposed to make driving behavior predictions, including traditional method such as hidden markov models (Deo, Rangesh, and Trivedi 2018), gaussian process (Zhang et al. 2021), and learning-based methods such as recurrent neural networks (Park et al. 2018), andgraph neural networks (Hu, Zhan, and Tomizuka 2020). In
terms of prediction objective, some works only focus on predicting intentions (Hu, Zhan, and Tomizuka 2018) or trajectories (Ma et al. 2019), while some other works make bothintention and trajectory predictions in a hierarchical manner (Wang et al. 2021a). While learning-based methods haveshown greater expressiveness with little human effort, taking
both intention and trajectory predictions into account could
simplify learning and make the model more interpretable. A
detailed survey can be found in (Rudenko et al. 2020).
To address individual differences, many works have beenutilizing feedback from historic observations to make adaptable predictions: adapting reward function parameters tomake driving-style-aware prediction via bayesian inferenceSemantic
Graph

Semantic Graph Network

Intention
SignalOnline Adaptation
Decoder

Encoder

GRU

Historic
State

...

GRU

Context
Vector

GRU...

Encoder Decoder Network

Error

Generated
Trajectory
GRU

Observed
Trajectory

Figure 1: Network architecture in our framework.
algorithm (Wang et al. 2021b), adapting the last layer of afeedforward neural network using the recursive least squareparameter adaptation algorithm (RLS-PAA) (Cheng et al.
2019), adapting the last layer of a recurrent neural network
using RLS-PAA (Si, Wei, and Liu 2019), and adapting any
layer of recurrent neural network using modified Extended
Kalman Filter algorithm (MEKFλ ) (Abuduweili and Liu2021).
To account for heterogeneous behaviors among agentsand scenarios, this paper extends the hierarchical prediction model (Wang et al. 2021a) to include adaptability. The
model conducts both intention and trajectory prediction onreal human driving data, which is a more complicated model
addressing more a complex problem compared to previous
works on online adaptation. Also, a new set of metrics is
proposed for systematic evaluation of online adaptation.

Algorithm 1: τ -step online adaptation with MEKFλInput: Offline trained EDN network with parameter θ, initial variance Qt and Rt for measurement noise and process
noise respectively, forgetting factor λOutput: A sequence of generated future behavior {Ŷt }Tt=1
1: for t = 1, 2, ..., T do
2:
if t ≥ τ then3:
stack recent τ -step observations:4:Yt−τ,t = [yt−τ +1 , ..., yt ]5:
stack recent τ -step generated trajectory:6:Ŷt−τ,t = [ŷt−τ +1 , ..., ŷt ]7:adapt model parameter via MEKFλ :ˆ∂Y8:Ht = ∂ θ̂t−τ,tt−1

9:
Kt = Pt−1 · HTt · (Ht · Pt−1 · HTt + Rt )−1
10:
Pt = λ−1 (Pt − Kt · Ht · Pt−1 + Qt )11:θ̂t = θ̂t−1 + Kt · (Yt−τ,t − Ŷt−τ,t )
12:
else
13:
initialization: θ̂t = θ
14:
end if15:
collect goal state gt from SGN and historic states St .
16:
generate future behavior:
17:Ŷt = [ŷt+1 , ...ŷt+Tf ] = fEDN (St , gt , θ̂t ).
18: end for
19: return sequence of predicted behaviors {Ŷt }T
t=1

Problem Formulation
Our goal is to generate more accurate driving behavior predictions in multi-agent traffic-dense scenarios. Formally, we
focus on making predictions for any selected car and its interacting cars in next Tf seconds Ŷt+1,t+Tf , based on observations of last Th seconds Ot−Th ,t :Ŷt+1,t+Tf = f (Ot−Th ,t ).

(1)

Specifically, we first make intention prediction with a highlevel semantic graph network (SGN), which takes in the semantic graph (SG) Gt−Tf ,t extracted from raw observation
Ot−Th ,t , and predicts the goal state gt :Gt−Tf ,t = fSG (Ot−Tf ,t ).

(2)

gt = fSGN (Gt−Tf ,t ).
(3)
The goal state gt is then delivered as the intention signal along with historic dynamics St−Th ,t to the low-level
encoder-decoder network (EDN) for trajectory prediction:Ŷt+1,t+Tf = fEDN (St−Th ,t , gt , θ).

Methodology
Our model consists of a high-level intention prediction
model, a low-level trajectory prediction model, and an online adaptation module.

Intention Prediction
During driving in intense-traffic environment, humans will
intentionally identify which slot is spatially and temporally
suitable to insert into. Then a determined slot and associatedgoal position is generated, which is then used to guide lowlevel actions such as acceleration or steering. Based on thisinsight, we first adopt the dynamic insertion areas (DIA) todescribe the slots on the road, which are regarded as nodes toconstruct a semantic graph (SG) Gt−Tf ,t . With SG, a semantic graph network (SGN) is then adopted to conduct relationship reasoning among the nodes in the graph and generate
future goal state gt . Detailed introduction on semantic graphand semantic graph network can be found at (Hu, Zhan, andTomizuka 2020; Wang et al. 2021a).

(4)

Trajectory Prediction

To endow the trajectory prediction model with the capability to capture individual and scenario differences, we set up
an online adaptation module, where a modified Extended
Kalman Filter (MEKFλ ) algorithm is utilized to adapt theparameter of the model in real time. Specifically, we regard
EDN as a dynamic system and estimate its parameter θ byminimizing the error between ground-truth trajectory in pastτ steps Yt−τ and predicted trajectory τ steps earlier Ŷt−τ,t :To generate future behaviors of arbitrary length with sufficient expressiveness, we use the encoder-decoder network(EDN) (Cho et al. 2014; Neubig 2017) as the low-level trajectory prediction policy. The EDN consists of two GRU
networks, namely the encoder and the decoder. At any time
step t, the encoder takes in the sequence of historic states
St = [st−Th , ..., st ] and compresses all these information
into a context vector ct :θt = fM EKFλ (θt−1 , Yt−τ,t , Ŷt−τ,t ).

ct = fenc (St ; θE ),

(5)

(6)

E

where θ denotes the parameter of encoder. The context
vector ct , the current state st , and the goal state gt generated by high-level policy are then fed into the decoder torecursively generate future behavior predictions:Ŷt = [ŷt+1 , ..., ŷt+Tf ] = fdec (ct , st , gt ; θD ),(7)

where θE denotes the parameter of encoder. Specifically, thedecoder takes vehicle’s current state st as the initial input
to generate the first-step trajectory. In every following step,
the decoder takes the output value of last step as the input
to generate a new output. In this paper, we choose graphrecurrent unit (GRU) as the basic RNN cell and stack threedense layers on the decoder for stronger decoding capability.
According to the empirical results in (Wang et al. 2021a),
we append the goal state feature gt to the end of original
input feature vector. The loss function is designed to canbe simply designed minimize the error between the groundtruth trajectory and the generated trajectory:L=

N
X||Ŷi − Yi ||2 ,

(8)

i=0

where N denotes the number of training trajectory samples.

Online Adaptation
Adaptable prediction to individuals requires the adaptationof parameters in the neural network in real time. Due to thehuge time delay in obtaining ground-truth intention label, in
this paper, we only consider the online adaptation for the
low-level behavior-prediction policy (EDN). Formally, we

5
4

3
2

1

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Percentage

Percentage
Percentage

Percentage

Percentage
Wih E Whh E Wih D Whh D W1F W2 F W3 F

Percentage Percentage

Figure 2: An illustration for the online adaptation process
and the proposed new set of metrics: 1) online adaptation
process: at time step t, the model parameters can be adaptedby minimizing prediction error of the trajectory in past τsteps Yt−τ,t . The adapted parameter is then used to generateprediction Yt on current time t; 2) a new set of 4 metrics areproposed to systematically evaluate the adaptation performance. ADE 1 can verify how the adaptation works on thesource trajectory Yt−τ,t . ADE 2 can verify whether adaptation can benefit short-term prediction in the presence of
behavior gap between earlier and current time. ADE 3 verifies whether we have obtained enough information from thesource trajectory Yt−τ,t . ADE 4 verifies whether adaptation
can benefit long-term prediction at current time t.

Adaptation Step

ADE 3: Error on Y t - τ

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Percentage

6

ADE 1: Error on Y t - τ, t

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Percentage

Historic TrajectoryYt-τ

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Percentage Percentage

7

Adapt Stepsτ

Percentage

Historic Timet-τ

Percentage Percentage

Yt

Percentage

8

Current Trajectory

Percentage

ADE 2: Error on Y t, t + τPercentage Percentage

ADE 4: Error on Y t
Current Timet

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Wih E Whh E Wih D Whh D W1F W2 F W3 F

Adapted Parameter

Adapted Parameter

Intersection

Roundabout

Scenario

Error Reduced

Error Raised

Figure 3: Percentage of error (ADE 2) reduced or raised after online adaptation, under 1) different adaptation step τ ;2) different layer of parameters adapted; 2) different scenarios. Three conclusions from the results: 1) online adaptation
works the best around the adaptation step of 2 or 3; 2) prediction accuracy was improved by a higher percentage inthe roundabout scenario (transferred) than in the intersection scenario (trained); 3) the best adaptation performance isobtained usually by adapting W3F , the last layer of the FC
network in the decoder.

regard the adaptation of the neural network as a parameter
estimation process of a nonlinear system with noise:
Yt = fEDN (St , gt , θ̂t ) + ut

(9)θ̂t = θ̂t−1 + ωt

(10)where θ denotes the parameter of EDN; the measurement
noise ut ∼ N (0, Rt ) and the process noise ωt ∼ N (0, Qt )are assumed to be Gaussian with zero mean and white noise.
For simplicity, we assume Qt = σq I and Rt = σr I where
σq > 0 and σr > 0.
A modified Extended Kalman Filter (MEKFλ ) algorithm
is then developed as in Algorithm 1 to make linearization of
the model and adapt arbitrary layer in the neural network. To
realize feedback decay from older observations, we modified
EKF to include the forgetting factor λ. Note that a multi-stepADE1: Error on Historic Steps

ADE2: Error on Current Steps

ADE3: Error on Historic Traj

ADE4: Error on Current Traj(a) Adaptation Results on Intersection Scenario
ADE1: Error on Historic Steps

ADE2: Error on Current Steps

ADE3: Error on Historic Traj

ADE4: Error on Current Traj(b) Adaptation Results on Roundabout Scenario
Before Adaptation

Improved After Adaptation

Declined After Adaptation

Figure 4: Online adaptation performance analysis. 1) ADE 1 and ADE 3: as adaptation step τ increased, the online adaptation
could get more information and improve prediction accuracy by a higher percentage. 2) ADE 2: as τ increases, the improved
percentage first grew higher due to more information obtained, but then declined due to the behavior gap between earlier timeand current time. When τ was 3, the short-term prediction was improved by 20% on intersection scenario (customizability) and
28% on roundabout scenario(transferability). 3) ADE 4: online adaptation can barely benefit long-term prediction.
adaptation strategy is considered, meaning that prediction
error of τ step is taken as the feedback to adapt the model.

layer the FC network in the decoder, is the best to adapt.

Trade-off on adaptation step τExperiment
Experiment setting
We verified our method with real driving data from INTERACTION dataset (Zhan et al. 2019). Our model is trained on
the MA intersection scenario by splitting 19084 data points
to 80% of training data and 20% of testing data. The modelis then directly tested on the FT roundabout scenario with
9711 data points to evaluate the transferability. In our experiments, we choose the historic time steps Th as 10 and future time step Tf as 30 with frequency of 10Hz. We utilized
Adam optimizer and sweep over more then 20 combinationsof hyperparameter for best performance. We proposed a new
set of 4 metrics to systematically evaluate the performanceof online adaptation, as show in Figure 4.

The best layer to adapt
There exist many layers in the END. We denote the encoder
E
GRU’s input-hidden weights as Wih, the encoder GRU’s
E
hidden-hidden weights as Whh , the decoder GRU’s inputD
hidden weights as Wih, the decoder GRU’s hidden-hiddenDweights as Whh , and the weights of the three-layer FC
stacked on top of decoder as W1F , W2F , W3F respectively.
While it remains theoretically unknown which layer is thebest to adapt due to the lack of interpretability of neural networks, as in Figure 3 we empirically found that W3F , the last

When we increase the adaptation step τ , there is a trade-off
between how much information we can get and the behavior
gap between earlier time t − τ and current time t. Figure 4
shows our exploration on the best step τ to adapt. Several observation can be drawn: 1) prediction performance decayedwhen the model was directly transferred to the roundabout
scenario. 2) online adaptation could benefit short-term prediction as in ADE 2, and could barely improve long-term
predcition as in ADE 4. 3) When the τ increases the improved percentage in ADE 2 first rose, reached peak τ of 2
or 3, and then dropped. Specifically, the ADE 2 is reduced
by 20% in the intersection scenario (customizability), andby 28 % in the roundabout scenario (transferability).

Conclusion
This paper extended a hierarchical prediction model to include adaptability, where MEKFλ was used to adapt to different human drivers and driving scenarios. A new set of
metrics is proposed to systematically evaluate the performance of online adaption. Our experiments demonstrated
online adaptation could improve short-term prediction accuracy by 20% and 28% in the trained and transferred scenario respectively. We also empirically found the best choiceof layer is the last layer of FC in the decoder, and the best
adaptation step τ is around 2 or 3. Future efforts include online adaptation of intention predictions, online adaptation onmore driving scenarios, and interpretation of neural network.

References
Abuduweili, A.; and Liu, C. 2021. Robust nonlinear adaptation algorithms for multitask prediction networks. International Journal of Adaptive Control and Signal Processing,35(3): 314–341.
Cheng, Y.; Zhao, W.; Liu, C.; and Tomizuka, M. 2019.
Human motion prediction using semi-adaptable neural networks. In 2019 American Control Conference (ACC), 4884–
4890. IEEE.
Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau,
D.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014.
Learning phrase representations using RNN encoderdecoder for statistical machine translation. arXiv preprint
arXiv:1406.1078.
Deo, N.; Rangesh, A.; and Trivedi, M. M. 2018. How would
surround vehicles move? a unified framework for maneuverclassification and motion prediction. IEEE Transactions onIntelligent Vehicles, 3(2): 129–140.
Hu, Y.; Zhan, W.; and Tomizuka, M. 2018. A frameworkfor probabilistic generic traffic scene prediction. In 2018
21st International Conference on Intelligent Transportation
Systems (ITSC), 2790–2796. IEEE.
Hu, Y.; Zhan, W.; and Tomizuka, M. 2020. Scenariotransferable semantic graph reasoning for interaction-aware
probabilistic prediction. arXiv preprint arXiv:2004.03053.
Ma, Y.; Zhu, X.; Zhang, S.; Yang, R.; Wang, W.; andManocha, D. 2019. Trafficpredict: Trajectory prediction forheterogeneous traffic-agents. In Proceedings of the AAAI
Conference on Artificial Intelligence, volume 33, 6120–
6127.
Neubig, G. 2017.
Neural machine translation andsequence-to-sequence models: A tutorial. arXiv preprint
arXiv:1703.01619.
Park, S. H.; Kim, B.; Kang, C. M.; Chung, C. C.; andChoi, J. W. 2018. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In
2018 IEEE Intelligent Vehicles Symposium (IV), 1672–1678.
IEEE.
Rudenko, A.; Palmieri, L.; Herman, M.; Kitani, K. M.;
Gavrila, D. M.; and Arras, K. O. 2020. Human motion trajectory prediction: A survey. The International Journal of
Robotics Research, 39(8): 895–935.
Si, W.; Wei, T.; and Liu, C. 2019. AGen: Adaptable Generative Prediction Networks for Autonomous Driving. In 2019
IEEE Intelligent Vehicles Symposium (IV), 281–286.
Wang, L.; Hu, Y.; Sun, L.; Zhan, W.; Tomizuka, M.;
and Liu, C. 2021a. Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction.
arXiv:2111.00788.
Wang, L.; Sun, L.; Tomizuka, M.; and Zhan, W. 2021b.
Socially-compatible behavior design of autonomous vehicles with verification on real human data. IEEE Robotics
and Automation Letters, 6(2): 3421–3428.

Zhan, W.; Sun, L.; Wang, D.; Shi, H.; Clausse, A.; Naumann,
M.; Kümmerle, J.; Königshof, H.; Stiller, C.; de La Fortelle,A.; and Tomizuka, M. 2019. INTERACTION Dataset:
An INTERnational, Adversarial and Cooperative moTION
Dataset in Interactive Driving Scenarios with Semantic
Maps. arXiv:1910.03088 [cs, eess].
Zhang, C.; Zhu, J.; Wang, W.; and Xi, J. 2021. Spatiotemporal Learning of Multivehicle Interaction Patterns in LaneChange Scenarios. IEEE Transactions on Intelligent Transportation Systems, 1–14.