Inverse Reinforcement Learning with Model
Predictive Control
Liangjun Zhang
Baidu Research Institute
Sunnyvale, CA
liangjunzhang@baidu.com

Jinxin Zhao
Baidu Research Institute
Sunnyvale, CA
jinxinzhao@baidu.com

Abstract
Model-free learning based methods for planning and control application have been
proven promising by many existing results. However, such kind of planning andcontrol algorithms are rarely used in practical systems, due to the unpredictable
outputs. On the other hand, model-based methods are largely deployed in real-lifesystems, with guarantee of operational safety but complaint of lacking close-tohuman behavior. In this article we make an effort to leverage the benefits ofmodel-based optimization method and model-free learning method by introducing
a novel scheme of inverse reinforcement learning. We then present a framework
for combining human behavior model with model predictive control. The idea
is to take advantage of the feature identification capability of neural network
to determine the reward function of model predictive control. Furthermore, theproposed approach is implemented to solve the practical autonomous driving
longitudinal control problem. In such problem, safe execution and passenger
comfort are simultaneously preferred.

1

Introduction

Selecting the most fruitful option by foreseeing the future, describes the nature of model predictive
control (MPC). At each time instant, MPC aims to find the control input by solving an optimization
problem. This optimization considers the costs/rewards of the future steps and the prediction isachieved by exploiting a state-space model [Mayne et al., 2000]. MPC has drawn enormous attention
from both industrial and research perspectives. Industrial applications of MPC started from the fieldof chemical process control. In decades, many companies like Shell, Honeywell have been developing
MPC packages for industrial uses [Qin and Badgwell, 2003]. Nowadays successful implementationsof MPC, in addition, lie in the area of power electronics with various topics including active frontend (AFE), power converters connected to resistor-inductor (RL) loads [Vazquez et al., 2014]. Apartfrom industrial applications, MPC has also been an on-going topic in large amount of research
projects. A real-time MPC scheme is described in [Erez et al., 2013] for the control of humanoid
robots, where MPC generates the trajectories for full body dynamics based on the received subtasks. Aerial and ground mobile robotic researchers also implement MPC algorithms to resist the
dynamic environment and confront the system constrains [Shim et al., 2003, Künhe et al., 2005].
Moreover, with the surge of self-driving vehicle development in present-day, MPC is showing evenmore significance in such fields. [Schmied et al., 2015, Borrelli et al., 2006] describe longitudinal
control approaches for adaptive cruise control and vehicle tractor control to improve the vehicle
emission and fuel-consumption efficiency. Meanwhile active steering lateral vehicle controls focus
on collision avoidance, which is made possible by MPC implementation with reasonable dynamic
model for prediction [Borrelli et al., 2005, Falcone et al., 2008, Frasch et al., 2013].
Machine Learning for Autonomous Driving Workshop at the 33rd Conference on Neural Information Processing
Systems (NeurIPS 2019), Vancouver, Canada.

Despite all the advantages and successful applications of MPC, it still suffers certain drawbacks likedifficulty of choosing parameters and lack of adaptivity. During the implementation phase of MPC,
selection of the parameters such as prediction horizon and optimization gains require lots of trial anderror in either simulation or hardware in the loop (HIL) test. In addition, previously fine-tuned MPC
controller in most cases does not adapt to the changes of the system. For example, variation of theplant (under-controlled) system parameters results in the state prediction model failing to provide
meaningful prediction. Also, the change of the reference trajectory pattern, which may cause thepredefined cost function not able to conclude a satisfactory tracking performance anymore. Moreover,in the field of autonomous driving, another issue of model-based algorithm like MPC is that thegenerated behaviors do not always align with the expectation of a human, resulting into certain levelof discomfort. The root cause is that the cost/reward functions are predefined and lack of variation.

2

Related WorkOne option to overcome such issues is to exploit finite state machine (FSM) with MPC to update
the prediction model or cost function parameters as explained in [Rodriguez et al., 2012, Arahal
et al., 2009]. However, such simple approach of combining several sets of parameters, does not avoid
tremendous amount of labor of parameters tuning and still requires delicately manual design of states
and transitions. On the other hand, deep learning techniques have shown enormous potential in thefields such as object detection, prediction and classification [LeCun et al., 2015, Schmidhuber, 2015].
An active research topic of utilizing capability of deep learning techniques for building self-driving
vehicles is end-to-end driving as discussed in [Pomerleau, 1989, Xu et al., 2017], while such methods
suffer oscillating control input signals and are not friendly for real-life implementations.
On the other hand, utilizing the identification capacity of data driven method to complete the modelbased optimization algorithm provides another viable perspective. Similar idea was investigatedwithin the topic of inverse reinforcement learning (IRL) [Abbeel and Ng, 2004, Ng et al., 2000],
which aims to recover the cost function by observing a desired system trajectory or policy. Whereas
most problems formulated in the contexts of IRL are finite-state Markov decision process (MDP)[Zhifei and Joo, 2012] with Bayesian probability state transition model, MPC algorithm in most casedeals with continuous state space systems. An interesting result is presented in [Silver et al., 2010],
where the behavior model, that translates state features (observation) to cost functions, was learnt
from demonstration.
In this article, we propose a framework for combining neural network (NN) model with MPC
algorithm, where the NN is pre-trained to recover the cost function for MPC, with respect to theobservation of the environment. One major challenge of such method is the under-determination,
where the observation does not provide enough bijective mapping from observation to label. Anothermajor challenge is that a behavior model with zero parameters also satisfies the optimality condition
with observation data, but this case has to be avoided in order to provide meaningful cost/reward
function. Autonomous vehicle longitudinal control problem could largely benefit from the propose
framework by learning the human-intended behavior. We thus provide a demonstration of solving
such problem with the proposed method.

3

Inverse Reinforcement Learning of MPC

3.1

Problem Formulation

Here we formulate the problem of inverse reinforcement learning for a system with MPC framework.
Model predictive control generates the control input for the plant (under-controlled) system by solving
an optimization problem. This process is repeated at each time instance and the first element of theobtained control sequence is provided to the system. Compared with the classic feedback control law,
the main advantages of MPC is bi-fold. One is that the MPC control input is searched from moregeneral space instead of just the linear combination of the state errors in classical control, anotherone the is that the future system states and references are also considered rather that only considering
the current state and the current reference signal.
2

A MPC algorithm is commonly formulated as follows, providing the current state is xk and prediction
horizon is N ,k+N
X−1
minimizeuk:k+N −1
Cθ (xi , ui ) + F (xk+N )i=k(1)
subject toxi+1 = f (xi , ui )u ≤ ui ≤ ūnmwhere xi ∈ R , ui ∈ R ; Cθ is a cost function and θ represents the function parameters; F
represents a final cost function.
In this article, we propose a novel framework of making use of neural network to predict theappropriate cost function for the MPC algorithm, i.e.,Cθ = gψ (yk ),(2)
where yk is the observed information at step k.
As mentioned above, the benefit of such architect is that the neural network model is capable ofprovide suitable and various cost functions according to the observation of the environment. However,it also introduces the difficulty for the training process. The objective is to train this neural networkmodel with favorable recorded trajectories, so that the entire algorithm is capable of generating
desired behavior while guaranteeing optimality in later use. The problem is formulated as follows.
Problem 1 Given the MPC algorithm described in (1) and the neural network structure in (2),design the training process with pre-recorded desired trajectories. So that the cost function canbe reconstructed and the output of the algorithm shares similar behavior with compared with therecorded trajectory .
3.2

Approach

In a normal imitation learning problem, the recorded data can explicitly provide the corresponding
relation between the input/observation and output/label of a model. However, this is not the casefor Problem 1, since the direct outputs of the MPC cost function Cθ can not be explicitly known orrecorded.
We provide the solution to Problem 1 here by presenting how to enable model training process
through recorded trajectory data. The idea is to exploit the Karush–Kuhn–Tucker (KKT) condition,
so that the a bijective mapping between observations and labels can be constructed.
Consider the MPC system configuration in (1), we further restrict the cost function to have a quadratic
form and omit the final state cost function F (·),
1Cθ (xi , ui ) = (xTi Qk xi + uTi Rk ui );(3)2
following this format, the output of the neural network should be the matrices Qi and Ri , that isθ = (Qk , Rk ) = gψ (yk ),(4)
where ψ represents the parameters of the neural network gψ (·). Along the dimension of time, theunder-controlled system physically locates at step k, and the cost function parameters (Qk , Rk )remain constant for the prediction steps i in equation (1). After the control input is generated andapplied to the actual system and the actual system proceeds to step k + 1, the cost function parameters
will then re-adjust and the optimization problem will be solved again for step k + 1.
Now, the original Problem 1 has been further concretized that we need to introduce a mechanism ofusing neural network to predict the parameters (Qk , Rk ) of cost function based on the observation
sequence yk . Given the observed samples of data sequence, our goal is to train a neural network suchthat it varies the MPC cost function parameters according to the observation to imitate the behavior
encoded inside the data. Here we assume the hard constraint is always not active, which means
the inequality constraint has always been satisfied before applying it. Consider the optimization
described in (1) with cost function defined in (3), the Lagrangian is written as
L(X, U, λ) =k+N −1
1 X(xTi Qk xi + uTi Rk ui ) + λT (k)F(X, U)
2
i=k

3

(5)

where the variables are defined as follows,
X = col(xk , xk+1 , ..., xk+N −1 )U = col(uk , uk+1 , ..., uk+N −1 )λ(k) = col(λ0 (k), λ1 (k), ..., λN −1 (k)), and λi (k) ∈ Rn
F(X, U) = col(f (xk , uk ) − xk+1 , ..., f (xk+N −1 , uk+N −1 ) − xk+N ).
The necessary and sufficient conditions for optimality of solution to the problem (1) is the KKTconditions [Boyd and Vandenberghe, 2004] as follows,∂L∂FT (X, U)= QX +
λ(k) = 0,∂X∂X∂L∂FT (X, U)= RU +
λ(k) = 0,
∂U
∂U

(6)

where
Q = diag(Qk , Qk , ..., Qk ),R = diag(Rk , Rk , ..., Rk ).

Upon solving the MPC optimization problem at each time instant, a sequence of control inputs are
generated, whereas only the first one is actually fed to and executed by the system. Hence in therecorded trajectory, each control input data point only represents the first element of the solution tothe optimization problem (1) at each time step.
Suppose X̃ = col(x̃0 , x̃1 , ..., x̃l ) is the recorded sequence of the system trajectories and Ũ =col(ũ0 , ũ1 , ..., ũl ) is the recorded sequence of the recorded system control inputs. Assuming thoserecorded trajectories are generated through a MPC solver, then each data pair needs to satisfy thefollowing condition
Qk x̃k +∂f T (x, u)|x=x̃k ,u=ũk λ0 (k) = 0∂x

Rk ũk +∂f T (x, u)|x=x̃k ,u=ũk λ0 (k) = 0.
∂u

(7)

At this point, we can see the first challenge of solving problem 1. Even given known system dynamics
f (x, u), one data pair (x̃k , ũk ) is not enough to recover the matrices of Qk and Rk .
We further restrict the form of the matrices Qk and Rk to be diagonal, which means
Qk = diag(qk ),Rk = diag(rk ),where col(q, r) should be generated by the prediction neural network model. The diagonal form of
these matrices is commonly used in MPC applications. Thus the neural network model is represented
as follows,col(qk , rk ) = gψ (ỹk ).
3.3

Model Training

Given a sequence of data pairs ((ỹ1 , x̃1 , ũ1 ), (ỹ2 , x̃2 , ũ2 ), ..., (ỹl , x̃l , ũl )), towards satisfying the
optimality condition described in (3.2), the loss function L is chosen asL=lX

J(λ0 (k)∗ ), λ0 (k)∗ = argminJ(λ0 (k)),

J(λ0 (k)) = kGλ0 (k) + Hcol(qk , rk )k, (8)k=0where the matrices G and H are defined as
" T
#
∂f
|
∂xT x=x̃k ,u=ũk
G = ∂f∂u |x=x̃k ,u=ũk

H = diag(x̃k , ũk ).

In this case, matrix G can be obtained from the system dynamics and matrix H can be constructedfrom the recorded trajectory data.
Here it can be seen another challenge of solving the proposed problem, which is
λ0 (k)∗ = 0,

col(qk , rk ) = 0
4

is an optimal solution to the minimization problem (8).
To avoid the optimal solution being trapped at zero, we introduce the following procedure for updating
the parameters ψ of the neural network gψ (·). The loss function in (8) is re-written as
J(λ0 (k)) = kU vk,

H] ,U = [Gv = col(λ0 (k), qk , rk ).

(9)

The row dimension of U is n + m and the column dimension of U is 2n + m, where n is thedimension of system state space and m is the dimension of the system input space. From (9), it canbe seen that as long as v lies inside the null space of U , J is minimized. Now let the columns of amatrix W span the null space of U , i.e.,
Null(U ) = span(w1 , w2 , ...)
where w1 , w2 ,... are columns of matrix W . Hence, for any vector η, letv = W η.
Then v is an optimal solution to the problem (8).
In this article, we exploit the idea of expectation-maximization(EM) algorithm for the training process.
First of all, η is initialized with η = η0 . Then, given the known system dynamics and data pair, matrix
W is calculated. Following that the guess value of the neural network output can be computed asõ = W η. Finally, the loss of the neural network is defined asXL=kõ − ok,
o = gψ (yk ).
And the loss is back-propagated to update the parameters of the neural network. Meanwhile, η isupdated by solving the least squares problem
minimizeη kW η − ok.
These steps are then iterated until the neural network parameters converge. Summary of the algorithm
is illustrated in algorithm 1.

4Autonomous Vehicle Longitudinal Control

Algorithm 1: Inverse Reinforcement Learning
MPC
Input: X̃, Ũ and Ỹ
Output: gψ (·)
1 η = η0 , L = L0 and L̄;
2 while L > L̄ do
3forward propagation o = gψ (Ỹ);
4construct matrix U ;5compute the null space matrix W ;6update η as η = argminη kW η − ok calculatePthe loss L =kW η − ok ;7back propagation to update network
parameters gψ (·) ;8 end

In this section, the proposed approach is applied
for the design of a longitudinal controller forautonomous vehicle. First a data-set generated
through simulation is used for training and thena publicly available data-set is exploited for performance comparison.
4.1

Algorithm formulation

Model-based optimization method such as
linear-quadratic-regulator (LQR) and MPC arelargely developed and deployed for autonomous
longitudinal vehicle control. However, substantial amount of complaints are reported becauseof the inconsistency between the algorithm generated behavior and human-expected behavior.
One interesting example is how to approach astatic vehicle stopping ahead of the plant (undercontrolled) vehicle. Human driver may speed
down far away and approach to the static vehicle
at a lower speed; on the contrary, optimization
based method usually commands the plant (under-controlled) vehicle to approach the static vehicle
rather fast followed by a late braking to stop behind the static vehicle. The reason is that the costfunction penalizes on the arrival time so that the vehicle travels as fast as possible to reduce the cost.
Adjusting the cost function could be an option to improve the passenger experience, but manualparameters-tuning takes lots of effort and the same cost function may not necessarily improve thedriving performance for every driving scenario.
5The proposed scheme in this article on the other hands aims to alleviate such drawback by imitating
the behavior of human through a recorded data set in the training phase and reconstruct the costfunction based on the observation later in the deployment phase.
For such longitudinal control problem, we define the under-controlled autonomous vehicle as “egovehicle” and call the vehicle in front as “leading vehicle”. Furthermore, the states of the system andthe system update function can be described as
"
#
"
#
"
#
dl − dd
1 dt 00x i = vl − ve ,xi+1 = Axi + Bui ,A= 0 1 0 ,
B = −dt ;vd − ve
0 0 1−dt
where dt is the time step or sampling time of the system and the physical meanings of the variables
are defined as vl ∈ R represents the velocity of the leading vehicle; ve ∈ R represents the velocityof the ego vehicle; dl ∈ R represents the distance between ego vehicle and leading vehicle; dd ∈ R
represents a desired distance value between ego vehicle and leading vehicle; vd ∈ R represents a
desired speed of ego vehicle; ui ∈ R represents the acceleration of the ego vehicle, which is also thecontrol command generated by the algorithm.
As for the prediction model gψ (yk ), in this example, wechoose the observation yk as the trajectory history of theprevious 10 frames, i.e.,yk = col(xk , xk−1 , ..., xk−9 ).
We take use of a four-layer fully connected neural network model wit number of nodes shown in Figure 1. The
activation functions are chosen as T anh(·) for the first
three layers and Sigmoid(·) for the last output layer toguarantee that Qk and Rk are positive semi-definite. We
train the model with batch size of 100 and learning rateof 1e − 4. The size of the training data varies between
different experiments.
4.2

Figure 1: Neural Network Layout

Simulation result

In this part, the training data is generated through a simulation. The simulation scenario is defined
as follows. The leading vehicle is first placed 50 meters ahead of the ego vehicle with a speed of
20 m/s. Then at the time t = 100s, the leading vehicle is switched to another one locates 40 meters
ahead of the ego but with slower speed 15m/s. Again at time t = 200s, the leading vehicle changes
to one only 20 meters ahead with a speed 18m/s. During the human driving data generation period,
the leading vehicle speed deviates from the initial speed by a normal distribution. The ego vehicle
starts with a speed of 25 m/s.

Figure 2: Simulation example result. Left: The loss history during the training phase; Right: Resultcomparison between simulated data and algorithm generated command. The upper figure shows the
speed comparison and the lower figure shows the acceleration command comparison.
The history of the loss L is shown on the left hand side of Figure 2, where it can be seen that theparameters actually converge with a rather fast rate. The performance of the proposed method isdemonstrated through another simulation, where the constant MPC controller is replaced by the
proposed algorithm in this article. The comparison of the recorded and generated ego vehicle6

acceleration is shown on the right hand side of Figure 2. When confronting a similar scenario, theproposed algorithm is able to generate similar behavior with respect to the recorded trajectory.
4.3

Real-world data validation

To further demonstrate the algorithm, we use several data sets extracted from Next Generation
Simulation (NGSIM) data set [US Department of Transportation, 2018]. In the NGSIM project,detailed vehicle trajectory data is collected using camera devices at specific locations including
including southbound US 101 and Landershim Bounlevard, Los Angeles, CA, eastbound I-80 in
Emeryville, CA and Peachtree Street in Atlanta, GA. Each trajectory provides the information of
precise locations, speeds and relative distances of a vehicle with a time resolution of 0.1 second.
We extract 7 datasets from NGSIM data for the evaluation of the proposed algorithm. The training
process is similar to the simulation data case. However, here the desired distance dd is not explicitly
known from the data-set, thus another prediction model is trained simultaneously for this value during
the training process. Later to examine the algorithm performance, the sequences of the leading
vehicle speeds and locations are reconstructed through the recorded data. A simulation is run byplacing the ego vehicle at the same initial position, while it is fully controlled by the proposed control
algorithm. The scheme of the implemented algorithm is shown in on the left hand side of Figure
3. On the right hand side of Figure 3, an example of the comparison between recorded data andsimulation data is shown, where the simulated vehicle distance and speed are close to the ones in therecorded data.

Figure 3: Real-world data result. Left: Algorithm scheme; Right: Sample result comparison
We choose the benchmark results in [Kesting and Treiber, 2008] as the baseline method, whichinvestigates two of the popular car-following model, Intelligent Driver Model (IDM) and Velocity
Difference Model (VDIFF). The IDM can be described mathematically asvs∗ (v, ∆v) 2 v̇IDM (s, v, ∆v) = a 1 − ( )4 − (
) ,
v0
s
v∆v
s∗ (v, ∆v) = s0 + vT + √ ,2 abwhere v is the current vehicle velocity, v0 is the desired velocity, ∆v is the velocity difference v0 andv, s is the current distance to the preceding vehicle and s∗ is the desired distance; in this IDM model,
a, b, s0 , v0 , T are parameters. Meanwhile, the VDIFF model is defined as
vopt (s) − v
v̇V DIF F (s, v, ∆v) =
− λ∆v
τ

v0 
s
vopt (s) =tanh(− β) − tanh(−β) ,2lintwhere definitions of v, s, ∆v are same as IDM model and τ , λ, lint , β are model parameters.
In [Kesting and Treiber, 2008], the models are calibrated by approximately solving the nonlinear
optimization problem through a generic algorithm [GOLDBERG, 2006]. In this algorithm, eachnew generation individual are generated by recombined two scholastically selected old generationindividuals. This evolution terminates until convergence criterion reached.
7

We as well exploit the same performance indicators like [Kesting and Treiber, 2008] such as
relative error and absolute error to measure the similarity between the data as follows,ssh(ssim − sdata )2 i
ssim − sdata 2
Frel (s) =()
Fabs (s) =,datashsdata i2
where the symbol h·i represents the mean of a data sequence and s represents the position data. In
addition, we define absolute difference measurements as follows,
Emean (s) = h|ssim − sdata |i
Evar (s) = h(ssim − sdata )2 i.
Table 1: Performance Evaluation
Dataset
1
2
3
4
5
6
7

Frel (s)
13.9%
22.3%
13.1%
13.6%
22.1%
19.3%
17.5%

Fabs (s)
13.1%
17.3%
10.6%
13.5%
12.2%
16.3%
15.6%

Emean (s)[m]
1.352
3.384
2.234
3.050
3.205
2.152
2.418

Evar (s)[m]
1.626
4.091
2.638
3.490
3.713
2.683
2.965

Emean (v)[m/s]
0.627
1.071
0.631
0.531
0.606
0.533
0.855

Evar (v)[m/s]
0.791
1.350
0.859
0.702
0.838
0.841
1.049

The performance evaluation results are shown in Table 1. Compared with the result of [Kesting andTreiber, 2008] shown in Table 2. In the baseline result relative error and absolute error ranges
20% to 30%, our results show around 10% less. Besides, our result shows around 2 meters of positiondifference and less than 1 m/s speed difference between the recorded data and simulated trajectory.
Table 2: Baseline Result
Dataset
1
2
3

5

IDM
Frel (s) Fabs (s)
24.0%
20.7%
28.7%
25.6%
18.0%
11.2%

VDIFF
Frel (s) Fabs (s)
25.5%
21.4%
29.1%
21.4%
28.2%
14.5%

Conclusion and Discussion

In this article, we propose a framework for inverse reinforcement learning with MPC by combining
the neural network prediction with MPC control algorithm to imitate the behavior encoded inside
a recorded data. The main challenge of such approach lies in the ambiguity of labeling and model
parameters, where we approach problem by building a bijective mapping between the recorded dataand prediction model output. Optimization and dynamic model-based autonomous vehicle longitudinal control algorithm nowadays suffers from the in-alignment with human intention. Whereas,
the proposed method provides a solution. We present an implementation of such control algorithm
using the proposed method. The result is compared with an existing approach and shows improved
performance by reproducing similar behavior encoded within human-driven vehicle trajectory.
Current theory development faces several drawbacks. One is that the final cost function F (·) in (1)has to be dropped, otherwise it cannot be recovered anyway. Another one is that the hard constraints
are assumed to be inactive. The main cause of such limitations is the commonly adopted design
decision of MPC, that the optimization problem is solved recursively at each step and only the firstpart of control sequence is executed. Such set-up introduces large amount of ambiguity to solve
the dual problem of optimization. Thus one of the further investigation direction is to break suchsetting and assume that more than one or all the generated control signals can be observed, in order tofully recover the cost functions and constraints. On the autonomous driving application side, feature
extraction can be potentially much enhanced to increase the similarity of driving style between ahuman driver and an autonomous vehicle. One potential approach is to provide more surrounding
traffic information not limited to the leading vehicle, as well as to provide local map informationsuch as distances to lane boundaries and positions of the traffic intersections. The idea is that to makean autonomous vehicle drive like human, we may need to feed comparable observation as humanreceive when driving.
8

References
David Q Mayne, James B Rawlings, Christopher V Rao, and Pierre OM Scokaert. Constrained model
predictive control: Stability and optimality. Automatica, 36(6):789–814, 2000.
S Joe Qin and Thomas A Badgwell. A survey of industrial model predictive control technology.
Control engineering practice, 11(7):733–764, 2003.
Sergio Vazquez, Jose Leon, Leopoldo Franquelo, Jose Rodriguez, Hector A Young, Abraham
Marquez, and Pericle Zanchetta. Model predictive control: A review of its applications in powerelectronics. IEEE Industrial Electronics Magazine, 8(1):16–31, 2014.
Tom Erez, Kendall Lowrey, Yuval Tassa, Vikash Kumar, Svetoslav Kolev, and Emanuel Todorov.
An integrated system for real-time model predictive control of humanoid robots. In 2013 13th
IEEE-RAS International Conference on Humanoid Robots (Humanoids), pages 292–299. IEEE,
2013.
David H Shim, H Jin Kim, and Shankar Sastry. Decentralized nonlinear model predictive control ofmultiple flying robots. In 42nd IEEE International Conference on Decision and Control (IEEECat. No. 03CH37475), volume 4, pages 3621–3626. IEEE, 2003.
F Künhe, J Gomes, and W Fetter. Mobile robot trajectory tracking using model predictive control. In
II IEEE latin-american robotics symposium, volume 51, 2005.
Roman Schmied, Harald Waschl, Rien Quirynen, Moritz Diehl, and Luigi del Re. Nonlinear mpc
for emission efficient cooperative adaptive cruise control. IFAC-PapersOnLine, 48(23):160–165,
2015.
Francesco Borrelli, Alberto Bemporad, Michael Fodor, and Davor Hrovat. An mpc/hybrid systemapproach to traction control. IEEE Transactions on Control Systems Technology, 14(3):541–552,
2006.
Francesco Borrelli, Paolo Falcone, Tamas Keviczky, Jahan Asgari, and Davor Hrovat. Mpc-based
approach to active steering for autonomous vehicle systems. International Journal of Vehicle
Autonomous Systems, 3(2):265–291, 2005.
Paolo Falcone, H Eric Tseng, Francesco Borrelli, Jahan Asgari, and Davor Hrovat. Mpc-based yaw
and lateral stabilisation via active front steering and braking. Vehicle System Dynamics, 46(S1):
611–628, 2008.
Janick V Frasch, Andrew Gray, Mario Zanon, Hans Joachim Ferreau, Sebastian Sager, Francesco
Borrelli, and Moritz Diehl. An auto-generated nonlinear mpc algorithm for real-time obstacle
avoidance of ground vehicles. In 2013 European Control Conference (ECC), pages 4136–4141.
IEEE, 2013.
9

Jose Rodriguez, Marian P Kazmierkowski, Jose R Espinoza, Pericle Zanchetta, Haitham Abu-Rub,
Hector A Young, and Christian A Rojas. State of the art of finite control set model predictive
control in power electronics. IEEE Transactions on Industrial Informatics, 9(2):1003–1016, 2012.
MR Arahal, F Barrero, S Toral, M Duran, and R Gregor. Multi-phase current control using finite-statemodel-predictive control. Control Engineering Practice, 17(5):579–587, 2009.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436, 2015.
Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85–117,
2015.
Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In Advances in neural
information processing systems, pages 305–313, 1989.
Huazhe Xu, Yang Gao, Fisher Yu, and Trevor Darrell. End-to-end learning of driving models fromlarge-scale video datasets. In Proceedings of the IEEE conference on computer vision and patternrecognition, pages 2174–2182, 2017.
Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learning. In
Proceedings of the twenty-first international conference on Machine learning, page 1. ACM, 2004.
Andrew Y Ng, Stuart J Russell, et al. Algorithms for inverse reinforcement learning. In Icml,
volume 1, page 2, 2000.
Shao Zhifei and Er Meng Joo. A review of inverse reinforcement learning theory and recent advances.
In 2012 IEEE Congress on Evolutionary Computation, pages 1–8. IEEE, 2012.
David Silver, J Andrew Bagnell, and Anthony Stentz. Learning from demonstration for autonomous
navigation in complex unstructured terrain. The International Journal of Robotics Research, 29(12):1565–1592, 2010.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
US Department of Transportation. NGSIM – Next Generation Simulation. http://www.ngsim.
fhwa.dot.gov, 2018.
Arne Kesting and Martin Treiber. Calibrating car-following models by using trajectory data: Methodological study. Transportation Research Record, 2088(1):148–156, 2008.
DD GOLDBERG. Genetic algorithms in search, optimization, and machine learning genetic algorithms in search, optimization, and machine learning, 1989. IEEJ Transactions on Electronics,
Information and Systems, 126(7):857–864, 2006.

10