<figure><img src="image_1.png" /></figure>

HAL
open science

# Reinforcement Learning with Model-Based Static
Output Feedback for Vehicle Lateral Control

Eslam Mahmoud, Saïd Mammar, Mohand Smaïli

► To cite this version:

Eslam Mahmoud, Saïd Mammar, Mohand Smaïli. Reinforcement Learning with Model-Based Static
Output Feedback for Vehicle Lateral Control. 36th IEEE Intelligent Vehicles Symposium (IV 2025),
Jun 2025, Cluj-Napoca, Romania. hal-05095739

HAL Id: hal-05095739

https://univ-evry.hal.science/hal-05095739v1

Submitted on 4 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Reinforcement Learning with Model-Based Static Output Feedback for Vehicle Lateral Control

Eslam Mahmoud

IBISC-EA4526

University of Paris-Saclay

Evry, France

eslam.mahmoud@univ-evry.fr

Saïd Mammar

IBISC-EA4526

University of Paris-Saclay

Evry, France

said.mammar@univ-evry.fr

Mohand Smaïli

IBISC-EA4526

University of Paris-Saclay

Evry, France

mohand.smaili@univ-evry.fr

**Abstract**— This paper investigates Reinforcement Learning (RL) for model-based static output feedback, with a focus on vehicle lateral control applications such as lane-keeping and lane-change maneuvers. Emphasizing stability during learning to prevent unsafe actions, the study reviews recent RL advancements and compares policy iteration (PI) and policy search (PS) methods in terms of convergence speed and stability. A two-degree-of-freedom bicycle model is employed to validate the proposed PI algorithm for static output feedback, highlighting its faster convergence compared to PS and its competitive performance against traditional Linear Quadratic Regulator (LQR) control. Results demonstrate the practicality and efficiency of RL for stable real-time vehicle control.

**Index Terms**— Reinforcement learning, Policy iteration algorithm, Policy search algorithm, Stability, State-Feedback, Output-feedback, Lane-keeping.

## I. INTRODUCTION

Reinforcement Learning (RL) has received more attention from researchers because RL can learn automatically by interacting with the environment, eliminating the need for labeled data, unlike supervised and unsupervised learning [1]. However, applying RL in real systems can be challenging due to the risk of unsafe actions during learning [2]. While most control methods are model-based, requiring system dynamics for controller design, RL's ability to work in both model-based and model-free settings makes it attractive for control applications.

Real-world applications often rely on output feedback due to limited sensor data, whereas most controllers are designed for state feedback. This limitation motivates the use of observers combined with state feedback, as demonstrated in [3], or directly employ output feedback controllers that utilize sensor data. Among these, static output feedback is particularly appealing for its implementation simplicity.

RL methods for optimal policy discovery include Policy Iteration (PI), Value Iteration (VI), and Policy Search (PS) [4]. Because VI can start with any arbitrary controller gains even if not stabilizable, our study focuses on stability-oriented PI and PS, where PS optimizes policy via optimization methods whether they are gradient-based or gradient-free, such as Genetic Algorithms (GA) and Particle Swarm Optimization

(PSO) [5]. Ensuring stability by using PS can be achieved in many ways: Offline training followed by stability checks as in [6], [7], or online methods use pre-designed stable controllers to switch between them as in [8] or using filters to ensure safe actions [9], [10]. Our previous work embedded stability checks during training to ensure safety [11]. For policy iterations (PI) model-based approaches, stability can be guaranteed by using Lyapunov functions [12] or Riccati equations [13] included in the algorithm, also in model-free methods like Integral RL (IRL) stability is guaranteed by formulating Riccati equation in a way such that the model dynamics are included in the learning terms of the algorithm to be model-free as in [14], [15].

Policy iteration can be on-policy, where the same policy generates and improves actions, or off-policy, where the behavior and target policies differ. Zhang et al. [16] found off-policy methods more effective, as they reduce bias from noise added to the behavior policy for satisfying the persistence of excitation (PE) condition.

A key challenge in policy iteration (PI) is selecting an initial stabilizing gain. Chen et al. [17] addressed this by starting with an arbitrary controller and using collected data to find a stabilizable controller before continuing with PI. While PI converges faster than policy search [4], the initial condition plays a significant role. Policy search, though slower, seeks the optimal solution.

We extend our previous work [11], where the controller was PID control and the RL method was PS with guaranteed stability. In this work, we transition to state feedback (SF) and Static Output Feedback (SOF) controllers. An algorithm will be developed for SOF in the policy iteration framework. This approach will be compared with policy search (PS). The algorithm will be tested on vehicle lateral control, specifically lane-keeping and lane-change.

The main contribution of this work is to develop a policy iteration algorithm for static output feedback and treat SOF tuning as an RL problem. Also, a comparison of RL methods (PI vs. PS) for vehicle lateral control with validation on lane-keeping and lane-change maneuvers with robustness tests is conducted.

The paper is structured as follows: Section II presents the developed PI-SOF algorithm. Simulation tests are provided in section III, and the results and discussions are in section

<footnote>This research is supported by the French ARTISMO (Intelligent Estimation Algorithms for Mobility) financed by ANR (National Research Agency).</footnote>
IV. Section V is devoted to conclusions and future work.

## II. PROPOSED METHODOLOGY

Before explaining our policy iteration algorithm for static output feedback a brief review of policy search and policy iteration is presented.

### A. Policy search approach

It searches for an optimal policy that maximizes the expected cumulative long-term reward. A very popular PS agent known as Twin Delayed Deep Deterministic Policy Gradient (TD3) is used for continuous systems and will be used here for the PS approach. The neural network for the policy is called the actor, which maps the states (outputs) of the system to control action, where the weights of the network are the controller gains to be tuned. The initial gains (weights) for the actor-network can be random or stabilizable. The stabilizing gains can be obtained by using the Linear Matrix Inequalities (LMI) method [18] for the state feedback problem or [19] for the static output feedback controller. The critic network is used as a value function estimator, evaluating the quality of the current policy by estimating the expected return from a given state-action pair [20]. Linear Quadratic Regulator (LQR) and Linear Quadratic Gaussian (LQG) are used as a reward equation for state feedback and output feedback problems, respectively.

The modified TD3 for stability guarantee [11] will be used here again for comparison study. For a multi-input multi-output (MIMO) system, stability can be checked with Lyapunov or eigenvalues of the closed-loop system [5].

### B. Policy iteration approach

1) State feedback synthesis: Let us consider a linear time-invariant system described by the state-space equation:

$$ \begin{cases} \dot{x}(t) = Ax(t) + Bu(t) \\ y(t) = Cx(t) \end{cases} \qquad (1) $$

with $x \in \mathbb{R}^n$ is the state vector, $u \in \mathbb{R}^m$ is the input vector and $y \in \mathbb{R}^p$ is the measurement vector. Matrices A, B and C are of compatible dimensions.

The algorithm for policy iteration for state feedback is developed in [13]. It aims to design a state feedback gain matrix $K \in \mathbb{R}^{m \times n}$ obtained from an iterative process starting with an initial stabilizing gain $K^0$ and solving Riccati Equation till convergence to optimal $K$.

$$ (A - BK^0)^T P^0 + P^0(A - BK^0) + Q + (K^0)^T R K^0 = 0 \quad (2) $$

where $Q > 0$ and $R > 0$ are the symmetric definite positive weight matrices. The solution to this equation exists under the condition that the pair $(A, B)$ is stabilizable. In the following, it is in addition assumed that the pair $(A, C)$ is detectable.

The procedure is summarized in Algorithm 1 in which the Riccati equation is solved for $P^j$ and is followed by the state feedback $K^j$ update for the next iteration. The stopping criterion is based on the closeness of two consecutive values of $P^j$, with an error less than $\epsilon > 0$, a user-defined threshold value.

### Algorithm 1 Model-Based Policy Iteration for State Feedback [13]

**Initialization:** Choose any initial stabilizing state feedback gain matrix $K^0$.

**Execution:**

Let iteration index $j = 0$. Solve the following equation:

$$ (A - BK^0)^T P^0 + P^0(A - BK^0) + Q + (K^0)^T R K^0 = 0 \quad (3) $$

**repeat**

Increment iteration index $j = j + 1$.

Calculate the state feedback gain matrix:

$$ K^j = R^{-1} B^T P^{j-1} \quad (4) $$

Solve the following equation:

$$ (A - BK^j)^T P^j + P^j(A - BK^j) + Q + (K^j)^T R K^j = 0 \quad (5) $$

**until** $\|P^j - P^{j-1}\| \le \epsilon$ for $j \ge 1$

**Return:** $P^j, K^j$

2) Output feedback synthesis: Designing a static output feedback is a challenging problem, as used conditions are generally sufficient ones and there is no guarantee for a solution. In this paper, the iterative Riccati equation of Algorithm 1 is extended for static output feedback. In this case, one seeks the control matrix gains $K_{out} \in \mathbb{R}^{m \times p}$ such that $u(t) = -K_{out}y(t)$. The proposed method is summarized in Algorithm 2.

### Algorithm 2 Model-Based Policy Iteration for Output Feedback

**Initialization:** Choose any initial stabilizing static output feedback gain matrix $K_{out}^0$.

**Execution:**

Let iteration index $j = 0$. Solve the following equation:

$$ (A - BK_{out}^0 C)^T P^0 + P^0(A - BK_{out}^0 C) + Q + C^T(K_{out}^0)^T R K_{out}^0 C = 0 \quad (6) $$

**repeat**

Increment iteration index $j = j + 1$.

Calculate the output feedback gain matrix:

$$ K_{out}^j = R^{-1} B^T P^{j-1} C^T \quad (7) $$

Solve the following equation:

$$ (A - BK_{out}^j C)^T P^j + P^j(A - BK_{out}^j C) + Q + C^T(K_{out}^j)^T R K_{out}^j C = 0 \quad (8) $$

**until** $\|P^j - P^{j-1}\| \le \epsilon$ for $j \ge 1$

**Return:** $P^j, K_{out}^j$

**Remark 1.** The equations (6) and (7) were originally developed in [3] as part of a model-based online learning approach. In that framework, an embedded estimator was
designed to estimate the matrix $P^*$, using measured outputs to iteratively update the estimate online to correct estimation errors. However, in this work, we adapt these equations to formulate an iterative offline model-based algorithm, which simplifies implementation.

3) *Proof convergence of Algorithm 2:* For the iteration index $j = 0, 1, \dots$, the gain matrices $K_{out}^j$ and the matrix $P^j$ can be obtained from Algorithm 2. Let us start by showing that if $K_{out}^j$ is a stabilizing matrix, then $K_{out}^{j+1}$ is also a stabilizing matrix.

If the gain matrix $K_{out}^j$ is stabilizing, we define the Lyapunov function as

$$L^j = x^T P^j x \quad (9)$$

where $P^j$ is a positive definite matrix and is obtained from equation (8). Taking the derivative of (9) along the gain matrices $K_{out}^{j+1}$, we have:

$$\begin{aligned} \dot{L}^j &= x^T P^j \dot{x} + \dot{x}^T P^j x \\ &= x^T \left( (A - BK_{out}^{j+1}C)^T P^j + P^j(A - BK_{out}^{j+1}C) \right) x \end{aligned} \quad (10)$$

where $K_{out}^{j+1}$ is obtained from Algorithm 2 and satisfies the condition:

$$(A - BK_{out}^{j+1}C)^T P^j + P^j(A - BK_{out}^{j+1}C) + Q + C^T(K_{out}^{j+1})^T R K_{out}^{j+1}C \le 0. \quad (11)$$

Therefore, we have:

$$\begin{aligned} \dot{L}_j &\le -x^T (Q + C^T (K_{out}^{j+1})^T R K_{out}^{j+1} C) x \\ &\le 0. \end{aligned} \quad (12)$$

Hence, the gain matrix $K_{out}^{j+1}$ is stabilizing.

In addition, one can derive the that $\dot{L}^0 \le \dot{L}^1$, and $L^1 \le L^0$ and thus $P^1 \le P^0$ which extends to $P^{j+1} \le P^j$ for $j = 0, 1, \dots$. The matrix $P^j \to P^\infty$ verifying

$$A_c^T P^\infty + P^\infty A_c + Q + C^T (K_{out}^\infty)^T R K_{out}^\infty C = 0 \quad (13)$$

and $A_c = A - BK_{out}^\infty C$, and the algorithm is converging. The proof is complete.

### III. VEHICLE MODELING

In this section, we study the controller design for vehicle lateral dynamics control. The study is carried out using a two-degree-of-freedom model that accounts for both lateral translation and yaw rotation represented by the well-known bicycle model [21]. This model is adapted for lane-keeping and lane-change purposes.

The state space representation is of fourth order given by [7]:

$$\left\{ \begin{aligned} \dot{x}(t) &= \bar{A}\bar{x}(t) + \bar{B}u + \bar{B}_d[F_w(t) \quad \rho(t)]^T \\ \bar{y}(t) &= \bar{C}\bar{x}(t) \end{aligned} \right. \quad (14)$$

where

<figure><img src="image_3.png" /><figcaption>Fig. 1. Vehicle single track bicycle model [22].</figcaption></figure>

$$\bar{A} = \begin{bmatrix} -\frac{2C_f+2C_r}{mV_x} & -V_x - \frac{2C_f l_f - 2C_r l_r}{mV_x} & 0 & 0 \\ -\frac{2C_f l_f - 2C_r l_r}{I_z V_x} & -\frac{2C_f l_f^2 + 2C_r l_r^2}{I_z V_x} & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 1 & l_s & V_x & 0 \end{bmatrix}$$

$$\bar{x} = \begin{bmatrix} \dot{y} \\ \dot{\psi} \\ \psi_L \\ y_L \end{bmatrix}, \quad \bar{B} = \begin{bmatrix} \frac{2C_f}{I_z} \\ \frac{2C_f l_f}{I_z} \\ 0 \\ 0 \end{bmatrix}, \quad \bar{B}_d = \begin{bmatrix} \frac{1}{l_w} & 0 \\ \frac{m}{I_z} & 0 \\ 0 & -V_x \\ 0 & 0 \end{bmatrix}$$

$$\bar{C} = \begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$$

Where $u = \delta_f$ is the control input represented by the front steering angle, $F_w$ is the disturbance force due to wind, $m, I_z$ are the vehicle mass and yaw moment inertia, $v_x$ and $\dot{y} = v_y$ are longitudinal and lateral velocities, $\dot{\psi}$ is the yaw rate, $l_f, l_r$ are distances from the vehicle center of gravity (CG) to the front and rear axle. Vehicle parameters and nominal values are provided in Table I. $\rho$ is the road curvature acting as a disturbance, and $l_s$ is the look-ahead distance, which is the preview distance of the lateral displacement (Fig. 2). The relative yaw rate $\psi_L$ and the lateral displacement $y_L$ are obtained by using a camera that detects the lane. They are measured at a look ahead distance $l_s$, here assumed constant ($l_s = 10m$).

The wind disturbance is modeled as a 1000 N force applied from the fifth second of the 20-second simulation.

The lateral velocity $\dot{y}$ is not available when using standard sensors at an affordable cost.

<table><caption>TABLE I VEHICLE PARAMETERS [7]</caption><thead><tr><th>Parameter</th><th>Nominal Value</th></tr></thead><tbody><tr><td>Distance between CG and front axle $l_f$ (m)</td><td>1.2</td></tr><tr><td>Distance between CG and rear axle $l_r$ (m)</td><td>1.6</td></tr><tr><td>Distance between CG and wind force $l_w$ (m)</td><td>1</td></tr><tr><td>Front cornering stiffness $c_f$ (kN / rad)</td><td>19</td></tr><tr><td>Rear cornering stiffness $c_r$ (kN / rad)</td><td>33</td></tr><tr><td>Vehicle mass $m$ (kg)</td><td>1575</td></tr><tr><td>Yaw inertia $I_z$ (kg.m<sup>2</sup>)</td><td>2875</td></tr></tbody></table>

To track the reference signal $y_{lref}$ for lane keeping or lane change and ensure zero steady-state error, an integral action
is added to the system output. An additional state variable $z$ is added. It verifies $\dot{z}(t) = y_L(t) - y_{lref}(t)$. The equation model (14) is augmented with $z$.

$$ \begin{cases} \dot{x}(t) = Ax(t) + Bu(t) + B_d[F_w \quad \rho \quad y_{lref}]^T \\ y(t) = Cx(t) \end{cases} \qquad (15) $$

where

$$ A = \begin{bmatrix} \bar{A} & 0 \\ C_z & 0 \end{bmatrix}, B = \begin{bmatrix} \bar{B} \\ 0 \end{bmatrix}, B_d = \begin{bmatrix} \bar{B}_d & 0 \\ 0 & -1 \end{bmatrix}, C = \begin{bmatrix} \bar{C} & 0 \\ 0 & 1 \end{bmatrix} $$

$$ C_z = \begin{bmatrix} 0 & 0 & 0 & 1 \end{bmatrix} $$

$$ x(t) = [\dot{y} \quad \dot{\Psi} \quad \Psi_L \quad y_L \quad z]^T, \quad y(t) = [\dot{\Psi} \quad \Psi_L \quad y_L \quad z]^T $$

The measurement vector $y$ is the state vector without the lateral speed which can not be measured with usual vehicle sensors. Other variables are easily obtained from a video camera and yaw rate sensor. Figure 3 shows the block

<figure><img src="image_4.png" /><figcaption>Fig. 4. State feedback gain convergence.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 2. Vehicle positioning on the lane [22].</figcaption></figure>

diagram for the policy search algorithm with state (output) feedback controller. Two scenarios will be tested: Lane

<figure><img src="image_6.png" /><figcaption>Fig. 3. Block diagram for policy search implementation.</figcaption></figure>

Keeping Assistant (LKA) and Lane Change Maneuver. These advanced driver assistance systems (ADAS) are designed to enhance vehicle safety and comfort. LKA helps maintain the vehicle centered within its lane by providing corrective steering inputs, while the Lane Change Maneuver involves moving the vehicle to an adjacent lane by activating steering during the maneuver.

## IV. RESULTS AND DISCUSSIONS

1) Policy search approach: The weighting parameters $Q$ and $R$ values are chosen to be $2 \times I_5$ and 1 receptively for state feedback, where $I_5$ is the identity matrix of dimension 5. The training required approximately 1.5 hours to converge with the stopping criteria that have been set (the simulations were conducted on a computer with Inter(R) Core(TM) i7-10700 CPU 2.90GHz,16GB of RAM, and MATLAB R2023b version). For SOF, $Q$ and $R$ values are chosen to be $I_4$ and 1 receptively. It takes two and a half hours to converge. Note that the reference signal here is the zero value for the desired lateral deviation.

2) Policy iteration approach: The same parameters and initial gains are used here for policy iteration, and the threshold value $\epsilon$ is taken as 1e-8. Figure 4 and Fig. 5 show the convergence of controller gains. Both state feedback and output feedback take 0.004s, which is very fast compared to policy search, making it ideal for real-time adaptive control. For example, one can use the data-driven technique to construct the model, such as in [23], and use our algorithm to tune the controller in case of system evolution.

Figure 6 shows the comparison of different approaches: policy search, policy iteration, initial gains obtained from the Lyapunov approach for state feedback and LQR, showing how the RL policy iteration or policy search enhanced the performance over the initial gains, PI approach has a similar response as LQR, the gains almost typical as shown in Table II, demonstrating the alignment of model-based reinforcement learning with the optimal control theory. Similarly, the static output feedback case is shown in Fig. 7.

Table III shows that the controller gains obtained from policy search and policy iteration are larger in magnitude compared to the initial gains. This indicates a more aggressive control action, which accelerates the system convergence, as also demonstrated in Fig. 7. Also, for validation for our PI-SOF algorithm, a lane change maneuver used for testing, Fig. 8 shows how the policy iteration output feedback is
<figure><img src="image_7.png" /><figcaption>Fig. 5. Static output feedback gain convergence.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. Comparison between Policy search, Policy iteration, and Lyapunov (Initial Gains) response for LKA state feedback control.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 7. Comparison between Policy search, Policy iteration, Lyapunov (Initial Gains), and LQR response for LKA static output feedback control.</figcaption></figure>

<table><caption>TABLE II
SF CONTROLLER GAINS FOR EACH APPROACH</caption><thead><tr><th rowspan="2">Approach</th><th colspan="5">Parameters</th></tr><tr><th>K<sub>1</sub></th><th>K<sub>2</sub></th><th>K<sub>3</sub></th><th>K<sub>4</sub></th><th>K<sub>5</sub></th></tr></thead><tbody><tr><td>Lyapunov (Initial Gains)</td><td>-0.49</td><td>0.47</td><td>1.4</td><td>0.13</td><td>0.14</td></tr><tr><td>Policy Search</td><td>0.79</td><td>0.4</td><td>0.92</td><td>0.69</td><td>0.42</td></tr><tr><td>Policy Iteration</td><td>1.182</td><td>1.676</td><td>2.899</td><td>1.595</td><td>1.414</td></tr><tr><td>LQR</td><td>1.182</td><td>1.676</td><td>2.899</td><td>1.595</td><td>1.414</td></tr></tbody></table>

<table><caption>TABLE III
SOF CONTROLLER GAINS FOR EACH APPROACH</caption><thead><tr><th rowspan="2">Approach</th><th colspan="4">Parameters</th></tr><tr><th>K<sub>1</sub></th><th>K<sub>2</sub></th><th>K<sub>3</sub></th><th>K<sub>4</sub></th></tr></thead><tbody><tr><td>Lyapunov (Initial Gains)</td><td>0.19</td><td>0.83</td><td>0.14</td><td>0.08</td></tr><tr><td>Policy Search</td><td>1.38</td><td>2.86</td><td>0.36</td><td>0.54</td></tr><tr><td>Policy Iteration</td><td>1.314</td><td>2.338</td><td>1.145</td><td>1</td></tr></tbody></table>

similar to LQR state feedback. Also, for the robustness of the static output feedback system, under variations in the front and rear cornering stiffness ($c_f, c_r$), which simulate the changes in tire-road interaction such as wet surfaces, Fig. 9, show that the controller maintains stability during the lane-change maneuver, even if the stiffness values deviate from the nominal conditions. Another test is conducted by varying the longitudinal velocity $V_x$ to evaluate the robustness of the static output feedback system. Figure 10 illustrates the controller's performance under these conditions. The results show that the controller effectively tracks the desired lane-change trajectory with minimal deviation across different speed levels. Even at high speeds, the system demonstrates strong robustness, ensuring smooth vehicle maneuvers. These findings validate the controller's reliability and suitability for real-time applications.

V. CONCLUSIONS

This paper investigates the application of Reinforcement Learning (RL) for vehicle lateral control, focusing on

<figure><img src="image_10.png" /><figcaption>Fig. 8. Comparison between Policy iteration static output feedback and LQR response for lane change maneuver problem.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>Fig. 9. Policy iteration static output feedback response with varying cornering stiffness during lane change maneuver.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 10. Policy iteration static output feedback response with varying longitudinal speed for lane change maneuver problem.</figcaption></figure>

stability-preserving policy search and policy iteration methods. We propose a new policy iteration approach for static output feedback, enhancing the initial stabilizing controller. The proposed methods are validated in two driving scenarios: lane-keeping and lane-change maneuvers. A comparative study between the two RL approaches demonstrates that policy iteration achieves faster convergence, making it more suitable for real-time tuning. Robustness was assessed by introducing parameter variations and uncertainties, and the proposed method demonstrated strong performance. Future work will extend this research to nonlinear vehicle dynamics and explore model-free RL approaches to eliminate the dependence on system dynamics.

## REFERENCES

[1] R. S. Sutton and A. G. Barto, *Reinforcement learning: An introduction*. MIT press, 2018.

[2] H. Atoui, O. Sename, V. Milanés, and J. J. Martinez, "Intelligent control switching for autonomous vehicles based on reinforcement learning," in *2022 IEEE Intelligent Vehicles Symposium (IV)*. IEEE, 2022, pp. 792-797.

[3] J. Zhao, J. Na, G. Gao, S. Han, Q. Chen, and S. Wang, "Online output-feedback optimal control of linear systems based on data-driven adaptive learning," *IFAC-PapersOnLine*, vol. 53, no. 2, pp. 1596-1601, 2020.

[4] L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst, *Reinforcement learning and dynamic programming using function approximators*. CRC press, 2017.

[5] R. Toscano, *Structured controllers for uncertain systems*. Springer, 2013.

[6] J. Yang, W. Peng, and C. Sun, "A learning control method of automated vehicle platoon at straight path with ddpg-based pid," *Electronics*, vol. 10, no. 21, p. 2580, 2021.

[7] E. Mahmoud, D. Ichalal, and S. Mammar, "Reinforcement learning based autonomous vehicles lateral control," in *2023 IEEE International Conference on Networking, Sensing and Control (ICNSC)*, vol. 1. IEEE, 2023, pp. 1-6.

[8] E. Abramova, "Combining reinforcement learning and optimal control for the control of nonlinear dynamical systems," Ph.D. dissertation, Imperial College London, 2015.

[9] A. I. Lakhani, M. A. Chowdhury, and Q. Lu, "Stability-preserving automatic tuning of pid control with reinforcement learning," *arXiv preprint arXiv:2112.15187*, 2021.

[10] S. Alighanbari and N. L. Azad, "Safe adaptive deep reinforcement learning for autonomous driving in urban environments. additional filter? how and where?" *IEEE Access*, vol. 9, pp. 141347-141359, 2021.

[11] E. Mahmoud and S. Mammar, "Pid control by safe reinforcement learning for vehicle lateral control," in *2024 IEEE Int. Conf. on Networking, Sensing and Control (ICNSC)*. IEEE, 2024, pp. 1-6.

[12] F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, "Safe model-based reinforcement learning with stability guarantees," *Advances in neural information processing systems*, vol. 30, 2017.

[13] H. Li and Q. Wei, "Data-driven optimal control for half-vehicle suspension system via adaptive dynamic programming," in *2022 IEEE 11th Data Driven Control and Learning Systems Conference (DDCLS)*. IEEE, 2022, pp. 525-530.

[14] G. Wang, B. Luo, and S. Xue, "Integral reinforcement learning-based optimal output feedback control for linear continuous-time systems with input delay," *Neurocomputing*, vol. 460, pp. 31-38, 2021.

[15] L. M. Zhu, H. Modares, G. O. Peen, F. L. Lewis, and B. Yue, "Adaptive suboptimal output-feedback control for linear systems using integral reinforcement learning," *IEEE Trans. on Cont. Sys. Tech.*, vol. 23, no. 1, pp. 264-273, 2014.

[16] L. Zhang, J. Fan, W. Xue, V. G. Lopez, J. Li, T. Chai, and F. L. Lewis, "Data-driven h optimal output feedback control for linear discrete-time systems based on off-policy q-learning," *IEEE Trans. on Neural Networks and Learning Systems*, vol. 34, no. 7, pp. 3553-3567, 2021.

[17] Y. Chen, C. Chen, K. Xie, and F. L. Lewis, "Online policy iteration algorithms for linear continuous-time h-infinity regulation with completely unknown dynamics," *IEEE Trans. on Aut. Sci. and Eng.*, 2023.

[18] K. Tanaka and H. O. Wang, *Fuzzy control systems design and analysis: a linear matrix inequality approach*. John Wiley & Sons, 2004.

[19] M. Chadli and P. Borne, *Multiple models approach in automation: Takagi-Sugeno fuzzy systems*. John Wiley & Sons, 2012.

[20] I. The MathWorks, *Reinforcement Learning Toolbox-User's Guide*. The MathWorks, Inc., 2022.

[21] R. Rajamani, *Vehicle dynamics and control*. Springer Science & Business Media, 2011.

[22] S. Mammar and H. Arioui, "Static output feedback control for lane change maneuver," in *2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC)*. IEEE, 2018, pp. 1-6.

[23] J. S. Kim, Y. S. Quan, and C. C. Chung, "Data-driven modeling and control for lane keeping system of automated driving vehicles: Koopman operator approach," in *2022 22nd International Conference on Control, Automation and Systems (ICCAS)*. IEEE, 2022, pp. 1049-1055.