# Data-Driven Robust Control Using Reinforcement Learning
## 1 Introduction  
Control of unknown dynamic systems with uncertainties is a challenge because exact mathematical models are often required. Since many processes are complicated, nonlinear, and varying with time, a control algorithm that does not depend on a mathematical model and can adapt to time-varying conditions is required. A popular approach is to develop a universal approximator for predicting the output of unknown systems [1]. Control algorithms can then be designed based on the parameters of the approximator. Based on this approach, many control techniques have been proposed using machine learning models such as neural networks and fuzzy logic. For example, [PERSON] et al. [2] proposed a robust sliding mode controller which can be designed from Chebyshev neural networks. [PERSON] and [PERSON] [3] introduced a robust static output feedback controller for Takagi Sugeno fuzzy models. [PERSON] and [PERSON] [4] proposed a method to model unstructured uncertainties and a new Takagi Sugeno fuzzy controller using type-2 fuzzy neural networks. However, obtaining a good approximator requires a significant amount of training data, especially for a complicated model with high-dimensional state spaces or with many inputs and outputs. The data-driven model must also be updated frequently for time-varying systems. In addition, many control design techniques assume uncertainties as functions of system parameters. However, in many cases, the causes of uncertainties are unknown and unstructured. With the development of data science and machine learning, model-free approaches such as reinforcement learning (RL) have emerged as an effective method to control unknown nonlinear systems [5; 6; 7; 8]. The principle of RL is based on the interaction between a decision-making agent and its environment [9], and the actor-critic method is often used as the RL framework for many control algorithms. In the actor-critic framework, the critic agent uses current state information of the environment in order toupdate the value or action value function. The actor agent then uses the value or action value function to calculate the optimal action. It can be seen that many data-driven algorithms lack stability analysis of the closed-loop systems. Among recent techniques focusing on the robustness of control algorithms, [PERSON] et al. [10] presented an off-policy reinforcement learning (RL) solution to solve robust control problems for a certain class of unknown systems with structured uncertainties. In [11], a robust data-driven controller was proposed based on the frequency response of multivariable systems and convex optimization. Based on data-driven tuning, [PERSON] et al. [12] introduced a detection algorithm suitable for massive overloaded multiple-input multiple-output systems. In more recent works, [PERSON] et al. [13] proposed an approach to address the output-feedback robust control for continuous-time uncertain systems using online data-driven learning, while [PERSON] et al. [14] used data-driven techniques for iterative feedback tuning of a proportional-integral-derivative controller's parameters. However, in many cases, stability can only be ensured for specific systems where uncertainties are structured. In addition, the value function must be estimated accurately, which is difficult to achieve, especially at the beginning of the control process when the agent has just started interacting with the environment. Additionally, in many applications, the state space is either continuous or high-dimensional. In these cases, the value function approximation is often inaccurate, potentially leading to instability. Therefore, new RL approaches for which stability can be guaranteed under uncertain conditions are essential if algorithms are to be used in critical and safety-demanding systems. Type 1 diabetes is a disease caused by the lack of insulin secretion. The condition results in uncontrolled increase of blood glucose level if the patients are not provided with insulin doses.