## 1 Introduction  
In control systems, the main task of the controller is to obtain an admissible control law when certain conditions are satisfied according to the dynamic characteristics of the plant. Then, the plant cooperates with other controllers to achieve the optimization of performance indicators (maximum or minimum), so as to solve the optimal control problem [1]. Solving the optimal control problem is equivalent to solving the Hamilton-Jacobi-Bellman (HJB) equation. Because of the nonlinearity and partial derivative of the HJB equation, it is difficult to acquire its analytical solution. In recent years, the adaptive dynamic programming (ADP) method has received widespread attention for obtaining the approximate solutions of the HJB equation [2; 3; 4; 5]. The ADP method combines the idea of dynamic programming (DP) and reinforcement learning (RL). The RL-based agent interacts with the environment and takes action to obtain cumulative rewards, and it overcomes the problem of dimensional disasters generated in DP. The implementation of the ADP method approximates the HJB equation by the approximation principle of neural network function, and obtains the optimal value function and optimal control law by RL. In [6], the structure of ADP is proposed for the first time. Subsequently, in [7] uses the actor-critic NN structure to deal with the nonlinear system that depends on the dynamic information. In [8; 9; 10], the policy iteration (PI) consisting of policy evaluation and policy improvement iterative techniques is used for continuous time and discrete time dynamic systems to iteratively update the control policy online with state and input information. However, the PI learning algorithm relies on a precise dynamic system. Both drift dynamic and input dynamic are needed in the iterative process. Therefore, it is based on the improvement of the PI algorithm and uses the iterative technology of PI approximation to the optimal solution. An integral reinforcement learning (IRL) algorithm in [11] is proposed to release drift dynamics by adding integral operations. In [12], for a locally unknown continuous-time nonlinear system with actuator saturation, an IRL algorithm with an actor-critic network is proposed to solve the HJB equation online. Then, the experience replay technique is used to update the critic weights to solve the IRL-Bellman equation. The IRL method is used for the unknown \(H_{\infty}\) tracking control problem of a linear system with actuator saturation in [13]. The designed controller not only makes the system state trajectory of convergence but the control strategy can also make the tracking error asymptotically stable. The complexity and uncertainty of the system increase the difficulty of solving the HJB solution. Especially the nonlinear system in practical problems, \(H_{\infty}\) control problem is an alternative method to solve the robust optimal problem. The purpose of the \(H_{\infty}\) control problems is to design a controller that can effectively suppress the impact of external disturbances on system performance. Therefore, many studies propose the robust optimal control method to transform the \(H_{\infty}\) optimal control problem into the zero-sum game problem, which is a max/min optimization problem essentially [14; 15]. Therefore, the Nash saddle point of the two-player zero-sum problem is regarded as solving the solution of the Hamilton-Jacobi-Isaacs (HJI) equation. In [16; 17], an online policy iteration (PI) algorithm is presented to solve the two-player zero-sum game. An ADP-based critic-actor network is used to approximate the solution of the HJI equation. For the system containing external disturbances, a new ADP-based online IRL algorithm is proposed to approximate the HJI equation. Use current data and historical data to update network weights to improve data utilization efficiency when solving the HJI equation [18]. In [19], an \(H_{\infty}\) tracking controller is designed to solve the zero-sum game with completely unknown dynamics and constrained input.