Prescribed Performance Optimal Tracking Control for Nonlinear Systems

Authors: zongsheng  huang
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-21 07:33:04

Abstract: In this paper, based on reinforcement learning (RL) technique, the optimal tracking control problem is considered for a class of strict-feedback nonlinear systems with prescribed performance (PP). The radial-basis-function (RBF) neural network (NN) is introduced to identify the unknown nonlinearities. Depending on the PP technique, the tracking error can be limited in the prescribed area. To guarantee the output of the system tracking the reference signal synchronously in an optimal way in strict-feedback nonlinear system, the adaptive backstepping control scheme is firstly established. Subsequently, the optimal controller is derived via policy iteration. Therefore, the whole controller consists of the adaptive controller and optimal controller. The stability analysis shows that all signals in the closed-loop system are bounded. The effectiveness and advantages of the designed control strategy are verified by the simulation examples.

Main paper: 
 i. introduction: 
 In practice, many classical systems in real physical world can be accurately modeled as strict-feedback nonlinear systems (SFNSs), such as the manned submarine vehicles [1] , the quadrotor unmanned aerial vehicles [2] , the flexible joint robots [3] , [4] and so on. Therefore, the control problems of SFNSs have become one of the hot issues in the control field due to the significant engineering requirements. A powerful technique to cope with the control problems for SFNSs is backstepping control technique [5] , [6] , [7] . Recently, due to the rapid development of neural control technique, the integration of backstepping control method and neural network (NN) control method has been widely utilized in nonlinear system control problems.Even though these combination of backstepping method and neural control method results mentioned above play an important role for SFNSs, the optimization performance has not been considered in these results. As a popular control method, optimal control can guarantee the optimal performance of the systems while saving energy [8] . Therefore, it is of great significance for practical control tasks. As we all know, one of the classical optimal control methods is dynamic programming (DP), which obtains the optimal solution by solving the Ricatti equation in linear systems. Moreover, by solving the Hamilton-Jacobian-Bellman (HJB) equation in nonlinear systems, the optimal controller is derived [9] . However, the HJB equation is a partial differential equation which is tough to deal with. To break through this limitation, many significant methods based on RL comes into being, such as the temporal-difference learning method [10] , Q-learning method [11] , adaptive dynamic programming (ADP) method [12] , and so on. As one of the most classical methods in RL-based methods, ADP method has been applied widely in nonlinear control field [13] . It should be noted that the RL method can interact with environment and modify control policies so that the successful control decisions can be remembered or used again by constructing a reinforcement signal [14] . In the existing results of optimal control problems, the RL was widely considered for optimal control [15] . In [16] , the RL control scheme was presented to study the H ∞ optimization problems. In [17] , the optimal control problem was solved by using RL algorithm for marine surface vessel system. In [18] , a robust control strategy was designed for multi-agent systems based on RL method. In [19] , an RL-based control method was proposed for nonlinear systems with input constraints and disturbances. These results made a great contribution on obtaining an optimal solution based on RL method. However, the steady state and transient state tracking error performances have not been considered in these results. Therefore, how to ensure the steady state and transient state tracking error performance of the nonlinear systems is of great significance [20] .By adopting a performance function, the PP is a valid technique to improve the steady state and transient state tracking error performance [21] . By transforming the restricted error to an unrestricted error, an expected tracking performance can be obtained accordingly [22] . The PP method has been widely applied in nonlinear systems control. In [23] , the PP method and sliding mode control technique were integrated to guarantee the tracking performance. However, these works paid little attention in consideration of the PP technique and optimal control problem simultaneously for SFNSs, which motives us to further study the RL-based PP optimal tracking control problem for SFNSs.Inspired by these results, in this paper, we are motivated to investigate the PP optimal tracking control based on RL for a class of SFNSs. To guarantee the error within the predefined range, the PP method is introduced. The adaptive controller is derived based on the backstepping method and the optimal controller is established via an RL-based policy iteration algorithm to guarantee the system output tracking the reference signal optimally. The main contributions of this paper are summarized as follows:(1) To guarantee the convergence rate, tracking error and the maximum overshoot within the predefined range, a performance function is adopted in the controller design.Compared with [24] , [25] , by combining the PP method and RL technique, the designed controller can make a balance between cost and tracking performance, which compensates the weakness that the transient state and steady state performance of the tracking error of the system has not been considered in the tracking control problem.(2) In this paper, the backstepping method is integrated with the policy iteration strategy in SFNSs. The whole controller consists of the adaptive controller and optimal controller, which guarantee the stability of the system while achieving the optimal tracking performance. The control scheme proposed in this paper takes the optimization into consideration and ensures the cost function to be minimized. (3) By utilizing the RBF NN to handle the unknown nonlinearity, we incorporate the adaptive controller and the optimal controller. Different from the tracking problems without considering the optimization and PP technique, the tracking error can be limited in the performance bounds optimally by the proposed controller. The remainder of the paper is as follows. In Section II, the PP technique is introduced. In Section III, the adaptive controller and the RL-based policy iteration algorithm are established. The stability analysis is given in Section IV. Finally, the simulation results and conclusions are shown in Section V and Section VI.In this section, the necessary knowledge of RBF NN and prescribed performance technique are introduced and the model of controlled plant is formulated.Consider the SFNS as followsẋi = x i+1 + f i (x i ), 1 ≤ i ≤ n -1 ẋn = u + f n (x n ) y = x 1 (1)where xi = [x 1 , x 2 , . . . , x i ] T ∈ R i , i = 1, . . . , n, denotes the state vector, u ∈ R and y ∈ R respectively represent the input and output of the system. f i (x i ) ∈ R, i = 1, . . . , n denotes the unknown smooth continuous nonlinear function.Define the y r as the desired signal and z 1 = x 1 -y r as the tracking error of system, respectively. 
 
 Appendix: