# A Review of Fuzzy Logic and Neural Network Based Intelligent Control Design for Discrete-Time Systems

Authors: Yiming Jiang; Chenguang Yang; Hongbin Ma
Journal: Discrete Dynamics in Nature and Society
 (2016)

---

## Abstract

Over the last few decades, the intelligent control methods such as fuzzy logic control (FLC) and neural network (NN) control have been successfully used in various applications. The rapid development of digital computer based control systems requires control signals to be calculated in a digital or discrete-time form. In this background, the intelligent control methods developed for discrete-time systems have drawn great attentions. This survey aims to present a summary of the state of the art of the design of FLC and NN-based intelligent control for discrete-time systems. For discrete-time FLC systems, numerous remarkable design approaches are introduced and a series of efficient methods to deal with the robustness, stability, and time delay of FLC discrete-time systems are recommended. Techniques for NN-based intelligent control for discrete-time systems, such as adaptive methods and adaptive dynamic programming approaches, are also reviewed. Overall, this paper is devoted to make a brief summary for recent progresses in FLC and NN-based intelligent control design for discrete-time systems as well as to present our thoughts and considerations of recent trends and potential research directions in this area.


---

## Body

## 1. Introduction

It is well known that control design is critical to the performance of the closed-loop system response, while an accurate system model is usually necessary for a high quality control design. But there are inevitable uncertainties during modeling of any practical systems. These modeling uncertainties may result in poor performance and may even lead to instability of the closed-loop systems. To improve control performance, many control strategies have been developed to consider these uncertainties in the control design stage. As one of the major control approaches, adaptive control has been developed for more than half a century with intense research activities involving rigorous problem formulation, stability analysis, robustness design, performance analysis, and applications [1–9].Early progress of adaptive control focused on identification and closed-loop system analysis of linear systems. Self-tuning regulator and model reference adaptive control are two typical adaptive controllers based on different design philosophies, and they have attracted renown scholars such as Åström and Wittenmark [10], Ljung [11], and Goodwin et al. [12] due to the challenges in closed-loop stability analysis. In fact, even for the discrete-time linear systems, the closed-loop system will become highly nonlinear due to the injection of adaptive controller such as Aström-Wittenmark self-tuning regulator, whose closed-loop stability was eventually resolved by Guo based on his cooperation with Chen et al. [6, 13, 14].Besides linear systems, discrete-time adaptive control has also been developed for various different classes of nonlinear systems, such as nonlinear systems with linear growth rate [15], nonlinear systems with convex/concave parameterization [16, 17], nonlinear systems with polynomial-form nonlinear growth rate [18–21], nonlinear systems with multiple/finite model structure uncertainty [22–25], nonlinear systems with nonlinear parameterization [26, 27], nonlinear systems with semiparametric uncertainties [28], and so on. Many challenges have been seen in these existing studies to cope with the interaction of nonlinearity and adaptation. Now it is clear that discrete-time adaptive control can be extended to nonlinear systems with linear growth rate by using the modification technique such as projection [15]; however, we also know that there exist intrinsic difficulties or even impossibility for discrete-time adaptive control when the nonlinear growth rate is high enough [29, 30], which is quantitatively characterized by fundamental limitations of feedback mechanism in terms of certain introduced uncertainty measure.In addition to the progress from linear systems to the nonlinear systems, rigorous stability analysis of the closed-loop adaptive system has also been well established [31]. Compared with robust control, the advantage of adaptive control lies in its ability in estimating and compensating for parametric uncertainties in a large range. However, towards the increasingly complexity of systems with complicated nonlinear functional uncertainties, it is necessary to develop more powerful control design tools. To this end, researchers noticed some limitations of traditional adaptive control in mainly coping with parametric uncertainties. And fuzzy logic (FL) and NN-based intelligent control have been introduced in the early 1990s [6].The concept of fuzzy set was initially proposed by Zadeh [32], and since then, fuzzy technique gained a rapid development. FL offers human reasoning capabilities to capture nonlinearities and uncertainties, which cannot be described by precise mathematical model. In [33], Wang and Mendel found that linear combinations of a series of fuzzy basis functions are able to approximate smooth nonlinear functions and proved such feature of universal approximator by using the Stone-Weierstrass theorem. Based on this work, an adaptive fuzzy control method was further proposed by combining the universal approximation of the FL controller and learning ability of the adaptive method [34]. And later, a stable adaptive fuzzy controller was successfully designed for a class of nonlinear systems. In this approach, the adaptive law was constructed to update the parameters of the FL controller during the adaptation procedure while it was not necessary to have the accurate mathematical model of the system. From then on, fuzzy control has attracted ever increasing research interest, since it is able to provide an effective solution to control complex and uncertain plants by employing the knowledge of specific experts for the controller design.NN is inspired by biological neuronal systems witch consist of a number of simple processing neurons interconnected to each other. McCulloch and Pitts introduced an idea to study the computational abilities of networks composed of simple models of neurons in the 1940s [35]. With highly parallel structure, NN is of great computational power and learning ability to emulate various systems dynamics. It is well established that NN is capable of universally approximating unknown smooth functions with errors that can be made arbitrarily small [36–38]. In addition to system modeling and control, NN has also been successfully applied in many other fields such as learning, pattern recognition, and signal processing. In NN control methodology, NN has been extensively studied as function approximators to compensate for the system uncertain nonlinearities in control design. In the last two decades, it has been verified that NN control is very useful for controlling highly uncertain nonlinear systems and NN control has demonstrated superiority over traditional control methods. Particularly, the marriage of adaptive control theories and NN techniques gives birth to adaptive NN control, which guarantees stability, robustness, and convergence of the closed-loop NN control systems without beforehand offline NN training [6].Nowadays, most of the control algorithms are realized by digital computers, and thus the desired control signals are calculated in a digital manner. Digital control systems have advantages such as easy to build, less sensitive to environment variation, flexible to change, and less expensive. A model based digital controller is designed in discrete-time form and operated in continuous-time with analogy signals. Since the data are generally processed at discrete-time instants, it is necessary for us to build the system model in discrete-time for ease of control design [6].Yet in the past decades, many significant progresses of FLC and NN control for nonlinear systems were in continuous-time form and there is considerable lag in the development for nonlinear discrete-time systems [39, 40]. Although for linear systems there are numerous results of NN and fuzzy controls in continuous-time systems, the NN and fuzzy control of nonlinear discrete-time systems have been considerately less studied than their counterparts in continuous-time. As a matter of fact, many techniques developed for continuous-time systems cannot be directly applied in discrete-time, especially when the systems are nonlinear. Discrete-time systems can be expressed by various equations, which in great contrast to the differential equations of continuous-time systems involve states at different time steps. Due to the different nature of difference equation and differential equation, some concepts in discrete-time have very different meanings from those in continuous-time and the stability analysis techniques become much more intractable. For example, the linearity property of the derivative of a continuous-time Lyapunov function is no longer existed in a discrete-time Lyapunov function [41]. Therefore, in order to design discrete-time fuzzy or NN controllers and implement them into digital control systems, they deserve in-depth investigation for fuzzy control and NN control of nonlinear discrete-time systems.The remainder of this paper is organized as follows. In Section2, a brief review of fuzzy control for discrete-time systems is presented. Some research problems of fuzzy control and discrete-time systems, such as adaptive control, robustness issue, stability analysis, and time-delay systems, are concerned and introduced for they are both theoretically challenging and practically meaningful. In Section 3, NN control methods for discrete-time systems are briefly reviewed. Research of adaptive NN control for discrete-time systems is introduced and NN-based adaptive dynamic programming methods for discrete-time systems are also discussed. Finally, in Section 4 a brief conclusion is given.


## 2. Fuzzy Logic Control for Discrete-Time Systems

Nowadays, techniques for FLC have developed rapidly, especially in modeling complex nonlinear systems. Since Wang first proved that the linear combinations of a series of fuzzy basis functions are universal approximators of any nonlinear systems, the universal approximation property of the fuzzy logic systems has been extensively studied [42–45]. The combination of adaptive control and fuzzy logic system allows adaptive laws to update parameters of the FL controller during the adaptation procedure while adaptive fuzzy control provides an efficient method to model complex nonlinear systems. In practice, the difficulty of design a fuzzy controller lies in the fact that there are usually various requirement imposed on the systems to ensure stability and performance, while most of complex control systems today are using digital computers to calculate the control signals in a digital form. Hence, the continuous-time methods could not be directly applied to most practical systems. The modeling/control for discrete-time systems is crucial to take the controllers into real plants. Thus, it is significant to study the discrete-time fuzzy systems as well as their molding and control. To this end, researchers have paid efforts to develop the discrete-time FLC. A typical procedure of the multiple-input multiple-output (MIMO) fuuzy logic system (FLS) approximating an unknown function F(x) is comprised of three primary components as follows [46].(1) Fuzzification. Take X as the input of the fuzzy system, and F^(X) is the estimation of the output. Both the input and the output are fuzzified into fuzzy linguistic terms with fuzzy membership function.(2) Fuzzy Rules. The collection of the fuzzy MIMO IF-THEN rules are designed to comprise the knowledge base for constructing the FLS. Using prior expert knowledge, the fuzzy rules can be obtained as (1)Rulel:Ifx1isA1l,x2isA2l,…,xnisAnl,theny1isB1l,y2isB2l,…,ymisBml,where x1~xn and y1~ym are the premise variables consisting of X and F^(X), respectively; Al and Bl are the linguistic variables of the fuzzy sets; l=1,2,…,L.(3) Fuzzy Inference Engine and Defuzzification. Using the strategy of the sum-product inference engine and the center-average defuzzifier, the output of the FLS can be described as(2)yj=∑l=1Lφ¯jl∏i=1nμAilxi∑l=1L∏i=1nμAilxi,where φ¯j(l) is the point where fuzzy membership function μBj(l)(φ¯j) achieves its maximum value. Define(3)wjx=∏i=1nμAilxi∑l=1L∏i=1nμAilxi,Wx=w1x,w2x,…,wLxT, and (Φ)j=[φ¯j(1),φ¯j(2),…,φ¯j(L)]T. Then, the output of MIMO FLS can be rewritten as(4)yj=ΦjTWX,where the subscript j represents the jth column vector of the matrix. Therefore, the estimation of F^(X) is(5)F^X=Φ1TWX,Φ2TWX,…,ΦmTWXT.Then F(X) can be written as(6)F^X=ΦTWX.


### 2.1. Adaptive FLC of Discrete-Time Systems

FLC has been extensively investigated in both academic and industrial communities. However, problems with high dimensionality also existed in higher-dimensional fuzzy systems due to the mathematical complications and intuitive limitations. Therefore, it required more systematic methods for advanced FLC design and synthesis, such as the adaptive FLC. Originally adaptive control was proposed for aircraft autopilots to deal with parameter variations during changing flight conditions. In the 1960s, the advances in stability theory and the progress of control theory improved the understanding of adaptive control. By the early 1980s, several adaptive approaches have been proven to provide stable operation and asymptotic tracking. Since then, the adaptive control problems have been rigorously formulated.Generally, adaptive FLC design for nonlinear discrete-time systems is much more difficult than those in continuous-time. The stability analysis techniques become much more intractable for difference equations than those for differential equations. Many nice Lyapunov adaptive control design methodologies developed in continuous-time are not applicable to discrete-time systems. Thus, it is significant to consider the adaptive FLC in discrete-time form.In recent years, the adaptive FLC for discrete-time nonlinear systems has been extensively studied. Jagannathan et al. [45, 47] developed an adaptive fuzzy logic controller for a general class of discrete-time nonlinear systems using basis vectors and provided rigorous proofs to stability of the discrete-time adaptive FLC. In this work, a multiloop structure controller was constructed with a fuzzy approximate linearization loop, an outer tracking loop, and a robustifying loop. By selecting FLC’s structure and the adaptation laws and giving a number of assumptions on a class of discrete-time nonlinear systems, closed-loop signals can achieve uniform ultimate boundedness. This method is significant in sense that no persistence of excitation is used and no certainty equivalence assumption is required. The controller has also been extended to a class of unknown feedback linearizable nonlinear dynamical systems under persistence of excitation in [48], in which rigorous stability proofs of discrete-time adaptive FLC for feedback linearizable unknown nonlinear systems were presented. Indirect fuzzy control of uncertain discrete-time nonlinear system was shown by Qi and Brdys [49]. This work presented an indirect adaptive FLC for the uncertainties in nonlinear plants and employed a Takagi-Sugeno (T-S) model to deal with the unknown dynamics in input-output form. A feedback linearization control law was designed by using structure, states, and parameters of the model. Gradient descent algorithm and recursive least square estimation method were used to online update the modeling parameters. The extension of sliding mode control (SMC), adaptive fuzzy SMC for a class of uncertain discrete-time nonlinear systems, was investigated by Lin et al. [50]. This work presented an adaptive interval type-2 fuzzy sliding mode controller for a class of unknown nonlinear discrete-time systems with training data disturbed by external disturbances. It employed an adaptive interval type-2 fuzzy control scheme and SMC approach to control the plant tracking a reference trajectory and prevented big chattering of the control effort. While adaptive fuzzy control has also been applied in discrete-time chaotic systems. In [51], Feng and Chen presented a novel adaptive control algorithm for discrete-time chaotic systems. The basic idea is representing the chaotic system as a T-S fuzzy model and designing a local linear adaptive controller in each local region. Meanwhile a global adaptive controller on the entire domain was reconstructed and the stability of the closed-loop adaptive control system was proved.


### 2.2. Robustness Issue in Discrete-Time Fuzzy Control

In practice, nonlinearities and uncertainties exist in almost all the industrial plants, while uncertainties in the modeling and control of nonlinear systems are still one of the most challenging problems in the control field. The presence of nonlinearities and uncertainties brings difficulties to controller design. In order to conquer these problems, a number of schemes have been developed, and among these, robust control is one of the most effective methods. Robustness is regarded as one of the most important requirements for a control system. In the past two decades, considerable attention has been paid to the robustness in fuzzy model-based control of discrete-time systems and many modification techniques were proposed to enhance the robustness of FLC for discrete-time systems [52–57].The work of Lee et al. [52] illustrated an approach of robust fuzzy control for nonlinear discrete-time systems. In this work, a systematic control structure was presented for nonlinear discrete-time T-S fuzzy systems with parametric uncertainties using the H∞ control design approach. This work also gave a novel solution to the robust stabilization problem of nonlinear systems by using basis-dependent Lyapunov function.H
∞ disturbance attenuation is one of the most important requirements for a fuzzy control system. Cao and Frank [53] have applied the H∞ control to address the robust stability for a class of uncertain discrete-time fuzzy systems via linear matrix inequality approach. They studied both the robust stabilization and the H∞ disturbance attenuation, while they presented several sufficient conditions to ensure robust stability of the fuzzy models based on coupled linear matrix inequalities. They also analyzed the robust H∞ performance of the fuzzy model-based discrete-time nonlinear systems by addressing the robust H∞ disturbance attenuation problems. The robust stability of the models is achieved independently of the uncertainties.How to develop a robustH∞ controller for discrete-time systems using the basis-dependent Lyapunov function was also shown by Zhou et al. [54]. In their work, a novel linear matrix inequalities characterization with H∞ norm bound was presented for discrete-time fuzzy systems. While additional matrix variables were used to decouple the Lyapunov function and system matrices as well as facilitate the design approach of the controller, this approach provides some sufficient results in the shape of strict linear matrix inequalities.Studied by Choi and Park, the state feedbackH∞ control for discrete-time systems was investigated by constructing Lyapunov functions with fuzzy weights. The fuzzy weighting Lyapunov functions were designed with both current-time and one-step-past information while designing controller parameterized with linear matrix inequalities (PLMIs). By selecting the structures of variables appropriately in the PLMIs, a special case LMI formulation was obtained. The H∞ control with output feedback for discrete-time fuzzy systems has also been rigorously studied by Xu and Lam [55], while Wu et al. [58] investigated the reliable H∞ control for discrete-time systems with delays and stochastic actuator faults by representing the stochastic behavior with a discrete-time homogeneous Markov chain.Tseng and Chen [56] used a fuzzy observer to estimate the premise variables which depend on the state variables, while the fuzzy observer was also used to address the nonlinear l∞-gain control problem. This work extended the l∞ control from linear discrete-time systems to nonlinear discrete-time systems. By using the T-S fuzzy model to represent the nonlinear discrete-time system, an observer-based fuzzy controller, which minimized the upper bound of l∞-gain and attenuated the peak of perturbation was designed. Xu et al. [57] gave stability analysis for discrete-time singular fuzzy systems in the presence of time-varying uncertainties while they ensured the system to be regular, causal, and stable. It has been shown that, for these systems, robust stability condition can be obtained by giving a sufficient condition in terms of a set of linear matrix inequalities.


### 2.3. Stability Issue in Discrete-Time Fuzzy Control

Stability is one of the most important properties for a system while FLC are used to address the stability problems for discrete-time systems as well. The quadratic stability for uncertain discrete-time fuzzy dynamic systems was shown by Feng and Ma [59]. This work gave some sufficient conditions of the quadratic stabilization for an uncertain fuzzy dynamic system. Stability was achieved when the suitable Riccati equation or a set of Riccati equations were solved. Stabilizing feedback control laws were also obtained by the developing algorithms. The nonquadratic stabilization conditions for nonlinear discrete-time fuzzy systems were also analyzed by Kruszewski et al. [60]. They considered discrete-time uncertain nonlinear models in a T-S form and studied the stability through a nonquadratic Lyapunov function. The stabilization conditions were developed by considering the Lyapunov function with a k-sample variation and extended to uncertain T-S models. Based on a switching fuzzy model and piecewise Lyapunov function, Wang et al. [61] proposed two stabilization criteria for discrete-time T-S fuzzy systems.Zhou et al. [54] used a basis-dependent Lyapunov-Krasovskii function to give stabilization analysis for discrete-time fuzzy systems. A robust control design approach was also developed by using the Lyapunov-Krasovskii function and facilitated by introducing additional instrumental matrix variables. Robust control problem for systems with time delay was concerned as well. Feng [62] used a piecewise smooth Lyapunov function to analyze the stability of discrete-time T-S fuzzy dynamic systems. In this study, stability of the system guaranteed by constructing a piecewise Lyapunov function. Meanwhile the Lyapunov function was obtained using the linear matrix inequalities method. This work showed that the piecewise quadratic Lyapunov functions based stability is less conservative than the common quadratic Lyapunov function based stability.The stabilization for discrete-time T-S fuzzy systems with state time-varying delay was shown by Gao et al. [63]. In their work, a fuzzy Lyapunov function was constructed to improve the delay-dependent stability condition. By avoiding the utilization of the bounding inequalities for the cross products between two vectors, reduction of the conservatism of stability condition was achieved. A delay-dependent stabilization algorithm was also developed for both state feedback and observer-based output feedback cases using a parallel distributed compensation scheme.


### 2.4. Fuzzy Control for Discrete-Time Systems with Time Delays

In practice, time delays are the intrinsic nature of various physical systems, such as communication, hydraulic, chemical processes, and electronics. The existence of time delays could generally lower the system performance and even cause instability and oscillation. Hence, remarkable attention has been devoted to the analysis and synthesis of time delay nonlinear systems as well as the discrete-time fuzzy systems [55, 64–69].The stability and stabilization problems of systems time-varying delay were investigated by Wu et al. [64]. In their work, the existing stability analysis approaches were divided into two types: delay-dependent (including the delay size) and delay-independent (irrelevant to the delay size). The stability of discrete-time T-S fuzzy systems with time-varying state delay was also investigated. They developed a delay partitioning method and employed a fuzzy Lyapunov-Krasovskii function to analyze the stability of delay-dependent time-varying state delay systems. Through delay partitioning, the less conservative stability condition is obtained and solved by LMI optimization techniques.The work of Su et al. [65] also gave a solution to the dynamic output feedback control problem for fuzzy systems with time-varying delays. They proposed a comparison model by approximating the time-varying delay state, while they gave a sufficient condition to ensure the asymptotical stability by adopting the scaled small-gain theorem as well as the delay partitioning method. This method allows designing desired dynamic output feedback controller by using optimization techniques. A filter design technique for discrete-time systems with time-varying delay has also been presented by Su et al. [66]. Both full-order and reduced-order filters were designed while approximators were employed to derive an input-output based sufficient condition. Tseng [67] studied the time delay problem for a nonlinear discrete-time system based on model reference fuzzy tracking control. In this work, the T-S fuzzy model was applied to approximate a time delay discrete-time system while the fuzzy controller was designed to reduce the tracking error based on this fuzzy model. This method has advantage in that no feedback linearization or adaptive approaches are used in the controller design.The robustH∞ control problem for uncertain discrete-time delay systems has been ivestigated by Xu and Lam [55]. They studied the state-space T-S fuzzy model with time delays under the assumption that the parameter uncertainties were norm-bounded. The output feedback controller was designed with full-order fuzzy dynamic to guarantee the closed-loop system is robust asymptotic stable as well as the admissible uncertainties are norm bound constraint. While solvability of this problem was addressed using a sufficient condition given by the linear matrix inequalities. The induced l2 filter design of T-S fuzzy discrete-time stochastic systems with time-varying delays has been investigated by Su et al. [68]. Wu et al. [69] studied the H∞ model approximation for discrete-time state delay fuzzy T-S systems.


## 2.1. Adaptive FLC of Discrete-Time Systems

FLC has been extensively investigated in both academic and industrial communities. However, problems with high dimensionality also existed in higher-dimensional fuzzy systems due to the mathematical complications and intuitive limitations. Therefore, it required more systematic methods for advanced FLC design and synthesis, such as the adaptive FLC. Originally adaptive control was proposed for aircraft autopilots to deal with parameter variations during changing flight conditions. In the 1960s, the advances in stability theory and the progress of control theory improved the understanding of adaptive control. By the early 1980s, several adaptive approaches have been proven to provide stable operation and asymptotic tracking. Since then, the adaptive control problems have been rigorously formulated.Generally, adaptive FLC design for nonlinear discrete-time systems is much more difficult than those in continuous-time. The stability analysis techniques become much more intractable for difference equations than those for differential equations. Many nice Lyapunov adaptive control design methodologies developed in continuous-time are not applicable to discrete-time systems. Thus, it is significant to consider the adaptive FLC in discrete-time form.In recent years, the adaptive FLC for discrete-time nonlinear systems has been extensively studied. Jagannathan et al. [45, 47] developed an adaptive fuzzy logic controller for a general class of discrete-time nonlinear systems using basis vectors and provided rigorous proofs to stability of the discrete-time adaptive FLC. In this work, a multiloop structure controller was constructed with a fuzzy approximate linearization loop, an outer tracking loop, and a robustifying loop. By selecting FLC’s structure and the adaptation laws and giving a number of assumptions on a class of discrete-time nonlinear systems, closed-loop signals can achieve uniform ultimate boundedness. This method is significant in sense that no persistence of excitation is used and no certainty equivalence assumption is required. The controller has also been extended to a class of unknown feedback linearizable nonlinear dynamical systems under persistence of excitation in [48], in which rigorous stability proofs of discrete-time adaptive FLC for feedback linearizable unknown nonlinear systems were presented. Indirect fuzzy control of uncertain discrete-time nonlinear system was shown by Qi and Brdys [49]. This work presented an indirect adaptive FLC for the uncertainties in nonlinear plants and employed a Takagi-Sugeno (T-S) model to deal with the unknown dynamics in input-output form. A feedback linearization control law was designed by using structure, states, and parameters of the model. Gradient descent algorithm and recursive least square estimation method were used to online update the modeling parameters. The extension of sliding mode control (SMC), adaptive fuzzy SMC for a class of uncertain discrete-time nonlinear systems, was investigated by Lin et al. [50]. This work presented an adaptive interval type-2 fuzzy sliding mode controller for a class of unknown nonlinear discrete-time systems with training data disturbed by external disturbances. It employed an adaptive interval type-2 fuzzy control scheme and SMC approach to control the plant tracking a reference trajectory and prevented big chattering of the control effort. While adaptive fuzzy control has also been applied in discrete-time chaotic systems. In [51], Feng and Chen presented a novel adaptive control algorithm for discrete-time chaotic systems. The basic idea is representing the chaotic system as a T-S fuzzy model and designing a local linear adaptive controller in each local region. Meanwhile a global adaptive controller on the entire domain was reconstructed and the stability of the closed-loop adaptive control system was proved.


## 2.2. Robustness Issue in Discrete-Time Fuzzy Control

In practice, nonlinearities and uncertainties exist in almost all the industrial plants, while uncertainties in the modeling and control of nonlinear systems are still one of the most challenging problems in the control field. The presence of nonlinearities and uncertainties brings difficulties to controller design. In order to conquer these problems, a number of schemes have been developed, and among these, robust control is one of the most effective methods. Robustness is regarded as one of the most important requirements for a control system. In the past two decades, considerable attention has been paid to the robustness in fuzzy model-based control of discrete-time systems and many modification techniques were proposed to enhance the robustness of FLC for discrete-time systems [52–57].The work of Lee et al. [52] illustrated an approach of robust fuzzy control for nonlinear discrete-time systems. In this work, a systematic control structure was presented for nonlinear discrete-time T-S fuzzy systems with parametric uncertainties using the H∞ control design approach. This work also gave a novel solution to the robust stabilization problem of nonlinear systems by using basis-dependent Lyapunov function.H
∞ disturbance attenuation is one of the most important requirements for a fuzzy control system. Cao and Frank [53] have applied the H∞ control to address the robust stability for a class of uncertain discrete-time fuzzy systems via linear matrix inequality approach. They studied both the robust stabilization and the H∞ disturbance attenuation, while they presented several sufficient conditions to ensure robust stability of the fuzzy models based on coupled linear matrix inequalities. They also analyzed the robust H∞ performance of the fuzzy model-based discrete-time nonlinear systems by addressing the robust H∞ disturbance attenuation problems. The robust stability of the models is achieved independently of the uncertainties.How to develop a robustH∞ controller for discrete-time systems using the basis-dependent Lyapunov function was also shown by Zhou et al. [54]. In their work, a novel linear matrix inequalities characterization with H∞ norm bound was presented for discrete-time fuzzy systems. While additional matrix variables were used to decouple the Lyapunov function and system matrices as well as facilitate the design approach of the controller, this approach provides some sufficient results in the shape of strict linear matrix inequalities.Studied by Choi and Park, the state feedbackH∞ control for discrete-time systems was investigated by constructing Lyapunov functions with fuzzy weights. The fuzzy weighting Lyapunov functions were designed with both current-time and one-step-past information while designing controller parameterized with linear matrix inequalities (PLMIs). By selecting the structures of variables appropriately in the PLMIs, a special case LMI formulation was obtained. The H∞ control with output feedback for discrete-time fuzzy systems has also been rigorously studied by Xu and Lam [55], while Wu et al. [58] investigated the reliable H∞ control for discrete-time systems with delays and stochastic actuator faults by representing the stochastic behavior with a discrete-time homogeneous Markov chain.Tseng and Chen [56] used a fuzzy observer to estimate the premise variables which depend on the state variables, while the fuzzy observer was also used to address the nonlinear l∞-gain control problem. This work extended the l∞ control from linear discrete-time systems to nonlinear discrete-time systems. By using the T-S fuzzy model to represent the nonlinear discrete-time system, an observer-based fuzzy controller, which minimized the upper bound of l∞-gain and attenuated the peak of perturbation was designed. Xu et al. [57] gave stability analysis for discrete-time singular fuzzy systems in the presence of time-varying uncertainties while they ensured the system to be regular, causal, and stable. It has been shown that, for these systems, robust stability condition can be obtained by giving a sufficient condition in terms of a set of linear matrix inequalities.


## 2.3. Stability Issue in Discrete-Time Fuzzy Control

Stability is one of the most important properties for a system while FLC are used to address the stability problems for discrete-time systems as well. The quadratic stability for uncertain discrete-time fuzzy dynamic systems was shown by Feng and Ma [59]. This work gave some sufficient conditions of the quadratic stabilization for an uncertain fuzzy dynamic system. Stability was achieved when the suitable Riccati equation or a set of Riccati equations were solved. Stabilizing feedback control laws were also obtained by the developing algorithms. The nonquadratic stabilization conditions for nonlinear discrete-time fuzzy systems were also analyzed by Kruszewski et al. [60]. They considered discrete-time uncertain nonlinear models in a T-S form and studied the stability through a nonquadratic Lyapunov function. The stabilization conditions were developed by considering the Lyapunov function with a k-sample variation and extended to uncertain T-S models. Based on a switching fuzzy model and piecewise Lyapunov function, Wang et al. [61] proposed two stabilization criteria for discrete-time T-S fuzzy systems.Zhou et al. [54] used a basis-dependent Lyapunov-Krasovskii function to give stabilization analysis for discrete-time fuzzy systems. A robust control design approach was also developed by using the Lyapunov-Krasovskii function and facilitated by introducing additional instrumental matrix variables. Robust control problem for systems with time delay was concerned as well. Feng [62] used a piecewise smooth Lyapunov function to analyze the stability of discrete-time T-S fuzzy dynamic systems. In this study, stability of the system guaranteed by constructing a piecewise Lyapunov function. Meanwhile the Lyapunov function was obtained using the linear matrix inequalities method. This work showed that the piecewise quadratic Lyapunov functions based stability is less conservative than the common quadratic Lyapunov function based stability.The stabilization for discrete-time T-S fuzzy systems with state time-varying delay was shown by Gao et al. [63]. In their work, a fuzzy Lyapunov function was constructed to improve the delay-dependent stability condition. By avoiding the utilization of the bounding inequalities for the cross products between two vectors, reduction of the conservatism of stability condition was achieved. A delay-dependent stabilization algorithm was also developed for both state feedback and observer-based output feedback cases using a parallel distributed compensation scheme.


## 2.4. Fuzzy Control for Discrete-Time Systems with Time Delays

In practice, time delays are the intrinsic nature of various physical systems, such as communication, hydraulic, chemical processes, and electronics. The existence of time delays could generally lower the system performance and even cause instability and oscillation. Hence, remarkable attention has been devoted to the analysis and synthesis of time delay nonlinear systems as well as the discrete-time fuzzy systems [55, 64–69].The stability and stabilization problems of systems time-varying delay were investigated by Wu et al. [64]. In their work, the existing stability analysis approaches were divided into two types: delay-dependent (including the delay size) and delay-independent (irrelevant to the delay size). The stability of discrete-time T-S fuzzy systems with time-varying state delay was also investigated. They developed a delay partitioning method and employed a fuzzy Lyapunov-Krasovskii function to analyze the stability of delay-dependent time-varying state delay systems. Through delay partitioning, the less conservative stability condition is obtained and solved by LMI optimization techniques.The work of Su et al. [65] also gave a solution to the dynamic output feedback control problem for fuzzy systems with time-varying delays. They proposed a comparison model by approximating the time-varying delay state, while they gave a sufficient condition to ensure the asymptotical stability by adopting the scaled small-gain theorem as well as the delay partitioning method. This method allows designing desired dynamic output feedback controller by using optimization techniques. A filter design technique for discrete-time systems with time-varying delay has also been presented by Su et al. [66]. Both full-order and reduced-order filters were designed while approximators were employed to derive an input-output based sufficient condition. Tseng [67] studied the time delay problem for a nonlinear discrete-time system based on model reference fuzzy tracking control. In this work, the T-S fuzzy model was applied to approximate a time delay discrete-time system while the fuzzy controller was designed to reduce the tracking error based on this fuzzy model. This method has advantage in that no feedback linearization or adaptive approaches are used in the controller design.The robustH∞ control problem for uncertain discrete-time delay systems has been ivestigated by Xu and Lam [55]. They studied the state-space T-S fuzzy model with time delays under the assumption that the parameter uncertainties were norm-bounded. The output feedback controller was designed with full-order fuzzy dynamic to guarantee the closed-loop system is robust asymptotic stable as well as the admissible uncertainties are norm bound constraint. While solvability of this problem was addressed using a sufficient condition given by the linear matrix inequalities. The induced l2 filter design of T-S fuzzy discrete-time stochastic systems with time-varying delays has been investigated by Su et al. [68]. Wu et al. [69] studied the H∞ model approximation for discrete-time state delay fuzzy T-S systems.


## 3. NN Control for Discrete-Time Systems

NN is of powerful computing ability and learning ability to emulate various systems dynamics and is capable of approximating an unknown function with arbitrary accuracy. NN is very successful in system modeling and control by its capacity of universal approximate highly nonlinear and uncertain, nonlinear and complex dynamic of systems. NN’s approximation ability has been shown by the Stone-Weierstrass theorem, which states that a universal approximator can approximate, to an arbitrary degree of accuracy, any real continuous function on a compact set. Besides the universal approximation abilities, NN also shows its excellence in parallel distributed processing abilities, learning, adaptation abilities, natural fault tolerance, and feasibility for hardware implementation. These advantages make NN particularly attractive and prospective for nonlinear control and modeling. NN has been successfully applied to robot manipulators control [70], distillation column control [71], chemical processes identification [72], flight control [73–75], and so forth. To extend the NN technique from continuous-time to discrete-time, many efforts have been made to study the NN controlled discrete-time systems [76–78].


### 3.1. Adaptive NN Control for Discrete-Time Systems

In the early stage, backpropagation (BP) algorithm [79] greatly boosted the development of NN control. It is noted that, in the early NN control design, the control performances were demonstrated through simulation or by particular experimental examples, and consequently there were shortage of analytical analysis. In addition, an offline identification procedure was essential for achieving a stable NN control system. Thereafter, the emergence of Lyapunov-based NN design makes it possible to use the available adaptive control theories to rigorously guarantee stability, robustness, and convergence of the closed-loop NN control systems. We call the control design combining adaptive control theories and NN techniques as adaptive NN control. It updates NN weights online and guarantees the stability of the closed-loop system. Adaptive NN control design has been elegantly developed for nonlinear systems with parametric uncertainties, while many efforts have been dedicated in adaptive NN control for nonlinear discrete-time systems [6].For high-order affine nonlinear system in normal form, adaptive NN controls using LPNN and MNN have been developed in [80, 81] using a filtered tracking error. In [82], the controller is designed incorporating reinforcement learning technique to improve control performance. In this work, a critic NN has been introduced to approximate the strategic utility function which is considered as the long-term system performance measure. For discrete-time systems in strict-feedback form, adaptive NN control has been developed via backstepping design after system transformation [83]. In [84], adaptive NN control has been investigated for discrete-time system in affine NARMAX form.In the above mentioned results, the adaptive NN control designs were carried out through either feedback linearization or backstepping. But these approaches are not applicable to nonaffine systems, especially feedback linearization based methods, which greatly depend on the affine appearance of control variables. As a matter of fact, adaptive NN control methods for nonaffine systems have been less studied in comparison with large amount of research work on affine nonlinear systems, because of the difficulty of control design caused by the nonaffine form of control input. To overcome the difficulty, linearization based NN controls have been put forward. In [17], the nonaffine discrete-time system has been divided into two parts: one is linear and another is nonlinear, and consequently a linear adaptive controller and a nonlinear adaptive NN controller have been designed with a switching rule. In [85], it directly utilized NN as emulator of the “inverse" of the nonlinear discrete-time systems. Furthermore, the study of discrete-time systems for adaptive NN control using implicit function to assert the existence of an ideal inverse control was investigated in [86]. Thereafter, the implicit function based adaptive NN control has been widely studied in discrete-time form [87, 88]. Block triangular discrete-time systems with normal form subsystems have been studied in [80, 81]. For the block triangular systems with strict-feedback subsystems, state feedback and output feedback adaptive NN control have been developed in [89, 90] by extending the systems transformation based backstepping technique proposed for SISO case in [83]. In [91], adaptive NN control has been used for sampled-data nonlinear MIMO systems in general affine form based on linearization. The control scheme is an integration of an NN approach and a variable structure method.An effort has been made in [92] to explore the adaptive NN control of a class of nonaffine systems in discrete-time. This work aimed to solve the nonaffine appearance and noncausal problems of the following pure-feedback discrete-time system: (7)ξik+1=fiξ¯i,ξi+1k,i=1,2,…,n-1,n≥2,ξnk+1=fnξ¯nk,uk,dk,yk=ξ1k,where ξ¯i(k)=[ξ1(k),ξ2(k),…,ξn(k)]T are system states, fi are unknown nonlinear functions, and u(k) and y(k) are system input and output, respectively, while d(k) denotes the external disturbance.Using a states prediction technique, the pure-feedback discrete-time systems in (7) are shown to be transformable to an n-step-ahead predictor as below:(8)yk+n=ϕξ¯nk,uk,dk=ϕsξnk,uk+dsk.And by future output predictions, the above system (8) can be further transformed into an input-output model for the output feedback control:(9)yk+n=Fn1y_k+1,u_k,dk,…,dk-n+2.This shows that the system presentation (9) in NARMAX form is a transformation of the pure-feedback system (8). After transformation, both state feedback and output feedback controls only need to employ a single NN in the controller design, rather than a number of NNs in previous researches.Based on the SISO pure-feedback system (8) and its transformation NARMAX model in (9), the control design has been further investigated by using high-order neural network to approximate unknown functions [93]. In the above mentioned control design, availability is assumed for knowledge of control directions, which are defined as “the signs of control variable gains in affine systems, or the signs of partial derivatives over control variables in nonaffine systems” [93].The paper overcame the problems of output-feedback control when the prior knowledge of the control directions is unknown. The main idea is to introduce a discrete Nussbaum gain to counter the lack of knowledge on control gain in adaptive NN control.A rigorous definition of discrete Nussbaum gain in [93] is given as follows.Remark 1.
Consider a discrete nonlinear functionN(x(k)) defined on a sequence x(k) with xs(k)=sup{(xk′)}. N(x(k)) is a discrete Nussbaum gain if and only if it satisfies the following two properties:
(i) ifxs(k) increases without bound, then(10)sup⁡1xskSNxk=+∞,inf⁡1xskSNxk=-∞;
(ii) ifxs(k)≥(k)δ1, then |SN(x(k))|≥δ2 with some positive constants δ1 and δ2, where SN(x(k)) is defined as(11)SNxk=∑k′=0kNxk′Δxk′,with Δx(k)=x(k+1)-x(k). In addition, neither the upper bounds nor the lower bounds of the control gains are required to be known.Unknown control directions problem for MIMO discrete-time nonlinear systems was solved by using adaptive output feedback NN control in [94]. The studies in [92, 93] all studied the discrete NN control for SISO systems, and [94] extended the study to a class of MIMO discrete-time systems with each subsystem in the nonaffine pure-feedback as follows:(12)ξj,ijk+1=fj,ijξ¯1,ij-mj1k,ξ¯2,ij-mj2k,…,ξ¯n,ij-mjnk,ξj,ij+1kij=1,2,…,nj-1,ξj,njk+1=fj,njΞk,u¯jk,djk,yjk=ξj,1k, whereξ¯j,ijk=[ξj,1(k),ξj,2(k),…,ξj,ij(k)]T are vectors of states variable of subsystem; Ξ(k) is vector of all state variables; fj,ij are unknown nonlinear functions; u¯j(k) and yj(k) are system inputs and outputs, respectively; while dj(k) denote the external disturbance.For nonlinear MIMO discrete-time higher order systems, a major drawback is that there are too many online-tuned adaptive parameters and large online computation burden. In [95], an adaptive neural output feedback adaptive controller was designed for MIMO nonlinear discrete-time systems with fewer adaptive parameters. The output feedback adaptive control for a class of nonlinear discrete-time systems with unknown control directions was investigated in [96].


### 3.2. NN-Based Dynamic Programming Algorithm for Discrete-Time Systems

In the past several decades, optimal control for nonlinear systems has been a key focus in the control fields. While adaptive dynamics programming (ADP) is a very useful approach to solve the optimal control problems. ADP is a reinforcement learning method to give solution to the dynamic programming utilizing function approximating the value function based on adaptive approach [97]. P. Werbos [98] classified the discrete-time ADP into a number of schemes: action-dependent HDP, dual HDP (DHP), heuristic dynamic programming (HDP), and action-dependent dual HDP, while NN has been widely used to solve the adaptive dynamic programming problems of discrete-time systems [97, 99–104].He and Jagannathan [99] showed the reinforcement learning with NN-based controller for nonlinear discrete-time systems. This work developed an adaptive-critic-base discrete-time NN controller to deliver a desired tracking performance for nonlinear systems in the presence of actuator constraints. The critic NN and action NN were designed to approximate the strategic utility function while uniformly ultimate boundedness of the closed-loop tracking error was proved by Lyapunov approach. The reinforcement learning NN control for SISO discrete-time pure-feedback systems was studied in [100]. The controller design is based on the transformed predictor and two NNs of control architecture. One critic NN to approximate the strategic utility function is shown as follows.By introducing a utility functionp(k) based on the tracking error e(k)=y(k)-yd(k):(13)pk=a0ek, where p(k)∈R, a0∈R is the positive design parameter. And the strategic utility function Q∈R is defined as(14)Qk=aNp+1+an-1pk+2+⋯+ak+1pN+⋯, where a∈R,0<a<1,N is the horizon. Then, the critic NN is used to approximate the strategic utility function Q(k):(15)Q^=W^cTkSczk,Sczk∈Rlc, where WcT(k)∈Rlc is the estimation of optimal NN Wc∗ weights. The uniformly ultimate boundedness of closed-loop signals is established via Lyapunov stability analysis.Al-Tamimi et al. [97] proved the convergence of a value function solution for nonlinear dynamical systems using HDP algorithm, and NNs were employed to approximate the value and the control action at each iteration. Liu et al. [101] used the HDP algorithm to develop a NN-based optimal controller for unknown discrete-time nonlinear systems. The main idea of this work is to introduce an iterative ADP algorithm and a globalized HDP technique into the controller design. And by using the cost function and control law, the convergence of the optimal control is guaranteed. In [102], an iterative ADP algorithm for discrete-time systems was developed as well. An optimal controller was designed to address the infinite-horizon discrete-time nonlinear systems with finite approximation errors. It was shown that when convergence conditions are satisfied, iterative performance index function can converge to the performance index functions’ lower bound, while the performance index functions were approximated by NNs to compute optimal control policy.The ADP control in the unknown discrete-time Markov jump systems was investigated by Zhong et al. [103]. And Wang et al. [105] also studied the ADP for discrete-time systems to obtain the ɛ-optimal control by using neural networks. In [106], iterative ADP algorithm was used to study the near-optimal control with control constraints in discrete-time systems. An iterative ADP for discrete-time systems was studied in [104]. In this work, the optimal controller was designed with control constraints while NN was used to identify the unknown dynamical systems with stability proof.


## 3.1. Adaptive NN Control for Discrete-Time Systems

In the early stage, backpropagation (BP) algorithm [79] greatly boosted the development of NN control. It is noted that, in the early NN control design, the control performances were demonstrated through simulation or by particular experimental examples, and consequently there were shortage of analytical analysis. In addition, an offline identification procedure was essential for achieving a stable NN control system. Thereafter, the emergence of Lyapunov-based NN design makes it possible to use the available adaptive control theories to rigorously guarantee stability, robustness, and convergence of the closed-loop NN control systems. We call the control design combining adaptive control theories and NN techniques as adaptive NN control. It updates NN weights online and guarantees the stability of the closed-loop system. Adaptive NN control design has been elegantly developed for nonlinear systems with parametric uncertainties, while many efforts have been dedicated in adaptive NN control for nonlinear discrete-time systems [6].For high-order affine nonlinear system in normal form, adaptive NN controls using LPNN and MNN have been developed in [80, 81] using a filtered tracking error. In [82], the controller is designed incorporating reinforcement learning technique to improve control performance. In this work, a critic NN has been introduced to approximate the strategic utility function which is considered as the long-term system performance measure. For discrete-time systems in strict-feedback form, adaptive NN control has been developed via backstepping design after system transformation [83]. In [84], adaptive NN control has been investigated for discrete-time system in affine NARMAX form.In the above mentioned results, the adaptive NN control designs were carried out through either feedback linearization or backstepping. But these approaches are not applicable to nonaffine systems, especially feedback linearization based methods, which greatly depend on the affine appearance of control variables. As a matter of fact, adaptive NN control methods for nonaffine systems have been less studied in comparison with large amount of research work on affine nonlinear systems, because of the difficulty of control design caused by the nonaffine form of control input. To overcome the difficulty, linearization based NN controls have been put forward. In [17], the nonaffine discrete-time system has been divided into two parts: one is linear and another is nonlinear, and consequently a linear adaptive controller and a nonlinear adaptive NN controller have been designed with a switching rule. In [85], it directly utilized NN as emulator of the “inverse" of the nonlinear discrete-time systems. Furthermore, the study of discrete-time systems for adaptive NN control using implicit function to assert the existence of an ideal inverse control was investigated in [86]. Thereafter, the implicit function based adaptive NN control has been widely studied in discrete-time form [87, 88]. Block triangular discrete-time systems with normal form subsystems have been studied in [80, 81]. For the block triangular systems with strict-feedback subsystems, state feedback and output feedback adaptive NN control have been developed in [89, 90] by extending the systems transformation based backstepping technique proposed for SISO case in [83]. In [91], adaptive NN control has been used for sampled-data nonlinear MIMO systems in general affine form based on linearization. The control scheme is an integration of an NN approach and a variable structure method.An effort has been made in [92] to explore the adaptive NN control of a class of nonaffine systems in discrete-time. This work aimed to solve the nonaffine appearance and noncausal problems of the following pure-feedback discrete-time system: (7)ξik+1=fiξ¯i,ξi+1k,i=1,2,…,n-1,n≥2,ξnk+1=fnξ¯nk,uk,dk,yk=ξ1k,where ξ¯i(k)=[ξ1(k),ξ2(k),…,ξn(k)]T are system states, fi are unknown nonlinear functions, and u(k) and y(k) are system input and output, respectively, while d(k) denotes the external disturbance.Using a states prediction technique, the pure-feedback discrete-time systems in (7) are shown to be transformable to an n-step-ahead predictor as below:(8)yk+n=ϕξ¯nk,uk,dk=ϕsξnk,uk+dsk.And by future output predictions, the above system (8) can be further transformed into an input-output model for the output feedback control:(9)yk+n=Fn1y_k+1,u_k,dk,…,dk-n+2.This shows that the system presentation (9) in NARMAX form is a transformation of the pure-feedback system (8). After transformation, both state feedback and output feedback controls only need to employ a single NN in the controller design, rather than a number of NNs in previous researches.Based on the SISO pure-feedback system (8) and its transformation NARMAX model in (9), the control design has been further investigated by using high-order neural network to approximate unknown functions [93]. In the above mentioned control design, availability is assumed for knowledge of control directions, which are defined as “the signs of control variable gains in affine systems, or the signs of partial derivatives over control variables in nonaffine systems” [93].The paper overcame the problems of output-feedback control when the prior knowledge of the control directions is unknown. The main idea is to introduce a discrete Nussbaum gain to counter the lack of knowledge on control gain in adaptive NN control.A rigorous definition of discrete Nussbaum gain in [93] is given as follows.Remark 1.
Consider a discrete nonlinear functionN(x(k)) defined on a sequence x(k) with xs(k)=sup{(xk′)}. N(x(k)) is a discrete Nussbaum gain if and only if it satisfies the following two properties:
(i) ifxs(k) increases without bound, then(10)sup⁡1xskSNxk=+∞,inf⁡1xskSNxk=-∞;
(ii) ifxs(k)≥(k)δ1, then |SN(x(k))|≥δ2 with some positive constants δ1 and δ2, where SN(x(k)) is defined as(11)SNxk=∑k′=0kNxk′Δxk′,with Δx(k)=x(k+1)-x(k). In addition, neither the upper bounds nor the lower bounds of the control gains are required to be known.Unknown control directions problem for MIMO discrete-time nonlinear systems was solved by using adaptive output feedback NN control in [94]. The studies in [92, 93] all studied the discrete NN control for SISO systems, and [94] extended the study to a class of MIMO discrete-time systems with each subsystem in the nonaffine pure-feedback as follows:(12)ξj,ijk+1=fj,ijξ¯1,ij-mj1k,ξ¯2,ij-mj2k,…,ξ¯n,ij-mjnk,ξj,ij+1kij=1,2,…,nj-1,ξj,njk+1=fj,njΞk,u¯jk,djk,yjk=ξj,1k, whereξ¯j,ijk=[ξj,1(k),ξj,2(k),…,ξj,ij(k)]T are vectors of states variable of subsystem; Ξ(k) is vector of all state variables; fj,ij are unknown nonlinear functions; u¯j(k) and yj(k) are system inputs and outputs, respectively; while dj(k) denote the external disturbance.For nonlinear MIMO discrete-time higher order systems, a major drawback is that there are too many online-tuned adaptive parameters and large online computation burden. In [95], an adaptive neural output feedback adaptive controller was designed for MIMO nonlinear discrete-time systems with fewer adaptive parameters. The output feedback adaptive control for a class of nonlinear discrete-time systems with unknown control directions was investigated in [96].


## 3.2. NN-Based Dynamic Programming Algorithm for Discrete-Time Systems

In the past several decades, optimal control for nonlinear systems has been a key focus in the control fields. While adaptive dynamics programming (ADP) is a very useful approach to solve the optimal control problems. ADP is a reinforcement learning method to give solution to the dynamic programming utilizing function approximating the value function based on adaptive approach [97]. P. Werbos [98] classified the discrete-time ADP into a number of schemes: action-dependent HDP, dual HDP (DHP), heuristic dynamic programming (HDP), and action-dependent dual HDP, while NN has been widely used to solve the adaptive dynamic programming problems of discrete-time systems [97, 99–104].He and Jagannathan [99] showed the reinforcement learning with NN-based controller for nonlinear discrete-time systems. This work developed an adaptive-critic-base discrete-time NN controller to deliver a desired tracking performance for nonlinear systems in the presence of actuator constraints. The critic NN and action NN were designed to approximate the strategic utility function while uniformly ultimate boundedness of the closed-loop tracking error was proved by Lyapunov approach. The reinforcement learning NN control for SISO discrete-time pure-feedback systems was studied in [100]. The controller design is based on the transformed predictor and two NNs of control architecture. One critic NN to approximate the strategic utility function is shown as follows.By introducing a utility functionp(k) based on the tracking error e(k)=y(k)-yd(k):(13)pk=a0ek, where p(k)∈R, a0∈R is the positive design parameter. And the strategic utility function Q∈R is defined as(14)Qk=aNp+1+an-1pk+2+⋯+ak+1pN+⋯, where a∈R,0<a<1,N is the horizon. Then, the critic NN is used to approximate the strategic utility function Q(k):(15)Q^=W^cTkSczk,Sczk∈Rlc, where WcT(k)∈Rlc is the estimation of optimal NN Wc∗ weights. The uniformly ultimate boundedness of closed-loop signals is established via Lyapunov stability analysis.Al-Tamimi et al. [97] proved the convergence of a value function solution for nonlinear dynamical systems using HDP algorithm, and NNs were employed to approximate the value and the control action at each iteration. Liu et al. [101] used the HDP algorithm to develop a NN-based optimal controller for unknown discrete-time nonlinear systems. The main idea of this work is to introduce an iterative ADP algorithm and a globalized HDP technique into the controller design. And by using the cost function and control law, the convergence of the optimal control is guaranteed. In [102], an iterative ADP algorithm for discrete-time systems was developed as well. An optimal controller was designed to address the infinite-horizon discrete-time nonlinear systems with finite approximation errors. It was shown that when convergence conditions are satisfied, iterative performance index function can converge to the performance index functions’ lower bound, while the performance index functions were approximated by NNs to compute optimal control policy.The ADP control in the unknown discrete-time Markov jump systems was investigated by Zhong et al. [103]. And Wang et al. [105] also studied the ADP for discrete-time systems to obtain the ɛ-optimal control by using neural networks. In [106], iterative ADP algorithm was used to study the near-optimal control with control constraints in discrete-time systems. An iterative ADP for discrete-time systems was studied in [104]. In this work, the optimal controller was designed with control constraints while NN was used to identify the unknown dynamical systems with stability proof.


## 4. Conclusion

In this short survey, despite the impossibility in identifying or listing all the related contributions, best efforts have been made to summarize the major achievements in the area of discrete-time adaptive control with modern techniques based on FL and NNs, which are also conventionally termed as important part of “soft-computing” approaches or “intelligent control” in the control communities. In particular, the marriage of “adaptive control” and “intelligent control” has been reviewed for the purpose of clarifying main contributions and outlining some possible trends for the development of this promising area.Generally speaking, adaptive control for discrete-time linear systems was extensively explored in the past decades, despite the fact that the complete closed-loop stability analysis for the so-called self-tuning regulator was ever regarded as a long-term challenging problem. Later it became clear that discrete-time adaptive control can be extended to nonlinear systems with linear growth rate. However, general nonlinear systems with parametric and/or nonparametric uncertainties are usually very difficult to cope with, which motivated various solutions for different certain classes of nonlinear systems.Among the various solutions, intelligent control emerged as one important way to resolve the challenges caused by the nonlinearity. The key for usability of intelligent control based on FL or NNs lies in their universal approximator property, at the cost of tuning rules of FL or updating parameters of NNs. FL and NNs are motivated by different background; hence their design philosophies look different; however, essentially speaking, both of them can be expressed as weighted sum of some kernel functions, where the weights can be tuned to approximate arbitrary smooth or continuous nonlinear function. The ideas borrowed from adaptive estimation and adaptive control provide such a way to adaptively tune the weight parameters in FL or NNs; thus the new area of adaptive FLC or NN control emerged with extensive studies.In a summary, a brief review on FLC for discrete-time systems is provided by highlighting the adaptive FLC, robustness issue, and stability issue, while NN control for discrete-time systems is also reviewed with focuses on adaptive NN control and NN-based dynamic programming. We believe this topic would promote increasing investigations in both theories and applications. And some emerging techniques, such as deep learning and big data, could also be merged with the intelligent control for discrete-time systems and give birth to brand new design approaches of intelligent control in the future.



---

*Source: 7217364-2016-03-31.xml*