adaptive control of non linear continuous time systems using neural networks general relative degree and mimo cases multilayer neural networks are used in a non linear adaptive control problem the plant is an unknown feedback linearizable continuous time system with relative degree 1 the single input single output system is studied first and then the methodology is extended to control square multi input multi output systems the control objective is for the plant to track a reference trajectory and the control law is defined in terms of the outputs of the neural networks the parameters of the networks are updated on line according to an augmented tracking error and the network derivatives a local convergence theorem is given on the convergence of the tracking error this control algorithm is applied to control a two input two output relative degree two system

direct adaptive nn control of a class of nonlinear systems in this paper direct adaptive neural network nn control is presented for a class of affine nonlinear systems in the strict feedback form with unknown nonlinearities by utilizing a special property of the affine term the developed scheme avoids the controller singularity problem completely all the signals in the closed loop are guaranteed to be semiglobally uniformly ultimately bounded and the output of the system is proven to converge to a small neighborhood of the desired trajectory the control performance of the closed loop system is guaranteed by suitably choosing the design parameters simulation results are presented to show the effectiveness of the approach

global target aggregation and state agreement of nonlinear multi agent systems with switching topologies in this paper we discuss coordination problems of a group of autonomous agents including the target aggregation to a convex set and the state agreement the aggregation of the whole agent group consisting of leaders informed agents and followers to a given set is investigated with switching interconnection topologies described by the connectivity assumptions on the joint topology in the time interval t for any time t and then the state agreement problem is studied in a similar way an approach based on set stability and limit set analysis is given to study the multi agent convergence problems with the help of graph theory and convex analysis coordination conditions are obtained in some important cases and the results show that simple local rules can make the networked agents with first order nonlinear individual dynamics achieve desired collective behaviors

approximate primal solutions and rate analysis for dual subgradient methods in this paper we study methods for generating approximate primal solutions as a byproduct of subgradient methods applied to the lagrangian dual of a primal convex possibly nondifferentiable constrained optimization problem our work is motivated by constrained primal problems with a favorable dual problem structure that leads to efficient implementation of dual subgradient methods such as the recent resource allocation problems in large scale networks for such problems we propose and analyze dual subgradient methods that use averaging schemes to generate approximate primal optimal solutions these algorithms use a constant stepsize in view of its simplicity and practical significance we provide estimates on the primal infeasibility and primal suboptimality of the generated approximate primal solutions these estimates are given per iteration thus providing a basis for analyzing the trade offs between the desired level of error and the selection of the stepsize value our analysis relies on the slater condition and the inherited boundedness properties of the dual problem under this condition it also relies on the boundedness of subgradients which is ensured by assuming the compactness of the constraint set

constrained consensus in continuous time multiagent systems under weighted graph this paper studies constrained consensus for continuous time multiagent systems where agents are subject to different state constraints a projection based consensus protocol is presented where a projection operator is used on top of the traditional consensus protocol in order to keep the state staying inside the constraint set this operator can ensure the smoothness of the state trajectory under projection and it will only be used when the state is close to the boundary of the constraint set and about to move outside under this protocol each individual agent only needs to know its own state and constraint set as well as the relative states to its neighboring agents with the assumption that all constraint sets are convex and have nonempty intersection it is proven that global consensus can be achieved without violating any state constraints during the transience

incremental stochastic subgradient algorithms for convex optimization this paper studies the effect of stochastic errors on two constrained incremental subgradient algorithms the incremental subgradient algorithms are viewed as decentralized network optimization algorithms as applied to minimize a sum of functions when each component function is known only to a particular agent of a distributed network first the standard cyclic incremental subgradient algorithm is studied in this the agents form a ring structure and pass the iterate in a cycle when there are stochastic errors in the subgradient evaluations sufficient conditions on the moments of the stochastic errors are obtained that guarantee almost sure convergence when a diminishing step size is used in addition almost sure bounds on the algorithm s performance with a constant step size are also obtained next the markov randomized incremental subgradient method is studied this is a noncyclic version of the incremental algorithm where the sequence of computing agents is modeled as a time nonhomogeneous markov chain such a model is appropriate for mobile networks as the network topology changes across time in these networks convergence results and error bounds for the markov randomized method in the presence of stochastic errors for diminishing and constant step sizes are obtained

zeroth order method for distributed optimization with approximate projections this paper studies the problem of minimizing a sum of possible nonsmooth convex functions that are corresponding to multiple interacting nodes subject to a convex state constraint set time varying directed network is considered here two types of computational constraints are investigated in this paper one where the information of gradients is not available and the other where the projection steps can only be calculated approximately we devise a distributed zeroth order method the implementation of which needs only functional evaluations and approximate projection in particular we show that the proposed method generates expected function value sequences that converge to the optimal value provided that the projection errors decrease at appropriate rates

design of networked control systems with packet dropouts this note is concerned with stability and controller design of networked control systems ncss with packet dropouts new ncs models are provided considering both single and multiple packet transmissions both sensor to controller s c and controller to actuator c a packet dropouts are modeled and their history behavior is described by different independent markov chains in term of the given models sufficient conditions for stochastic stability are derived in the form of linear matrix inequalities lmis and corresponding control laws are given numerical examples illustrate the effectiveness of the results

optimal estimation and control for lossy network stability convergence and performance in this paper we study the problems of optimal estimation and control i e the linear quadratic gaussian lqg control for systems with packet losses but without acknowledgment such acknowledgment is a signal sent by the actuator to inform the estimator of the incidence of control packet losses for such system which is usually called as a user datagram protocol udp like system the optimal estimation is nonlinear and its calculation is time consuming making its corresponding optimal lqg problem complicated we first propose two conditions 1 the sensor has some computation abilities and 2 the control command exerted to the plant is known to the sensor for a udp like system satisfying these two conditions we derive the optimal estimation by constructing the finite and infinite product probability measure spaces for the estimation error covariances eec we give the stability condition for the expected eec and show the existence of a measurable function to which the eec converges in distribution and propose some practical methods to evaluate the estimation performance finally the lqg controllers are derived and the conditions for the mean square stability of the closed loop system are established

a robust algorithm for random parameter tracking an algorithm for tracking time varying parameters is studied it is shown to be exponentially stable independently of the choice of algorithm parameters we derive some useful nonasymptotic estimates for the tracking error and develop the asymptotically optimal version of the method

stable adaptive control for a class of nonlinear systems using a modified lyapunov function investigates the adaptive control design for a class of nonlinear systems using lyapunov s stability theory the proposed method is developed based on a novel lyapunov function which removes the possible controller singularity problem in some of the existing adaptive control schemes using feedback linearization techniques the resulting closed loop system is proven to be globally stable and the output tracking error converges to an adjustable neighborhood of zero

on projection algorithms for solving convex feasibility problems due to their extraordinary utility and broad applicability in many areas of classical mathematics and modern physical sciences most notably computerized tomography algorithms for solving convex feasibility problems continue to receive great attention to unify generalize and review some of these algorithms a very broad and flexible framework is investigated several crucial new concepts which allow a systematic discussion of questions on behaviour in general hilbert spaces and on the quality of convergence are brought out numerous examples are given

persistency of excitation in possible unstable continuous time systems identification and indirect adaptive control the author proves persistency of excitation of the output of an output reachable possibly unstable linear system under certain input conditions and applies this result to adaptive identification and indirect adaptive control in adaptive identification he proves exponential parameter convergence regardless of stability of the identified plant in indirect adaptive control he proves exponential parameter convergence along with asymptotic time invariance and global experimental stability of the controlled closed loop system

brief stochastic optimal control and analysis of stability of networked control systems with long delay in this paper stochastic optimal controllers of networked control systems whose network induced delay is longer than a sampling period are designed for the two cases of the system with either full state information or partial state information the controllers are shown to render corresponding networked control systems exponentially mean square stable the optimal estimator of the system state is presented when the system has partial state information and network induced delay is longer than a sampling period the separation theorem is proved to still hold in such networked control systems

on the persistency of excitation in radial basis function network identification of nonlinear systems considers radial basis function rbf network approximation of a multivariate nonlinear mapping as a linear parametric regression problem linear recursive identification algorithms applied to this problem are known to converge provided the regressor vector sequence has the persistency of excitation pe property the main contribution of this paper is formulation and proof of pe conditions on the input variables in the rbf network identification the regressor vector is a nonlinear function of these input variables according to the formulated condition the inputs provide pe if they belong to domains around the network node centers for a two input network with gaussian rbf that have typical width and are centered on a regular mesh these domains cover about 25 of the input domain volume the authors further generalize the proposed solution of the standard rbf network identification problem and study affine rbf network identification that is important for affine nonlinear system control for the affine rbf network the author formulates and proves a pe condition on both the system state parameters and control inputs

robust adaptive neural control for a class of perturbed strict feedback nonlinear systems this paper presents a robust adaptive neural control design for a class of perturbed strict feedback nonlinear system with both completely unknown virtual control coefficients and unknown nonlinearities the unknown nonlinearities comprise two types of nonlinear functions one naturally satisfies the triangularity condition and can be approximated by linearly parameterized neural networks while the other is assumed to be partially known and consists of parametric uncertainties and known bounding functions with the utilization of iterative lyapunov design and neural networks the proposed design procedure expands the class of nonlinear systems for which robust adaptive control approaches have been studied the design method does not require a priori knowledge of the signs of the unknown virtual control coefficients leakage terms are incorporated into the adaptive laws to prevent parameter drifts due to the inherent neural network approximation errors it is proved that the proposed robust adaptive scheme can guarantee the uniform ultimate boundedness of the closed loop system signals the control performance can be guaranteed by an appropriate choice of the design parameters simulation studies are included to illustrate the effectiveness of the proposed approach

adaptive neural control for a class of switched nonlinear systems in this paper adaptive neural control is presented for a class of switched nonlinear systems with switching jumps and uncertainties in both system models and switching signals condition on limiting variation of the lyapunov function is given for input to state asymptotic stability of switched systems with switching jumps the control objective is achieved uniformly with respect to a class of switching signals the coupled difficulties from the discrepancy between control gains and switching jumps are overcome by discontinuous adaptive neural control combined with the classical adaptive control smooth approximations of the discontinuous controls are then presented for a systematic design procedure

approximate optimal control of affine nonlinear continuous time systems using event sampled neurodynamic programming this paper presents an approximate optimal control of nonlinear continuous time systems in affine form by using the adaptive dynamic programming adp with event sampled state and input vectors the knowledge of the system dynamics is relaxed by using a neural network nn identifier with event sampled inputs the value function which becomes an approximate solution to the hamilton jacobi bellman equation is generated by using event sampled nn approximator subsequently the nn identifier and the approximated value function are utilized to obtain the optimal control policy both the identifier and value function approximator weights are tuned only at the event sampled instants leading to an aperiodic update scheme a novel adaptive event sampling condition is designed to determine the sampling instants such that the approximation accuracy and the stability are maintained a positive lower bound on the minimum inter sample time is guaranteed to avoid accumulation point and the dependence of inter sample time upon the nn weight estimates is analyzed a local ultimate boundedness of the resulting nonlinear impulsive dynamical closed loop system is shown finally a numerical example is utilized to evaluate the performance of the near optimal design the net result is the design of an event sampled adp based controller for nonlinear continuous time systems

computational adaptive optimal control for continuous time linear systems with completely unknown dynamics this paper presents a novel policy iteration approach for finding online adaptive optimal controllers for continuous time linear systems with completely unknown system dynamics the proposed approach employs the approximate adaptive dynamic programming technique to iteratively solve the algebraic riccati equation using the online information of state and input without requiring the a priori knowledge of the system matrices in addition all iterations can be conducted by using repeatedly the same state and input information on some fixed time intervals a practical online algorithm is developed in this paper and is applied to the controller design for a turbocharged diesel engine with exhaust gas recirculation finally several aspects of future work are discussed

a novel robust adaptive fuzzy tracking control for a class of nonlinearmulti input multi output systems robust adaptive fuzzy tracking control of a class of uncertain multi input multi output nonlinear systems with coupled interconnections is considered in this paper takagi sugeno t s fuzzy systems are used to approximate the unknown system functions a novel adaptive control scheme is developed on the basis of the so called dynamic surface control and minimal learning parameters techniques the proposed scheme has following two key features first the number of parameters updated online for each subsystem is reduced to one and both problems of curse of dimension for high dimensional systems and explosion of complexity inherent in the conventional backstepping methods are circumvented second the potential controller singularity problem in some of the existing adaptive control schemes with feedback linearization techniques is overcome it is shown via lyapunov theory that all the signals in the closed loop system are semiglobally uniformly ultimately bounded finally simulation results via two examples are presented to demonstrate the effectiveness and advantages of the proposed scheme

sufficiency of excitation in the context of adaptive identification and adaptive control of linear systems the concept of a sufficiently exciting input play a key role in this note necessary and sufficient conditions are presented for an input function of a single input linear stationary system to be sufficiently exciting these conditions are independent of the stability properties of the linear system

cooperative convex optimization in networked systems augmented lagrangian algorithms with directed gossip communication we study distributed optimization in networked systems where nodes cooperate to find the optimal quantity of common interest x x the objective function of the corresponding optimization problem is the sum of private known only by a node convex nodes objectives and each node imposes a private convex constraint on the allowed values of x we solve this problem for generic connected network topologies with asymmetric random link failures with a novel distributed de centralized algorithm we refer to this algorithm as al g augmented lagrangian gossiping and to its variants as al mg augmented lagrangian multi neighbor gossiping and al bg augmented lagrangian broadcast gossiping the al g algorithm is based on the augmented lagrangian dual function dual variables are updated by the standard method of multipliers at a slow time scale to update the primal variables we propose a novel gauss seidel type randomized algorithm at a fast time scale al g uses unidirectional gossip communication only between immediate neighbors in the network and is resilient to random link failures for networks with reliable communication i e no failures the simplified al bg augmented lagrangian broadcast gossiping algorithm reduces communication computation and data storage cost we prove convergence for all proposed algorithms and demonstrate by simulations the effectiveness on two applications l1 regularized logistic regression for classification and cooperative spectrum sensing for cognitive radio networks

approximate projected consensus for convex intersection computation convergence analysis and critical error angle in this paper we study an approximate projected consensus algorithm for a network to cooperatively compute the intersection of convex sets where each set corresponds to one network node instead of assuming exact convex projection that each node can compute we allow each node to compute an approximate projection with respect to its own set after receiving the approximate projection information nodes update their states by weighted averaging with the neighbors over a directed and time varying communication graph the approximate projections are related to projection angle errors which introduces state dependent disturbance in the iterative algorithm projection accuracy conditions are presented for the considered algorithm to converge the results indicate how much projection accuracy is required to ensure global consensus to a point in the intersection set when the communication graph is uniformly jointly strongly connected in addition we show that pi 4 is a critical angle for the error of the projection approximation to ensure the boundedness finally the results are illustrated by simulations

distributed gradient algorithm for constrained optimization with application to load sharing in power systems abstract in this paper a distributed constrained optimization problem is discussed to achieve the optimal point of the sum of agents local objective functions while satisfying local constraints here neither the local objective function nor local constraint functions of each agent can be shared with other agents to solve the problem a novel distributed continuous time algorithm is proposed by using the kkt condition combined with the lagrangian multiplier method and the convergence is proved with the help of lyapunov functions and an invariance principle for hybrid systems furthermore this distributed algorithm is applied to optimal load sharing control problem in power systems both theoretical and numerical results show that the optimal load sharing can be achieved within both generation and delivering constraints in a distributed way

nonasymptotic results for finite memory wls filters a nonasymptotic analysis of properties of weighted least squares wls adaptive filters used for identification of time varying systems is presented it is shown that the problem of mean square boundedness of wls estimates is closely related to the problem of invertibility in the mean sense of the corresponding regression matrix necessary and sufficient conditions are discussed for such invertibility to hold based on that a number of results are derived paralleling those already obtained for least mean square lms filters and the problem of statistical robustness of the wls estimator is briefly discussed

distributed continuous time approximate projection protocols for shortest distance optimization problems in this paper we investigate a distributed shortest distance optimization problem for a multi agent network to cooperatively minimize the sum of the quadratic distances from some convex sets where each set is only associated with one agent to deal with this optimization problem with projection uncertainties we propose a distributed continuous time dynamical protocol where each agent can only obtain an approximate projection and communicate with its neighbors over a time varying communication graph first we show that no matter how large the approximate angle is system states are always bounded for any initial condition and uniformly bounded with respect to all initial conditions if the inferior limit of the stepsize is greater than zero then in both cases of nonempty and empty intersection of convex sets we provide stepsize and approximate angle conditions to ensure the optimal convergence respectively moreover we also give some characterizations about the optimal solutions for the empty intersection case

stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses in this paper the stochastic optimal control of linear networked control system ncs with uncertain system dynamics and in the presence of network imperfections such as random delays and packet losses is derived the proposed stochastic optimal control method uses an adaptive estimator ae and ideas from q learning to solve the infinite horizon optimal regulation of unknown ncs with time varying system matrices next a stochastic suboptimal control scheme which uses ae and q learning is introduced for the regulation of unknown linear time invariant ncs that is derived using certainty equivalence property update laws for online tuning the unknown parameters of the ae to obtain the q function are derived lyapunov theory is used to show that all signals are asymptotically stable as and that the estimated control signals converge to optimal or suboptimal control inputs simulation results are included to show the effectiveness of the proposed schemes the result is an optimal control scheme that operates forward in time manner for unknown linear systems in contrast with standard riccati equation based schemes which function backward in time

stability condition for sampled data based control of linear continuous switched systems abstract most practical systems are continuous in nature but with discrete sampled feedback when digital control is utilized this paper investigates the stabilization problem of switched linear continuous time systems with sampled data based control for both known and unknown arbitrary switching processes on the basis of lyapunov stability theory a sufficient global exponential stability condition related to dwell time and sampling period is established for the latter case on line one step adaptive estimation algorithm is derived and integrated with sampled feedback for control design validation and verification of the established result are conducted through cruise control of train systems

randomized optimal consensus of multi agent systems in this paper we formulate and solve a randomized optimal consensus problem for multi agent systems with stochastically time varying interconnection topology the considered multi agent system with a simple randomized iterating rule achieves an almost sure consensus meanwhile solving the optimization problem min z r email protected i 1 nf i z in which the optimal solution set of objective function f i can only be observed by agent i itself at each time step simply determined by a bernoulli trial each agent independently and randomly chooses either taking an average among its neighbor set or projecting onto the optimal solution set of its own optimization component both directed and bidirectional communication graphs are studied connectivity conditions are proposed to guarantee an optimal consensus almost surely with proper convexity and intersection assumptions the convergence analysis is carried out using convex analysis we compare the randomized algorithm with the deterministic one via a numerical example the results illustrate that a group of autonomous agents can reach an optimal opinion by each node simply making a randomized trade off between following its neighbors or sticking to its own opinion at each time step

distributed optimization for linear multiagent systems edge and node based adaptive designs this paper studies the distributed optimization problem for continuous time multiagent systems with general linear dynamics the objective is to cooperatively optimize a team performance function formed by a sum of convex local objective functions each agent utilizes only local interaction and the gradient of its own local objective function to achieve the cooperative goal a couple of fully distributed optimal algorithms are designed first an edge based adaptive algorithm is developed for linear multiagent systems with a class of convex local objective functions then a node based adaptive algorithm is constructed to solve the distributed optimization problem for a class of agents satisfying the bounded input bounded state stable property sufficient conditions are given to ensure that all agents reach a consensus while minimizing the team performance function finally numerical examples are provided to illustrate the theoretical results

on parameter convergence in adaptive control it is well known that the parameter error as well as the model plant mismatch error in a model reference adaptive scheme tends exponentially to zero iff a certain sufficient richness condition holds for signals inside the time varying plant control loop in this paper we give conditions on the reference signal the exogenous input to the adaptive loop namely that it have as many spectral lines as there are unknown parameters in order to guarantee parameter convergence

the adiabatic limit of schrodinger operators on fibre bundles we consider schrodinger operators h delta g varepsilon v on a fibre bundle m mathop rightarrow limits pi b with compact fibres and a metric g varepsilon that blows up directions perpendicular to the fibres by a factor varepsilon 1 gg 1 we show that for an eigenvalue lambda of the fibre wise part of h satisfying a local gap condition and every n in mathbb n there exists a subspace of l 2 m that is invariant under h up to errors of order varepsilon n 1 the dynamical and spectral features of h on this subspace can be described by an effective operator on the fibre wise lambda eigenspace bundle mathcal e rightarrow b giving detailed asymptotics for h

on the discreteness of the spectrum of the laplacian on noncompact riemannian manifolds necessary and sufficient conditions for the discreteness of the laplacian on a noncompact riemannian manifold m are established in terms of the isocapacitary function of m the relevant capacity takes a different form according to whether m has finite or infinite volume conditions involving the more standard isoperimetric function of m can also be derived but they are only sufficient in general as we demonstrate by concrete examples

recursive identification of overparametrized systems a recursive identification algorithm based on extended least squares is proposed to deal with the contingency of overparametrization the algorithm is relatively simple compared to those involving online order determination being based on adaptively introducing suitable excitation into the algorithm to avoid ill conditioning in the case of extended least squares based adaptive estimation then the regressors are appropriately stochastically perturbed the algorithm is shown to converge to a uniquely defined signal model with any pole zero cancellations at the origin

a robust event triggered approach for fast sampled data extremization and learning this paper presents a general framework for the analysis and design of a class of model free robust and efficient sampled data based algorithms for extremization and learning in continuous time nonlinear systems that generate response maps with an optimal operational set in particular we consider plants described by differential inclusions interconnected in a sampled data setting with a robust learning algorithm characterized by a constrained difference inclusion in contrast to standard sampled data based approaches where the learning dynamics are updated after a fixed sufficiently long sampling time has passed we design a robust dynamic event based mechanism that triggers the control action as soon as the rate of change of the output of the plant is sufficiently small by using this event based update rule a significant improvement in the convergence time of the closed loop system can be achieved using the framework of set valued hybrid dynamical systems we establish for the closed loop system the existence of a uniformly asymptotically stable compact set which by an appropriate tuning of the control parameters can be made arbitrarily close to the optimal operational set our results generalize existing results for periodic sampled data extremum seeking and can be used to solve model free multivariable smooth nonsmooth constrained optimization problems as well as learning problems in game theoretical scenarios

connectivity and set tracking of multi agent systems guided by multiple moving leaders in this paper we investigate distributed multi agent tracking of a convex set specified by multiple moving leaders with unmeasurable velocities various jointly connected interaction topologies of the follower agents with uncertainties are considered in the study of set tracking based on the connectivity of the time varying multi agent system necessary and sufficient conditions are obtained for set input to state stability and set integral input to state stability for a nonlinear neighbor based coordination rule with switching directed topologies conditions for asymptotic set tracking are also proposed with respect to the polytope spanned by the leaders

brief paper target containment control of multi agent systems with random switching interconnection topologies in this paper the distributed containment control is considered for a second order multi agent system guided by multiple leaders with random switching topologies the multi leader control problem is investigated via a combination of convex analysis and stochastic process the interaction topology between agents is described by a continuous time irreducible markov chain a necessary and sufficient condition is obtained to make all the mobile agents almost surely asymptotically converge to the static convex leader set moreover conditions on the tracking estimation are provided for the convex target set determined by moving multiple leaders

convergence rates of continuous time stochastic els parameter estimation discrete time convergence rates for extended least squares els algorithms are generalized to the continuous time case an essential difference in the estimation is the appropriate prefiltering while in the theory the existence of solutions of the stochastic equations is a concern

technical communique on event design in event triggered feedback systems this paper studies event design in event triggered feedback systems a novel event triggering scheme is presented to ensure exponential stability of the resulting sampled data system the scheme postpones the triggering of events over previously proposed methods and therefore enlarges the intersampling period the resulting intersampling periods and deadlines are bounded strictly away from zero when the continuous time system is input to state stable with respect to measurement errors

observer design for a class of nonlinear systems a viable design methodology to construct observers for a class of nonlinear systems is developed the proposed technique is based on the off line solution of a riccati equation and can be solved using commercially available software packages for globally valid results the class of systems considered is characterized by globally lipschitz nonlinearities local results relax this assumption to only a local requirement for a more general description of nonlinear systems the methodology yields approximate observers locally the proposed theory is used to design an observer for a single link flexible joint robot this observer estimates the robot link variables based on the joint measurements

persistence of excitation in linear systems abstract this paper develops output reachability characterizations of linear finite dimensional multivariate systems so as to translate excitation properties of system inputs to excitation properties of system outputs states or associated regression vectors such properties are of fundamental concern for convergence of algorithms involving on line identification adaptive state estimation prediction and control persistence of excitation guarantees convergence without a priori stability assumptions and ensures robustness properties

neural network based finite horizon optimal control of uncertain affine nonlinear discrete time systems in this paper the finite horizon optimal control design for nonlinear discrete time systems in affine form is presented in contrast with the traditional approximate dynamic programming methodology which requires at least partial knowledge of the system dynamics in this paper the complete system dynamics are relaxed utilizing a neural network nn based identifier to learn the control coefficient matrix the identifier is then used together with the actor critic based scheme to learn the time varying solution referred to as the value function of the hamilton jacobi bellman hjb equation in an online and forward in time manner since the solution of hjb is time varying nns with constant weights and time varying activation functions are considered to properly satisfy the terminal constraint an additional error term is incorporated in the novel update law such that the terminal constraint error is also minimized over time policy and or value iterations are not needed and the nn weights are updated once a sampling instant the uniform ultimate boundedness of the closed loop system is verified by standard lyapunov stability theory under nonautonomous analysis numerical examples are provided to illustrate the effectiveness of the proposed method

neural network based near optimal control for a class of discrete time affine nonlinear systems with control constraints in this paper the near optimal control problem for a class of nonlinear discrete time systems with control constraints is solved by iterative adaptive dynamic programming algorithm first a novel nonquadratic performance functional is introduced to overcome the control constraints and then an iterative adaptive dynamic programming algorithm is developed to solve the optimal feedback control problem of the original constrained system with convergence analysis in the present control scheme there are three neural networks used as parametric structures for facilitating the implementation of the iterative algorithm two examples are given to demonstrate the convergence and feasibility of the proposed optimal control scheme

constrained consensus and optimization in multi agent networks we present distributed algorithms that can be used by multiple agents to align their estimates with a particular value over a network with time varying connectivity our framework is general in that this value can represent a consensus value among multiple agents or an optimal solution of an optimization problem where the global objective function is a combination of local agent objective functions our main focus is on constrained problems where the estimates of each agent are restricted to lie in different convex sets to highlight the effects of constraints we first consider a constrained consensus problem and present a distributed projected consensus algorithm in which agents combine their local averaging operation with projection on their individual constraint sets this algorithm can be viewed as a version of an alternating projection method with weights that are varying over time and across agents we establish convergence and convergence rate results for the projected consensus algorithm we next study a constrained optimization problem for optimizing the sum of local objective functions of the agents subject to the intersection of their local constraint sets we present a distributed projected subgradient algorithm which involves each agent performing a local averaging operation taking a subgradient step to minimize its own objective function and projecting on its constraint set we show that with an appropriately selected stepsize rule the agent estimates generated by this algorithm converge to the same optimal solution for the cases when the weights are constant and equal and when the weights are time varying but all agents have the same constraint set

approximate representation of the pareto frontier in multiparty negotiations decentralized methods and privacy preservation abstract multiparty negotiations have drawn much research attention in recent years and an important problem is how to find a pareto optimal solution or the entire pareto frontier in a decentralized way privacy preservation is also important in negotiation analysis the main aim of this paper is to find an approximate representation of the pareto frontier in a decentralized manner and meanwhile all parties privacy can be effectively protected in this paper we propose a decentralized discrete time algorithm based on a weight sum method and the well known subgradient optimization algorithm where a mediator works as a coordinator to help negotiators the proposed algorithm is easily executable and it only requires the mediator to compute a weighted average of the noisy estimates received from negotiators and negotiators to follow a subgradient optimization iteration at this weighted average the proposed algorithm can generate an approximate pareto optimal solution for one particular weight vector and an approximate representation of the pareto frontier by varying appropriately weight vectors the approximation error between the obtained approximate representation and the pareto frontier can be controlled by the number of iterations and the step size moreover it also reveals that the proposed algorithm is privacy preserving as a result of the random disturbance technique and the weighted average scheme used in this algorithm

transfer function identification with filtering techniques this paper addresses the problem of identifying the transfer function of a plant with the aid of filtering techniques conditions for obtaining fair estimates of a transfer function are derived it is shown that to be able to reduce the effect of noise and other stochastic effects highly tuned filters are required it is also found that proper excitation signals are necessary for effective implementation of the proposed scheme some methods for designing the required highly tuned filters are proposed the relationship between the proposed scheme and existing methods is discussed it is shown that for some specific cases the proposed scheme is in effect a computationally efficient implementation of existing methods however for some other cases the proposed scheme gives more accurate results the theoretical predictions are substantiated with computer simulations

estimating time varying parameters by the kalman filter based algorithm stability and convergence convergence and stability properties of the kalman filter based parameter estimator are established for linear stochastic time varying regression models the main features are both the variances and sample path averages of the parameter tracking error are shown to be bounded the regression vector includes both stochastic and deterministic signals and no assumptions of stationarity or independence are requires and the unknown parameters are only assumed to have bounded variations in an average sense

a randomized incremental subgradient method for distributed optimization in networked systems we present an algorithm that generalizes the randomized incremental subgradient method with fixed stepsize due to nedic and bertsekas siam j optim 12 2001 pp 109 138 our novel algorithm is particularly suitable for distributed implementation and execution and possible applications include distributed optimization e g parameter estimation in networks of tiny wireless sensors the stochastic component in the algorithm is described by a markov chain which can be constructed in a distributed fashion using only local information we provide a detailed convergence analysis of the proposed algorithm and compare it with existing both deterministic and randomized incremental subgradient methods

adaptive neural control for a class of nonlinearly parametric time delay systems in this paper an adaptive neural controller for a class of time delay nonlinear systems with unknown nonlinearities is proposed based on a wavelet neural network wnn online approximation model a state feedback adaptive controller is obtained by constructing a novel integral type lyapunov krasovskii functional which also efficiently overcomes the controller singularity problem it is shown that the proposed method guarantees the semiglobal boundedness of all signals in the adaptive closed loop systems an example is provided to illustrate the application of the approach

exponential convergence of adaptive identification and control algorithms output and equation error adaptive identification algorithms are shown to be exponentially convergent under a deterministic or stochastic persistently exciting or spanning condition on the system inputs together with several other standard conditions an adaptive control algorithm is shown to be exponentially convergent under a deterministic or stochastic persistently exciting condition on the reference trajectory together with some standard conditions

gossip algorithms for convex consensus optimization over networks in many applications nodes in a network desire not only a consensus but an optimal one to date a family of subgradient algorithms have been proposed to solve this problem under general convexity assumptions this technical note shows that for the scalar case and by assuming a bit more novel non gradient based algorithms with appealing features can be constructed specifically we develop pairwise equalizing pe and pairwise bisectioning pb two gossip algorithms that solve unconstrained separable convex consensus optimization problems over undirected networks with time varying topologies where each local function is strictly convex continuously differentiable and has a minimizer we show that pe and pb are easy to implement bypass limitations of the subgradient algorithms and produce switched nonlinear networked dynamical systems that admit a common lyapunov function and asymptotically converge moreover pe generalizes the well known pairwise averaging and randomized gossip algorithm while pb relaxes a requirement of pe allowing nodes to never share their local functions

on hold or drop out of order packets in networked control systems holding or dropping out of order packets in the communications are two typical signal choosing schemes in networked control systems however neither of the two schemes can be claimed always superior to the other to improve the control performance a smart signal choosing scheme is proposed to determine whether to use the out of order packets it is theoretically proved that improved performance can be obtained based on the proposed smart signal choosing scheme two numerical examples are given to demonstrate the effectiveness of the proposed approach

globally convergent adaptive iir filters based on fixed pole locations a new class of adaptive filters dubbed fixed pole adaptive filters fpaf s is introduced these adaptive filters have infinite impulse responses yet their adaptation exhibits provable global convergence good filter performance with a relatively small number of adapted parameters is permitted by the new filter structure thus reducing the computational burden needed to implement adaptive filters the implementation and computational complexity of the fpaf is described and its modeling capabilities are determined excitation conditions on the filter input are established that guarantee global convergence of a standard set of adaptive algorithms some methods are described for selecting the fixed pole locations based on a priori information regarding the operating environment of the adaptive filter the fpaf is tailored to applications by such a procedure enabling improved performance in examples the fpaf is shown to achieve a smaller minimum mean square error given an equal number of adapted parameters in comparison with adaptive fir filters and adaptive filters based on laguerre and kautz models

central tendency adaptive pole assignment the concept of central tendency adaptive control for achieving minimum variance objectives proposed by the authors in a companion paper 1989 is applied to adaptive pole assignment at any iteration given plant parameter estimate and their uncertainty a controller is designed which is most likely to achieve the pole assignment objectives simulations show a factor of 100 improvement in transient response over certainty equivalent adaptive pole assignment schemes for one example

central tendency minimum variance adaptive control the authors propose designing indirect adaptive controllers using measures of central tendency of the a posteriori probability function of the control for parameters that is given a knowledge of plant uncertainty at each time instant from estimation algorithms and a controller design rule controller parameters are sought which maximize the likelihood of achieving the control objectives the particular case of adaptive minimum variance control is studied in detail

feedback linearization using neural networks abstract for a class of single input single output continuous time nonlinear systems a multilayer neural network based controller that feedback linearizes the system is presented control action is used to achieve tracking performance for a state feedback linearizable but unknown nonlinear system the control structure consists of a feedback linearization portion provided by two neural networks plus a robustifying portion that keeps the control magniture bounded a stability proof is given in the sense of lyapunov it is shown that all the signals in the closed loop system are uniformly ultimately bounded no off line learning phase is needed initialization of the network weights is straightforward

asymptotically optimal parameter estimation with scheduled measurements to reduce the communication cost of a sensor node this paper is concerned with an estimation framework with scheduled measurements for a linear system a scheduler is designed to control the transmission of measurements from sensor to estimator which results in that only a subset of measurements is transmitted to the estimator we propose an innovation based scheduler and derive an analytical expression for the cramer rao lower bound crlb for the given scheduling strategy under a communication constraint an adaptive scheduler and a corresponding recursive estimator are jointly designed to asymptotically attain the crlb the structure of the estimator bears close resemblance to the standard least square estimator lse with the full set of sensor measurements moreover we prove that the estimation performance in terms of mean square estimation error is comparable to the standard lse even under a moderate communication cost the theoretical results are verified by simulations

on the local stability properties of adaptive parameter estimators with composite errors and split algorithms the stability characteristics of a generalized error system structure are examined for adaptive parameter estimation systems this error system form encompasses the structure of a number of particular applications of adaptive parameter estimation theory which fall outside the framework of more familiar error system models the consequences for error system stability which derive from the added generality are analyzed it is demonstrated that when using averaging theory techniques to analyze the error system stability conditions must be augmented for the basic error system in order to ensure stability for the generalized error system the inadequacy of regressor spectral restrictions and persistent spanning conditions to guarantee local error system stability of the generalized structure is rigorously established alternative conditions which will yield such local stability are demonstrated to illustrate the concepts involved the recursive identification of parameters is examined in a parallel form realization of a linear system

a universality advantage of stochastic excitation signals for adaptive control abstract adaptive control schemes usually depend on estimation of system parameters which in turn depends excitation of modes associated with these parameters such excitation is supplied by plant noise the adaptive control signal itself and any external excitation such as a reference signal in this note it is shown that there is a universality advantage for any externally applied signal to be stochastic rather than deterministic the crucial property of a stochastic signal exploited in this note is its unpredictability by any causal system such as an adaptive control scheme when such unpredictable signals excite an adaptive control scheme there is no need to deliberately constrain the adaptation to be slow or excitation maintaining to ensure adequate identification

persistence of excitation in extended least squares in least squares parameter estimation schemes persistency of excitation conditions on the plant states are required for consistent estimation in the case of extended least squares the persistency conditions are on the state estimates here these persistency of excitation conditions are translated into sufficiently rich conditions on the plant noise and inputs in the case of adaptive minimum variance control schemes the sufficiently rich conditions are on the noise and specified output trajectory with sufficiently rich input signals guaranteed convergence rates of prediction errors improve and it is conjectured that the algorithms are consequently more robust

adaptive control of time varying linear systems the authors present a global bounded input bounded state stability theory for a class of continuously adapting controllers applied to time varying linear systems this gives theoretical support to the application of many of the algorithms used in practice a key feature of the analysis is that no persistence of excitation requirement is needed

nash equilibrium computation in subnetwork zero sum games with switching communications in this paper we investigate a distributed nash equilibrium computation problem for a time varying multi agent network consisting of two subnetworks where the two subnetworks share the same objective function we first propose a subgradient based distributed algorithm with heterogeneous stepsizes to compute a nash equilibrium of a zero sum game we then prove that the proposed algorithm can achieve a nash equilibrium under uniformly jointly strongly connected ujsc weight balanced digraphs with homogenous stepsizes moreover we demonstrate that for weighted unbalanced graphs a nash equilibrium may not be achieved with homogenous stepsizes unless certain conditions on the objective function hold we show that there always exist heterogeneous stepsizes for the proposed algorithm to guarantee that a nash equilibrium can be achieved for ujsc digraphs finally in two standard weight unbalanced cases we verify the convergence to a nash equilibrium by adaptively updating the stepsizes along with the arc weights in the proposed algorithm

consensus seeking in multiagent systems under dynamically changing interaction topologies this note considers the problem of information consensus among multiple agents in the presence of limited and unreliable information exchange with dynamically changing interaction topologies both discrete and continuous update schemes are proposed for information consensus this note shows that information consensus under dynamically changing interaction topologies can be achieved asymptotically if the union of the directed interaction graphs have a spanning tree frequently enough as the system evolves

reaching a consensus in a dynamically changing environment a graphical approach this paper presents new graph theoretic results appropriate for the analysis of a variety of consensus problems cast in dynamically changing environments the concepts of rooted strongly rooted and neighbor shared are defined and conditions are derived for compositions of sequences of directed graphs to be of these types the graph of a stochastic matrix is defined and it is shown that under certain conditions the graph of a sarymsakov matrix and a rooted graph are one and the same as an illustration of the use of the concepts developed in this paper graph theoretic conditions are obtained which address the convergence question for the leaderless version of the widely studied vicsek consensus problem

on schrodinger operators with discrete spectrum on etudie le caractere discret des equations de schrodinger a valeur operateur on considere l operateur differentiel x 2 q avec un potentiel localement integrable q agissant sur c 0 r r 0

a lyapunov proof of an improved maximum allowable transfer interval for networked control systems simple lyapunov proofs are given for an improved relative to previous results that have appeared in the literature bound on the maximum allowable transfer interval to guarantee global asymptotic or exponential stability in networked control systems and also for semiglobal practical asymptotic stability with respect to the length of the maximum allowable transfer interval

value iteration and adaptive dynamic programming for data driven adaptive optimal control design this paper presents a novel non model based data driven adaptive optimal controller design for linear continuous time systems with completely unknown dynamics inspired by the stochastic approximation theory a continuous time version of the traditional value iteration vi algorithm is presented with rigorous convergence analysis this vi method is crucial for developing new adaptive dynamic programming methods to solve the adaptive optimal control problem and the stochastic robust optimal control problem for linear continuous time systems fundamentally different from existing results the a priori knowledge of an initial admissible control policy is no longer required the efficacy of the proposed methodology is illustrated by two examples and a brief comparative study between vi and earlier policy iteration methods

adaptive neural network based event triggered control of single input single output nonlinear discrete time systems this paper presents a novel adaptive neural network nn control of single input and single output uncertain nonlinear discrete time systems under event sampled nn inputs in this control scheme the feedback signals are transmitted and the nn weights are tuned in an aperiodic manner at the event sampled instants after reviewing the nn approximation property with event sampled inputs an adaptive state estimator se consisting of linearly parameterized nns is utilized to approximate the unknown system dynamics in an event sampled context the se is viewed as a model and its approximated dynamics and the state vector during any two events are utilized for the event triggered controller design an adaptive event trigger condition is derived by using both the estimated nn weights and a dead zone operator to determine the event sampling instants this condition both facilitates the nn approximation and reduces the transmission of feedback signals the ultimate boundedness of both the nn weight estimation error and the system state vector is demonstrated through the lyapunov approach as expected during an initial online learning phase events are observed more frequently over time with the convergence of the nn weights the inter event times increase thereby lowering the number of triggered events these claims are illustrated through the simulation results

distributed optimal coordination for multiple heterogeneous euler lagrangian systems abstract in this paper we consider the distributed optimal coordination doc problem for multi agent systems with the agents in the form of euler lagrangian el dynamics we propose two different distributed protocols for the heterogeneous continuous time el agents to achieve the optimization task by constructing suitable lyapunov functions we prove the global convergence to the optimal coordination of the el systems in the case with parametric uncertainties and the global exponential convergence in the case without parametric uncertainties furthermore we estimate the regret bound for an uncertain doc problem with time varying cost functions and inexact gradients finally we provide a numerical example to validate the theoretical results

lq optimal sampled data control under stochastic delays gridding approach for stabilizability and detectability we solve a linear quadratic optimal control problem for sampled data systems with stochastic delays the delays are stochastically determined by the last few delays the proposed optimal controller

q learning for continuous time linear systems a model free infinite horizon optimal control approach abstract in this paper we propose an online q learning algorithm to solve the infinite horizon optimal control problem of a linear time invariant system with completely uncertain unknown dynamics we first formulate the q function by using the hamiltonian and the optimal cost an integral reinforcement learning approach is used to develop an actor critic approximator structure to estimate the parameters of the q function online while also guaranteeing closed loop asymptotic stability and convergence to the optimal solution

reaching an optimal consensus dynamical systems that compute intersections of convex sets in this paper multi agent systems minimizing a sum of objective functions where each component is only known to a particular node is considered for continuous time dynamics with time varying interconnection topologies assuming that each node can observe a convex solution set of its optimization component and the intersection of all such sets is nonempty the considered optimization problem is converted to an intersection computation problem by a simple distributed control rule the considered multi agent system with continuous time dynamics achieves not only a consensus but also an optimal agreement within the optimal solution set of the overall optimization objective directed and bidirectional communications are studied respectively and connectivity conditions are given to ensure a global optimal consensus in this way the corresponding intersection computation problem is solved by the proposed decentralized continuous time algorithm we establish several important properties of the distance functions with respect to the global optimal solution set and a class of invariant sets with the help of convex and non smooth analysis

consensus control for linear systems with optimal energy cost in this paper we design an optimal energy cost controller for linear systems asymptotic consensus given the topology of the graph the controller depends only on relative information of the agents

discreteness conditions for the laplacian on complete non compact riemannian manifolds soit m une variete de riemann connexe complete lisse de dimension d sans bord et soit le laplacien non negatif associe on donne une condition necessaire et suffisante pour que le spectre de ne contienne que des valeurs propres isolees de multiplicite finie

stochastic strongly convex optimization via distributed epoch stochastic gradient algorithm this article considers the problem of stochastic strongly convex optimization over a network of multiple interacting nodes the optimization is under a global inequality constraint and the restriction that nodes have only access to the stochastic gradients of their objective functions we propose an efficient distributed non primal dual algorithm by incorporating the inequality constraint into the objective via a smoothing technique we show that the proposed algorithm achieves an optimal o 1 t t is the total number of iterations convergence rate in the mean square distance from the optimal solution in particular we establish a high probability bound for the proposed algorithm by showing that with a probability at least 1 the proposed algorithm converges at a rate of o ln ln t t finally we provide numerical experiments to demonstrate the efficacy of the proposed algorithm

policy iteration adaptive dynamic programming algorithm for discrete time nonlinear systems this paper is concerned with a new discrete time policy iteration adaptive dynamic programming adp method for solving the infinite horizon optimal control problem of nonlinear systems the idea is to use an iterative adp technique to obtain the iterative control law which optimizes the iterative performance index function the main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete time nonlinear systems for the first time it shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the hamilton jacobi bellman equation it is also proven that any of the iterative control laws can stabilize the nonlinear systems neural networks are used to approximate the performance index function and compute the optimal control law respectively for facilitating the implementation of the iterative adp algorithm where the convergence of the weight matrices is analyzed finally the numerical results and analysis are presented to illustrate the performance of the developed method