optimal tracking control of nonlinear partially unknown constrained input systems using integral reinforcement learning in this paper a new formulation for the optimal tracking control problem otcp of continuous time nonlinear systems is presented this formulation extends the integral reinforcement learning irl technique a method for solving optimal regulation problems to learn the solution to the otcp unlike existing solutions to the otcp the proposed method does not need to have or to identify knowledge of the system drift dynamics and it also takes into account the input constraints a priori an augmented system composed of the error system dynamics and the command generator dynamics is used to introduce a new nonquadratic discounted performance function for the otcp this encodes the input constrains into the optimization problem a tracking hamilton jacobi bellman hjb equation associated with this nonquadratic performance function is derived which gives the optimal control solution an online irl algorithm is presented to learn the solution to the tracking hjb equation without knowing the system drift dynamics convergence to a near optimal control solution and stability of the whole system are shown under a persistence of excitation condition simulation examples are provided to show the effectiveness of the proposed method

a sparse collocation method for solving time dependent hjb equations using multivariate b splines this paper presents a sparse collocation method for solving the time dependent hamilton jacobi bellman hjb equation associated with the continuous time optimal control problem on a fixed finite time horizon with integral cost functional through casting the problem in a recursive framework using the value iteration procedure the value functions of every iteration step is approximated with a time varying multivariate simplex b spline on a certain state domain of interest in the collocation scheme the time dependent coefficients of the spline function are further approximated with ordinary univariate b splines to yield a discretization for the value function fully in terms of piece wise polynomials the b spline coefficients are determined by solving a sequence of highly sparse quadratic programming problems the proposed algorithm is demonstrated on a pair of benchmark example problems simulation results indicate that the method can yield increasingly more accurate approximations of the value function by refinement of the triangulation

adaptive dynamic programming for a class of complex valued nonlinear systems in this brief an optimal control scheme based on adaptive dynamic programming adp is developed to solve infinite horizon optimal control problems of continuous time complex valued nonlinear systems a new performance index function is established on the basis of complex valued state and control using system transformations the complex valued system is transformed into a real valued one which overcomes cauchy riemann conditions effectively with the transformed system and the performance index function a new adp method is developed to obtain the optimal control law by using neural networks a compensation controller is developed to compensate the approximation errors of neural networks stability properties of the nonlinear system are analyzed and convergence properties of the weights for neural networks are presented finally simulation results demonstrate the performance of the developed optimal control scheme for complex valued nonlinear systems

sharp bounds on treatment effects in a binary triangular system this paper considers the evaluation of the average treatment effect ate in a triangular system with binary dependent variables i impose a threshold crossing model on both the endogenous regressor and the outcome the bounds proposed by shaikh and vytlacil 2011 sv on the ate are sharp only under a restrictive condition on the support of the covariates and the instruments which rules out a wide range of models and many relevant applications in this setting i provide a methodology that allows the construction of sharp bounds on the ate by efficiently using the variation of covariates without imposing support restrictions

brief paper adaptive optimal control for continuous time linear systems based on policy iteration in this paper we propose a new scheme based on adaptive critics for finding online the state feedback infinite horizon optimal control solution of linear continuous time systems using only partial knowledge regarding the system dynamics in other words the algorithm solves online an algebraic riccati equation without knowing the internal dynamics model of the system being based on a policy iteration technique the algorithm alternates between the policy evaluation and policy update steps until an update of the control policy will no longer improve the system performance the result is a direct adaptive control algorithm which converges to the optimal control solution without using an explicit a priori obtained model of the system internal dynamics the effectiveness of the algorithm is shown while finding the optimal load frequency controller for a power system

identification and estimation of triangular simultaneous equations models without additivity this paper uses control variables to identify and estimate models with nonseparable multidimensional disturbances triangular simultaneous equations models are considered with instruments and disturbances that are independent and a reduced form that is strictly monotonic in a scalar disturbance here it is shown that the conditional cumulative distribution function of the endogenous variable given the instruments is a control variable also for any control variable identification results are given for quantile average and policy effects bounds are given when a common support assumption is not satisfied estimators of identified objects and bounds are provided and a demand analysis empirical example is given

local structural quantile effects in a model with a nonseparable control variable abstract i consider a semiparametric version of the nonseparable triangular model of chesher chesher a 2003 identification in nonseparable models econometrica 71 1405 1441 the proposed model is linear in coefficients where the coefficients are unknown functions of unobserved latent variables using a control variable idea and quantile regression methods i propose a simple two step estimator for the coefficients evaluated at particular values of the latent variables under the condition that the instruments are locally relevant i e they affect a particular conditional quantile of interest of the endogenous variable i establish consistency and asymptotic normality simulation experiments confirm the theoretical results

optimal approximation schedules for a class of iterative algorithms with an application to multigrid value iteration many iterative algorithms employ operators which are difficult to evaluate exactly but for which a graduated range of approximations exist in such cases coarse to fine algorithms are often used in which a crude initial operator approximation is gradually refined with new iterations in such algorithms because the computational complexity increases over iterations the algorithm s convergence rate is properly calculated with respect to cumulative computation time this suggests the problem of determining an optimal rate of refinement for the operator approximation this paper shows that for linearly convergent algorithm the optimal rate of refinement approaches the rate of convergence of the exact algorithm itself regardless of the tolerance complexity relationship we illustrate this result with an analysis of coarse to fine grid algorithms for markov decision processes with continuous state spaces using the methods proposed here we deduce an algorithm that presents optimal complexity results and consists solely of a non adaptive schedule for the gradual decrease of grid size

nearly optimal control laws for nonlinear systems with saturating actuators using a neural network hjb approach the hamilton jacobi bellman hjb equation corresponding to constrained control is formulated using a suitable nonquadratic functional it is shown that the constrained optimal control law has the largest region of asymptotic stability ras the value function of this hjb equation is solved for by solving for a sequence of cost functions satisfying a sequence of lyapunov equations le a neural network is used to approximate the cost function associated with each le using the method of least squares on a well defined region of attraction of an initial stabilizing controller as the order of the neural network is increased the least squares solution of the hjb equation converges uniformly to the exact solution of the inherently nonlinear hjb equation associated with the saturating control inputs the result is a nearly optimal constrained state feedback controller that has been tuned a priori off line

converse optimality for discrete time systems given an optimal value function for a discrete time infinite horizon optimal control problem how does one derive a nonlinear system dynamics model which satisfies this optimal value function this problem is called converse optimality and was successfully addressed in the continuous time case there is however no similar derivation in the discrete time case and this is the purpose of this paper a class of discrete time systems satisfying the given optimality condition is constructed and its properties are analyzed

computational adaptive optimal control for continuous time linear systems with completely unknown dynamics this paper presents a novel policy iteration approach for finding online adaptive optimal controllers for continuous time linear systems with completely unknown system dynamics the proposed approach employs the approximate adaptive dynamic programming technique to iteratively solve the algebraic riccati equation using the online information of state and input without requiring the a priori knowledge of the system matrices in addition all iterations can be conducted by using repeatedly the same state and input information on some fixed time intervals a practical online algorithm is developed in this paper and is applied to the controller design for a turbocharged diesel engine with exhaust gas recirculation finally several aspects of future work are discussed

multi agent differential graphical games online adaptive learning solution for synchronization with optimality multi agent systems arise in several domains of engineering and can be used to solve problems which are difficult for an individual agent to solve strategies for team decision problems including optimal control n player games h infinity control and non zero sum and so on are normally solved for off line by solving associated matrix equations such as the coupled riccati equations or coupled hamilton jacobi equations however using that approach players cannot change their objectives online in real time without calling for a completely new off line solution for the new strategies therefore in this paper we bring together cooperative control reinforcement learning and game theory to present a multi agent formulation for the online solution of team games the notion of graphical games is developed for dynamical systems where the dynamics and performance indices for each node depend only on local neighbor information it is shown that standard definitions for nash equilibrium are not sufficient for graphical games and a new definition of interactive nash equilibrium is given we give a cooperative policy iteration algorithm for graphical games that converges to the best response when the neighbors of each agent do not update their policies and to the cooperative nash equilibrium when all agents update their policies simultaneously this is used to develop methods for online adaptive learning solutions of graphical games in real time along with proofs of stability and convergence

reinforcement learning in continuous time and space this article presents a reinforcement learning framework for continuous time dynamical systems without a priori discretization of time state and action based on the hamilton jacobi bellman hjb equation for infinite horizon discounted reward problems we derive algorithms for estimating value functions and improving policies with the use of function approximators the processof value function estimation is formulated asthe minimization of a continuous time form of the temporal difference td error update methods based on backward euler approximation and exponential eligibility traces are derived and their correspondences with the conventional residual gradient td 0 and td lambda algorithms are shown for policy improvement two methods a continuous actor critic method and a value gradient based greedy policy are formulated as a special case of the latter a nonlinear feedback control law using the value gradient and the model of the input gain is derived the advantage updating a model free algorithm derived previously is also formulated in the hjbbased framework the performance of the proposed algorithms is first tested in a nonlinear control task of swinging a pendulum up with limited torque it is shown in the simulations that 1 the task is accomplished by the continuous actor critic method in a number of trials several times fewer than by the conventional discrete actor critic method 2 among the continuous policy update methods the value gradient based policy with a known or learned dynamic model performs several times better than the actor critic method and 3 a value function update using exponential eligibility traces is more efficient and stable than that based on euler approximation the algorithms are then tested in a higher dimensional task cartpole swing up this task is accomplished in several hundred trials using the value gradient based policy with a learned dynamic model

online adaptive optimal control for continuous time nonlinear systems with completely unknown dynamics an online adaptive optimal control is proposed for continuous time nonlinear systems with completely unknown dynamics which is achieved by developing a novel identifier critic based approximate dynamic programming algorithm with a dual neural network nn approximation structure first an adaptive nn identifier is designed to obviate the requirement of complete knowledge of system dynamics and a critic nn is employed to approximate the optimal value function then the optimal control law is computed based on the information from the identifier nn and the critic nn so that the actor nn is not needed in particular a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier nn and critic nn simultaneously which converge to small neighbourhoods around their ideal values the closed loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the lyapunov theory the proposed ada

adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro dynamic programming highly dissipative nonlinear partial differential equations pdes are widely employed to describe the system dynamics of industrial spatially distributed processes sdps in this paper we consider the optimal control problem of the general highly dissipative sdps and propose an adaptive optimal control approach based on neuro dynamic programming ndp initially karhunen loeve decomposition is employed to compute empirical eigenfunctions eefs of the sdp based on the method of snapshots these eefs together with singular perturbation technique are then used to obtain a finite dimensional slow subsystem of ordinary differential equations that accurately describes the dominant dynamics of the pde system subsequently the optimal control problem is reformulated on the basis of the slow subsystem which is further converted to solve a hamilton jacobi bellman hjb equation hjb equation is a nonlinear pde that has proven to be impossible to solve analytically thus an adaptive optimal control method is developed via ndp that solves the hjb equation online using neural network nn for approximating the value function and an online nn weight tuning law is proposed without requiring an initial stabilizing control policy moreover by involving the nn estimation error we prove that the original closed loop pde system with the adaptive optimal control policy is semiglobally uniformly ultimately bounded finally the developed method is tested on a nonlinear diffusion convection reaction process and applied to a temperature cooling fin of high speed aerospace vehicle and the achieved results show its effectiveness

nonparametric identification under discrete variation this paper provides weak conditions under which there is nonparametric interval identification of local features of a structural function that depends on a discrete endogenous variable and is nonseparable in latent variates the function delivers values of a discrete or continuous outcome and instruments may be discrete valued application of the analog principle leads to quantile regression based interval estimators of values and partial differences of structural functions the results are used to investigate the nonparametric identifying power of the quarter of birth instruments used in angrist and krueger s 1991 study of the returns to schooling

sampling moments and reconstructing signals of finite rate of innovation shannon meets strang fix consider the problem of sampling signals which are not bandlimited but still have a finite number of degrees of freedom per unit of time such as for example nonuniform splines or piecewise polynomials and call the number of degrees of freedom per unit of time the rate of innovation classical sampling theory does not enable a perfect reconstruction of such signals since they are not bandlimited recently it was shown that by using an adequate sampling kernel and a sampling rate greater or equal to the rate of innovation it is possible to reconstruct such signals uniquely these sampling schemes however use kernels with infinite support and this leads to complex and potentially unstable reconstruction algorithms in this paper we show that many signals with a finite rate of innovation can be sampled and perfectly reconstructed using physically realizable kernels of compact support and a local reconstruction algorithm the class of kernels that we can use is very rich and includes functions satisfying strang fix conditions exponential splines and functions with rational fourier transform this last class of kernels is quite general and includes for instance any linear electric circuit we thus show with an example how to estimate a signal of finite rate of innovation at the output of an rc circuit the case of noisy measurements is also analyzed and we present a novel algorithm that reduces the effect of noise by oversampling

integral q learning and explorized policy iteration for adaptive optimal control of continuous time linear systems this paper proposes an integral q learning for continuous time ct linear time invariant lti systems which solves a linear quadratic regulation lqr problem in real time for a given system and a value function without knowledge about the system dynamics a and b here q learning is referred to as a family of reinforcement learning methods which find the optimal policy by interaction with an uncertain environment in the evolution of the algorithm we first develop an explorized policy iteration pi method which is able to deal with known exploration signals then the integral q learning algorithm for ct lti systems is derived based on this pi and the variants of q functions derived from the singular perturbation of the control input the proposed q learning scheme evaluates the current value function and the improved control policy at the same time and are proven stable and convergent to the lq optimal solution provided that the initial policy is stabilizing for the proposed algorithms practical online implementation methods are investigated in terms of persistency of excitation pe and explorations finally simulation results are provided for the better comparison and verification of the performance

neural network observer based optimal control for unknown nonlinear systems using adaptive dynamic programming in this paper an observer based optimal control scheme is developed for unknown nonlinear systems using adaptive dynamic programming adp algorithm first a neural network nn observer is designed to estimate system states then based on the observed states a neuro controller is constructed via adp method to obtain the optimal control in this design two nn structures are used a three layer nn is used to construct the observer which can be applied to systems with higher degrees of nonlinearity and without a priori knowledge of system dynamics and a critic nn is employed to approximate the value function the optimal control law is computed using the critic nn and the observer nn uniform ultimate boundedness of the closed loop system is guaranteed the actor critic and observer structures are all implemented in real time continuously and simultaneously finally simulation results are presented to demonstrate the effectiveness of the proposed control scheme

l sub 2 gain analysis of nonlinear systems and nonlinear state feedback h sub infinity control previously obtained results on l2 gain analysis of smooth nonlinear systems are unified and extended using an approach based on hamilton jacobi equations and inequalities and their relation to invariant manifolds of an associated hamiltonian vector field on the basis of these results a nonlinear analog is obtained of the simplest part of a state space approach to linear h sub infinity control namely the state feedback h sub infinity optimal control problem furthermore the relation with h sub infinity control of the linearized system is dealt with

tighter bounds in triangular systems abstract we study a nonparametric triangular system with potentially discrete endogenous regressors and nonseparable errors like in other work in this area the parameter of interest is the structural function evaluated at particular values we impose a global exclusion and exogeneity condition in contrast to chesher 2005 but develop a rank condition which is weaker than chesher s the alternative rank condition can be satisfied for binary endogenous regressors and it often leads to an identified interval tighter than chesher 2005 s minimum length interval we illustrate the potential of the new rank condition using the angrist and krueger 1991 data

optimal control for unknown discrete time nonlinear markov jump systems using adaptive dynamic programming in this paper we develop and analyze an optimal control method for a class of discrete time nonlinear markov jump systems mjss with unknown system dynamics specifically an identifier is established for the unknown systems to approximate system states and an optimal control approach for nonlinear mjss is developed to solve the hamilton jacobi bellman equation based on the adaptive dynamic programming technique we also develop detailed stability analysis of the control approach including the convergence of the performance index function for nonlinear mjss and the existence of the corresponding admissible control neural network techniques are used to approximate the proposed performance index function and the control law to demonstrate the effectiveness of our approach three simulation studies one linear case one nonlinear case and one single link robot arm case are used to validate the performance of the proposed optimal control method

hamiltonian driven adaptive dynamic programming for continuous nonlinear dynamical systems this paper presents a hamiltonian driven framework of adaptive dynamic programming adp for continuous time nonlinear systems which consists of evaluation of an admissible control comparison between two different admissible policies with respect to the corresponding the performance function and the performance improvement of an admissible control it is showed that the hamiltonian can serve as the temporal difference for continuous time systems in the hamiltonian driven adp the critic network is trained to output the value gradient then the inner product between the critic and the system dynamics produces the value derivative under some conditions the minimization of the hamiltonian functional is equivalent to the value function approximation an iterative algorithm starting from an arbitrary admissible control is presented for the optimal control approximation with its convergence proof the implementation is accomplished by a neural network approximation two simulation studies demonstrate the effectiveness of hamiltonian driven adp

data based robust optimal control of continuous time affine nonlinear systems with matched uncertainties in this paper the robust optimal control of continuous time affine nonlinear systems with matched uncertainties is investigated by using a data based integral policy iteration approach it is a natural extension of the traditional optimal control design under the framework of adaptive dynamic programming adp method to robust optimal control of nonlinear systems with matched uncertainties in theoretical aspect by increasing a feedback gain to the optimal controller of the nominal system the robust controller of the matched uncertain system is obtained which also achieves optimality with a newly well defined cost function when regarding the implementation the data based integral policy iteration algorithm is used to solve the hamilton jacobi bellman equation corresponding to the nominal system with completely unknown dynamics information then the actor critic technique based on neural networks and least squares implementation method are employed to facilitate deriving the optimal control law iteratively so that the closed form expression of the robust optimal controller is available additionally two simulation examples with application backgrounds are presented to illustrate the effectiveness of the established robust optimal control scheme in summary it is important to note that the result developed in this paper broadens the application scope of adp based optimal control approach to more general nonlinear systems possessing dynamical uncertainties

reinforcement learning for adaptive optimal control of unknown continuous time nonlinear systems with input constraints in this paper an adaptive reinforcement learning based solution is developed for the infinite horizon optimal control problem of constrained input continuous time nonlinear systems in the presence of nonlinearities with unknown structures two different types of neural networks nns are employed to approximate the hamilton jacobi bellman equation that is an recurrent nn is constructed to identify the unknown dynamical system and two feedforward nns are used as the actor and the critic to approximate the optimal control and the optimal cost respectively based on this framework the action nn and the critic nn are tuned simultaneously without the requirement for the knowledge of system drift dynamics moreover by using lyapunov s direct method the weights of the action nn and the critic nn are guaranteed to be uniformly ultimately bounded while keeping the closed loop system stable to demonstrate the effectiveness of the present approach simulation results are illustrated

h _ infty tracking control of completely unknown continuous time systems via off policy reinforcement learning this paper deals with the design of an h infty tracking controller for nonlinear continuous time systems with completely unknown dynamics a general bounded l 2 gain tracking problem with a discounted performance function is introduced for the h infty tracking a tracking hamilton jacobi isaac hji equation is then developed that gives a nash equilibrium solution to the associated min max optimization problem a rigorous analysis of bounded l 2 gain and stability of the control solution obtained by solving the tracking hji equation is provided an upper bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics an off policy reinforcement learning algorithm is used to learn the solution to the tracking hji equation online without requiring any knowledge of the system dynamics convergence of the proposed algorithm to the solution to the tracking hji equation is shown simulation examples are provided to verify the effectiveness of the proposed method

complexity of shapes embedded in z n with a bias towards squares shape complexity is a hard to quantify quality mainly due to its relative nature biased by euclidean thinking circles are commonly considered as the simplest however their constructions as digital images are only approximations to the ideal form consequently complexity orders computed in reference to circle are unstable unlike circles which lose their circleness in digital images squares retain their qualities hence we consider squares hypercubes in mathbb z n to be the simplest shapes relative to which complexity orders are constructed using the connection between l infty norm and squares we effectively encode squareness adapted simplification through which we obtain multi scale complexity measure where scale determines the level of interest to the boundary the emergent scale above which the effect of a boundary feature appendage disappears is related to the ratio of the contacting width of the appendage to that of the main body we discuss what zero complexity implies in terms of information repetition and constructibility and what kind of shapes in addition to squares have zero complexity

neural network based online simultaneous policy update algorithm for solving the hji equation in nonlinear h_ infty control it is well known that the nonlinear h state feedback control problem relies on the solution of the hamilton jacobi isaacs hji equation which is a nonlinear partial differential equation that has proven to be impossible to solve analytically in this paper a neural network nn based online simultaneous policy update algorithm spua is developed to solve the hji equation in which knowledge of internal system dynamics is not required first we propose an online spua which can be viewed as a reinforcement learning technique for two players to learn their optimal actions in an unknown environment the proposed online spua updates control and disturbance policies simultaneously thus only one iterative loop is needed second the convergence of the online spua is established by proving that it is mathematically equivalent to newton s method for finding a fixed point in a banach space third we develop an actor critic structure for the implementation of the online spua in which only one critic nn is needed for approximating the cost function and a least square method is given for estimating the nn weight parameters finally simulation studies are provided to demonstrate the effectiveness of the proposed algorithm

optimal adaptive fuzzy ftc design for strict feedback nonlinear uncertain systems with actuator faults abstract in this paper the optimal fault tolerant control ftc design problem is investigated for strict feedback nonlinear systems the considered nonlinear systems contain actuator faults which are bias fault and gain fault the problem of actuator faults is solved by utilizing the adaptive compensation technique applying the backstepping design technique a main fault tolerant controller is established via the designed main fault tolerant control strategy the controlled nonlinear system can be converted to an equivalent nonlinear system in affine form subsequently a fuzzy logic system fls is introduced to approximate the unknown cost function and an adaptive fuzzy feedback optimal controller is developed the whole adaptive fuzzy fault tolerant optimal control scheme consists of a main controller and a feedback optimal controller based on the lyapunov theory it is proved that all the signals in the closed loop system are bounded the system output can track a bounded reference signal in addition the proposed control method can guarantee the cost function is the smallest simulation results are provided to illustrate the effectiveness of the proposed fault tolerant optimal control scheme

sampling signals with finite rate of innovation the authors consider classes of signals that have a finite number of degrees of freedom per unit of time and call this number the rate of innovation examples of signals with a finite rate of innovation include streams of diracs e g the poisson process nonuniform splines and piecewise polynomials even though these signals are not bandlimited we show that they can be sampled uniformly at or above the rate of innovation using an appropriate kernel and then be perfectly reconstructed thus we prove sampling theorems for classes of signals and kernels that generalize the classic bandlimited and sinc kernel case in particular we show how to sample and reconstruct periodic and finite length streams of diracs nonuniform splines and piecewise polynomials using sinc and gaussian kernels for infinite length signals with finite local rate of innovation we show local sampling and reconstruction based on spline kernels the key in all constructions is to identify the innovative part of a signal e g time instants and weights of diracs using an annihilating or locator filter a device well known in spectral analysis and error correction coding this leads to standard computational procedures for solving the sampling problem which we show through experimental results applications of these new sampling results can be found in signal processing communications systems and biological systems

model based reinforcement learning for approximate optimal regulation reinforcement learning rl based online approximate optimal control methods applied to deterministic systems typically require a restrictive persistence of excitation pe condition for convergence this paper develops a concurrent learning cl based implementation of model based rl to solve approximate optimal regulation problems online under a pe like rank condition the development is based on the observation that given a model of the system rl can be implemented by evaluating the bellman error at any number of desired points in the state space in this result a parametric system model is considered and a cl based parameter identifier is developed to compensate for uncertainty in the parameters uniformly ultimately bounded regulation of the system states to a neighborhood of the origin and convergence of the developed policy to a neighborhood of the optimal policy are established using a lyapunov based analysis and simulation results are presented to demonstrate the performance of the developed controller

quantile regression methods for recursive structural equation models abstract two classes of quantile regression estimation methods for the recursive structural equation models of chesher 2003 identification in nonseparable models econometrica 71 1405 1441 are investigated a class of weighted average derivative estimators based directly on the identification strategy of chesher is contrasted with a new control variate estimation method the latter imposes stronger restrictions achieving an asymptotic efficiency bound with respect to the former class an application of the methods to the study of the effect of class size on the performance of dutch primary school students shows that i reductions in class size are beneficial for good students in language and for weaker students in mathematics ii larger classes appear beneficial for weaker language students and iii the impact of class size on both mean and median performance is negligible

on integral generalized policy iteration for continuous time linear quadratic regulations this paper mathematically analyzes the integral generalized policy iteration i gpi algorithms applied to a class of continuous time linear quadratic regulation lqr problems with the unknown system matrix a gpi is the general idea of interacting policy evaluation and policy improvement steps of policy iteration pi for computing the optimal policy we first introduce the update horizon and then show that i all of the i gpi methods with the same can be considered equivalent and that ii the value function approximated in the policy evaluation step monotonically converges to the exact one as this reveals the relation between the computational complexity and the update or time horizon of i gpi as well as between i pi and i gpi in the limit we also provide and discuss two modes of convergence of i gpi i gpi behaves like pi in one mode and in the other mode it performs like value iteration for discrete time lqr and infinitesimal gpi 0 from these results a new classification of the integral reinforcement learning is formed with respect to two matrix inequality conditions for stability the region of local monotone convergence and data driven adaptive implementation methods are also provided with detailed discussion numerical simulations are carried out for verification and further investigations

online actor critic algorithm to solve the continuous time infinite horizon optimal control problem in this paper we discuss an online algorithm based on policy iteration for learning the continuous time ct optimal control solution with infinite horizon cost for nonlinear systems with known dynamics that is the algorithm learns online in real time the solution to the optimal control design hj equation this method finds in real time suitable approximations of both the optimal cost and the optimal control policy while also guaranteeing closed loop stability we present an online adaptive algorithm implemented as an actor critic structure which involves simultaneous continuous time adaptation of both actor and critic neural networks we call this synchronous policy iteration a persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function novel tuning algorithms are given for both critic and actor networks with extra nonstandard terms in the actor tuning law being required to guarantee closed loop dynamical stability the convergence to the optimal controller is proven and the stability of the system is also guaranteed simulation examples show the effectiveness of the new algorithm

multi player non zero sum games online adaptive learning solution of coupled hamilton jacobi equations in this paper we present an online adaptive control algorithm based on policy iteration reinforcement learning techniques to solve the continuous time ct multi player non zero sum nzs game with infinite horizon for linear and nonlinear systems nzs games allow for players to have a cooperative team component and an individual selfish component of strategy the adaptive algorithm learns online the solution of coupled riccati equations and coupled hamilton jacobi equations for linear and nonlinear systems respectively this adaptive control method finds in real time approximations of the optimal value and the nzs nash equilibrium while also guaranteeing closed loop stability the optimal adaptive algorithm is implemented as a separate actor critic parametric network approximator structure for every player and involves simultaneous continuous time adaptation of the actor critic networks a persistence of excitation condition is shown to guarantee convergence of every critic to the actual optimal value function for that player a detailed mathematical analysis is done for 2 player nzs games novel tuning algorithms are given for the actor critic networks the convergence to the nash equilibrium is proven and stability of the system is also guaranteed this provides optimal adaptive control solutions for both non zero sum games and their special case the zero sum games simulation examples show the effectiveness of the new algorithm

sampling curves with finite rate of innovation in this paper we extend the theory of sampling signals with finite rate of innovation fri to a specific class of two dimensional curves which are defined implicitly as the zeros of a mask function here the mask function has a parametric representation as a weighted summation of a finite number of complex exponentials and therefore has finite rate of innovation an associated edge image which is discontinuous on the predefined parametric curve is proved to satisfy a set of linear annihilation equations we show that it is possible to reconstruct the parameters of the curve i e to detect the exact edge positions in the continuous domain based on the annihilation equations robust reconstruction algorithms are also developed to cope with scenarios with model mismatch moreover the annihilation equations that characterize the curve are linear constraints that can be easily exploited in optimization problems for further image processing e g image up sampling we demonstrate one potential application of the annihilation algorithm with examples in edge preserving interpolation experimental results with both synthetic curves as well as edges of natural images clearly show the effectiveness of the annihilation constraint in preserving sharp edges and improving snrs

convergence analysis of alternating direction method of multipliers for a family of nonconvex problems the alternating direction method of multipliers admm is widely used to solve large scale linearly constrained optimization problems convex or nonconvex in many engineering fields however there is a general lack of theoretical understanding of the algorithm when the objective function is nonconvex in this paper we analyze the convergence of the admm for solving certain nonconvex consensus and sharing problems we show that the classical admm converges to the set of stationary solutions provided that the penalty parameter in the augmented lagrangian is chosen to be sufficiently large for the sharing problems we show that the admm is convergent regardless of the number of variable blocks our analysis does not impose any assumptions on the iterates generated by the algorithm and is broadly applicable to many admm variants involving proximal update rules and various flexible block selection rules

multi agent discrete time graphical games and reinforcement learning solutions this paper introduces a new class of multi agent discrete time dynamic games known in the literature as dynamic graphical games for that reason a local performance index is defined for each agent that depends only on the local information available to each agent nash equilibrium policies and best response policies are given in terms of the solutions to the discrete time coupled hamilton jacobi equations since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of nash equilibrium it is proved that this notion holds if all agents are in nash equilibrium and the graph is strongly connected a novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence the policies of the agents form a nash equilibrium when all the agents in the neighborhood update their policies and a best response outcome when the agents in the neighborhood are kept constant the paper brings together discrete hamiltonian mechanics distributed multi agent control optimal control theory and game theory to formulate and solve these multi agent dynamic graphical games a simulation example shows the effectiveness of the proposed approach in a leader synchronization case along with optimality guarantees

adaptive control of uncertain nonaffine nonlinear systems with input saturation using neural networks this paper presents a tracking control methodology for a class of uncertain nonlinear systems subject to input saturation constraint and external disturbances unlike most previous approaches on saturated systems which assumed affine nonlinear systems in this paper tracking control problem is solved for uncertain nonaffine nonlinear systems with input saturation to deal with the saturation constraint an auxiliary system is constructed and a modified tracking error is defined then by employing implicit function theorem mean value theorem and modified tracking error updating rules are derived based on the well known back propagation bp algorithm which has been proven to be the most relevant updating rule to control problems however most of the previous approaches on bp algorithm suffer from lack of stability analysis by injecting a damping term to the standard bp algorithm uniformly ultimately boundedness of all the signals of the closed loop system is ensured via lyapunov s direct method furthermore the presented approach employs nonlinear in parameter neural networks hence the proposed scheme is applicable to systems with higher degrees of nonlinearity using a high gain observer to reconstruct the states of the system an output feedback controller is also presented finally the simulation results performed on a duffing holmes chaotic system a generalized pendulum type system and a numerical system are presented to demonstrate the effectiveness of the suggested state and output feedback control schemes

finite horizon control constrained nonlinear optimal control using single network adaptive critics to synthesize fixed final time control constrained optimal controllers for discrete time nonlinear control affine systems a single neural network nn based controller called the finite horizon single network adaptive critic is developed in this paper inputs to the nn are the current system states and the time to go and the network outputs are the costates that are used to compute optimal feedback control control constraints are handled through a nonquadratic cost function convergence proofs of 1 the reinforcement learning based training method to the optimal solution 2 the training error and 3 the network weights are provided the resulting controller is shown to solve the associated time varying hamilton jacobi bellman equation and provide the fixed final time optimal solution performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite time attitude maneuver subject to control bounds the new formulation has great potential for implementation since it consists of only one nn with single set of weights and it provides comprehensive feedback solutions online though it is trained offline

approximate n player nonzero sum game solution for an uncertain continuous nonlinear system an approximate online equilibrium solution is developed for an n player nonzero sum game subject to continuous time nonlinear unknown dynamics and an infinite horizon quadratic cost a novel actor critic identifier structure is used wherein a robust dynamic neural network is used to asymptotically identify the uncertain system with additive disturbances and a set of critic and actor nns are used to approximate the value functions and equilibrium policies respectively the weight update laws for the actor neural networks nns are generated using a gradient descent method and the critic nns are generated by least square regression which are both based on the modified bellman error that is independent of the system dynamics a lyapunov based stability analysis shows that uniformly ultimately bounded tracking is achieved and a convergence analysis demonstrates that the approximate control policies converge to a neighborhood of the optimal solutions the actor critic and identifier structures are implemented in real time continuously and simultaneously simulations on two and three player games illustrate the performance of the developed method

low rank modeling of local k space neighborhoods loraks for constrained mri recent theoretical results on low rank matrix reconstruction have inspired significant interest in low rank modeling of mri images existing approaches have focused on higher dimensional scenarios with data available from multiple channels timepoints or image contrasts the present work demonstrates that single channel single contrast single timepoint k space data can also be mapped to low rank matrices when the image has limited spatial support or slowly varying phase based on this we develop a novel and flexible framework for constrained image reconstruction that uses low rank matrix modeling of local k space neighborhoods loraks a new regularization penalty and corresponding algorithm for promoting low rank are also introduced the potential of loraks is demonstrated with simulated and experimental data for a range of denoising and sparse sampling applications loraks is also compared against state of the art methods like homodyne reconstruction l1 norm minimization and total variation minimization and is demonstrated to have distinct features and advantages in addition while calibration based support and phase constraints are commonly used in existing methods the loraks framework enables calibrationless use of these constraints

infinite horizon self learning optimal control of nonaffine discrete time nonlinear systems in this paper a novel iterative adaptive dynamic programming adp based infinite horizon self learning optimal control algorithm called generalized policy iteration algorithm is developed for nonaffine discrete time dt nonlinear systems generalized policy iteration algorithm is a general idea of interacting policy and value iteration algorithms of adp the developed generalized policy iteration algorithm permits an arbitrary positive semidefinite function to initialize the algorithm where two iteration indices are used for policy improvement and policy evaluation respectively it is the first time that the convergence admissibility and optimality properties of the generalized policy iteration algorithm for dt nonlinear systems are analyzed neural networks are used to implement the developed algorithm finally numerical examples are presented to illustrate the performance of the developed algorithm

partial identification in triangular systems of equations with binary dependent variables this paper studies the special case of the triangular system of equations in vytlacil and yildiz 2007 where both dependent variables are binary but without imposing the restrictive support condition required by vytlacil and yildiz 2007 for identification of the average structural function asf and the average treatment effect ate under weak regularity conditions we derive upper and lower bounds on the asf and the ate we show further that the bounds on the asf and ate are sharp under some further regularity conditions and an additional restriction on the support of the covariates and the instrument

boundary control of linear uncertain 1 d parabolic pde using approximate dynamic programming this paper develops a near optimal boundary control method for distributed parameter systems governed by uncertain linear 1 d parabolic partial differential equations pde by using approximate dynamic programming a quadratic surface integral is proposed to express the optimal cost functional for the infinite dimensional state space accordingly the hamilton jacobi bellman hjb equation is formulated in the infinite dimensional domain without using any model reduction subsequently a neural network identifier is developed to estimate the unknown spatially varying coefficient in pde dynamics novel tuning law is proposed to guarantee the boundedness of identifier approximation error in the pde domain a radial basis network rbn is subsequently proposed to generate an approximate solution for the optimal surface kernel function online the tuning law for near optimal rbn weights is created such that the hjb equation error is minimized while the dynamics are identified and closed loop system remains stable ultimate boundedness ub of the closed loop system is verified by using the lyapunov theory the performance of the proposed controller is successfully confirmed by simulation on an unstable diffusion reaction process

sampling and reconstruction of shapes with algebraic boundaries we present a sampling theory for a class of binary images with finite rate of innovation fri every image in our model is the restriction of mathbb 1 p leq 0 to the image plane where mathbb 1 denotes the indicator function and p is some real bivariate polynomial this particularly means that the boundaries in the image form a subset of an algebraic curve with the implicit polynomial p we show that the image parameters i e the polynomial coefficients satisfy a set of linear annihilation equations with the coefficients being the image moments the inherent sensitivity of the moments to noise makes the reconstruction process numerically unstable and narrows the choice of the sampling kernels to polynomial reproducing kernels as a remedy to these problems we replace conventional moments with more stable generalized moments that are adjusted to the given sampling kernel the benefits are threefold 1 it relaxes the requirements on the sampling kernels 2 produces annihilation equations that are robust at numerical precision and 3 extends the results to images with unbounded boundaries we further reduce the sensitivity of the reconstruction process to noise by taking into account the sign of the polynomial at certain points and sequentially enforcing measurement consistency we consider various numerical experiments to demonstrate the performance of our algorithm in reconstructing binary images including low to moderate noise levels and a range of realistic sampling kernels

a novel actor critic identifier architecture for approximate optimal control of uncertain nonlinear systems an online adaptive reinforcement learning based solution is developed for the infinite horizon optimal control problem for continuous time uncertain nonlinear systems a novel actor critic identifier aci is proposed to approximate the hamilton jacobi bellman equation using three neural network nn structures actor and critic nns approximate the optimal control and the optimal value function respectively and a robust dynamic neural network identifier asymptotically approximates the uncertain system dynamics an advantage of using the aci architecture is that learning by the actor critic and identifier is continuous and simultaneous without requiring knowledge of system drift dynamics convergence of the algorithm is analyzed using lyapunov based adaptive control methods a persistence of excitation condition is required to guarantee exponential convergence to a bounded region in the neighborhood of the optimal control and uniformly ultimately bounded uub stability of the closed loop system simulation results demonstrate the performance of the actor critic identifier method for approximate optimal control

dummy endogenous variables in weakly separable models in this paper we consider the nonparametric identification and estimation of the average effect of a dummy endogenous regressor in models where the regressors are weakly but not additively separable from the error term the model is not required to be strictly increasing in the error term and the class of models considered includes limited dependent variable models such as discrete choice models conditions are established conditions under which it is possible to identify the average effect of the dummy endogenous regressor in a weakly separable model without relying on parametric functional form or distributional assumptions and without the use of large support conditions

online optimal control of affine nonlinear discrete time systems with unknown internal dynamics by using time based policy update in this paper the hamilton jacobi bellman equation is solved forward in time for the optimal control of a class of general affine nonlinear discrete time systems without using value and policy iterations the proposed approach referred to as adaptive dynamic programming uses two neural networks nns to solve the infinite horizon optimal regulation control of affine nonlinear discrete time systems in the presence of unknown internal dynamics and a known control coefficient matrix one nn approximates the cost function and is referred to as the critic nn while the second nn generates the control input and is referred to as the action nn the cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time based adp novel update laws for tuning the unknown weights of the nns online are derived lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small bounded error over time in the absence of disturbances an optimal control is demonstrated simulation results are included to show the effectiveness of the approach the end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation

identification in nonseparable models weak nonparametric restrictions are developed sufficient to identify the values of derivatives of structural functions in which latent random variables are nonseparable these derivatives can exhibit stochastic variation in a microeconometric context this allows the impact of a policy intervention as measured by the value of a structural derivative to vary across people who are identical as measured by covariates when the restrictions are satisfied quantiles of the distribution of a policy impact across people can be identified the identification restrictions are local in the sense that they are specific to the values of the covariates and the specific quantiles of latent variables at which identification is sought the conditions do not include the commonly required independence of latent variables and covariates they include local versions of the classical rank and order conditions and local quantile insensitivity conditions values of structural derivatives are identified by functionals of quantile regression functions and can be estimated using the same functionals applied to estimated quantile regression functions

iterative adaptive dynamic programming for solving unknown nonlinear zero sum game based on online data h infty control is a powerful method to solve the disturbance attenuation problems that occur in some control systems the design of such controllers relies on solving the zero sum game zsg but in practical applications the exact dynamics is mostly unknown identification of dynamics also produces errors that are detrimental to the control performance to overcome this problem an iterative adaptive dynamic programming algorithm is proposed in this paper to solve the continuous time unknown nonlinear zsg with only online data a model free approach to the hamilton jacobi isaacs equation is developed based on the policy iteration method control and disturbance policies and value are approximated by neural networks nns under the critic actor disturber structure the nn weights are solved by the least squares method according to the theoretical analysis our algorithm is equivalent to a gauss newton method solving an optimization problem and it converges uniformly to the optimal solution the online data can also be used repeatedly which is highly efficient simulation results demonstrate its feasibility to solve the unknown nonlinear zsg when compared with other algorithms it saves a significant amount of online measurement time

distributed cooperative h optimal tracking control of mimo nonlinear multi agent systems in strict feedback form via adaptive dynamic programming the design of distributed cooperative h optimal controllers for multi agent systems is a major challenge when the agents models are uncertain multi input and multi output nonlinear systems in str

recovery of damped exponentials using structured low rank matrix completion we introduce a structured low rank matrix completion algorithm to recover a series of images from their under sampled measurements where the signal along the parameter dimension at every pixel is described by a linear combination of exponentials we exploit the exponential behavior of the signal at every pixel along with the spatial smoothness of the exponential parameters to derive an annihilation relation in the fourier domain this relation translates to a low rank property on a structured matrix constructed from the fourier samples we enforce the low rank property of the structured matrix as a regularization prior to recover the images since the direct use of current low rank matrix recovery schemes to this problem is associated with high computational complexity and memory demand we adopt an iterative re weighted least squares algorithm which facilitates the exploitation of the convolutional structure of the matrix novel approximations involving 2 d fast fourier transforms are introduced to drastically reduce the memory demand and computational complexity which facilitates the extension of structured low rank methods to large scale 3 d problems we demonstrate our algorithm in the mr parameter mapping setting and show improvement over the state of the art methods

inference on regressions with interval data on a regressor or outcome this paper examines inference on regressions when interval data are available on one variable the other variables being measured precisely let a population be characterized by a distribution p y x v v 0 v 1 where y r 1 x r k and the real variables v v 0 v 1 satisfy v 0 v v 1 let a random sample be drawn from p and the realizations of y x v 0 v 1 be observed but not those of v the problem of interest may be to infer e y x v or e v x this analysis maintains interval i monotonicity m and mean independence mi assumptions i p v 0 v v 1 1 m e y x v is monotone in v mi e y x v v 0 v 1 e y x v no restrictions are imposed on the distribution of the unobserved values of v within the observed intervals v 0 v 1 it is found that the immi assumptions alone imply simple nonparametric bounds on e y x v and e v x these assumptions invoked when y is binary and combined with a semiparametric binary regression model yield an identification region for the parameters that may be estimated consistently by a modified maximum score mms method the immi assumptions combined with a parametric model for e y x v or e v x yield an identification region that may be estimated consistently by a modified minimum distance mmd method monte carlo methods are used to characterize the finite sample performance of these estimators empirical case studies are performed using interval wealth data in the health and retirement study and interval income data in the current population survey

partial distributional policy effects in this paper we propose a method to evaluate the effect of a counterfactual change in the unconditional distribution of a single covariate on the unconditional distribution of an outcome variable of interest both fixed and infinitesimal changes are considered we show that such effects are point identified under general conditions if the covariate affected by the counterfactual change is continuously distributed but are typically only partially identified if its distribution is discrete for the latter case we derive informa tive bounds making use of the available information we also discuss estimation and inference

shapes from pixels continuous domain visual signals are usually captured as discrete digital images this operation is not invertible in general in the sense that the continuous domain signal cannot be exactly reconstructed based on the discrete image unless it satisfies certain constraints e g bandlimitedness in this paper we study the problem of recovering shape images with smooth boundaries from a set of samples thus the reconstructed image is constrained to regenerate the same samples consistency as well as forming a shape bilevel image we initially formulate the reconstruction technique by minimizing the shape perimeter over the set of consistent binary shapes next we relax the non convex shape constraint to transform the problem into minimizing the total variation over consistent non negative valued images we also introduce a requirement called reducibility that guarantees equivalence between the two problems we illustrate that the reducibility property effectively sets a requirement on the minimum sampling density we also evaluate the performance of the relaxed alternative in various numerical experiments

finite horizon h_ infty tracking control for unknown nonlinear systems with saturating actuators in this paper a neural network nn based online model free integral reinforcement learning algorithm is developed to solve the finite horizon h infty optimal tracking control problem for completely unknown nonlinear continuous time systems with disturbance and saturating actuators constrained control input an augmented system is constructed with the tracking error system and the command generator system a time varying hamilton jacobi isaacs hji equation is formulated for the augmented problem which is extremely difficult or impossible to solve due to its time dependent property and nonlinearity then an actor critic disturbance nn structure based scheme is proposed to learn the time varying solution to the hji equation in real time without using the knowledge of system dynamics since the solution to the hji equation is time dependent the form of nns representation with constant weights and time dependent activation functions is considered furthermore an extra error is incorporated in order to satisfy the terminal constraints in the weight update law convergence and stability proofs are given based on the lyapunov theory for nonautonomous systems two simulation examples are provided to demonstrate the effectiveness of the designed algorithm

reconstruction of binary shapes from blurred images via hankel structured low rank matrix recovery with the dominance of digital imaging systems we are often dealing with discrete domain samples of an analog image due to physical limitations all imaging devices apply a blurring kernel on the input image before taking samples to form the output pixels in this paper we focus on the reconstruction of binary shape images from few blurred samples this problem has applications in medical imaging shape processing and image segmentation our method relies on representing the analog shape image in a discrete grid much finer than the sampling grid we formulate the problem as the recovery of a rank r matrix that is formed by a hankel structure on the pixels we further propose efficient admm based algorithms to recover the low rank matrix in both noiseless and noisy settings we also analytically investigate the number of required samples for successful recovery in the noiseless case for this purpose we study the problem in the random sampling framework and show that with mathcal o r log 4 n 1 n 2 random samples where the size of the image is assumed to be n 1 times n 2 we can guarantee the perfect reconstruction with high probability under mild conditions we further prove the robustness of the proposed recovery in the noisy setting by showing that the reconstruction error in the noisy case is bounded when the input noise is bounded simulation results confirm that our proposed method outperform the conventional total variation minimization in the noiseless settings

towards generalized fri sampling with an application to source resolution in radioastronomy it is a classic problem to estimate continuous time sparse signals like point sources in a direction of arrival problem or pulses in a time of flight measurement the earliest occurrence is the estimation of sinusoids in time series using prony s method this is at the root of a substantial line of work on high resolution spectral estimation the estimation of continuous time sparse signals from discrete time samples is the goal of the sampling theory for finite rate of innovation fri signals both spectral estimation and fri sampling usually assume uniform sampling but not all measurements are obtained uniformly as exemplified by a concrete radioastronomy problem we set out to solve thus we develop the theory and algorithm to reconstruct sparse signals typically sum of sinusoids from nonuniform samples we achieve this by identifying a linear transformation that relates the unknown uniform samples of sinusoids to the given measurements these uniform samples are known to satisfy the annihilation equations a valid solution is then obtained by solving a constrained minimization such that the reconstructed signal is consistent with the given measurements and satisfies the annihilation constraint thanks to this new approach we unify a variety of fri based methods we demonstrate the versatility and robustness of the proposed approach with five fri reconstruction problems namely dirac reconstructions with irregular time or fourier domain samples fri curve reconstructions dirac reconstructions on the sphere and point source reconstructions in radioastronomy the proposed algorithm improves substantially over state of the art methods and is able to reconstruct point sources accurately from irregularly sampled fourier measurements under severe noise conditions

efficient model based reinforcement learning for approximate online optimal control an infinite horizon optimal regulation problem is solved online for a deterministic control affine nonlinear dynamical system using a state following staf kernel method to approximate the value function unlike traditional methods that aim to approximate a function over a large compact set the staf kernel method aims to approximate a function in a small neighborhood of a state that travels within a compact set simulation results demonstrate that stability and approximate optimality of the control system can be achieved with significantly fewer basis functions than may be required for global approximation methods

finite horizon approximate optimal guaranteed cost control of uncertain nonlinear systems with application to mars entry guidance this paper studies the finite horizon optimal guaranteed cost control gcc problem for a class of time varying uncertain nonlinear systems the aim of this problem is to find a robust state feedback controller such that the closed loop system has not only a bounded response in a finite duration of time for all admissible uncertainties but also a minimal guaranteed cost a neural network nn based approximate optimal gcc design is developed initially by modifying the cost function to account for the nonlinear perturbation of system the optimal gcc problem is transformed into a finite horizon optimal control problem of the nominal system subsequently with the help of the modified cost function together with a parametrized bounding function for all admissible uncertainties the solution to the optimal gcc problem is given in terms of a parametrized hamilton jacobi bellman phjb equation then a nn method is developed to solve offline the phjb equation approximately and thus obtain the nearly optimal gcc policy furthermore the convergence of approximate phjb equation and the robust admissibility of nearly optimal gcc policy are also analyzed finally by applying the proposed design method to the entry guidance problem of the mars lander the achieved simulation results show the effectiveness of the proposed controller

geometry based image retrieval in binary image databases in this paper a geometry based image retrieval system is developed for multiobject images we model both shape and topology of image objects using a structured representation called curvature tree ct the hierarchy of the ct reflects the inclusion relationships between the image objects to facilitate shape based matching triangle area representation tar of each object is stored at the corresponding node in the ct the similarity between two multiobject images is measured based on the maximum similarity subtree isomorphism mssi between their cts for this purpose we adopt a recursive algorithm to solve the mssi problem and a very effective dynamic programming algorithm to measure the similarity between the attributed nodes our matching scheme agrees with many recent findings in psychology about the human perception of multiobject images experiments on a database of 13 500 real and synthesized medical images and the mpeg 7 ce 1 database of 1 400 shape images have shown the effectiveness of the proposed method

data based robust adaptive control for a class of unknown nonlinear constrained input systems via integral reinforcement learning this paper presents a data based robust adaptive control methodology for a class of nonlinear constrained input systems with completely unknown dynamics by introducing a value function for the nominal system the robust control problem is transformed into a constrained optimal control problem due to the unavailability of system dynamics a data based integral reinforcement learning rl algorithm is developed to solve the constrained optimal control problem based on the present algorithm the value function and the control policy can be updated simultaneously using only system data the convergence of the developed algorithm is proved via an established equivalence relationship to implement the integral rl algorithm an actor neural network nn and a critic nn are separately utilized to approximate the control policy and the value function and the least squares method is employed to estimate the unknown parameters by using lyapunov s direct method the obtained approximate optimal control is verified to guarantee the unknown nonlinear system to be stable in the sense of uniform ultimate boundedness two examples are provided to demonstrate the effectiveness and applicability of the theoretical results

neural network based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming in this paper the neural network based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming approach is investigated first the robust controller of the original uncertain system is derived by adding a feedback gain to the optimal controller of the nominal system it is also shown that this robust controller can achieve optimality under a specified cost function which serves as the basic idea of the robust optimal control design then a critic network is constructed to solve the hamilton jacobi bellman equation corresponding to the nominal system where an additional stabilizing term is introduced to verify the stability the uniform ultimate boundedness of the closed loop system is also proved by using the lyapunov approach moreover the obtained results are extended to solve decentralized optimal control problem of continuous time nonlinear interconnected large scale systems finally two simulation examples are presented to illustrate the effectiveness of the established control scheme

policy iteration adaptive dynamic programming algorithm for discrete time nonlinear systems this paper is concerned with a new discrete time policy iteration adaptive dynamic programming adp method for solving the infinite horizon optimal control problem of nonlinear systems the idea is to use an iterative adp technique to obtain the iterative control law which optimizes the iterative performance index function the main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete time nonlinear systems for the first time it shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the hamilton jacobi bellman equation it is also proven that any of the iterative control laws can stabilize the nonlinear systems neural networks are used to approximate the performance index function and compute the optimal control law respectively for facilitating the implementation of the iterative adp algorithm where the convergence of the weight matrices is analyzed finally the numerical results and analysis are presented to illustrate the performance of the developed method

galerkin approximations of the generalized hamilton jacobi bellman equation in this paper we study the convergence of the galerkin approximation method applied to the generalized hamilton jacobi bellman ghjb equation over a compact set containing the origin the ghjb equation gives the cost of an arbitrary control law and can be used to improve the performance of this control the ghjb equation can also be used to successively approximate the hamilton jacobi bellman equation we state sufficient conditions that guarantee that the galerkin approximation converges to the solution of the ghjb equation and that the resulting approximate control is stabilizing on the same region as the initial control the method is demonstrated on a simple nonlinear system and is compared to a result obtained by using exact feedback linearization in conjunction with the lqr design method