on model reduction for quantum dynamics symmetries and invariant subspaces simulation of quantum dynamics is a grand challenge of computational physics in this work we investigate methods for reducing the demands of such simulation by identifying reduced order models for dynamics generated by parameterized quantum hamiltonians in particular we first formulate an algebraic condition that certifies the existence of invariant subspaces for a model defined by a parameterized hamiltonian and an initial state following this we develop and analyze two methods to explicitly construct a reduced order model if one exists in addition to general results characterizing invariant subspaces of arbitrary finite dimensional hamiltonians by exploiting properties of the generalized pauli group we develop practical tools to speed up simulation of dynamics generated by certain spin hamiltonians to illustrate the methods developed we apply them to several paradigmatic spin models

common eigenvectors and quasicommutativity of sets of simultaneously triangularizable matrices abstract a set of simultaneously triangularizable square matrices over an arbitrary field is considered if the matrices are also quasicommutative then they have a common eigenvector for every distinct set of corresponding eigenvalues conversely if the set of matrices has this common eigenvector property hereditarily i e for every set of corresponding blocks in every simultaneous block triangularization then the matrices are quasicommutative

reinforcement learning in continuous time and space this article presents a reinforcement learning framework for continuous time dynamical systems without a priori discretization of time state and action based on the hamilton jacobi bellman hjb equation for infinite horizon discounted reward problems we derive algorithms for estimating value functions and improving policies with the use of function approximators the processof value function estimation is formulated asthe minimization of a continuous time form of the temporal difference td error update methods based on backward euler approximation and exponential eligibility traces are derived and their correspondences with the conventional residual gradient td 0 and td lambda algorithms are shown for policy improvement two methods a continuous actor critic method and a value gradient based greedy policy are formulated as a special case of the latter a nonlinear feedback control law using the value gradient and the model of the input gain is derived the advantage updating a model free algorithm derived previously is also formulated in the hjbbased framework the performance of the proposed algorithms is first tested in a nonlinear control task of swinging a pendulum up with limited torque it is shown in the simulations that 1 the task is accomplished by the continuous actor critic method in a number of trials several times fewer than by the conventional discrete actor critic method 2 among the continuous policy update methods the value gradient based policy with a known or learned dynamic model performs several times better than the actor critic method and 3 a value function update using exponential eligibility traces is more efficient and stable than that based on euler approximation the algorithms are then tested in a higher dimensional task cartpole swing up this task is accomplished in several hundred trials using the value gradient based policy with a learned dynamic model

online adaptive optimal control for continuous time nonlinear systems with completely unknown dynamics an online adaptive optimal control is proposed for continuous time nonlinear systems with completely unknown dynamics which is achieved by developing a novel identifier critic based approximate dynamic programming algorithm with a dual neural network nn approximation structure first an adaptive nn identifier is designed to obviate the requirement of complete knowledge of system dynamics and a critic nn is employed to approximate the optimal value function then the optimal control law is computed based on the information from the identifier nn and the critic nn so that the actor nn is not needed in particular a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier nn and critic nn simultaneously which converge to small neighbourhoods around their ideal values the closed loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the lyapunov theory the proposed ada

operators on positive semidefinite inner product spaces abstract let u be a semiunitary space i e a complex vector space with scalar product given by a positive semidefinite hermitian form if a linear operator a u u is bounded i e a u c u for some c r and all u u then the subspace u 0 u u u u 0 is invariant and so a defines the linear operators a 0 u 0 u 0 and a 1 u u 0 u u 0 let a be an indecomposable bounded operator on u such that 0 u 0 u let be an eigenvalue of a 0 we prove that the algebraic multiplicity of in a 1 is not less than the geometric multiplicity of in a 0 and the geometric multiplicity of in a 1 is not less than the number of jordan blocks j t of each fixed size t t in the jordan canonical form of a 0 we give canonical forms of selfadjoint and isometric operators on u and of hermitian forms on u for an arbitrary system of semiunitary spaces and linear mappings on between them we give an algorithm that reduces their matrices to canonical form its special cases are belitskii s and littlewood s algorithms for systems of linear operators on vector spaces and unitary spaces respectively

band diagonal operators abstract we show that there are operators on a five dimensional hilbert space which are not tridiagonal and that there are compact operators and normal operators on separable infinite dimensional spaces which are not band diagonal

model based reinforcement learning for infinite horizon approximate optimal tracking this brief paper provides an approximate online adaptive solution to the infinite horizon optimal tracking problem for control affine continuous time nonlinear systems with unknown drift dynamics to relax the persistence of excitation condition model based reinforcement learning is implemented using a concurrent learning based system identifier to simulate experience by evaluating the bellman error over unexplored areas of the state space tracking of the desired trajectory and convergence of the developed policy to a neighborhood of the optimal policy are established via lyapunov based stability analysis simulation results demonstrate the effectiveness of the developed technique

linear spanning sets for matrix spaces abstract necessary and sufficient conditions are given on matrices a b and s having entries in some field f and suitable dimensions such that the linear span of the terms a i s b j over f is equal to the whole matrix space this result is then used to determine the cardinality of subsets of f a s f b when f is a finite field

on the congruence transformation of a pencil of real symmetric matrices to a pencil with identical characteristic polynomial abstract simultaneous nonorthogonal congruence transformations for pairs a b of 2 2 real symmetric matrices which leave the characteristic polynomial of the pencil x 1 a x 2 b invariant are studied

tridiagonal forms in low dimensions abstract pati showed that every 4 4 matrix is unitarily similar to a tridiagonal matrix we give a simple proof in addition we show that in an appropriate sense there are generically precisely 12 ways to do this when the real part is diagonal it is shown that the unitary can be chosen with the form u pd where d is diagonal and p is real orthogonal however even if both real and imaginary parts are real symmetric there may be no real orthogonal matrices which tridiagonalize it on the other hand if the matrix belongs to the lie algebra sp 4 c then it can be tridiagonalized by a unitary in the symplectic group sp 2 in dimension 5 or greater there are always rank three matrices which are not tridiagonalizable

h _ infty tracking control of completely unknown continuous time systems via off policy reinforcement learning this paper deals with the design of an h infty tracking controller for nonlinear continuous time systems with completely unknown dynamics a general bounded l 2 gain tracking problem with a discounted performance function is introduced for the h infty tracking a tracking hamilton jacobi isaac hji equation is then developed that gives a nash equilibrium solution to the associated min max optimization problem a rigorous analysis of bounded l 2 gain and stability of the control solution obtained by solving the tracking hji equation is provided an upper bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics an off policy reinforcement learning algorithm is used to learn the solution to the tracking hji equation online without requiring any knowledge of the system dynamics convergence of the proposed algorithm to the solution to the tracking hji equation is shown simulation examples are provided to verify the effectiveness of the proposed method

online actor critic algorithm to solve the continuous time infinite horizon optimal control problem in this paper we discuss an online algorithm based on policy iteration for learning the continuous time ct optimal control solution with infinite horizon cost for nonlinear systems with known dynamics that is the algorithm learns online in real time the solution to the optimal control design hj equation this method finds in real time suitable approximations of both the optimal cost and the optimal control policy while also guaranteeing closed loop stability we present an online adaptive algorithm implemented as an actor critic structure which involves simultaneous continuous time adaptation of both actor and critic neural networks we call this synchronous policy iteration a persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function novel tuning algorithms are given for both critic and actor networks with extra nonstandard terms in the actor tuning law being required to guarantee closed loop dynamical stability the convergence to the optimal controller is proven and the stability of the system is also guaranteed simulation examples show the effectiveness of the new algorithm

simultaneous similarity and triangularization of sets of 2 by 2 matrices abstract let a a 1 a n be a finite or infinite sequence of 2 2 matrices with entries in an integral domain we show that except in a very special case a is simultaneously triangularizable if and only if all pairs a j a k are triangularizable for 1 j k we also provide a simple numerical criterion for triangularization using constructive methods in invariant theory we define a map with the minimal number of invariants that distinguishes simultaneous similarity classes for non commutative sequences over a field of characteristic 2 we also describe canonical forms for sequences of 2 2 matrices over algebraically closed fields and give a method for finding sequences with a given set of invariants

recursive estimation in piecewise affine systems using parameter identifiers and concurrent learning abstractpiecewise affine systems constitute a popular framework for the approximation of non linear systems and the modelling of hybrid systems this paper addresses the recursive subsystem estimation in continuous time piecewise affine systems parameter identifiers are extended from continuous time state space models to piecewise linear and piecewise affine systems the convergence rate of the presented identifiers is improved further using concurrent learning which makes concurrent use of current and recorded measurements in concurrent learning assumptions on persistence of excitation are replaced by the less restrictive linear independence of the recorded data the introduction of memory however reduces the tracking ability of concurrent learning because errors in the recorded measurements prevent convergence to the true parameters in order to overcome this limitation an algorithm is proposed to detect and remove erroneous measurements at run time and thereby restore the tracking ability detai

simultaneous reduction of matrices in the presence of a nonderogatory matrix abstract some known results on the question when are two matrices simultaneously similar to matrices in upper triangular form are simplified in the case when one of the matrices is nonderogatory

robust identification based state derivative estimation for nonlinear systems a robust identification based state derivative estimation method for uncertain nonlinear systems is developed the identifier architecture consists of a recurrent multilayer dynamic neural network which approximates the system dynamics online and a continuous robust feedback robust integral of the sign of the error rise term which accounts for modeling errors and exogenous disturbances numerical simulations provide comparisons with existing robust derivative estimation methods including a high gain observer a 2 sliding mode robust exact differentiator and numerical differentiation methods such as backward difference and central difference

brief paper pseudospectral methods for solving infinite horizon optimal control problems an important aspect of numerically approximating the solution of an infinite horizon optimal control problem is the manner in which the horizon is treated generally an infinite horizon optimal control problem is approximated with a finite horizon problem in such cases regardless of the finite duration of the approximation the final time lies an infinite duration from the actual horizon at t in this paper we describe two new direct pseudospectral methods using legendre gauss lg and legendre gauss radau lgr collocation for solving infinite horizon optimal control problems numerically a smooth strictly monotonic transformation is used to map the infinite time domain t 0 onto a half open interval t 1 1 the resulting problem on the finite interval is transcribed to a nonlinear programming problem using collocation the proposed methods yield approximations to the state and the costate on the entire horizon including approximations at t these pseudospectral methods can be written equivalently in either a differential or an implicit integral form in numerical experiments the discrete solution exhibits exponential convergence as a function of the number of collocation points it is shown that the map f 1 1 0 can be tuned to improve the quality of the discrete approximation

geometry of the numerical range of matrices abstract some techniques for the study of the algebraic curve c a which generates the numerical range w a of an n n matrix a as its convex hull are developed these enable one to give an explicit point equation of c a and a formula for the curvature of c a at a boundary point of w a applied to the case of a nonnegative matrix a a simple relation is found between the curvature of the function a p 1 a a t p being the perron root at 1 2 and the curvature of w a at the perron root of 1 2 a a t a connection with 2 dimensional pencils of hermitian matrices is mentioned and a conjecture formulated

approximate optimal trajectory tracking for continuous time nonlinear systems adaptive dynamic programming has been investigated and used as a method to approximately solve optimal regulation problems however the extension of this technique to optimal tracking problems for continuous time nonlinear systems has remained a non trivial open problem the control development in this paper guarantees ultimately bounded tracking of a desired trajectory while also ensuring that the enacted controller approximates the optimal controller

a survey of canonical forms and invariants for unitary similarity abstract matrices a and b are said to be unitarily similar if u au b for some unitary matrix u this expository paper surveys results on canonical forms and invariants for unitary similarity the first half gives a detailed description of methods developed by several authors brenner littlewood mitchell mcrae radjavi serge chuk and benedetti and cragnolini using inductively defined reduction procedures to transform matrices to canonical form the matrix is partitioned and successive unitary similarities applied to reduce the submatrices to some nice form at each stage one refines the partition and restricts the set of permissible unitary similarities to those that preserve the already reduced blocks the process ends in a finite number of steps producing both the canonical form and the subgroup of the unitary group that preserves that form depending on the initial step various canonical forms may be defined the method can also be used to define canonical forms relative to certain subgroups of the unitary group and canonical forms for finite sets of matrices under simultaneous unitary similarity the remainder of the paper surveys results on unitary invariants and other topics related to unitary similarity such as the specht pearcy trace invariants the numerical range and unitary reducibility

complementary triangular forms of pairs of matrices realizations with prescribed main matrices and complete factorization of rational matrix functions abstract the problem considered is the following given two square matrices a and z when does there exist an invertible matrix s such that s 1 as is upper triangular and s 1 zs is lower triangular several sufficient conditions for such an s to exist are presented in some cases the conditions are also necessary attention is paid to the order in which the eigenvalues of a and z may appear on the diagonals of s 1 as and s 1 zs respectively there is an intimate connection with the notions of realization and minimal factorization from systems theory

hermitian pencils with a cubic minimal polynomial let a be an n n complex matrix and write a h ik where h and k are hermitian matrices we show that if the minimal polynomial of the pencil xh yk has degree 3 then there is a unitary matrix u such that u 1au is block diagonal with blocks of size 3 3 or smaller this is a special case of a conjecture made by kippenhahn in 1951

on a conjecture of kippenhahn about the characteristic polynomial of a pencil generated by two hermitian matrices i abstract let a be an n n matrix write a h ik where i 2 1 and h and k are hermitian let f x y z det zi xh yk we first show that a pair of matrices over an algebraically closed field which satisfy quadratic polynomials can be put into block upper triangular form with diagonal blocks of size 1 1 or 2 2 via a simultaneous similarity this is used to prove that if f x y z g x y z n 2 where g has degree 2 then for some unitary matrix u the matrix u au is the direct sum of n 2 copies of a 2 2 matrix a 1 where a 1 is determined up to unitary similarity by the polynomial g x y z we use the connection between f x y z and the numerical range of a to investigate the case where f x y z has the form z ax y r g x y z s where g x y z is irreducible of degree 2

numerical ranges of cube roots of the identity abstract the numerical range of a bounded linear operator t on a hilbert space h is defined to be the subset w t tv v v h v 1 of the complex plane for operators on a finite dimensional hilbert space it is known that if w t is a circular disk then the center of the disk must be a multiple eigenvalue of t in particular if t has minimal polynomial z 3 1 then w t cannot be a circular disk in this paper we show that this is no longer the case when h is infinite dimensional the collection of 3 3 matrices with three fold symmetry about the origin are also classified

nonexistence of limit cycles in two classes of predator prey systems abstract this paper deals with the question of nonexistence of limit cycles in two famous classes of predator prey systems we present some sufficient conditions for the nonexistence of limit cycles in these systems our results extend and improve the results presented by moghadas

least squares approximation by elements from matrix orbits achieved by gradient flows on compact lie groups let s a denote the orbit of a complex or real matrix a under a certain equivalence relation such as unitary similarity unitary equivalence unitary congruences etc efficient gradient flow algorithms are constructed to determine the best approximation of a given matrix a0 by the sum of matrices in s a1 s an in the sense of finding the euclidean least squares distance

common eigenvectors of two matrices abstract a computable criterion is given for two square matrices to possess a common eigenvector as well as a criterion for one matrix to have an eigenvector lying in a given subspace some applications are discussed

on tridiagonalization of matrices abstract we consider the question is every n n complex matrix unitarily similar to a tridiagonal one it is shown that the answer is negative if n 6 and is affirmative if n 3 additionally some positive partial answers and related results are given for example 1 every pair of hermitian projections is simultaneously unitarily similar to a pair of tridiagonal matrices 2 if a a has a rank one then a is unitarily similar to a tridiagonal matrix

on unitary similarity of matrices abstract we obtain a set of polynomials in a a a an n n matrix whose traces completely characterize the unitary similarity classes of a generic set of matrices the number of invariants required is an order of magnitude smaller than earlier examples of such sets of invariants

moving zeros among matrices abstract we investigate the zero patterns that can be created by unitary similarity in a given matrix and the zero patterns that can be created by simultaneous unitary similarity in a given sequence of matrices the latter framework allows a simultaneous hessenberg formulation of pati s tridiagonal result for 4 4 matrices this formulation appears to be a strengthening of pati s theorem our work depends at several points on the simplified proof of pati s result by davidson and djokovic the hessenberg approach allows us to work with ordinary similarity and suggests an extension from the complex to arbitrary algebraically closed fields this extension is achieved and related results for 5 5 and larger matrices are formulated and proved

a counterexample to kippenhahn s conjecture on hermitian pencils abstract a counterexample is constructed to a conjecture of kippenhahn math nachr 6 193 288 1951 52 a pair of hermitian 8 8 matrices h k is found such that 1 h k generate m 8 c and 2 the minimal polynomial of the pencil xh yk has degree 4 recent work of h shapiro e g linear algebra appl 43 201 221 1982 has established the conjecture for n n matrices n 5

reinforcement learning for adaptive optimal control of unknown continuous time nonlinear systems with input constraints in this paper an adaptive reinforcement learning based solution is developed for the infinite horizon optimal control problem of constrained input continuous time nonlinear systems in the presence of nonlinearities with unknown structures two different types of neural networks nns are employed to approximate the hamilton jacobi bellman equation that is an recurrent nn is constructed to identify the unknown dynamical system and two feedforward nns are used as the actor and the critic to approximate the optimal control and the optimal cost respectively based on this framework the action nn and the critic nn are tuned simultaneously without the requirement for the knowledge of system drift dynamics moreover by using lyapunov s direct method the weights of the action nn and the critic nn are guaranteed to be uniformly ultimately bounded while keeping the closed loop system stable to demonstrate the effectiveness of the present approach simulation results are illustrated

concurrent learning for parameter estimation using dynamic state derivative estimators a concurrent learning cl based parameter estimator is developed to identify the unknown parameters in a nonlinear system unlike state of the art cl techniques that assume knowledge of the state derivative or rely on numerical smoothing cl is implemented using a dynamic state derivative estimator a novel purging algorithm is introduced to discard possibly erroneous data recorded during the transient phase for cl asymptotic convergence of the error states to the origin is established under a persistent excitation condition and the error states are shown to be uniformly ultimately bounded under a finite excitation condition

a new adaptive law for robust adaptation without persistent excitation a new adaptive law motivated by the work of loannou and kokotovic 1983 is proposed for the robust adaptive control of plants with unknown parameters in this adaptive law the output error plays a dual role in the adjustment of the control parameter vector the advantages of using the adaptive law over others proposed in the literature are discussed in the ideal case the adaptive system has bounded solutions in addition the origin of the error equations is exponentially stable when the reference input is persistently exciting and has a sufficiently large amplitude the adaptive system is also shown to be robust under bounded external disturbances finally it is shown that by suitably modifying the adaptive law the overall system can be made robust in the presence of a class of unmodeled dynamics of the plant simulation results are presented throughout the paper to complement the theoretical developments

unitarily achievable zero patterns and traces of words in a and a abstract our primary objective is to identify a natural and substantial problem about unitary similarity on arbitrary complex matrices which 0 patterns may be achieved for any given n by n complex matrix via some unitary similarity of it to this end certain restrictions on achievable 0 patterns are mentioned both positional and more important on the maximum number of achievable 0 s prior results fitting this general question are mentioned as well as the first unresolved pattern for 3 by 3 matrices in the process a recent question is answered a closely related additional objective is to mention the best known bound for the maximum length of words necessary for the application of specht s theorem about which pairs of complex matrices are unitarily similar which seems not widely known to matrix theorists in the process we mention the number of words necessary for small size matrices

neural network based solutions for stochastic optimal control using path integrals in this paper an offline approximate dynamic programming approach using neural networks is proposed for solving a class of finite horizon stochastic optimal control problems there are two approaches available in the literature one based on stochastic maximum principle smp formalism and the other based on solving the stochastic hamilton jacobi bellman hjb equation however in the presence of noise the smp formalism becomes complex and results in having to solve a couple of backward stochastic differential equations hence current solution methodologies typically ignore the noise effect on the other hand the inclusion of noise in the hjb framework is very straightforward furthermore the stochastic hjb equation of a control affine nonlinear stochastic system with a quadratic control cost function and an arbitrary state cost function can be formulated as a path integral pi problem however due to curse of dimensionality it might not be possible to utilize the pi formulation for obtaining comprehensive solutions over the entire operating domain a neural network structure called the adaptive critic design paradigm is used to effectively handle this difficulty in this paper a novel adaptive critic approach using the pi formulation is proposed for solving stochastic optimal control problems the potential of the algorithm is demonstrated through simulation results from a couple of benchmark problems

canonical matrices for linear matrix problems abstract we consider a large class of matrix problems which includes the problem of classifying arbitrary systems of linear mappings for every matrix problem from this class we construct belitskii s algorithm for reducing a matrix to a canonical form which is the generalization of the jordan normal form and study the set c mn of indecomposable canonical m n matrices considering c mn as a subset in the affine space of m n matrices we prove that either c mn consists of a finite number of points and straight lines for every m n or c mn contains a 2 dimensional plane for a certain m n

an upper bound for the length of a finite dimensional algebra abstract letfbe a field and letabe a finite dimensionalf algebra writed dimf a and letebe the largest degree of the minimal polynomial for anya a define the function f d e e 2d e 1 1 4 e 2 2 we prove that ifsis any finite generating set foraas anf algebra the words insof length less thanf d e spanaas anf vector space in the special case ofn by nmatrices this bound becomes f n 2 n n 2n 2 n 1 1 4 n 2 2 o n 3 2 this is a substantial improvement over previous bounds which have all beeno n2 we also prove that for particular setssof matrices the bound can be sharpened to one that is linear inn as an application of these results we reprove a theorem of small stafford and warfield about semiprime affinef algebras

multi player non zero sum games online adaptive learning solution of coupled hamilton jacobi equations in this paper we present an online adaptive control algorithm based on policy iteration reinforcement learning techniques to solve the continuous time ct multi player non zero sum nzs game with infinite horizon for linear and nonlinear systems nzs games allow for players to have a cooperative team component and an individual selfish component of strategy the adaptive algorithm learns online the solution of coupled riccati equations and coupled hamilton jacobi equations for linear and nonlinear systems respectively this adaptive control method finds in real time approximations of the optimal value and the nzs nash equilibrium while also guaranteeing closed loop stability the optimal adaptive algorithm is implemented as a separate actor critic parametric network approximator structure for every player and involves simultaneous continuous time adaptation of the actor critic networks a persistence of excitation condition is shown to guarantee convergence of every critic to the actual optimal value function for that player a detailed mathematical analysis is done for 2 player nzs games novel tuning algorithms are given for the actor critic networks the convergence to the nash equilibrium is proven and stability of the system is also guaranteed this provides optimal adaptive control solutions for both non zero sum games and their special case the zero sum games simulation examples show the effectiveness of the new algorithm

finite horizon control constrained nonlinear optimal control using single network adaptive critics to synthesize fixed final time control constrained optimal controllers for discrete time nonlinear control affine systems a single neural network nn based controller called the finite horizon single network adaptive critic is developed in this paper inputs to the nn are the current system states and the time to go and the network outputs are the costates that are used to compute optimal feedback control control constraints are handled through a nonquadratic cost function convergence proofs of 1 the reinforcement learning based training method to the optimal solution 2 the training error and 3 the network weights are provided the resulting controller is shown to solve the associated time varying hamilton jacobi bellman equation and provide the fixed final time optimal solution performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite time attitude maneuver subject to control bounds the new formulation has great potential for implementation since it consists of only one nn with single set of weights and it provides comprehensive feedback solutions online though it is trained offline

a novel actor critic identifier architecture for approximate optimal control of uncertain nonlinear systems an online adaptive reinforcement learning based solution is developed for the infinite horizon optimal control problem for continuous time uncertain nonlinear systems a novel actor critic identifier aci is proposed to approximate the hamilton jacobi bellman equation using three neural network nn structures actor and critic nns approximate the optimal control and the optimal value function respectively and a robust dynamic neural network identifier asymptotically approximates the uncertain system dynamics an advantage of using the aci architecture is that learning by the actor critic and identifier is continuous and simultaneous without requiring knowledge of system drift dynamics convergence of the algorithm is analyzed using lyapunov based adaptive control methods a persistence of excitation condition is required to guarantee exponential convergence to a bounded region in the neighborhood of the optimal control and uniformly ultimately bounded uub stability of the closed loop system simulation results demonstrate the performance of the actor critic identifier method for approximate optimal control

adaptive critic based optimal neuro control synthesis for distributed parameter systems a neural network based optimal control synthesis approach is presented for systems modeled by partial differential equations the problem is formulated via discrete dynamic programming and the necessary conditions of optimality are derived for synthesis of the controller we propose two sets of neural networks the set of action networks captures the mapping between the state and control while the set of critic networks captures the mapping between the state and costate we illustrate the solution process with a parabolic equation involving a nonlinear term for comparison we consider the linear quadratic regulator problem for the diffusion equation for which the ricatti operator based solution is known results show that this adaptive critic based systematic approach holds promise for obtaining the optimal control design of both linear and nonlinear distributed parameter systems

positive periodic solutions in delayed gause type predator prey systems abstract by using a continuation theorem based on coincidence degree theory we establish easily verifiable criteria for the existence of positive periodic solutions in delayed gause type predator prey systems some known results are shown to be special cases of the presented paper

specht s criterion for systems of linear mappings abstract w specht 1940 proved that two n n complex matrices a and b are unitarily similar if and only if trace w a a trace w b b for every word w x y in two noncommuting variables we extend his criterion and its generalizations by n a wiegmann 1961 and n jing 2015 to an arbitrary system a consisting of complex or real inner product spaces and linear mappings among them we represent such a system by the directed graph q a whose vertices are inner product spaces and arrows are linear mappings denote by q a the directed graph obtained by enlarging to q a the adjoint linear mappings we prove that a system a is transformed by isometries of its spaces to a system b if and only if the traces of all closed directed walks in q a and q b coincide

the intersection of the similarity and conjunctivity equivalence classes abstract let a be an n n complex matrix let sim a denote the similarity equivalence class of a let conj a denote the conjunctivity equivalence class of a and let u a denote the unitary similarity equivalence class of a define cs a sim a conj a we seek to classify the matrices that have cs a u a and show by an example that this is not true in general but we show that it is true when a is hermitian or is a scalar multiple of a hermitian for n 3 we reduce the general n n case to the case when a is non singular and not a multiple of a hermitian matrix we completely classify the 2 2 case and find that cs a u a when a is non singular or normal or both

on actor critic algorithms in this article we propose and analyze a class of actor critic algorithms these are two time scale algorithms in which the critic uses temporal difference learning with a linearly parameterized approximation architecture and the actor is updated in an approximate gradient direction based on information provided by the critic we show that the features for the critic should ideally span a subspace prescribed by the choice of parameterization of the actor we study actor critic algorithms for markov decision processes with polish state and action spaces we state and prove two results regarding their convergence

simultaneous reduction of sets of matrices under similarity abstract let s be a set of n n matrices over a field f and a the algebra generated by s over f the problem of deciding whether the elements of s can be simultaneously reduced to block triangular form with the diagonal blocks of some specified size is considered and an account is given of various methods used to attack the problem most of the techniques use representation theory to obtain information on a the problems of simultaneous triangularization existence of common eigenvectors etc are also considered the aim of the paper is to survey the methods used to attack these problems and to give some typical results the paper does not contain many new results

global stability for a class of predator prey systems this paper deals with the question of global stability of the positive locally asymptotically stable equilibrium in a class of predator prey systems the dulac s criterion is applied and liapunov functions are constructed to establish the global stability

a conjectured property of hermitian pencils abstract if a b is any pair of hermitian matrices the power of dividing det i xa yb will be given by the number of basic singular summands in the pair contrary to conjecture this power can be greater than one even when the pair is unitarily irreducible

converse optimality for discrete time systems given an optimal value function for a discrete time infinite horizon optimal control problem how does one derive a nonlinear system dynamics model which satisfies this optimal value function this problem is called converse optimality and was successfully addressed in the continuous time case there is however no similar derivation in the discrete time case and this is the purpose of this paper a class of discrete time systems satisfying the given optimality condition is constructed and its properties are analyzed

on invertibility preserving linear mappings simultaneous triangularization and property l abstract let a be a unital matrix algebra a m n c a unital linear mapping and b the algebra generated by a the mapping is a homomorphism modulo the jacobson radical in b if and only if for k dim b dim a 3 the mapping id a m k c b m k c preserves invertibility this result is closely related to the question on the relationship between property l and simultaneous triangularization we introduce a property kl and give criteria in terms of conditions on matrix pencils with coefficients in m k c rather than in c which ensure simultaneous triangularization

the constrained bilinear form and the c numerical range abstract let v be an n dimentional unitary space with inner product and s the set x v x x 1 for any a hom v v and q c with q 1 we define w a q ax y x y s x y q if q 1 then w a q is just the classical numerical range ax x x s the convexity of which is well known another generalization of the numerical range is the c numerical range which is defined to be the set w c a tr cu au u unitary where c hom v v in this note we prove that w a q is always convex and that wc a is convex for all a if rank c 1 or n 2

model based reinforcement learning for approximate optimal regulation reinforcement learning rl based online approximate optimal control methods applied to deterministic systems typically require a restrictive persistence of excitation pe condition for convergence this paper develops a concurrent learning cl based implementation of model based rl to solve approximate optimal regulation problems online under a pe like rank condition the development is based on the observation that given a model of the system rl can be implemented by evaluating the bellman error at any number of desired points in the state space in this result a parametric system model is considered and a cl based parameter identifier is developed to compensate for uncertainty in the parameters uniformly ultimately bounded regulation of the system states to a neighborhood of the origin and convergence of the developed policy to a neighborhood of the optimal policy are established using a lyapunov based analysis and simulation results are presented to demonstrate the performance of the developed controller

matrices with some extremal properties abstract for a complex matrix a there are many inequalities related to its eigenvalues diagonal elements and three types of singular values we investigate the conditions on the matrices for which those inequalities become equalities it is shown that in many cases the conditions are both necessary and sufficient for the matrices to be unitarily similar to matrices of lower orders moreover we get alternative proofs of various characterization theorems of different classes of matrices

equality of higher numerical ranges of matrices and a conjecture of kippenhahn on hermitian pencils let mn be the algebra of all n n complex matrices for 1 k n the kth numerical range of a mn is defined by wk a 1 k jk 1xj axj x1 xk is an orthonormal set in n it is known that tr a n wn a wn 1 a w1 a we study the condition on a under which wm a wk a for some given 1 m k n it turns out that this study is closely related to a conjecture of kippenhahn on hermitian pencils a new class of counterexamples to the conjecture is constructed based on the theory of the numerical range

on the gau wu number for some classes of matrices abstract for a given n n matrix a let k a stand for the maximal number of orthonormal vectors x j such that the scalar products a x j x j lie on the boundary of the numerical range w a this number was recently introduced by gau and wu and we therefore call it the gau wu number of the matrix a we compute k a for two classes of n n matrices a a simple and explicit expression for k a for tridiagonal toeplitz matrices a is derived furthermore we prove that k a 2 for every pure almost normal matrix a note that for every matrix a we have k a 2 and for normal matrices a we have k a n so our results show that pure almost normal matrices are in fact as far from normal as possible with respect to the gau wu number finally matrices with maximal gau wu number k a n are considered

unitary and euclidean representations of a quiver abstract a unitary euclidean representation of a quiver is given by assigning to each vertex a unitary euclidean vector space and to each arrow a linear mapping of the corresponding vector spaces we recall an algorithm for reducing the matrices of a unitary representation to canonical form give a certain description of the representations of canonical form and reduce the problem of classifying euclidean representations to the problem of classifying unitary representations we also describe the set of dimensions of all indecomposable unitary euclidean representations of a quiver and establish the number of parameters in an indecomposable unitary representation of a given dimension

on robust parameter estimation in finite time without persistence of excitation the problem of adaptive estimation of constant parameters in the linear regressor model is studied without the hypothesis that regressor is persistently excited first the initial vector estimation problem is transformed to a series of the scalar ones using the method of dynamic regressor extension and mixing second several adaptive estimation algorithms are proposed for the scalar scenario in such a case if the regressor is nullified asymptotically or in finite time then the problem of estimation is also posed on a finite interval of time robustness of the proposed algorithms with respect to measurement noise and exogenous disturbances is analyzed the efficiency of the designed estimators is demonstrated in numeric experiments for academic examples

optimal tracking control of nonlinear partially unknown constrained input systems using integral reinforcement learning in this paper a new formulation for the optimal tracking control problem otcp of continuous time nonlinear systems is presented this formulation extends the integral reinforcement learning irl technique a method for solving optimal regulation problems to learn the solution to the otcp unlike existing solutions to the otcp the proposed method does not need to have or to identify knowledge of the system drift dynamics and it also takes into account the input constraints a priori an augmented system composed of the error system dynamics and the command generator dynamics is used to introduce a new nonquadratic discounted performance function for the otcp this encodes the input constrains into the optimization problem a tracking hamilton jacobi bellman hjb equation associated with this nonquadratic performance function is derived which gives the optimal control solution an online irl algorithm is presented to learn the solution to the tracking hjb equation without knowing the system drift dynamics convergence to a near optimal control solution and stability of the whole system are shown under a persistence of excitation condition simulation examples are provided to show the effectiveness of the proposed method

arveson s criterion for unitary similarity abstract this paper is an exposition of w b arveson s complete invariant for the unitary similarity of complex irreducible matrices

simultaneous reduction to triangular forms after extension with zeroes abstract this paper is concerned with pairs of m m matrices a z for which there exists an invertible m m matrix s such that s 1 as is upper triangular and s1zs is lower triangular such pairs are said to admit simultaneous reduction to complementary triangular forms in particular if such a similariy s does not exist for the pair a z tollowing question is taken into consideration does there exist a positive integer r such that the pair of m r m r matrices a o o o r z o o o r admits simultaneous reduction to complementary triangular forms so that the pair a z obtains the property after extension with zeroes it is shown that the answer to this question is mixed an example of a pair is given which obtains the property after extension with one zero this example is put into contrast with a large number of results that answers the question negatively i e that describes situations where the pair of extended matrices admits simultaneous reduction to complementary triangular forms if and only if the original pair does

aspects of nonnormality for iterative methods abstract recently new optimal krylov subspace methods have been discovered for normal matrices in light of this novel ways to quantify nonnormality are considered in connection with various families of matrices we use as a criterion how for a given matrix these iterative methods introduced can be employed via e g inexpensive matrix factorizations the unitary orbit of the set of binormal matrices provides a natural extension of normal matrices its elements yield polynomially normal matrices of moderate degree in this context several matrix nearness problems arise

multi agent differential graphical games online adaptive learning solution for synchronization with optimality multi agent systems arise in several domains of engineering and can be used to solve problems which are difficult for an individual agent to solve strategies for team decision problems including optimal control n player games h infinity control and non zero sum and so on are normally solved for off line by solving associated matrix equations such as the coupled riccati equations or coupled hamilton jacobi equations however using that approach players cannot change their objectives online in real time without calling for a completely new off line solution for the new strategies therefore in this paper we bring together cooperative control reinforcement learning and game theory to present a multi agent formulation for the online solution of team games the notion of graphical games is developed for dynamical systems where the dynamics and performance indices for each node depend only on local neighbor information it is shown that standard definitions for nash equilibrium are not sufficient for graphical games and a new definition of interactive nash equilibrium is given we give a cooperative policy iteration algorithm for graphical games that converges to the best response when the neighbors of each agent do not update their policies and to the cooperative nash equilibrium when all agents update their policies simultaneously this is used to develop methods for online adaptive learning solutions of graphical games in real time along with proofs of stability and convergence

a predator prey model with ivlev s functional response abstract the uniqueness of limit cycles is proved for a two dimensional predator prey system with a functional response of ivlev type the system of planar autonomous ode s is transformed to a lienard system to which a modified theorem of zhang is applied

algebras generated by two idempotents abstract it is shown that a noncommutative simple algebra generated over a field f by two idempotents is necessarily the ring of 2 2 matrices over a simple extension of f and that every matrix ring over a field k can be generated over k by three idempotents

pairs of matrices with quadratic minimal polynomials abstract let a b be n n matrices over a field f and suppose a b have quadratic minimal polynomials then the algebra generated by a b has dimension at most 2n 1 if n is odd and 2n if n is even these bounds are exact

a structure theorem for some matrix algebras abstract let f be a field and m n f the algebra of n n matrices over f it is in general a very difficult and tedious problem to determine the structure of the subalgebra x of m n f generated by a given subset s of m n f we show that if x contains a matrix a with n distinct eigenvalues in f then x can be determined up to similarity very quickly by a graph theoretic method as a consequence we show that any such x can be generated by a pair of elements one of which can be taken to be a and the other a 0 1 matrix as an application we obtain a specht type similarity theorem

irredundant generating sets for matrix algebras abstract we prove the following main theorem let s be an irredundant generating set for the algebra m n f of n n matrices over a field f then the cardinality of s is at most 3 if n 2 and at most 2 n 2 if n 2

numerical shadows measures and densities on the numerical range abstract for any operator m acting on an n dimensional hilbert space h n we introduce its numerical shadow which is a probability measure on the complex plane supported by the numerical range of m the shadow of m at point z is defined as the probability that the inner product mu u is equal to z where u stands for a random complex vector from h n satisfying u 1 in the case of n 2 the numerical shadow of a non normal operator can be interpreted as a shadow of a hollow sphere projected on a plane a similar interpretation is provided also for higher dimensions for a hermitian m its numerical shadow forms a probability distribution on the real axis which is shown to be a one dimensional b spline in the case of a normal m the numerical shadow corresponds to a shadow of a transparent solid simplex in r n 1 onto the complex plane numerical shadow is found explicitly for jordan matrices j n direct sums of matrices and in all cases where the shadow is rotation invariant results concerning the moments of shadow measures play an important role a general technique to study numerical shadow via the cartesian decomposition is described and a link of the numerical shadow of an operator to its higher rank numerical range is emphasized

on the number of matrices to generate a matrix algebra over the real field abstract in this paper we discuss generating a algebra over the real field with a set of symmetric matrices this is motivated by an application in structural engineering we show that any algebra can be generated with at most four randomly chosen symmetric matrices the proof relies on the structure theorem for algebras and the notion of genericity in eigenvalue structure

a generalization of the motzkin taussky theorem abstract in this paper we generalize the motzkin taussky theorem to matrices with polynomial entries

reinforcement q learning for optimal tracking control of linear discrete time systems with unknown dynamics abstract in this paper a novel approach based on the q learning algorithm is proposed to solve the infinite horizon linear quadratic tracker lqt for unknown discrete time systems in a causal manner it is assumed that the reference trajectory is generated by a linear command generator system an augmented system composed of the original system and the command generator is constructed and it is shown that the value function for the lqt is quadratic in terms of the state of the augmented system using the quadratic structure of the value function a bellman equation and an augmented algebraic riccati equation are for solving the lqt are derived in contrast to the standard solution of the lqt which requires the solution of an are and a noncausal difference equation simultaneously in the proposed method the optimal control input is obtained by only solving an augmented are a q learning algorithm is developed to solve online the augmented are without any knowledge about the system dynamics or the command generator convergence to the optimal solution is shown a simulation example is used to verify the effectiveness of the proposed control scheme

on the reduction of pairs of hermitian or symmetric matrices to diagonal form by congruence abstract let a 1 a 2 be given n by n hermitian or symmetric matrices and consider the simultaneous transformations a i sa i s if a i is hermitian or a i sa i s t if a i is symmetric we give necessary and sufficient conditions for the existence of a unitary s which reduces both a 1 and a 2 to diagonal form in this way when at least one of a 1 or a 2 is nonsingular we give necessary and sufficient conditions for a reduction of this sort with a nonsingular s these results are a generalization of the classical theorem from mechanics that a positive definite matrix and a hermitian matrix can always be diagonalized simultaneously by a nonsingular congruence

tridiagonal canonical matrices of bilinear or sesquilinear forms and of pairs of symmetric skew symmetric or hermitian forms abstract tridiagonal canonical forms of square matrices under congruence or congruence pairs of symmetric or skew symmetric matrices under congruence and pairs of hermitian matrices under congruence are given over an algebraically closed field of characteristic different from 2

efficient model based reinforcement learning for approximate online optimal control an infinite horizon optimal regulation problem is solved online for a deterministic control affine nonlinear dynamical system using a state following staf kernel method to approximate the value function unlike traditional methods that aim to approximate a function over a large compact set the staf kernel method aims to approximate a function in a small neighborhood of a state that travels within a compact set simulation results demonstrate that stability and approximate optimality of the control system can be achieved with significantly fewer basis functions than may be required for global approximation methods

a complete unitary similarity invariant for unicellular matrices abstract we present necessary and sufficient conditions for an n n complex matrix b to be unitarily similar to a fixed unicellular i e indecomposable by similarity n n complex matrix a

simultaneous similarity of matrices abstract in this paper we solve completely and explicitly the long standing problem of classifying pairs of n n complex matrices a b under the simultaneous similarity tat 1 tbt 1 roughly speaking the classification decomposes to a finite number of steps in each step we consider an open algebraic set m 0n 2 r mn mn mn the set of n n complex valued matrices here r and are two positive integers then we construct a finite number of rational functions o1 os in the entries of a and b whose values are constant on all pairs similar in m n 2 r to a b the values of the functions oi a b i 1 s determine a finite number at most n 2 r of similarity classes in m n 2 r let sn be the subspace of complex symmetric matrices in mn for a b sn sn we consider the similarity class tatt tbtt where t ranges over all complex orthogonal matrices then the characteristic polynomial i a xb determines a finite number of similarity classes for almost all pairs a b sn sn