linear quadratic mean field control the invariant subspace method this note studies linear quadratic lq mean field social optimization problems with non uniform agents and indefinite state weight and analyzes the resulting linear system of ordinary differential equations odes subject to a partial initial condition and a growth condition we apply a subspace decomposition method for existence analysis and computation this is accomplished by carefully choosing the unspecified initial condition in the ode system such that the modes exceeding the admissible growth rate are not activated we further extend this method to lq mean field games

idempotency of linear combinations of an idempotent matrix and a tripotent matrix abstract the problem of characterizing situations in which a linear combination c c 1 a c 2 b of an idempotent matrix a and a tripotent matrix b is an idempotent matrix is thoroughly studied in two particular cases of this problem when either b or b is an idempotent matrix a complete solution follows from the main result in linear algebra appl 321 2000 3 in the present paper a complete solution is established in all the remaining cases when b is an essentially tripotent matrix in the sense that both idempotent matrices b 1 and b 2 constituting its unique decomposition b b 1 b 2 are nonzero the problem is considered also under the additional assumption that the differences a b 1 and a b 2 are hermitian matrices this obviously covers the case when a b 1 and b 2 are hermitian themselves when the problem can be interpreted from a statistical point of view

robust mean field linear quadratic gaussian games with unknown l 2 disturbance this paper considers a class of mean field linear quadratic gaussian games with model uncertainty the drift term in the dynamics of the agents contains a common unknown function we take a robust

random dynamical systems rough paths and rough flows abstract we analyze common lifts of stochastic processes to rough paths rough drivers valued processes and give sufficient conditions for the cocycle property to hold for these lifts we show that random rough differential equations driven by such lifts induce random dynamical systems in particular our results imply that rough differential equations driven by the lift of fractional brownian motion in the sense of friz victoir 26 induce random dynamical systems

approximations of two dimensional mean field games with nonsymmetric controls abstract the numerical methods are presented for solving economic problems formulated in the mean field game mfg form the mean field equilibrium is a solution of the coupled system of two parabolic partial differential equations the fokker planck kolmogorov equation and the hamilton jacobi bellman one the description focuses on the discrete approximation of these equations and on the application of the mfg theory directly at discrete level this approach results in an efficient algorithm for finding the corresponding grid control functions contrary to other difference schemes here the semi lagrangian approximation is applied which improves properties of a discrete problem of this type this implies the fast convergence of an iterative algorithm for the minimization of the cost functional the constructed algorithms are implemented to the problem of carbon dioxide pollution

large population cost coupled lqg problems with nonuniform agents individual mass behavior and decentralized varepsilon nash equilibria we consider linear quadratic gaussian lqg games in large population systems where the agents evolve according to nonuniform dynamics and are coupled via their individual costs a state aggregation technique is developed to obtain a set of decentralized control laws for the individuals which possesses an epsiv nash equilibrium property a stability property of the mass behavior is established and the effect of inaccurate population statistics on an isolated agent is also analyzed by variational techniques

probabilistic analysis of mean field games the purpose of this paper is to provide a complete probabilistic analysis of a large class of stochastic differential games with mean field interactions we implement the mean field game strategy developed analytically by lasry and lions in a purely probabilistic framework relying on tailor made forms of the stochastic maximum principle while we assume that the state dynamics are affine in the states and the controls and the costs are convex our assumptions on the nature of the dependence of all the coefficients upon the statistical distribution of the states of the individual players remains of a rather general nature our probabilistic approach calls for the solution of systems of forward backward stochastic differential equations of a mckean vlasov type for which no existence result is known and for which we prove existence and regularity of the corresponding value function finally we prove that a solution of the mean field game problem as formulated by lasry and lions does indeed provide appr

nash social and centralized solutions to consensus problems via mean field control theory the purpose of this paper is to synthesize initial mean consensus behavior of a set of agents from the fundamental optimization principles of i stochastic dynamic games and ii optimal control in the stochastic dynamic game model each agent seeks to minimize its individual quadratic discounted cost function involving the mean of the states of all other agents in this formulation we derive the limiting infinite population mean field equation system and explicitly compute its unique solution the resulting mean field mf control strategies drive each agent to track the overall population s initial state distribution mean and by applying these decentralized strategies any finite population system reaches mean consensus asymptotically as time goes to infinity furthermore these control laws possess an en nash equilibrium property where en goes to zero as the population size n goes to infinity finally the analysis is extended to the case of random mean field couplings in the social cooperative formulation the basic objective is to minimize a social cost as the sum of the individual cost functions containing mean field coupling in this formulation we show that for any individual agent the decentralized mean field social mf social control strategy is the same as the mean field nash mf nash equilibrium strategy hence mf nash controls unashn mf social controls usocn on the other hand the solution to the centralized lqr optimal control formulation yields the standard consensus sc algorithm whenever the graph representing the corresponding topology of the network is completely connected cc hence cent lqr controls ucentn sc cc controls uscn moreover a system with centralized control laws reaches consensus on the initial state distribution mean as time and population size n go to infinity hence asymptotically in time mf nash controls unashn mf social controls usocn cent lqr controls ucent sc cc controls usc finally the analysis is extended to the long time average lta cost functions case

idempotency of linear combinations of three idempotent matrices two of which are commuting abstract the considerations of the present paper were inspired by baksalary o m baksalary idempotency of linear combinations of three idempotent matrices two of which are disjoint linear algebra appl 388 2004 67 78 who characterized all situations in which a linear combination p c 1 p 1 c 2 p 2 c 3 p 3 with c i i 1 2 3 being nonzero complex scalars and p i i 1 2 3 being nonzero complex idempotent matrices such that two of them p 1 and p 2 say are disjoint i e satisfy condition p 1 p 2 0 p 2 p 1 is an idempotent matrix in the present paper by utilizing different formalism than the one used by baksalary the results given in the above mentioned paper are generalized by weakening the assumption expressing the disjointness of p 1 and p 2 to the commutativity condition p 1 p 2 p 2 p 1

idempotency of linear combinations of an idempotent matrix and a t potent matrix that commute abstract this paper deals with idempotent matrices i e a 2 a and t potent matrices i e b t b when both matrices commute we derive a list of all complex numbers c 1 and c 2 such that c 1 a c 2 b is an idempotent matrix in addition the real case is also analyzed

indefinite mean field stochastic linear quadratic optimal control this paper is concerned with the discrete time indefinite mean field linear quadratic optimal control problem the so called mean field type stochastic control problems refer to the problem of incorporating the means of the state variables into the state equations and cost functionals such as the mean variance portfolio selection problems a dynamic optimization problem is called to be nonseparable in the sense of dynamic programming if it is not decomposable by a stage wise backward recursion the classical dynamic programming based optimal stochastic control methods would fail in such nonseparable situations as the principle of optimality no longer applies in this paper we show that both the well posedness and the solvability of the indefinite mean field linear quadratic problem are equivalent to the solvability of two coupled constrained generalized difference riccati equations and a constrained linear recursive equation we characterize the optimal control set completely and obtain a set of necessary and sufficient conditions on the mean variance portfolio selection problem the results established in this paper offer a more accurate solution scheme in tackling directly the issue of nonseparability and deriving the optimal policies analytically for the mean variance type portfolio selection problems

on the linear quadratic gaussian nash game with one step delay observation sharing pattern this paper studies a two player nash dynamic discrete time linear quadratic game under the so called one step delay observation sharing pattern it is shown that under very weak assumptions the solution exists and is unique and linear in the information

wellposedness of mean field games with common noise under a weak monotonicity condition in this paper we consider mean field games in the presence of common noise relaxing the usual independence assumption of individual random noise we assume a simple linear model with terminal cost satisfying a convexity and a weak monotonicity property our main result is showing existence and uniqueness of a mean field game solution using the stochastic maximum principle the uniqueness is a result of a monotonicity property similar to that of lasry and lions we use the banach fixed point theorem to establish an existence over a small time duration and show that it can be extended over an arbitrary finite time duration

linear quadratic risk sensitive and robust mean field games this paper considers two classes of large population stochastic differential games connected to optimal and robust decentralized control of large scale multiagent systems the first problem p1 is one where each agent minimizes an exponentiated cost function capturing risk sensitive behavior whereas in the second problem p2 each agent minimizes a worst case risk neutral cost function where the worst case stems from the presence of an adversary entering each agent s dynamics characterized by a stochastic differential equation in both problems the individual agents are coupled through the mean field term included in each agent s cost function which captures the average or mass behavior of the agents we solve both p1 and p2 via mean field game theory specifically we first solve a generic risk sensitive optimal control problem and a generic stochastic zero sum differential game where the corresponding optimal controllers are applied by each agent to construct the mean field systems of p1 and p2 we then characterize an approximated mass behavior effect on an individual agent via a fixed point analysis of the mean field system for each problem p1 and p2 we show that the approximated mass behavior is in fact the best estimate of the actual mass behavior in various senses as the population size n goes to infinity moreover we show that for finite n there exist epsilon nash equilibria for both p1 and p2 where the corresponding individual nash strategies are decentralized in terms of local state information and the approximated mass behavior we also show that epsilon can be taken to be arbitrarily small when n is sufficiently large we show that the epsilon nash equilibria of p1 and p2 are partially equivalent in the sense that the individual nash strategies share identical control laws but the approximated mass behaviors for p1 and p2 are different since in p2 the mass behavior is also affected by the associated worst case disturbance finally we prove that the nash equilibria for p1 and p2 both feature robustness and as the parameter characterizing this robustness becomes infinite the two nash equilibria become identical and equivalent to that of the risk neutral case as in the one agent risk sensitive and robust control theory

mean field lqg control in leader follower stochastic multi agent systems likelihood ratio based adaptation we study large population leader follower stochastic multi agent systems where the agents have linear stochastic dynamics and are coupled via their quadratic cost functions the cost of each leader is based on a trade off between moving toward a certain reference trajectory which is unknown to the followers and staying near their own centroid on the other hand followers react by tracking a convex combination of their own centroid and the centroid of the leaders we approach this large population dynamic game problem by use of so called mean field mf linear quadratic gaussian lqg stochastic control theory in this model followers are adaptive in the sense that they use a likelihood ratio estimator on a sample population of the leaders trajectories to identify the member of a given finite class of models which is generating the reference trajectory of the leaders under appropriate conditions it is shown that the true reference trajectory model is identified by each follower in finite time with probability one as the leaders population goes to infinity furthermore we show that the resulting sets of mean field control laws for both leaders and adaptive followers possess an almost sure en nash equilibrium property for a system with population n where en goes to zero as n goes to infinity numerical experiments are presented illustrating the results

linear quadratic n person and mean field games with ergodic cost we consider stochastic differential games with n players linear gaussian dynamics in arbitrary state space dimension and long time average cost with quadratic running cost admissible controls

linear quadratic mean field games with a major player the multi scale approach this paper considers linear quadratic lq mean field games with a major player and analyzes an asymptotic solvability problem it starts with a large scale system of coupled dynamic programming equations and applies a re scaling technique introduced in huang and zhou 2018 and huang and zhou 2020 to derive a set of riccati equations in lower dimensions the solvability of which determines the necessary and sufficient condition for asymptotic solvability we next derive the mean field limit of the strategies and the value functions finally we show that the two decentralized strategies can be interpreted as the best responses of a major player and a representative minor player embedded in an infinite population which have the property of consistent mean field approximations

nonlinear filtering theory for mckean vlasov type stochastic differential equations we consider estimation theory for partially observed stochastic dynamical systems whose state equations are given by mckean vlasov type stochastic differential equations and hence contain a measure term corresponding to the distribution of the solution of the state process nonlinear filtering equations are derived in this framework based upon the classification that the measure term is either stochastic or deterministic and that either the state or the measure term is estimated when the measure term is deterministic the standard theory holds further when the measure is random the induced functions in the dynamics of the state become random and a similar recursion for the optimal filter is obtained the joint estimation of state and the measure term is next considered the extended state in this setup is shown to be a banach space valued stochastic process with random functions in its state dynamics and a nonlinear filtering equation for this setup is provided the first step of such an analysis requ

levy area of wiener processes in banach spaces the goal of this paper is to construct canonical levy area processes for banach space valued brownian motions via dyadic approximations the significance of the existence of canonical levy area processes is that a stochastic integration theory can be established for such brownian motions in banach spaces existence of flows for stochastic differential equations with infinite dimensional noise then follows via the results of lyons and lyons and qian see e g system control and rough paths 2000 oxford univ press this investigation involves a careful analysis on the choice of tensor norms motivated by the applications to infinite dimensional stochastic differential equations

large population lqg games involving a major player the nash certainty equivalence principle we consider linear quadratic gaussian lqg games with a major player and a large number of minor players the major player has a significant influence on others the minor players individually have negligible impact but they collectively contribute mean field coupling terms in the individual dynamics and costs to overcome the dimensionality difficulty and obtain decentralized strategies the so called nash certainty equivalence methodology is applied the control synthesis is preceded by a state space augmentation via a set of aggregate quantities giving the mean field approximation subsequently within the population limit the lqg game is decomposed into a family of limiting two player games as each is locally seen by a representative minor player next when solving these limiting two player games we impose certain interaction consistency conditions such that the aggregate quantities initially assumed coincide with the ones replicated by the closed loop of a large number of minor players this procedure leads to a set of decentralized strategies for the original lqg game which is an varepsilon nash equilibrium

on the existence of classical solutions for stationary extended mean field games abstract in this paper we consider extended stationary mean field games that is mean field games which depend on the velocity field of the players we prove various a priori estimates which generalize the results for quasi variational mean field games in gomes et al 2012 in addition we use adjoint method techniques to obtain higher regularity bounds then we establish the existence of smooth solutions under fairly general conditions by applying the continuity method when applied to standard stationary mean field games as in lasry and lions 2006 gomes and sanchez morgado 2011 or gomes et al 2012 this paper yields various new estimates and regularity properties not available previously we discuss additionally several examples where the existence of classical solutions can be proved

on linear combinations of two tripotent idempotent and involutive matrices abstract let a c 1 a 1 c 2 a 2 where c 1 c 2 are nonzero complex numbers and a 1 a 2 is a pair of two n n nonzero matrices we consider the problem of characterizing all situations where a linear combination of the form a c 1 a 1 c 2 a 2 is i a tripotent or an involutive matrix when a 1 and a 2 are commuting involutive or commuting tripotent matrices respectively ii an idempotent matrix when a 1 and a 2 are involutive matrices and iii an involutive matrix when a 1 and a 2 are involutive or idempotent matrices

on some linear combinations of commuting involutive and idempotent matrices abstract we consider the problem of characterizing situations where a linear combination of commuting involutive matrices is a k potent matrix where k is a positive integer such that k 2 also we consider the problem of characterizing situations where a linear combination of the form c 1 i n c 2 a c 3 b is a nonsingular or an involutive matrix when idempotent matrices a b c n n satisfy different conditions and c 1 c 2 c 3 c 0

mean field linear quadratic gaussian lqg games for stochastic integral systems in this technical note we formulate and investigate a class of mean field linear quadratic gaussian lqg games for stochastic integral systems unlike other literature on mean field games where the individual states follow the controlled stochastic differential equations sdes the individual states in our large population system are characterized by a class of stochastic volterra type integral equations we obtain the nash certainty equivalence nce equation and hence derive the set of associated decentralized strategies the epsilon nash equilibrium properties are also verified due to the intrinsic integral structure the techniques and estimates applied here are significantly different from those existing results in mean field lqg games for stochastic differential systems for example some fredholm equation in the mean field setup is introduced for the first time as for applications two types of stochastic delayed systems are formulated as the special cases of our stochastic integral system and relevant mean field lqg games are discussed

decentralized tracking type games for multi agent systems with coupled arx models asymptotic nash equilibria a class of decentralized tracking type games is considered for large population multi agent systems mas the agents are described by stochastic discrete time auto regressive models with exogenous inputs arx models and coupled together through their individual dynamics and performance indexes by terms of the unknown population state average psa the performance index of each agent to minimize is a stochastic long term averaged group tracking type functional in which there is a nonlinear term of the unknown psa the control law is decentralized and implemented via the nash certainty equivalence principle by probability limit theory under mild conditions it is shown that a the estimate of the psa is strongly consistent b the closed loop system is stable almost surely and the stability is independent of the number n of agents c the decentralized control law is an asymptotic nash equilibrium almost surely or in probability according to the property of the nonlinear coupling function in the performance indexes

a class of interference induced games we consider a multi agent system with linear stochastic individual dynamics and individual linear quadratic ergodic cost functions the agents partially observe their own states their cost functions and initial statistics are a priori independent but they are coupled through an interference term the mean of all agent states entering each of their individual measurement equations while in general for a finite number of agents the resulting optimal control law may be a non linear function of the available observations we establish that for certain classes of cost and dynamic parameters optimal separated control laws obtained by ignoring the interference coupling are asymptotically optimal when the number of agents goes to infinity thus forming for finite n an nash equilibrium more generally though optimal separated control laws may not be asymptotically optimal and can in fact result in unstable overall behavior thus we consider a class of parameterized decentralized control laws whereby the separated kalman gain is treated as the arbitrary gain of a luenberger like observer system stability regions are characterized and the nature of optimal cooperative control policies within the considered class is explored numerical results and an application example for wireless communications are reported

mean field game theory with a partially observed major agent mean field game mfg theory where there is a major agent and many minor agents mm mfg has been formulated for both the linear quadratic gaussian lqg case and for the case of nonlinear state dynamics and nonlinear cost functions in this framework even asymptotically as the population size n approaches infinity and in contrast to the situation without major agents the mean field term becomes stochastic due to the stochastic evolution of the state of the major agent furthermore the best response control actions of the minor agents depend on the state of the major agent as well as the stochastic mean field in a decentralized environment one is led to consider the situation where the agents are provided only with partial information on the major agent s state in this work such a scenario is considered for systems with nonlinear dynamics and cost functions and an epsilon nash mfg theory is developed for this mm mfg setup the approach to the problem of partially observed mm mfg systems ado

mean field stochastic h 2 h control with delay the finite horizon h 2 h control problem of mean field type for delayed stochastic systems is considered in this paper firstly we derive a sufficient mean field generalised stochastic bounded

on a disjoint idempotent decomposition for linear combinations produced from n commutative tripotent matrices abstract it has been established a 3 n term disjoint idempotent decomposition did for the linear combinations produced from n 2 commutative tripotent matrices their products and their products of power 2 at most the results obtained in this way generalize those in y tian a disjoint idempotent decomposition for linear combinations produced from two commutative tripotent matrices and its applications linear multilinear algebra 59 2011 1237 1246 moreover an algorithm to get a did has been provided finally a numerical example has been given to exemplify the results

a note on linear combinations of commuting tripotent matrices abstract the purpose of this note is to characterize all situations in which a linear combination of two commuting tripotent matrices is also a tripotent matrix in the case of real scalars and real symmetric matrices this problem admits an interesting statistical interpretation namely it is equivalent to the question of when a linear combination of two quadratic forms in normal variables each distributed as a difference of two independent 2 variables is also distributed as such a difference

risk sensitive zero sum differential games we consider two player risk sensitive zero sum differential games rszsdgs in our problem setup both the drift term and the diffusion term in the controlled stochastic differential equation are dependent on the state and controls of both players and the objective functional is of the risk sensitive type first a stochastic maximum principle type necessary condition for an open loop saddle point of the rszsdg is established via nonlinear transformations of the adjoint processes of the equivalent risk neutral stochastic zero sum differential game in particular we obtain two variational inequalities namely the pair of saddle point inequalities of the rszsdg next we obtain the hamilton jacobi isaacs partial differential equation for the rszsdg which provides a sufficient condition for a feedback saddle point of the rszsdg using a logarithmic transformation of the associated value function finally we study the extended linear quadratic rszsdg lq rszsdg we show intractability of the extended lq rszsdg with the state and or controls of both players appearing in the diffusion term this unexpected intractability could lead to nonlinear open loop and feedback saddle points even if the problem itself is essentially lq and the isaacs condition holds

discrete time mean field stochastic linear quadratic optimal control problems this paper firstly presents necessary and sufficient conditions for the solvability of discrete time mean field stochastic linear quadratic optimal control problems secondly the optimal control within a class of linear feedback controls is investigated using a matrix dynamical optimization method thirdly by introducing several sequences of bounded linear operators the problem is formulated as an operator stochastic linear quadratic optimal control problem by the kernel range decomposition representation of the expectation operator and its pseudo inverse the optimal control is derived using solutions to two algebraic riccati difference equations finally by completing the square the two riccati equations and the optimal control are also obtained

mean field stochastic linear quadratic optimal control with markov jump parameters abstract this paper considers a class of mean field stochastic linear quadratic optimal control problems with markov jump parameters the new feature of these problems is that means of state and control are incorporated into the systems and the cost functional based on the modes of markov chain the corresponding decomposition technique of augmented state and control is introduced it is shown that under some appropriate conditions there exists a unique optimal control which can be explicitly given via solutions of two generalized difference riccati equations a numerical example sheds light on the theoretical results established

consensus conditions of multi agent systems with time varying topologies and stochastic communication noises this paper investigates the average consensus problem of first order discrete time multi agent networks in uncertain communication environments each agent can only use its own and neighbors information to design its control input to attenuate the communication noises a distributed stochastic approximation type protocol is used by using probability limit theory and algebraic graph theory consensus conditions for this kind of protocols are obtained a for the case of fixed topologies a necessary and sufficient condition for mean square average consensus is given which is also sufficient for almost sure consensus b for the case of time varying topologies sufficient conditions for mean square average consensus and almost sure consensus are given respectively especially if the network switches between jointly containing spanning tree instantaneously balanced graphs then the designed protocol can guarantee that each individual state converges both almost surely and in mean square to a common random variable whose expectation is right the average of the initial states of the whole system and whose variance describes the static maximum mean square error between each individual state and the average of the initial states of the whole system

electric load model synthesis by diffusion approximation of a high order hybrid state stochastic system a statistical approach is used to model the dynamics of the electric demand of large aggregates of electric space heaters or air conditioners the importance of such loads is twofold first they account for a significant portion of power system dynamics following a power outage second because they are associated with energy storage they are often selected for load shedding within a load management program the derivation of the aggregate electrical dynamics is considered first for a homogeneous group of devices subsequently a perturbation analysis yields the dynamics for a nonhomogeneous group the homogeneons group aggregrate load model is a system of coupled ordinary and partial differential equations fokker planck equations it is obtained by writing evolution equations for the probability density of a hybrid state markov system used to model the switching dynamics of individual devices this result is new and could give a clue to the analysis of a broad class of hybrid state stochastic systems in turn this could provide a new impetus not only in the area of electric load modeling but other areas such as power system reliability and the design of relay control systems where stochastic hybrid state models occur frequently simulation results which illustrate the dynamical properties of the model are presented

hierarchically accelerated dynamic programming for finite state machines a procedure called hierarchically accelerated dynamic programming hadp is presented which at the cost of a degree of suboptimality can significantly accelerate dynamic programming algorithms for discrete event systems modeled by finite state machines fsms the methodology is based upon the possibly iterated dynamical abstraction of a given fsm by state aggregation in order to generate a so called partition machine hierarchy necessary and sufficient conditions for the hadp procedure to generate globally optimal solutions are given as well as bounds on the degree of suboptimality of the method a group of examples called the broken manhattan grid problems is used to illustrate an implementation of hadp with two and three level hierarchies a set of open problems is described concerning the construction and selection of the partition machine abstractions and the improvement of the estimation of hadp suboptimality

first order stationary mean field games with congestion abstract mean field games mfgs are models for large populations of competing rational agents that seek to optimize a suitable functional in the case of congestion this functional takes into account the difficulty of moving in high density areas here we study stationary mfgs with congestion with quadratic or power like hamiltonians first using explicit examples we illustrate two main difficulties the lack of classical solutions and the existence of areas with vanishing densities our main contribution is a new variational formulation for mfgs with congestion with this formulation we prove the existence and uniqueness of solutions finally we consider applications to numerical methods

linear quadratic mean field stochastic zero sum differential games abstract in this paper we consider the two player linear quadratic mean field stochastic differential game lq mf szsdg where the expected values of state and players control variables are included in the corresponding mean field stochastic system and the objective functional we characterize the explicit condition under which the feedback saddle point equilibrium for the lq mf szsdg exists the results are obtained by using the completion of squares method and the ordered interchangeability property of multiple saddle points we show that the corresponding saddle point equilibrium is linear in state which is characterized by the two coupled riccati differential equations crdes we also discuss the solvability conditions of the crdes numerical results are presented to verify the theoretical results of the paper

asymptotically optimal decentralized control for large population stochastic multiagent systems the interaction of interest coupled decision makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems mas how to break through the framework of conventional control theory which aims at single decision maker and single decision objective and to extend the methodology and tools in the stochastic adaptive control theory to analyze mas are of great significance in this paper a preliminary exploration is made in this direction and the decentralized control problem is considered for large population stochastic mas with coupled cost functions different from the deterministic discounted costs in the existing differential game models a time averaged stochastic cost function is adopted for each agent the decentralized control law is constructed based on the state aggregation method and tracking like quadratic optimal control by using probability limit theory the stability and optimality of the closed loop system are analyzed the main contributions of this paper include the following points 1 the concepts of asymptotic nash equilibrium in probability and almost surely respectively are introduced and the relationship between these concepts is illuminated which provide necessary tools for analyzing the optimality of the decentralized control laws 2 the closed loop system is shown to be almost surely uniformly stable and bounded independently of the number of agents n 3 the population state average psa is shown to converge to the infinite population mean ipm trajectory in the sense of both l2 norm and time average almost surely as n increases to infinity 4 the decentralized control law is designed and shown to be almost surely asymptotically optimal the cost of each agent based on local measurements converges to that based on global measurements almost surely as n increases to infinity

convex analysis for lqg systems with applications to major minor lqg mean field game systems abstract we develop a convex analysis approach for solving lqg optimal control problems and apply it to major minor mm lqg mean field game mfg systems the approach retrieves the best response strategies for the major agent and all minor agents that attain an nash equilibrium an important and distinctive advantage to this approach is that unlike the classical approach in the literature we are able to avoid imposing assumptions on the evolution of the mean field in particular this provides a tool for dealing with complex and non standard systems

ϵ nash mean field games for general linear quadratic systems with applications this paper is concerned with general mean field mf linear quadratic lq games of stochastic large population system where the individual diffusion coefficient can depend on both the state and the control of the agent moreover the control weight in the cost functional could be indefinite the asymptotic suboptimality property namely nash equilibrium of the decentralized strategies for the lq games is derived through the consistency condition the impact of the population s collective behaviors and the consistency of the mean field estimation are illustrated by the numerical results a pricing problem is also studied for which the decentralized suboptimal price is obtained

weakly coupled mean field game systems abstract here we prove the existence of solutions to first order mean field games mfgs arising in optimal switching first we use the penalization method to construct approximate solutions then we prove uniform estimates for the penalized problem finally by a limiting procedure we obtain solutions to the mfg problem

on idempotency and tripotency of linear combinations of two commuting tripotent matrices abstract let t 1 and t 2 be two nonzero commuting n n tripotent matrices and c 1 c 2 two nonzero complex numbers necessary and sufficient conditions for the tripotency and the idempotency of c 1 t 1 c 2 t 2 are obtained the problems considered here have also statistical importance when c 1 c 2 are real scalars and t 1 t 2 are real symmetric matrices

social optima in mean field lqg control centralized and decentralized strategies we study a class of linear quadratic gaussian lqg control problems with n decision makers where the basic objective is to minimize a social cost as the sum of n individual costs containing mean field coupling the exact socially optimal solution determining a particular pareto optimum requires centralized information for each agent and has high implementational complexity as an alternative we subsequently exploit a mean field structure in the centralized optimal control problem to develop decentralized cooperative optimization so that each agent only uses its own state and a function which may be computed offline the resulting set of strategies asymptotically achieves the social optimum as n a key feature in this scheme is to let each agent optimize a new cost as the sum of its own cost and another component capturing its social impact on all other agents we also discuss the relationship between the decentralized cooperative solution and the so called nash certainty equivalence based solution presented in previous work on mean field lqg games

ϵ nash mean field games for linear quadratic systems with random jumps and applications this paper studies linear quadratic gaussian lqg games of stochastic large population system with jump diffusion processes the most distinguishing feature compared with the well studied mean fi

rough paths analysis of general banach space valued wiener processes abstract in this article we carry out a rough paths analysis for banach space valued wiener processes we show that most of the features of the classical wiener process pertain to its rough path analog to be more precise the enhanced process has the same scaling properties and it satisfies a fernique type theorem a support theorem and a large deviation principle in the same holder topologies as the classical wiener process does moreover the canonical rough paths of finite dimensional approximating wiener processes converge to the enhanced wiener process finally a new criterion for the existence of the enhanced wiener process is provided which is based on compact embeddings this criterion is particularly handy when analyzing kunita flows by means of rough paths analysis which is the topic of a forthcoming article

properties of the combinations of commutative idempotents this paper deals with idempotent operators i e p2 p and n potent operators i e pn p we characterize the invertibility the group invertibility and idempotency of the linear combinations of idempotent operators or n potent operators under certain commutativity property imposed on them

short time solution to the master equation of a first order mean field game abstract the goal of this paper is to show existence of short time classical solutions to the so called master equation of first order mean field games which can be thought of as the limit of the corresponding master equation of a stochastic mean field game as the individual noises approach zero despite being the equation of an idealistic model its study is justified as a way of understanding mean field games in which the individual players randomness is negligible in this sense it can be compared to the study of ideal fluids we restrict ourselves to mean field games with smooth coefficients but do not impose any monotonicity conditions on the running and initial costs and we do not require convexity of the hamiltonian thus extending the result of gangbo and swiech to a considerably broader class of hamiltonians

epsilon nash equilibria for partially observed lqg mean field games with a major player huang 2010 and nguyen and huang 2012 solved the linear quadratic mean field systems and control problem in the case where there is a major agent i e non asymptotically vanishing as the population size goes to infinity together with a population of minor agents i e individually asymptotically negligible the new feature in this case is that the mean field becomes stochastic and then by minor agent state extension the existence of epsilon nash equilibria together with the individual agents control laws that yield the equilibria may be established this paper presents results initially announced by caines and kizilkale 2013 2014 where it is shown that if the major agent s state is partially observed by the minor agents and if the major agent completely observes its own state all agents can recursively generate estimates in general individually distinct of the major agent s state and the mean field and thence generate feedback controls yielding epsilon nash equilibria

asymptotic expansions for the laplace approximations for ito functionals of brownian rough paths abstract in this paper we establish asymptotic expansions for the laplace approximations for ito functionals of brownian rough paths under the condition that the phase function has finitely many non degenerate minima our main tool is the banach space valued rough path theory of t lyons we use a large deviation principle and the stochastic taylor expansion with respect to the topology of the space of geometric rough paths this is a continuation of a series of papers by inahama y inahama laplace s method for the laws of heat processes on loop spaces j funct anal 232 2006 148 194 and by inahama and kawabi y inahama h kawabi large deviations for heat kernel measures on loop spaces via rough paths j london math soc 73 3 2006 797 816 y inahama h kawabi on asymptotics of certain banach space valued ito functionals of brownian rough paths in proceedings of the abel symposium 2005 stochastic analysis and applications a symposium in honor of kiyosi ito springer berlin in press available at http www abelprisen no no abelprisen deltagere 2005 html

synchronization of coupled oscillators is a game the purpose of this paper is to understand phase transition in noncooperative dynamic games with a large number of agents applications are found in neuroscience biology and economics as well as traditional engineering applications the focus of analysis is a variation of the large population linear quadratic gaussian lqg model of huang et al 2007 comprised here of a controlled n dimensional stochastic differential equation model coupled only through a cost function the states are interpreted as phase angles for a collection of heterogeneous oscillators and in this way the model may be regarded as an extension of the classical coupled oscillator model of kuramoto a deterministic pde model is proposed which is shown to approximate the stochastic system as the population size approaches infinity key to the analysis of the pde model is the existence of a particular nash equilibrium in which the agents opt out of the game setting their controls to zero resulting in the incoherence equilibrium methods from dynamical systems theory are used in a bifurcation analysis based on a linearization of the partial differential equation pde model about the incoherence equilibrium a critical value of the control cost parameter is identified above this value the oscillators are incoherent and below this value when control is sufficiently cheap the oscillators synchronize these conclusions are illustrated with results from numerical experiments

control of admission and routing in loss networks hybrid dynamic programming equations call admission and routing control decisions in stochastic loss circuit switched networks with semi markovian multi class call arrival and general connection time processes are formulated as optimal stochastic control problems the resulting so called hybrid dynamic programming equation systems take the form of vectors of partial differential equations with each component associated to a distinct distribution of routed calls over the network i e distinct occupation states this framework reduces to that of a markov decision process when the traffic is poisson and the associated computational limitations are approximately those of linear programs examples are provided of i network state space constructions and controlled state transition processes ii a new closed form solution for a simple network and iii the analysis and illustrative numerical results for a three link network

on idempotency of linear combinations of idempotent matrices p 1 p 2 and p 3 being any three different nonzero mutually commutative nxn idempotent matrices and c 1 c 2 and c 3 being nonzero scalars the problem of characterizing some situations where a linear combination of the form p c 1p 1 c 2p 2 or p c 1p 1 c 2p 2 c 3p 3 is also an idempotent matrix has been considered moreover two interesting results about the idempotency of linear combinations of 2x2 idempotent matrices and 3x3 idempotent matrices have been given a statistical interpretation of the idempotency problem considered in this study has also been pointed out

duality of metric entropy abstractfor two convex bodies kand t in r n the covering number of kbyt denoted n k t is de ned as the minimal number of translates oft needed to cover k let us denote by k the polar body of k andby d the euclidean unit ball in r n we prove that the two functionsof t n k td and n d tk are equivalent in the appropriate sense uniformly over symmetric convex bodies k r n and over n n inparticular this veri es the duality conjecture for entropy numbers of linearoperators posed by pietsch in 1972 in the central case when either thedomain or the range of the operator is a hilbert space 1 introduction for two convex bodies kand t in r n the covering number of kby t denoted n k t is de ned as the minimal number of translates of tneeded to cover kn k t min n x 1 x n r n k i n x i t we denote by dthe euclidean unit ball in r n in this paper we prove thefollowing duality result for covering numbers theorem 1 main theorem there exist two universal constants

an open loop stackelberg strategy for the linear quadratic mean field stochastic differential game this paper is concerned with the open loop linear quadratic lq stackelberg game of the mean field stochastic systems in finite horizon by means of two generalized differential riccati equations the follower first solves a mean field stochastic lq optimal control problem then the leader turns to solve an optimization problem for a linear mean field forward backward stochastic differential equation by introducing new state and costate variables we present a sufficient condition for the existence and uniqueness of the stackelberg strategy in terms of the solvability of some riccati equations and a convexity condition furthermore it is shown that the open loop stackelberg equilibrium admits a feedback representation involving the new state and its mean finally two examples are given to show the effectiveness of the proposed results

epsilon nash mean field game theory for nonlinear stochastic dynamical systems with major and minor agents this paper studies large population dynamic games involving nonlinear stochastic dynamical systems with agents of the following mixed types i a major agent and ii a population of n minor agents where n is very large the major and minor agents are coupled via both i their individual nonlinear stochastic dynamics and ii their individual finite time horizon nonlinear cost functions this problem is analyzed by the so called epsilon nash mean field game theory a distinct feature of the mixed agent mean field game problem is that even asymptotically as the population size n approaches infinity the noise process of the major agent causes random fluctuation of the mean field behavior of the minor agents to deal with this the overall asymptotic n rightarrow infty mean field game problem is decomposed into i two nonstandard stochastic optimal control problems with random coefficient processes which yield forward adapted stochastic best response control processes determined from the

continuum and finite player noncooperative models of competition the anonymous interaction of large numbers of economic agents is a kind of noncooperative situation which is markedly different from small numbers strategic conflict the mathematical model of a nonatomic game or a game with a continuum of players has been introduced as a model for these many agent situations on the basis that its equilibria should closely approximate those of games with large finite numbers of players this paper contains a precise definition of what it means for a nonatomic game to be the limit of a sequence of finite player games and a theorem which states when the limit of equilibria of finite player games will be an equilibrium of the nonatomic limit game this is analogous to theorems prompted by edgeworth s conjecture in core theory it is derived from a general set of sufficient conditions for the graph of a noncooperative equilibrium correspondence to be closed

asymptotic solutions to weakly coupled stochastic teams with nonclassical information the authors develop a new iterative approach toward the solution of a class of two agent dynamic stochastic teams with nonclassical information when the coupling between the agents is weak either through the state dynamics or through the information channel in each case the weak coupling is characterized in terms of a small perturbation parameter when this parameter value say in is set equal to zero the original fairly complex dynamic team with a nonclassical information pattern is decomposed into or converted to relatively simple stochastic control or team problems the solution of which makes up the zeroth order approximation in a function space to the team optical solution of the original problem the fact that the zeroth order solution approximates the optimal cost up to at least o in is shown it is also shown that approximations of all orders can be obtained by solving a sequence of stochastic control and or simpler team problems

a mean field game synthesis of initial mean consensus problems a continuum approach for non gaussian behavior this technical note presents a continuum approach to a non gaussian initial mean consensus problem via mean field mf stochastic control theory in this problem formulation i each agent has simple stochastic dynamics with inputs directly controlling its state s rate of change and ii each agent seeks to minimize by continuous state feedback its individual discounted cost function involving the mean of the states of all other agents for this dynamic game problem a set of coupled deterministic hamilton jacobi bellman and fokker planck kolmogorov equations is derived approximating the stochastic system of agents as the population size goes to infinity in a finite population system analogous to the mf lqg framework i the resulting decentralized mf control strategies possess an en nash equilibrium property where en goes to zero as the population size n approaches infinity and ii these mf control strategies steer each individual s state toward the initial state population mean which is reached asymptotically as time goes to infinity hence the system with decentralized mf control strategies reaches mean consensus on the initial state population mean asymptotically as time goes to infinity

a stochastic maximum principle for delayed mean field stochastic differential equations and its applications in this technical note we discuss the stochastic optimal control problems of mean field stochastic differential delayed equations mfsddes which arise naturally from various backgrounds including economics finance engineering and physics etc to this end some new estimates are used to handle the complex structure of our controlled system due to the presence of both delay and mean field characters as the main result a stochastic maximum principle for the mean field stochastic optimal control with delay mfsocd is derived in terms of necessary and sufficient conditions in particular applying the convex variation and duality relation we obtain the necessary condition for optimality see theorem 1 in addition the sufficient condition of the optimality is also obtained under some convex condition see theorem 2 based on our maximum principle the related mean field linear quadratic delayed mflqd optimal control problems are also investigated the optimal control is derived and its existence is also verified refer theorem 3 as illustration an example is also proposed and its explicit optimal control is derived

risk sensitive mean field games in this paper we study a class of risk sensitive mean field stochastic differential games we show that under appropriate regularity conditions the mean field value of the stochastic differential game with exponentiated integral cost functional coincides with the value function satisfying a hamilton jacobi bellman hjb equation with an additional quadratic term we provide an explicit solution of the mean field best response when the instantaneous cost functions are log quadratic and the state dynamics are affine in the control an equivalent mean field risk neutral problem is formulated and the corresponding mean field equilibria are characterized in terms of backward forward macroscopic mckean vlasov equations fokker planck kolmogorov equations and hjb equations we provide numerical examples on the mean field behavior to illustrate both linear and mckean vlasov dynamics

on invertibility of combinations of k potent operators abstract some properties of combinations c 1 p 1 c 2 p 2 c 3 p 1 s p 2 k 1 s where p 1 and p 2 are two different nonzero k potents c 1 c 2 and c 3 are three nonzero complex numbers and positive integers k 2 and s k 1 are obtained furthermore the invertibility the group invertibility and the k potency of the linear combinations of k potents are investigated under certain commutativity properties imposed on them

linear quadratic gaussian mixed games with continuum parametrized minor players we consider a mean field linear quadratic gaussian game with a major player and a large number of minor players parametrized by a continuum set the mean field generated by the minor players is approximated by a random process depending only on the initial state and the brownian motion of the major player and this leads to two limiting optimal control problems with random coefficients which are solved subject to a consistency requirement on the mean field approximation the set of decentralized strategies constructed from the limiting control problems has an varepsilon nash equilibrium property when applied to the large but finite population model

mean field games convergence of a finite difference method mean field type models describing the limiting behavior of stochastic differential games as the number of players tends to infty have been recently introduced by lasry and lions numerical methods for the approximation of the stationary and evolutive versions of such models have been proposed by the authors in previous works here convergence theorems for these methods are proved under various assumptions on the coupling operator

quadratic forms in random variables theory and applications textbook for a one semester graduate course for students specializing in mathematical statistics or in multivariate analysis or reference for theoretical as well as applied statisticians confines its discussion to quadratic forms and second degree polynomials in real normal random vectors and matr

tripotency of a linear combination of two involutory matrices and a tripotent matrix that mutually commute this paper concerns with the tripotency of a linear combination of three matrices which has a background in statistical theory we demonstrate all the possible cases that lead to the tripotency of a linear combination of two involutory matrices and a tripotent matrix that mutually commute by utilizing block technique and returning the partitioned matrices to their original forms we derive the sufficient and necessary conditions such that a linear combination of three mutually commuting involutory matrices is tripotent

idempotency of linear combinations of two idempotent matrices abstract a complete solution is established to the problem of characterizing all situations where a linear combination of two different idempotent matrices p 1 and p 2 is also an idempotent matrix including naturally three such situations known in the literature viz if the combination is either the sum p 1 p 2 or one of the differences p 1 p 2 p 2 p 1 and appropriate additional conditions are fulfilled the solution asserts that in the particular case where p 1 and p 2 are complex matrices such that p 1 p 2 is hermitian these three situations exhaust the list of all possibilities and that this list extends when the above assumption on p 1 and p 2 is violated a statistical interpretation of the idempotency problem considered in this note is also pointed out

a singular perturbation approach to modeling and control of markov chains finite state continuous time markov processes with weak interactions are modeled as singularly perturbed systems aggregate states are obtained using a grouping algorithm two time scale expansions simplify cost equations and lead to decentralized optimization algorithms

social optima in mean field linear quadratic gaussian models with markov jump parameters this paper investigates social optima of mean field linear quadratic gaussian lqg control models with markov jump parameters the common objective of the agents is to minimize a social cost the cost average of the whole society in the cost functions there are coupled mean field terms first we consider the centralized case and get a parameterized equation of mean field effect then we design a set of distributed strategies by solving a limiting optimal control problem in an augmented state space subject to the consistency requirement for mean field approximation it is shown that the set of distributed strategies is asymptotically team optimal and the asymptotically optimal social cost value can be obtained explicitly the optimal social average cost is compared with the optimal individual cost in mean field games by virtue of the explicit expressions and the difference is further illustrated by a numerical example

open loop nash equilibrium in polynomial differential games via state dependent riccati equation this paper studies finite as well as infinite time horizon nonzero sum polynomial differential games in both cases we explore the so called state dependent riccati equations to find a set of strategies that guarantee an open loop nash equilibrium for this particular class of nonlinear games such a method presents advantages in simplicity of the design of equilibrium strategies and yields computationally effective solution algorithms we demonstrate that this solution leads the game to an e or quasi equilibrium and provide an upper bound for this e quantity the proposed solution is given as a set of n coupled polynomial riccati like state dependent differential equations where each equation includes a p linear form tensor representation for its polynomial part we provide an algorithm for finding the solution of the state dependent algebraic equation in the infinite time case based on a hamiltonian approach and give conditions on the existence of such stabilizing solutions for a third order polynomial a numerical example is presented to illustrate effectiveness of the approach

uplink power adjustment in wireless communication systems a stochastic control analysis this paper considers mobile to base station power control for lognormal fading channels in wireless communication systems within a centralized information stochastic optimal control framework under a bounded power rate of change constraint the stochastic control problem and its associated hamilton jacobi bellman hjb equation are analyzed by the viscosity solution method then the degenerate hjb equation is perturbed to admit a classical solution and a suboptimal control law is designed based on the perturbed hjb equation when a quadratic type cost is used without a bound constraint on the control the value function is a classical solution to the degenerate hjb equation and the feedback control is affine in the system power in addition in this case we develop approximate but highly scalable solutions to the hjb equation in terms of a local polynomial expansion of the exact solution when the channel parameters are not known a priori one can obtain on line estimates of the parameters and get adaptive versions of the control laws in numerical experiments with both of the above cost functions the following phenomenon is observed whenever the users have different initial conditions there is an initial convergence of the power levels to a common level and then subsequent approximately equal behavior which converges toward a stochastically varying optimum

adaptive tracking games for coupled stochastic linear multi agent systems stability optimality and robustness distributed adaptive tracking type games are investigated for a class of coupled stochastic linear multi agent systems with uncertainties of unknown structure parameters external stochastic disturbances unmodeled dynamics and unknown agents interactions the control goal is to make the states of all the agents converge to a desired function of the population state average psa due to the fact that only local information is available for each agent the control is distributed for the time invariant parameter case the extended least squares algorithm nash certainty equivalence nce principle and certainty equivalence ce principle are used to estimate the unknown parameters and the psa term and to design adaptive control respectively under some mild conditions it is shown that the closed loop system is almost surely uniformly stable with respect to the population number n the estimate for the psa term is strongly consistent the adaptive control is almost surely an asymptotic nash equilibrium when the dynamics of each agent contains time varying parameters and unmodeled dynamics the projected least mean square lms algorithm nce principle and ce principle are adopted to estimate the unknown time varying parameters and the unknown psa term and to design robust adaptive control respectively in addition to stability of the closed loop system and consistency of the psa estimate the control law is shown to be robust nash equilibrium with respect to the unmodeled dynamics the variation of the unknown parameters and the external disturbances two numerical examples are given to illustrate the methods and results of this paper

on the convergence of finite state mean field games through γ convergence abstract in this study we consider the long term convergence trend toward an equilibrium of finite state mean field games using convergence our techniques are based on the observation that an important class of mean field games can be viewed as the euler lagrange equation of a suitable functional therefore using a scaling argument one can convert a long term convergence problem into a convergence problem our results generalize previous results related to long term convergence for finite state problems