nash social and centralized solutions to consensus problems via mean field control theory the purpose of this paper is to synthesize initial mean consensus behavior of a set of agents from the fundamental optimization principles of i stochastic dynamic games and ii optimal control in the stochastic dynamic game model each agent seeks to minimize its individual quadratic discounted cost function involving the mean of the states of all other agents in this formulation we derive the limiting infinite population mean field equation system and explicitly compute its unique solution the resulting mean field mf control strategies drive each agent to track the overall population s initial state distribution mean and by applying these decentralized strategies any finite population system reaches mean consensus asymptotically as time goes to infinity furthermore these control laws possess an en nash equilibrium property where en goes to zero as the population size n goes to infinity finally the analysis is extended to the case of random mean field couplings in the social cooperative formulation the basic objective is to minimize a social cost as the sum of the individual cost functions containing mean field coupling in this formulation we show that for any individual agent the decentralized mean field social mf social control strategy is the same as the mean field nash mf nash equilibrium strategy hence mf nash controls unashn mf social controls usocn on the other hand the solution to the centralized lqr optimal control formulation yields the standard consensus sc algorithm whenever the graph representing the corresponding topology of the network is completely connected cc hence cent lqr controls ucentn sc cc controls uscn moreover a system with centralized control laws reaches consensus on the initial state distribution mean as time and population size n go to infinity hence asymptotically in time mf nash controls unashn mf social controls usocn cent lqr controls ucent sc cc controls usc finally the analysis is extended to the long time average lta cost functions case

hilbert coefficients and the gorenstein property of the associated graded ring in this article we investigate the gorenstein property of the associated graded ring ga i of an ideal i in a gorenstein local ring a m of positive dimension we especially concentrate on the case where i is m primary assuming that the rees algebra of i is cohen macaulay we then give necessary and sufficient conditions for the gorensteiness of ga i in terms of the hilbert coefficients of i

exact results for the kuramoto model with a bimodal frequency distribution we analyze a large system of globally coupled phase oscillators whose natural frequencies are bimodally distributed the dynamics of this system has been the subject of long standing interest in 1984 kuramoto proposed several conjectures about its behavior ten years later crawford obtained the first analytical results by means of a local center manifold calculation nevertheless many questions have remained open especially about the possibility of global bifurcations here we derive the system s stability diagram for the special case where the bimodal distribution consists of two equally weighted lorentzians using an ansatz recently discovered by ott and antonsen we show that in this case the infinite dimensional problem reduces exactly to a flow in four dimensions depending on the parameters and initial conditions the long term dynamics evolves to one of three states incoherence where all the oscillators are desynchronized partial synchrony where a macroscopic group of phase locked oscillators coexists with a sea of desynchronized ones and a standing wave state where two counter rotating groups of phase locked oscillators emerge analytical results are presented for the bifurcation boundaries between these states similar results are also obtained for the case in which the bimodal distribution is given by the sum of two gaussians

on the linear quadratic gaussian nash game with one step delay observation sharing pattern this paper studies a two player nash dynamic discrete time linear quadratic game under the so called one step delay observation sharing pattern it is shown that under very weak assumptions the solution exists and is unique and linear in the information

wellposedness of mean field games with common noise under a weak monotonicity condition in this paper we consider mean field games in the presence of common noise relaxing the usual independence assumption of individual random noise we assume a simple linear model with terminal cost satisfying a convexity and a weak monotonicity property our main result is showing existence and uniqueness of a mean field game solution using the stochastic maximum principle the uniqueness is a result of a monotonicity property similar to that of lasry and lions we use the banach fixed point theorem to establish an existence over a small time duration and show that it can be extended over an arbitrary finite time duration

mean field games for large population multiagent systems with markov jump parameters in this paper distributed games for large population multiagent systems with random time varying parameters are investigated where the agents are coupled via their individual costs and the structure parameters are a family of independent markov chains with identical generators the cost function of each agent is a long run average tracking type functional with an unknown mean field coupling nonlinear term as reference signal to reduce the computational complexity the mean field approach is applied to construct distributed strategies the population statistics effect pse is used to approximate the average effect of all the agents and the distributed strategies are given through solving a markov jump tracking problem here the pse is a deterministic quantity and can be obtained by solving the stackelberg equilibrium of an auxiliary two player game it is shown that the closed loop system is uniformly stable and the distributed strategies are asymptotically optimal in the sense of nash equilibrium

mean field lqg control in leader follower stochastic multi agent systems likelihood ratio based adaptation we study large population leader follower stochastic multi agent systems where the agents have linear stochastic dynamics and are coupled via their quadratic cost functions the cost of each leader is based on a trade off between moving toward a certain reference trajectory which is unknown to the followers and staying near their own centroid on the other hand followers react by tracking a convex combination of their own centroid and the centroid of the leaders we approach this large population dynamic game problem by use of so called mean field mf linear quadratic gaussian lqg stochastic control theory in this model followers are adaptive in the sense that they use a likelihood ratio estimator on a sample population of the leaders trajectories to identify the member of a given finite class of models which is generating the reference trajectory of the leaders under appropriate conditions it is shown that the true reference trajectory model is identified by each follower in finite time with probability one as the leaders population goes to infinity furthermore we show that the resulting sets of mean field control laws for both leaders and adaptive followers possess an almost sure en nash equilibrium property for a system with population n where en goes to zero as n goes to infinity numerical experiments are presented illustrating the results

linear quadratic n person and mean field games with ergodic cost we consider stochastic differential games with n players linear gaussian dynamics in arbitrary state space dimension and long time average cost with quadratic running cost admissible controls

linear quadratic mean field games with a major player the multi scale approach this paper considers linear quadratic lq mean field games with a major player and analyzes an asymptotic solvability problem it starts with a large scale system of coupled dynamic programming equations and applies a re scaling technique introduced in huang and zhou 2018 and huang and zhou 2020 to derive a set of riccati equations in lower dimensions the solvability of which determines the necessary and sufficient condition for asymptotic solvability we next derive the mean field limit of the strategies and the value functions finally we show that the two decentralized strategies can be interpreted as the best responses of a major player and a representative minor player embedded in an infinite population which have the property of consistent mean field approximations

technical communique feedback nash equilibria in the scalar infinite horizon lq game in this note we consider linear stationary feedback nash equilibria of the scalar linear quadratic differential game the planning horizon considered is assumed to be infinite we present both necessary and sufficient conditions on the system parameters for the existence of a unique solution of the associated algebraic riccati equations are that stabilizes the closed loop system for the case of more solutions singleton valued refinements of the equilibrium concept are studied

large population lqg games involving a major player the nash certainty equivalence principle we consider linear quadratic gaussian lqg games with a major player and a large number of minor players the major player has a significant influence on others the minor players individually have negligible impact but they collectively contribute mean field coupling terms in the individual dynamics and costs to overcome the dimensionality difficulty and obtain decentralized strategies the so called nash certainty equivalence methodology is applied the control synthesis is preceded by a state space augmentation via a set of aggregate quantities giving the mean field approximation subsequently within the population limit the lqg game is decomposed into a family of limiting two player games as each is locally seen by a representative minor player next when solving these limiting two player games we impose certain interaction consistency conditions such that the aggregate quantities initially assumed coincide with the ones replicated by the closed loop of a large number of minor players this procedure leads to a set of decentralized strategies for the original lqg game which is an varepsilon nash equilibrium

mean field linear quadratic control uniform stabilization and social optimality abstract this paper is concerned with uniform stabilization and social optimality for general mean field linear quadratic control systems where subsystems are coupled via individual dynamics and costs and the state weight is not assumed with the definiteness condition for the finite horizon problem we first obtain a set of forward backward stochastic differential equations fbsdes from variational analysis and construct a feedback type control by decoupling the fbsdes for the infinite horizon problem by using solutions to two riccati equations we design a set of decentralized control laws which is further proved to be asymptotically social optimal some equivalent conditions are given for uniform stabilization of the systems in different cases respectively finally the proposed decentralized controls are compared to the asymptotic optimal strategies in previous works

a briancon skoda theorem for isolated singularities abstract letabe a normal cohen macaulay local ring which is a homomorphic image of a gorenstein local ring supposing thatais pseudo rational in the punctured spectrum and that there exists a proper birational morphismx spec asuch thatxis pseudo rational we show how one can find an integerk 0 such that i n k infor all idealsi aand integersn 0

discrete time mean field stochastic linear quadratic optimal control problems this paper firstly presents necessary and sufficient conditions for the solvability of discrete time mean field stochastic linear quadratic optimal control problems secondly the optimal control within a class of linear feedback controls is investigated using a matrix dynamical optimization method thirdly by introducing several sequences of bounded linear operators the problem is formulated as an operator stochastic linear quadratic optimal control problem by the kernel range decomposition representation of the expectation operator and its pseudo inverse the optimal control is derived using solutions to two algebraic riccati difference equations finally by completing the square the two riccati equations and the optimal control are also obtained

electric load model synthesis by diffusion approximation of a high order hybrid state stochastic system a statistical approach is used to model the dynamics of the electric demand of large aggregates of electric space heaters or air conditioners the importance of such loads is twofold first they account for a significant portion of power system dynamics following a power outage second because they are associated with energy storage they are often selected for load shedding within a load management program the derivation of the aggregate electrical dynamics is considered first for a homogeneous group of devices subsequently a perturbation analysis yields the dynamics for a nonhomogeneous group the homogeneons group aggregrate load model is a system of coupled ordinary and partial differential equations fokker planck equations it is obtained by writing evolution equations for the probability density of a hybrid state markov system used to model the switching dynamics of individual devices this result is new and could give a clue to the analysis of a broad class of hybrid state stochastic systems in turn this could provide a new impetus not only in the area of electric load modeling but other areas such as power system reliability and the design of relay control systems where stochastic hybrid state models occur frequently simulation results which illustrate the dynamical properties of the model are presented

assessing the quality of identified models through the asymptotic theory when is the result reliable in this paper the problem of estimating uncertainty regions for identified models is considered a typical approach in this context is to resort to the asymptotic theory of prediction error methods for system identification by means of which ellipsoidal uncertainty regions can be constructed for the uncertain parameters we show that the uncertainty regions worked out through the asymptotic theory can be unreliable in certain situations precisely characterized in the paper then we critically analyze the theoretical conditions for the validity of the asymptotic theory and prove that the asymptotic theory also applies under new assumptions which are less restrictive than the usually required ones thanks to this result we single out the classes of models among standard ones arx armax box jenkins etc where the asymptotic theory can be safely used in practical applications to assess the quality of the identified model these results are of interest in many applications including iterative controller design schemes

hierarchically accelerated dynamic programming for finite state machines a procedure called hierarchically accelerated dynamic programming hadp is presented which at the cost of a degree of suboptimality can significantly accelerate dynamic programming algorithms for discrete event systems modeled by finite state machines fsms the methodology is based upon the possibly iterated dynamical abstraction of a given fsm by state aggregation in order to generate a so called partition machine hierarchy necessary and sufficient conditions for the hadp procedure to generate globally optimal solutions are given as well as bounds on the degree of suboptimality of the method a group of examples called the broken manhattan grid problems is used to illustrate an implementation of hadp with two and three level hierarchies a set of open problems is described concerning the construction and selection of the partition machine abstractions and the improvement of the estimation of hadp suboptimality

use hausdorff metric to analyze convergence of parameter estimation in system identification abstract for a linear time invariant system model this paper analyzes the convergence of parameter estimations as the length of the input output data tends to infinity through the prediction error method it is known that the sequence of the criterion functions converges uniformly in the parameter with probability one as the data length tends to infinity the parameter estimation is represented by a set in general instead of by a single point on which the criterion function takes its minimum thus a mathematical feature of the convergence problem of parameter estimation is in that we are needed from the convergence of a sequence of functions to infer the convergence of the sequence of their sets of minimizing arguments the hausdorff metric is suggested to measure the distance between sets and then is used to discuss the convergence problem here according to the hausdorff metric the convergence of parameter estimation is not guaranteed in general a condition guaranteeing such convergence is given

asymptotically optimal decentralized control for large population stochastic multiagent systems the interaction of interest coupled decision makers and the uncertainty of individual behavior are prominent characteristics of multiagent systems mas how to break through the framework of conventional control theory which aims at single decision maker and single decision objective and to extend the methodology and tools in the stochastic adaptive control theory to analyze mas are of great significance in this paper a preliminary exploration is made in this direction and the decentralized control problem is considered for large population stochastic mas with coupled cost functions different from the deterministic discounted costs in the existing differential game models a time averaged stochastic cost function is adopted for each agent the decentralized control law is constructed based on the state aggregation method and tracking like quadratic optimal control by using probability limit theory the stability and optimality of the closed loop system are analyzed the main contributions of this paper include the following points 1 the concepts of asymptotic nash equilibrium in probability and almost surely respectively are introduced and the relationship between these concepts is illuminated which provide necessary tools for analyzing the optimality of the decentralized control laws 2 the closed loop system is shown to be almost surely uniformly stable and bounded independently of the number of agents n 3 the population state average psa is shown to converge to the infinite population mean ipm trajectory in the sense of both l2 norm and time average almost surely as n increases to infinity 4 the decentralized control law is designed and shown to be almost surely asymptotically optimal the cost of each agent based on local measurements converges to that based on global measurements almost surely as n increases to infinity

convex analysis for lqg systems with applications to major minor lqg mean field game systems abstract we develop a convex analysis approach for solving lqg optimal control problems and apply it to major minor mm lqg mean field game mfg systems the approach retrieves the best response strategies for the major agent and all minor agents that attain an nash equilibrium an important and distinctive advantage to this approach is that unlike the classical approach in the literature we are able to avoid imposing assumptions on the evolution of the mean field in particular this provides a tool for dealing with complex and non standard systems

linear quadratic mean field games asymptotic solvability and relation to the fixed point approach mean field game theory has been developed largely following two routes one of them called the direct approach starts by solving a large scale game and next derives a set of limiting equations as the population size tends to infinity the second route is to apply mean field approximations and formalize a fixed point problem by analyzing the best response of a representative player this paper addresses the connection and difference of the two approaches in a linear quadratic lq setting we first introduce an asymptotic solvability notion for the direct approach which means for all sufficiently large population sizes the corresponding game has a set of feedback nash strategies in addition to a mild regularity requirement we provide a necessary and sufficient condition for asymptotic solvability and show that in this case the solution converges to a mean field limit this is accomplished by developing a re scaling method to derive a low dimensional ordinary differential equation ode system where a non symmetric riccati ode has a central role we next compare with the fixed point approach which determines a two point boundary value tpbv problem and show that asymptotic solvability implies feasibility of the fixed point approach but the converse is not true we further address non uniqueness in the fixed point approach and examine the long time behavior of the non symmetric riccati ode in the asymptotic solvability problem

necessary and sufficient conditions for uniqueness of the minimum in prediction error identification the contribution of this paper is to establish computable necessary and sufficient conditions on the model structure and on the experiment under which the prediction error identification pei criterion has a unique global minimum we consider a broad class of rational model structures whose numerator and denominator are affine in the unknown parameter vector this class encompasses all classical model structures used in system identification the main results in this paper rely on the standard assumption that the system is in the model set while some intermediate results are valid even when this assumption does not hold in particular theorem 4 2 and lemma 6 1 this is achieved by first establishing necessary and sufficient conditions on the model structure and on the experiment under which a global minimum is isolated these conditions must hold locally at the global minimum a second contribution is to show that these conditions are equivalent to the nonsingularity of the information matrix at that minimum for open loop identification and with some additional constraints also for closed loop identification the nonsingularity of the information matrix is then also equivalent to the uniqueness of the global minimum

social optima in mean field lqg control centralized and decentralized strategies we study a class of linear quadratic gaussian lqg control problems with n decision makers where the basic objective is to minimize a social cost as the sum of n individual costs containing mean field coupling the exact socially optimal solution determining a particular pareto optimum requires centralized information for each agent and has high implementational complexity as an alternative we subsequently exploit a mean field structure in the centralized optimal control problem to develop decentralized cooperative optimization so that each agent only uses its own state and a function which may be computed offline the resulting set of strategies asymptotically achieves the social optimum as n a key feature in this scheme is to let each agent optimize a new cost as the sum of its own cost and another component capturing its social impact on all other agents we also discuss the relationship between the decentralized cooperative solution and the so called nash certainty equivalence based solution presented in previous work on mean field lqg games

short time solution to the master equation of a first order mean field game abstract the goal of this paper is to show existence of short time classical solutions to the so called master equation of first order mean field games which can be thought of as the limit of the corresponding master equation of a stochastic mean field game as the individual noises approach zero despite being the equation of an idealistic model its study is justified as a way of understanding mean field games in which the individual players randomness is negligible in this sense it can be compared to the study of ideal fluids we restrict ourselves to mean field games with smooth coefficients but do not impose any monotonicity conditions on the running and initial costs and we do not require convexity of the hamiltonian thus extending the result of gangbo and swiech to a considerably broader class of hamiltonians

backward mean field linear quadratic gaussian lqg games full and partial information this paper introduces the backward mean field mf linear quadratic gaussian lqg games for short bmflqg of weakly coupled stochastic large population system in contrast to the well studied forward mean field lqg games the individual state in our large population system follows the backward stochastic differential equation bsde whose terminal instead initial condition should be prescribed two classes of bmflqg games are discussed here and their decentralized strategies are derived through the consistency condition in the first class the individual agents of large population system are weakly coupled in their state dynamics and the full information can be accessible to all agents in the second class the coupling structure lies in the cost functional with only partial information structure in both classes the asymptotic near optimality property namely epsilon nash equilibrium of decentralized strategies are verified to this end some estimates to bsde are presented in the large population setting

a survey of nonsymmetric riccati equations abstract we survey recent and also older results on nonsymmetric matrix riccati differential equations and in the time invariant case on the corresponding algebraic riccati equations in particular we cite various applications connected with matrix riccati equations

asymptotic solutions to weakly coupled stochastic teams with nonclassical information the authors develop a new iterative approach toward the solution of a class of two agent dynamic stochastic teams with nonclassical information when the coupling between the agents is weak either through the state dynamics or through the information channel in each case the weak coupling is characterized in terms of a small perturbation parameter when this parameter value say in is set equal to zero the original fairly complex dynamic team with a nonclassical information pattern is decomposed into or converted to relatively simple stochastic control or team problems the solution of which makes up the zeroth order approximation in a function space to the team optical solution of the original problem the fact that the zeroth order solution approximates the optimal cost up to at least o in is shown it is also shown that approximations of all orders can be obtained by solving a sequence of stochastic control and or simpler team problems

an open loop stackelberg strategy for the linear quadratic mean field stochastic differential game this paper is concerned with the open loop linear quadratic lq stackelberg game of the mean field stochastic systems in finite horizon by means of two generalized differential riccati equations the follower first solves a mean field stochastic lq optimal control problem then the leader turns to solve an optimization problem for a linear mean field forward backward stochastic differential equation by introducing new state and costate variables we present a sufficient condition for the existence and uniqueness of the stackelberg strategy in terms of the solvability of some riccati equations and a convexity condition furthermore it is shown that the open loop stackelberg equilibrium admits a feedback representation involving the new state and its mean finally two examples are given to show the effectiveness of the proposed results

continuum and finite player noncooperative models of competition the anonymous interaction of large numbers of economic agents is a kind of noncooperative situation which is markedly different from small numbers strategic conflict the mathematical model of a nonatomic game or a game with a continuum of players has been introduced as a model for these many agent situations on the basis that its equilibria should closely approximate those of games with large finite numbers of players this paper contains a precise definition of what it means for a nonatomic game to be the limit of a sequence of finite player games and a theorem which states when the limit of equilibria of finite player games will be an equilibrium of the nonatomic limit game this is analogous to theorems prompted by edgeworth s conjecture in core theory it is derived from a general set of sufficient conditions for the graph of a noncooperative equilibrium correspondence to be closed

risk sensitive mean field games in this paper we study a class of risk sensitive mean field stochastic differential games we show that under appropriate regularity conditions the mean field value of the stochastic differential game with exponentiated integral cost functional coincides with the value function satisfying a hamilton jacobi bellman hjb equation with an additional quadratic term we provide an explicit solution of the mean field best response when the instantaneous cost functions are log quadratic and the state dynamics are affine in the control an equivalent mean field risk neutral problem is formulated and the corresponding mean field equilibria are characterized in terms of backward forward macroscopic mckean vlasov equations fokker planck kolmogorov equations and hjb equations we provide numerical examples on the mean field behavior to illustrate both linear and mckean vlasov dynamics

linear quadratic gaussian mixed games with continuum parametrized minor players we consider a mean field linear quadratic gaussian game with a major player and a large number of minor players parametrized by a continuum set the mean field generated by the minor players is approximated by a random process depending only on the initial state and the brownian motion of the major player and this leads to two limiting optimal control problems with random coefficients which are solved subject to a consistency requirement on the mean field approximation the set of decentralized strategies constructed from the limiting control problems has an varepsilon nash equilibrium property when applied to the large but finite population model

mean field games convergence of a finite difference method mean field type models describing the limiting behavior of stochastic differential games as the number of players tends to infty have been recently introduced by lasry and lions numerical methods for the approximation of the stationary and evolutive versions of such models have been proposed by the authors in previous works here convergence theorems for these methods are proved under various assumptions on the coupling operator

identification and the information matrix how to get just sufficiently rich in prediction error identification the information matrix plays a central role specifically when the system is in the model set the covariance matrix of the parameter estimates converges asymptotically up to a scaling factor to the inverse of the information matrix the existence of a finite covariance matrix thus depends on the positive definiteness of the information matrix and the rate of convergence of the parameter estimate depends on its size the information matrix is also the key tool in the solution of optimal experiment design procedures which have become a focus of recent attention introducing a geometric framework we provide a complete analysis for arbitrary model structures of the minimum degree of richness required to guarantee the nonsingularity of the information matrix we then particularize these results to all commonly used model structures both in open loop and in closed loop in a closed loop setup our results provide an unexpected and precisely quantifiable trade off between controller degree and required degree of external excitation

a singular perturbation approach to modeling and control of markov chains finite state continuous time markov processes with weak interactions are modeled as singularly perturbed systems aggregate states are obtained using a grouping algorithm two time scale expansions simplify cost equations and lead to decentralized optimization algorithms

open loop nash equilibrium in polynomial differential games via state dependent riccati equation this paper studies finite as well as infinite time horizon nonzero sum polynomial differential games in both cases we explore the so called state dependent riccati equations to find a set of strategies that guarantee an open loop nash equilibrium for this particular class of nonlinear games such a method presents advantages in simplicity of the design of equilibrium strategies and yields computationally effective solution algorithms we demonstrate that this solution leads the game to an e or quasi equilibrium and provide an upper bound for this e quantity the proposed solution is given as a set of n coupled polynomial riccati like state dependent differential equations where each equation includes a p linear form tensor representation for its polynomial part we provide an algorithm for finding the solution of the state dependent algebraic equation in the infinite time case based on a hamiltonian approach and give conditions on the existence of such stabilizing solutions for a third order polynomial a numerical example is presented to illustrate effectiveness of the approach

uplink power adjustment in wireless communication systems a stochastic control analysis this paper considers mobile to base station power control for lognormal fading channels in wireless communication systems within a centralized information stochastic optimal control framework under a bounded power rate of change constraint the stochastic control problem and its associated hamilton jacobi bellman hjb equation are analyzed by the viscosity solution method then the degenerate hjb equation is perturbed to admit a classical solution and a suboptimal control law is designed based on the perturbed hjb equation when a quadratic type cost is used without a bound constraint on the control the value function is a classical solution to the degenerate hjb equation and the feedback control is affine in the system power in addition in this case we develop approximate but highly scalable solutions to the hjb equation in terms of a local polynomial expansion of the exact solution when the channel parameters are not known a priori one can obtain on line estimates of the parameters and get adaptive versions of the control laws in numerical experiments with both of the above cost functions the following phenomenon is observed whenever the users have different initial conditions there is an initial convergence of the power levels to a common level and then subsequent approximately equal behavior which converges toward a stochastically varying optimum

large population cost coupled lqg problems with nonuniform agents individual mass behavior and decentralized varepsilon nash equilibria we consider linear quadratic gaussian lqg games in large population systems where the agents evolve according to nonuniform dynamics and are coupled via their individual costs a state aggregation technique is developed to obtain a set of decentralized control laws for the individuals which possesses an epsiv nash equilibrium property a stability property of the mass behavior is established and the effect of inaccurate population statistics on an isolated agent is also analyzed by variational techniques