the minimum rank of symmetric matrices described by a graph a survey abstract the minimum rank of a simple graph g is defined to be the smallest possible rank over all symmetric real matrices whose ijth entry for i j is nonzero whenever i j is an edge in g and is zero otherwise this paper surveys the current state of knowledge on the problem of determining the minimum rank of a graph and related issues

mixed risk neutral minimax control of discrete time finite state markov decision processes addresses the control design problem for discrete time finite state markov decision processes when both risk neutral and minimax objectives are of interest we introduce the mixed risk neutral minimax objective and utilize results from risk neutral and minimax control to derive an information state process and dynamic programming equations for the value function we synthesize optimal control laws both on the finite and the infinite horizons we study the effectiveness of both the mixed risk neutral minimax family and the risk sensitive family of controllers as tools to tradeoff risk neutral and minimax objectives we conclude that the mixed risk neutral minimax family is more effective at the cost of increased controller complexity

nonstationary discrete time deterministic and stochastic control systems bounded and unbounded cases this paper is about nonstationary nonlinear discrete time deterministic and stochastic control systems with borel state and control spaces with either bounded or unbounded costs the control problem is to minimize an infinite horizon total cost performance index using dynamic programming arguments we show that under suitable assumptions the optimal cost functions satisfy optimality equations which in turn give a procedure to find optimal control policies we also prove the convergence of value iteration or successive approximations functions several examples illustrate our results under different sets of assumptions

a simulation based algorithm for ergodic control of markov chains conditioned on rare events we study the problem of long run average cost control of markov chains conditioned on a rare event in a related recent work a simulation based algorithm for estimating performance measures associated with a markov chain conditioned on a rare event has been developed we extend ideas from this work and develop an adaptive algorithm for obtaining online optimal control policies conditioned on a rare event our algorithm uses three timescales or step size schedules on the slowest timescale a gradient search algorithm for policy updates that is based on one simulation simultaneous perturbation stochastic approximation spsa type estimates is used deterministic perturbation sequences obtained from appropriate normalized hadamard matrices are used here the fast timescale recursions compute the conditional transition probabilities of an associated chain by obtaining solutions to the multiplicative poisson equation for a given policy estimate further the risk parameter associated with the value function for a given policy estimate is updated on a timescale that lies in between the two scales above we briefly sketch the convergence analysis of our algorithm and present a numerical application in the setting of routing multiple flows in communication networks

graphs whose positive semi definite matrices have nullity at most two abstract let g v e be a undirected graph containing n vertices and let m g be the set of all hermitian n n matrices m m i j with m i j 0 if i and j are connected by one edge of g with m i j c if i and j are connected by at least two edges with m i j 0 if i j and i and j are not connected by an edge of g and with m i i for i 1 n a real number what is the largest nullity attained by any positive semi definite matrix m m g in this paper we characterize for t 1 and 2 those graphs g for which the maximum nullity is not greater than t

a time aggregation approach to markov decision processes we propose a time aggregation approach for the solution of infinite horizon average cost markov decision processes via policy iteration in this approach policy update is only carried out when the process visits a subset of the state space as in state aggregation this approach leads to a reduced state space which may lead to a substantial reduction in computational and storage requirements especially for problems with certain structural properties however in contrast to state aggregation which generally results in an approximate model due to the loss of markov property time aggregation suffers no loss of accuracy because the markov property is preserved single sample path based estimation algorithms are developed that allow the time aggregation approach to be implemented on line for practical systems some numerical and simulation examples are presented to illustrate the ideas and potential computational savings

limiting average criteria for nonstationary markov decision processes this paper deals with the so called limiting average criteria for nonstationary markov decision processes with possibly unbounded rewards and borel state space a new set of conditions is provided under which the existence of both a solution to the optimality equations and the limiting average epsilon geq 0 optimal markov policies is derived also a rolling horizon algorithm for computing limiting average epsilon 0 optimal markov policies is developed furthermore the results in this paper are illustrated by several examples such as the water regulation problem

classifying thick subcategories of the stable category of cohen macaulay modules abstract various classification theorems of thick subcategories of a triangulated category have been obtained in many areas of mathematics in this paper as a higher dimensional version of the classification theorem of thick subcategories of the stable category of finitely generated representations of a finite p group due to benson carlson and rickard we consider classifying thick subcategories of the stable category of cohen macaulay modules over a gorenstein local ring the main result of this paper yields a complete classification of the thick subcategories of the stable category of cohen macaulay modules over a local hypersurface in terms of specialization closed subsets of the prime ideal spectrum of the ring which are contained in its singular locus

computation of minimal rank and path cover number for certain graphs for a given undirected graph g the minimum rank of g is defined to be the smallest possible rank over all real symmetric matrices a whose i j th entry is non zero whenever i j and i j is an edge in g the path cover number of g is the minimum number of vertexdisjoint paths occurring as induced subgraphs of g that cover all the vertices of g for trees the relationship between minimum rank and path cover number is completely understood however for non trees only sporadic results are known we derive formulae for the minimum rank and path cover number for graphs obtained from edge sums and formulae for minimum rank of vertex sums of graphs in addition we examine previously identified special types of vertices and attempt to unify the theory behind them

on the graph complement conjecture for minimum rank abstract the minimum rank of a graph has been an interesting and well studied parameter investigated by many researchers over the past decade or so one of the many unresolved questions on this topic is the so called graph complement conjecture which grew out of a workshop in 2006 this conjecture asks for an upper bound on the sum of the minimum rank of a graph and the minimum rank of its complement and may be classified as a nordhaus gaddum type problem involving the graph parameter minimum rank the conjectured bound is the order of the graph plus two other variants of the graph complement conjecture are introduced here for the minimum semidefinite rank and the colin de verdiere type parameter we show that if the graph complement conjecture is true for two graphs then it is true for the join of these graphs related results for the graph complement conjecture and the positive semidefinite version for joins of graphs are also established we also report on the use of recent results on partial k trees to establish the graph complement conjecture for graphs of low minimum rank

technical communique a unified approach to markov decision problems and performance sensitivity analysis we propose a simple approach that provides a unified formulation for the performance sensitivity analysis of markov chains and markov decision problems with both infinite horizon average cost and discounted performance criteria

optimal average value convergence in nonhomogeneous markov decision processes abstract we address the undiscounted nonhomogeneous markov decision process with average reward criterion and prove two structural results first we establish equivalence of this problem to a discounted markov decision process by means of an ergodic coefficient embedded in the original problem second we prove for the original problem that the optimal finite horizon average values converge to the infinite horizon optimal average value under an ergodic condition

kochen specker sets and the rank 1 quantum chromatic number the quantum chromatic number of a graph g is sandwiched between its chromatic number and its clique number which are well known np hard quantities we restrict our attention to the rank 1 quantum chromatic number q 1 g which upper bounds the quantum chromatic number but is defined under stronger constraints we study its relation with the chromatic number g and the minimum dimension of orthogonal representations g it is known that g q 1 g g we answer three open questions about these relations we give a necessary and sufficient condition to have g q 1 g we exhibit a class of graphs such that g q 1 g and we give a necessary and sufficient condition to have q 1 g g our main tools are kochen specker sets collections of vectors with a traditionally important role in the study of contextuality of physical theories and more recently in the quantification of quantum zero error capacities finally as a corollary of our results and a result by avis et al on the quantum chromatic number we give a family of kochen specker sets of growing dimension

a basic formula for online policy gradient algorithms this note presents a new basic formula for sample path based estimates for performance gradients for markov systems called policy gradients in reinforcement learning literature with this basic formula many policy gradient algorithms including those that have previously appeared in the literature can be easily developed the formula follows naturally from a sensitivity equation in perturbation analysis new research direction is discussed

nonhomogeneous markov decision processes with borel state space the average criterion with nonuniformly bounded rewards this paper deals with nonhomogeneous markov decision processes with borel state space and nonuniformly bounded rewards under the average criterion first under the minorant type assumption we prove the existence of an appropriate solution to the optimality equations second from the optimality equations we also establish the existence of e 0 optimal markov policies under the additional conditions third some sufficient conditions for the validity of the assumptions in this paper and several examples such as inventory production systems are provided finally as an application of the optimality equations a rolling horizon algorithm is given

markov decision processes with a minimum variance criterion abstract we consider the optimization of the variance of the sum of costs as well as that of an average expected cost in markov decision processes with unbounded cost in case of general state and action space we find the stationary policy which makes the average variance as small as possible in the class of policies which are e optimal in an average expected cost

on the difference between the maximum multiplicity and path cover number for tree like graphs abstract for a given undirected graph g the maximum multiplicity of g is defined to be the largest multiplicity of an eigenvalue over all real symmetric matrices a whose i j th entry is non zero whenever i j and i j is an edge in g the path cover number of g is the minimum number of vertex disjoint paths occurring as induced subgraphs of g that cover all the vertices of g we derive a formula for the path cover number of a vertex sum of graphs and use it to prove that the vertex sum of so called non deficient graphs is non deficient for unicyclic graphs we provide a complete description of the path cover number and the maximum multiplicity and hence the minimum rank and we investigate the difference between path cover number and maximum multiplicity for a class of graphs referred to as block cycle graphs

unitary matrix digraphs and minimum semidefinite rank for an undirected simple graph g the minimum rank among all positive semidefinite matrices with graph g is called the minimum semidefinite rank msr of g in this paper we show that the msr of a given graph may be determined from the msr of a related bipartite graph finding the msr of a given bipartite graph is then shown to be equivalent to determining which digraphs encode the zero nonzero pattern of a unitary matrix we provide an algorithm to construct unitary matrices with a certain pattern and use previous results to give a lower bound for the msr of certain bipartite graphs

completion of squares revisited and extended due to its simplicity the completion of squares technique is quite popular in linear optimal control however this simple technique is limited to linear quadratic gaussian systems in this note by interpreting the completion of squares from a new angle we extend this technique to performance optimization of general markov systems with the long run average criterion leading to a new approach to performance optimization based on direct comparisons of the performance of two policies

simulation based optimization of markov reward processes this paper proposes a simulation based algorithm for optimizing the average reward in a finite state markov reward process that depends on a set of parameters as a special case the method applies to markov decision processes where optimization takes place within a parametrized set of policies the algorithm relies on the regenerative structure of finite state markov processes involves the simulation of a single sample path and can be implemented online a convergence result with probability 1 is provided

discrete time controlled markov processes with average cost criterion a survey this work is a survey of the average cost control problem for discrete time markov processes the authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades the exposition ranges from finite to borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies the authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography the authors have also identified several important questions that are still open to investigation

optimal control of ergodic continuous time markov chains with average sample path rewards in this paper we study continuous time markov decision processes with the average sample path reward aspr criterion and possibly unbounded transition and reward rates we propose conditions on the system s primitive data for the existence of epsilon aspr optimal deterministic stationary policies in a class of randomized markov policies satisfying some additional continuity assumptions the proof of this fact is based on the time discretization technique the martingale stability theory and the concept of potential we also provide both policy and value iteration algorithms for computing or at least approximating the epsilon aspr optimal stationary policies we illustrate with examples our main results as well as the difference between the aspr and the average expected reward criteria

zero forcing sets and the minimum rank of graphs the minimum rank of a simple graph g is defined to be the smallest possible rank over all symmetric real matrices whose ijth entry for i 6 j is nonzero whenever i j is an edge in g and is zero otherwise this paper introduces a new graph parameter z g that is the minimum size of a zero forcing set of vertices and uses it to bound the minimum rank for numerous families of graphs often enabling computation of the minimum rank

linearly independent vertices and minimum semidefinite rank we study the minimum semidefinite rank of a graph using vector representations of the graph and of certain subgraphs we present a sufficient condition for when the vectors corresponding to a set of vertices of a graph must be linearly independent in any vector representation of that graph and conjecture that the resulting graph invariant is equal to minimum semidefinite rank rotation of vector representations by a unitary matrix allows us to find the minimum semidefinite rank of the join of two graphs we also improve upon previous results concerning the effect on minimum semidefinite rank of the removal of a vertex

on minimum rank and zero forcing sets of a graph abstract for a graph g on n vertices and a field f the minimum rank of g over f written as mr f g is the smallest possible rank over all n n symmetric matrices over f whose i j th entry for i j is nonzero whenever ij is an edge in g and is zero otherwise the maximum nullity of g over f is m f g n mr f g the minimum rank problem of a graph g is to determine mr f g or equivalently m f g this problem has received considerable attention over the years in f barioli w barrett s butler s m cioab d cvetkovic s m fallat c godsil w haemers l hogben r mikkelson s narayan o pryporova i sciriha w so d stevanovic h van der holst k v meulen a w wehe aim minimum rank special graphs work group zero forcing sets and the minimum rank of graphs linear algebra appl 428 2008 1628 1648 a new graph parameter z g the zero forcing number was introduced to bound m f g from above the authors posted an attractive question what is the class of graphs g for which z g m f g for some field f this paper focuses on exploring the above question

the inverse inertia problem for the complements of partial k trees let f be an infinite field with characteristic not equal to two for a graph g v e with v 1 n let s g f be the set of all symmetric n n matrices a ai j over f with ai j 0 i j if and only if ij e we show that if g is the complement of a partial k tree and m k 2 then for all nonsingular symmetric m m matrices k over f there exists an m n matrix u such that utku s g f as a corollary we obtain that if k 2 m n and g is the complement of a partial k tree then for any two nonnegative integers p and q with p q m there exists a matrix in s g r with p positive and q negative eigenvalues

quillen equivalences for stable categories abstract for an abelian category a we investigate when the stable categories gpro j a and gin j a are triangulated equivalent to this end we realize these stable categories as homotopy categories of certain non trivial model categories and give conditions on a that ensure the existence of a quillen equivalence between the model categories in question we also study when such a quillen equivalence transfers from a to the category of chain complexes in a

robust constrained model predictive control using linear matrix inequalities abstract the primary disadvantage of current design techniques for model predictive control mpc is their inability to deal explicitly with plant model uncertainty in this paper we present a new approach for robust mpc synthesis that allows explicit incorporation of the description of plant uncertainty in the problem formulation the uncertainty is expressed in both the time and frequency domains the goal is to design at each time step a state feedback control law that minimizes a worst case infinite horizon objective function subject to constraints on the control input and plant output using standard techniques the problem of minimizing an upper bound on the worst case objective function subject to input and output constraints is reduced to a convex optimization involving linear matrix inequalities lmis it is shown that the feasible receding horizon state feedback control design robustly stabilizes the set of uncertain plants several extensions such as application to systems with time delays problems involving constant set point tracking trajectory tracking and disturbance rejection which follow naturally from our formulation are discussed the controller design is illustrated with two examples

worst case formulations of model predictive control for systems with bounded parameters two different predictive control formulations are developed based on minimization of the worst case quadratic cost for systems with bounded parameters the two formulations differ on the assumptions made about the future inputs in optimizing the current input one assumes open loop control while the other considers closed loop control their closed loop properties such as asymptotic stability are examined we then focus on a moving average model with an integrator and derive computationally simpler suboptimal algorithms

optimization of a special case of continuous time markov decision processes with compact action set performance optimization is considered for average cost multichain markov decision processes mdps with compact action set since for a general compact multichain model the optimality equation system may have no solution and also a policy iteration algorithm may yield a suboptimal policy rather than an optimal one we concentrate only on a special case of multichain models in this paper where we assume that the classifications of states are fixed identically rather than varying with policies by using the concept of performance potentials the existence of solutions to the optimality equation system is established and then a potential based policy iteration algorithm is supposed to solve this system in addition the optimality convergence for recurrent classes of the algorithm has been proved finally a numerical example is provided

another set of conditions for markov decision processes with average sample path costs abstract this paper deals with discrete time markov decision processes with average sample path costs aspc in borel spaces the costs may have neither upper nor lower bounds we propose new conditions for the existence of e aspc optimal deterministic stationary policies in the class of all randomized history dependent policies our conditions are weaker than those in the previous literature moreover some sufficient conditions for the existence of aspc optimal stationary policies are imposed on the primitive data of the model in particular the stochastic monotonicity condition in this paper has first been used to study the aspc criterion also the approach provided here is slightly different from the optimality equation approach widely used in the previous literature on the other hand under mild assumptions we show that average expected cost optimality and aspc optimality are equivalent finally we use a controlled queueing system to illustrate our results

minimax control of discrete time stochastic systems this paper gives a unified self contained presentation of minimax control problems for discrete time stochastic systems on borel spaces with possibly unbounded costs the main results include conditions for the existence of minimax strategies for finite horizon problems and infinite horizon discounted and undiscounted average cost criteria the results are specialized to control systems with unknown disturbance distributions also known as games against nature two examples illustrate the theory one of them on the mold level control problem which is a key problem in the steelmaking industry

partially observable markov decision processes with reward information basic ideas and models in a partially observable markov decision process pomdp if the reward can be observed at each step then the observed reward history contains information on the unknown state this information in addition to the information contained in the observation history can be used to update the state probability distribution the policy thus obtained is called a reward information policy ri policy an optimal ri policy performs no worse than any normal optimal policy depending only on the observation history the above observation leads to four different problem formulations for pomdps depending on whether the reward function is known and whether the reward at each step is observable this exploratory work may attract attention to these interesting problems

a unified approach to markov decision problems and performance sensitivity analysis with discounted and average criteria multichain cases we propose a unified framework to markov decision problems and performance sensitivity analysis for multichain markov processes with both discounted and average cost performance criteria with the fundamental concept of performance potentials we derive both performance gradient and performance difference formulas which play the central role in performance optimization the standard policy iteration algorithms for both discounted and average reward mdps can be established using the performance difference formulas in a simple and intuitive way and the performance gradient formulas together with stochastic approximation may lead to new optimization schemes this sensitivity based point of view of performance optimization provides some insights that link perturbation analysis markov decision processes and reinforcement learning together the research is an extension of the previous work on ergodic markov chains cao automatica 36 2000 771

a brouwer fixed point mapping approach to communicating markov decision processes abstract the solvability of the functional equations of communicating markov decision processes is demonstrated by an elementary application of the brouwer fixed point mapping theorem the technical assumptions include finite state space and finite or compact action spaces

rolling horizon procedures in nonhomogeneous markov decision processes by far the most common planning procedure found in practice is to approximate the solution to an infinite horizon problem by a series of rolling finite horizon solutions although many empirical studies have been done this so called rolling horizon procedure has been the subject of few analytic studies we provide a cost error bound for a general rolling horizon algorithm when applied to infinite horizon nonhomogeneous markov decision processes both in the discounted and average cost cases we show that a doeblin coefficient of ergodicity acts much like a discount factor to reduce this error in particular we show that the error goes to zero for any fixed rolling horizon as this doeblin measure of control over the future decreases the theory is illustrated through an application to vehicle deployment

markov decision processes with a borel measurable cost function the average case we consider a markov decision process with a borel measurable cost function we introduce a p step contraction property for the average cost case by use of this method the validity of the optimality equation and the existence of e optimal stationary policies are proved as some applications the sequential replacement model and the inventory model are considered

brief minimax control for discrete time time varying stochastic systems this paper gives a self contained presentation of minimax control for discrete time time varying stochastic systems under finite and infinite horizon expected total cost performance criteria suitable conditions for the existence of minimax strategies are proposed also we prove that the values of the finite horizon problem converge to the values of the infinite horizon problems moreover for finite horizon problems an algorithm of calculation of minimax strategies is developed and tested by using time varying stochastic systems

the potential structure of sample paths and performance sensitivities of markov systems we study the structure of sample paths of markov systems by using performance potentials as the fundamental units with a sample path based approach we show that performance sensitivity formulas performance gradients and performance differences of markov systems can be constructed intuitively by first principles with performance potentials or equivalently perturbation realization factors as building blocks in particular we derive sensitivity formulas for two markov chains with possibly different state spaces the proposed approach can be used to obtain flexibly the sensitivity formulas for a wide range of problems including those with partial information these formulas are the basis for performance optimization of discrete event dynamic systems including perturbation analysis markov decision processes and reinforcement learning the approach thus provides insight on on line learning and performance optimization and opens up new research directions sample path based algorithms can be developed

policy iteration based feedback control it is well known that stochastic control systems can be viewed as markov decision processes mdps with continuous state spaces in this paper we propose to apply the policy iteration approach in mdps to the optimal control problem of stochastic systems we first provide an optimality equation based on performance potentials and develop a policy iteration procedure then we apply policy iteration to the jump linear quadratic problem and obtain the coupled riccati equations for their optimal solutions the approach is applicable to linear as well as nonlinear systems and can be implemented on line on real world systems without identifying all the system structure and parameters

gorenstein complexes and recollements from cotorsion pairs we describe a general correspondence between injective resp projective recollements of triangulated categories and injective resp projective cotorsion pairs this provides a model category description of these recollement situations our applications focus on displaying several recollements that glue together various full subcategories of k r the homotopy category of chain complexes of modules over a general ring r when r is left noetherian these recollements involve complexes built from the gorenstein injective modules when r is a left coherent ring for which all flat modules have finite projective dimension we obtain the duals these results extend to a general ring r by replacing the gorenstein modules with the gorenstein ac modules introduced in 9 we also see that in any abelian category with enough injectives the gorenstein injective objects enjoy a maximality property in that they contain every other class making up the right half of an injective cotorsion pair

a new optimality criterion for nonhomogeneous markov decision processes we propose a new definition of optimality for nonhomogeneous markov decision processes called periodic forecast horizon pfh optimality using measures of discounting and ergodicity we establish conditions under which pfh optimal strategies exist and pfh optimality implies the more conventional notions of optimality and average optimality finally we use this definition of optimality as the basis for a direct development of forecast horizon results for discounted and undiscounted nonhomogeneous markov decision processes

the n th order bias optimality for multichain markov decision processes in this paper we propose a new approach to the theory of finite multichain markov decision processes mdps with different performance optimization criteria we first propose the concept of nth order bias then using the average reward and bias difference formulas derived in this paper we develop an optimization theory for finite mdps that covers a complete spectrum from average optimality bias optimality to all high order bias optimality in a unified way the approach is simple direct natural and intuitive it depends neither on laurent series expansion nor on discounted mdps we also propose one phase policy iteration algorithms for bias and high order bias optimal policies which are more efficient than the two phase algorithms in the literature furthermore we derive high order bias optimality equations this research is a part of our effort in developing sensitivity based learning and optimization theory

orthogonal representations minimum rank and graph complements orthogonal representations are used to show that complements of certain sparse graphs have positive semidefinite minimum rank at most 4 this bound applies to the complement of a 2 tree and to the complement of a unicyclic graph hence for such graphs the sum of the minimum rank of the graph and the minimum rank of its complement is at most two more than the order of the graph the minimum rank of the complement of a 2 tree is determined exactly

markov decision processes with sample path constraints the communicating case we consider time average markov decision processes mdps which accumulate a reward and cost at each decision epoch a policy meets the sample path constraint if the time average cost is below a specified value with probability one the optimization problem is to maximize the expected average reward over all policies that meet the sample path constraint the sample path constraint is compared with the more commonly studied constraint of requiring the average expected cost to be less than a specified value although the two criteria are equivalent for certain classes of mdps their feasible and optimal policies differ for many nontrivial problems in general there does not exist optimal or nearly optimal stationary policies when the expected average cost constraint is employed assuming that a policy exists that meets the sample path constraint we establish that there exist nearly optimal stationary policies for communicating mdps a parametric linear programming algorithm is given to construct nearly optimal stationary policies the discussion relies on well known results from the theory of stochastic processes and linear programming the techniques lead to simple proofs of the existence of optimal and nearly optimal stationary policies for unichain and deterministic mdps respectively

brief min max predictive control techniques for a linear state space system with a bounded set of input matrices min max predictive control of a linear state space system with a bounded set of input matrices is studied based on a quadratic performance criterion systems with stable and integrating dynamics as well as time varying and time invariant uncertainties are treated algorithms that are computationally manageable and ensure closed loop stability are developed and tested

exact model structures and recollements abstract we show how to obtain recollements of triangulated categories using the theory of exact model structures from 13 after noting how the theory relates to well known notions in the simplest case of frobenius categories we apply these ideas to recollements of homotopy categories of chain complexes in short we give model categorical explanations for the classical verdier localization recollement as well as several recollement situations due to neeman and murfet

stratifying modular representations of finite groups we classify localising subcategories of the stable module category of a finite group that are closed under tensor product with simple or equiv alently all modules one application is a proof of the telescope conjecture in this context others include new proofs of the tensor product theorem and of the classification of thick subcategories of the finitely generated modules which avoid the use of cyclic shifted subgroups along the way we establish similar classifications for differential graded modules over graded polynomial rings and over graded exterior algebras

a liapounov bound for solutions of the poisson equation in this paper we consider psi irreducible markov processes evolving in discrete or continuous time on a general state space we develop a liapounov function criterion that permits one to obtain explicit bounds on the solution to the poisson equation and in particular obtain conditions under which the solution is square integrable these results are applied to obtain sufficient conditions that guarantee the validity of a functional central limit theorem for the markov process as a second consequence of the bounds obtained a perturbation theory for markov processes is developed which gives conditions under which both the solution to the poisson equation and the invariant probability for the process are continuous functions of its transition kernel the techniques are illustrated with applications to queueing theory and autoregressive processes

continuous time markov decision processes with nth bias optimality criteria in this paper we study the nth bias optimality problem for finite continuous time markov decision processes mdps with a multichain structure we first provide nth bias difference formulas for two policies and present some interesting characterizations of an nth bias optimal policy by using these difference formulas then we prove the existence of an nth bias optimal policy by using nth bias optimal policy iteration algorithms and show that such an nth bias optimal policy can be obtained in a finite number of policy iterations

ring homomorphisms and finite gorenstein dimension the local structure of homomorphisms of commutative noetherian rings is investigated from the point of view of dualizing complexes a concept of finite gorenstein dimension which substantially weakens the notion of finite flat dimension is introduced for homomorphisms it is shown to impose structural constraints due to a remarkable equivalence of subcategories of the derived category of all modules an essential part of this study is the development of relative notions of dualizing complexes and bass numbers it is proved that the bass numbers of local homomorphisms are rigid extending a known result for local rings quasi gorenstein homomorphisms are introduced as local homomorphisms that base change a dualizing complex for the source ring into one for the target they are shown to have the stability properties of the gorenstein homomorphism that they generalize 1991 mathematics subject classification primary 13h10 13d23 14e40 secondary 13c15

brief minimax controller design for a class of uncertain linear systems this paper considers the design of minimax controllers for a class of linear systems the system is perturbed by a known initial condition on the state and by a disturbance process constrainted by an l 2 norm bound structured undertainty is represented by perturbation feedback through l 2 induced norm bounded operators the performance is measured by an integral quadratic cost function the resultant controllers minimise the worst case cost for all admissible distrubances and for all admissible systems the design involves the optimal solution of a parametric algebraic riccati equation a new proof of minimax optimality is given using standard methods for linear systems the multivariable optimisation problem is shown to be convex and the set of feasible parameter values is shown to be compact thus the design procedure is computationally tractable

percentile performance criteria for limiting average markov decision processes addresses the following basic feasibility problem for infinite horizon markov decision processes mdps can a policy be found that achieves a specified value target of the long run limiting average reward at a specified probability level percentile related optimization problems of maximizing the target for a specified percentile and vice versa are also considered the authors present a complete and discrete classification of both the maximal achievable target levels and of their corresponding percentiles the authors also provide an algorithm for computing a deterministic policy corresponding to any feasible target percentile pair next the authors consider similar problems for an mdp with multiple rewards and or constraints this case presents some difficulties and leads to several open problems an lp based formulation provides constructive solutions for most cases

the minimum semidefinite rank of a triangle free graph abstract we employ a result of moshe rosenfeld to show that the minimum semidefinite rank of a triangle free graph with no isolated vertex must be at least half the number of its vertices we define a rosenfeld graph to be such a graph that achieves equality in this bound and we explore the structure of these special graphs their structure turns out to be intimately connected with the zero nonzero patterns of the unitary matrices finally we suggest an exploration of the connection between the girth of a graph and its minimum semidefinite rank and provide a conjecture in this direction

the stable category and invertible modules for infinite groups abstract we construct a well behaved stable category of modules for a large class of infinite groups we then consider its picard group which is the group of invertible or endotrivial modules we show how this group can be calculated when the group acts on a tree with finite stabilisers

forbidden minors for the class of graphs g with ξ g 2 abstract for a given simple graph g s g is defined to be the set of real symmetric matrices a whose i j th entry is nonzero whenever i j and ij is an edge in g in f barioli s fallat l hogben a variant on the graph parameters of colin de verdiere implications to the minimum rank of graphs electron j linear algebra 13 2005 387 404 g is defined to be the maximum corank i e nullity among a s g having the strong arnold property is used to study the minimum rank maximum eigenvalue multiplicity problem for g since is minor monotone the graphs g such that g k can be described by a finite set of forbidden minors we determine the forbidden minors for g 2 and present an application of this characterization to computation of minimum rank among matrices in s g

structural infinitesimal perturbation analysis sipa for derivative estimation of discrete event dynamic systems in this paper we present an algorithm for derivative estimation of discrete event dynamic systems ded s this algorithm is applicable to derivative estimation with respect to most structural parameters in ded s it extends the applicability of existing perturbation analysis techniques including infinitesimal perturbation analysis on structural parameters furthermore it offers an alternative approach to derivative estimation in ded s which sheds additional light on discrete event simulation finally unifying relationships with existing techniques are discussed

graphs associated with vector spaces of even dimension a link with differential geometry abstract we define a new association between graphs and orthonormal bases of even dimensional euclidean vector spaces endowed with an special isomorphism motivated by the recently introduced theory of submanifolds associated with graphs we provide several interesting examples and we analyze the shape of such graphs by proving some general results

a new graph parameter related to bounded rank positive semidefinite matrix completions the gram dimension mathrm gd g of a graph g is the smallest integer k ge 1 such that any partial real symmetric matrix whose entries are specified on the diagonal and at the off diagonal positions corresponding to edges of g can be completed to a positive semidefinite matrix of rank at most k assuming a positive semidefinite completion exists for any fixed k the class of graphs satisfying mathrm gd g le k is minor closed hence it can be characterized by a finite list of forbidden minors we show that the only minimal forbidden minor is k k 1 for k le 3 and that there are two minimal forbidden minors k 5 and k 2 2 2 for k 4 we also show some close connections to euclidean realizations of graphs and to the graph parameter nu g of van der holst combinatorica 23 4 633 651 2003 in particular our characterization of the graphs with mathrm gd g le 4 implies the forbidden minor characterization of the 3 realizable graphs of belk discret comput geom 37 139 162 2007 and belk and connelly discret comput geom 37 125 137 2007 and of the graphs with nu g le 4 of van der holst combinatorica 23 4 633 651 2003

the minimum semidefinite rank of the complement of partial k trees abstract for a graph g v e with v 1 n let s g be the set of all real symmetric n n matrices a a i j with a i j 0 i j if and only if ij e we prove the following results if g is the complement of a partial k tree h then there exists a positive semidefinite matrix a s g with rank a k 2 if in addition k 3 or g is k connected then there exist positive semidefinite matrices a s g and b s h such that rank a rank b n 2

perturbation realization potentials and sensitivity analysis of markov processes two fundamental concepts and quantities realization factors and performance potentials are introduced for markov processes the relations among these two quantities and the group inverse of the infinitesimal generator are studied it is shown that the sensitivity of the steady state performance with respect to the change of the infinitesimal generator can be easily calculated by using either of these three quantities and that these quantities can be estimated by analyzing a single sample path of a markov process based on these results algorithms for estimating performance sensitivities on a single sample path of a markov process can be proposed the potentials in this paper are defined through realization factors and are shown to be the same as those defined by poisson equations the results provide a uniform framework of perturbation realization for infinitesimal perturbation analysis ipa and non ipa approaches to the sensitivity analysis of steady state performance they also provide a theoretical background for the pa algorithms developed in recent years

on gorenstein projective injective and flat dimensions a functorial description with applications gorenstein homological dimensions are refinements of the classical homological dimensions and finiteness singles out modules with amenable properties reflecting those of modules over gorenstein rings as opposed to their classical counterparts these dimensions do not immediately come with practical and robust criteria for finiteness not even over commutative noetherian local rings in this paper we enlarge the class of rings known to admit good criteria for finiteness of gorenstein dimensions

robust model predictive control of stable linear systems a new robust model predictive control mpc algorithm is presented for stable constrained linear plants that is a direct generalization of the nominally stabilizing regulator presented by rawlings and muske model uncertainty is parametrized by a list of possible plants robust stability is achieved through the addition of constraints that prevent the sequence of optimal controller costs from increasing for the true plant asymptotic stability is demonstrated through a lyapunov argument simulation experiments demonstrate the performance of the algorithm for two example processes

decentralized learning in finite markov chains revisited the convergence proof in the paper ldquodecentralized learning in finite markov chains rdquo published in the ieee transactions on automatic control vol ac 31 no 6 pp 519 526 1986 is incomplete this note first provides a sufficient condition for the existence of a unique optimal policy for infinite horizon average cost markov decision processes mdps making the convergence result established by wheeler and narendra preserved with the condition we then present a novel simulation based decentralized algorithm called ldquosampled joint strategy fictitious play for mdprdquo for average mdps based on the recent study by garcia of a decentralized approach to discrete optimization via fictitious play applied to games with identical payoffs we establish a stronger almost sure convergence result than wheeler and narendra s showing that the sequence of probability distributions over the policy space for a given mdp generated by the algorithm converges to a unique optimal policy with probability one

policy iteration algorithm for singular controlled diffusion processes in this paper the infinite horizon optimal control problems for singular diffusion processes are considered from the viewpoints of markov decision processes and perturbation analysis where the singularity of diffusion means that the covariance matrix of the system noise is allowed to be degenerate a formula of performance difference under two different controls is derived and leads to a comparison theorem by the comparison theorem starting from a control a so called better control can be selected therefore a control policy iteration algorithm is developed by which the performance improves step by step and converges to the optimal one when this applies to the stochastic affine nonlinear regulator and stochastic linear quadratic optimal control problems better control can be constructed in a closed form it is also shown that when the considered stochastic systems degenerate to the deterministic ones the proposed algorithm reduces to the adaptive dynamic programming algorithm j j murray c j

denumerable state nonhomogeneous markov decision processes we consider denumerable state nonhomogeneous markov decision processes and extend results from both denumerable state homogeneous and finite state nonhomogeneous problems we show that under weak ergodicity accumulation points of finite horizon optima termed algorithmic optima are average cost optimal we also establish the existence of solution horizons finally an algorithm is presented to solve problems of this class for the case where there is a unique algorithmic optimum

cotorsion pairs model category structures and representation theory we make a general study of quillen model structures on abelian categories we show that they are closely related to cotorsion pairs which were introduced by salce sal79 and have been much studied recently by enochs and coauthors ej00 this gives a method of constructing model structures on abelian categories which we illustrate by building two model structures on the category of modules over a possibly noncommutative gorenstein ring the homotopy category of these model structures is a generalization of the stable module category much used in modular representation theory this stable module category has also been studied by benson ben97

semi markov decision problems and performance sensitivity analysis recent research indicates that markov decision processes mdps can be viewed from a sensitivity point of view and the perturbation analysis pa mdps and reinforcement learning rl are three closely related areas in optimization of discrete event dynamic systems that can be modeled as markov processes the goal of this paper is two fold first we develop the pa theory for semi markov processes smps and then we extend the aforementioned results about the relation among pa mdp and rl to smps in particular we show that performance sensitivity formulas and policy iteration algorithms of semi markov decision processes can be derived based on the performance potential and realization matrix both the long run average and discounted cost problems are considered this approach provides a unified framework for both problems and the long run average problem corresponds to the discounted factor being zero the results indicate that performance sensitivities and optimization depend only on first order statistics single sample path based implementations are discussed