location on tree networks p centre and n dispersion problems we consider two problems of locating facilities on trees in the first we wish to determine the location of n points so as to maximize the minimum distance between any pair in the second we wish to locate p centers of a single type of facility so as to minimize the maximum distance between any point and the center nearest to it we provide polynomial algorithms for both problems of o n 2 log n log n for the first problem and of o n 2 log p for the second where n is the number of nodes in the tree

self adaptive operator splitting methods for monotone variational inequalities solving a variational inequality problem vi f is equivalent to finding a solution of a system of nonsmooth equations a hard problem the peaceman rachford and or douglas rachford operator splitting methods are advantageous when they are applied to solve variational inequality problems because they solve the original problem via solving a series of systems of nonlinear smooth equations a series of easy problems although the solution of vi f is invariant under multiplying f by some positive scalar yet the numerical experiment has shown that the number of iterations depends significantly on the positive parameter which is a constant in the original operator splitting methods in general it is difficult to choose a proper parameter for individual problems in this paper we present a modified operator splitting method which adjusts the scalar parameter automatically per iteration based on the message of the iterates exact and inexact forms of the modified method with self adaptive variable parameter are suggested and proved to be convergent under mild assumptions finally preliminary numerical tests show that the self adaptive adjustment rule is proper and necessary in practice

denumerable constrained markov decision processes and finite approximations the purpose of this paper is two fold first to establish the theory of discounted constrained markov decision processes with a countable state and action spaces with general multi chain structure second to introduce finite approximation methods we define the occupation measures and obtain properties of the set of all achievable occupation measures under the different admissible policies we establish the optimality of stationary policies for the constrained control problem and obtain an lp with a countable number of decision variables through which stationary optimal policies are computed since for such an lp one cannot expect to find an optimal solution in a finite number of operations we present two schemes for finite approximations and establish the convergence of optimal values and policies for both the discounted and the expected average cost with unbounded cost sometimes it turns out to be easier to solve the problem with infinite state space than the problem with finite yet large state space based on the optimal policy for the problem with infinite state space we construct policies which are almost optimal for the problem with truncated state space this method is applied to obtain an e optimal policy for a problem of optimal priority assignment under constraints for a system of k finite queues

constrained markov decision models with weighted discounted rewards this paper deals with constrained optimization of markov decision processes both objective function and constraints are sums of standard discounted rewards but each with a different discount factor such models arise e g in production and in applications involving multiple time scales we prove that it a feasible policy exists then there exists an optimal policy which is i stationary nonrandomized from some step onward ii randomized markov before this step but the total number of actions which are added by randomization is bounded by the number of constraints optimality of such policies for multi criteria problems is also established these new policies have the pleasing aesthetic property that the amount of randomization they require over any trajectory is restricted by the number of constraints this result is new even for constrained optimization with a single discount factor where the optimality of randomized stationary policies is known however a randomized stationary policy may req

continuity of optimal values and solutions for control of markov chains with constraints we consider in this paper constrained markov decision processes this type of control model has many applications in telecommunications and other fields e altman and a shwartz ieee trans automat control 34 1989 pp 1089 1102 e a feinberg and m i reiman probab engrg inform sci 8 1994 pp 463 489 a hordijk and f spieksma adv in appl probab 21 1989 pp 409 431 a lazar ieee trans automat control 28 1983 pp 1001 1007 p nain and k w ross ieee trans automat control 31 1986 pp 883 888 k w ross and b chen ieee trans automat control 33 1988 pp 261 267 we address the issue of the convergence of the value and optimal policies of the problem with discounted costs to the ones for the problem with expected average cost we consider the general multichain ergodic structure we present two stability results in this paper we establish the continuity of optimal values and solutions of as well as some type of robustness of some suboptimal solutions in the discount factor our proof relies on same general theory on continuity of values and solutions in convex optimization that relies on well known notions of gamma convergence

on the solution value of the continuous p center location problem on a graph let g v e be an undirected graph with positive edge lengths let rp denote the solution value to the continuous p center location problem on g we prove that rp is of the form rp t 2q where t is the length of an euler tour of a subgraph of g which belongs to one of four possible types and q is an integer 1 q 2p we also discuss algorithmic implications of this result

locating an absolute center on graphs that are almost trees abstract physical models that exhibit a treelike graph structure frequently arise in telecommunication networks interstate highway systems and computer communication networks treelike graphs are characterized and properties related to the absolute center problem are analyzed that are preserved under two transformations of the graph utilizing these preserved properties a linear time algorithm is presented for the absolute center problem whenever the underlying graph is homeomorphic to a treelike 3 cactus graph

backup 2 center on interval graphs a graph g is called an interval graph if there exists a set of intervals corresponding to the vertex set of g and two vertices are adjacent to each other if and only if the two corresponding intervals are intersecting with each other in this paper we apply the reliability based backup 2 center modal proposed by wang wu and chao in which each server may fail with a given probability to interval graphs and present an o n time algorithm

adaptive control of constrained markov chains adaptive control of finite state markov chains is discussed the optimal performance is characterized through the minimization of a long run average cost functional subject to constraints on several other such functionals under mild structural and feasibility conditions two explicit adaptive control policies are exhibited for the case where the transition probabilities are unknown the policies are optimal under the constrained optimization criterion they rely on a powerful estimation scheme which provides consistent estimators for the transition probabilities this scheme is of independent interest as it provides strong consistency under a large number of adaptive schemes and is independent of any identifiability conditions as an application an optimal adaptive policy is derived for a system of k competing queues with countable state space for which the constrained criteria arise naturally in the context of communication networks

a family of operator splitting methods revisited abstract a family of operator splitting methods for maximal monotone operators is investigated it generalizes the douglas peaceman rachford varga class of methods in the way that it allows the scaling parameters to vary from iteration to iteration non monotonically conditions for convergence of methods within this family and for obtaining a linear rate of convergence are given these conditions cover more general cases than the existing ones

solving np hard problems on graphs that are almost trees and an application to facility location problems description d une technique generale pour resoudre certains problemes de graphes np durs en un temps qui est exponentiel dans un parametre k defini comme le maximum sur tous les composants c non separables du graphe d un nombre de bords qui peut etre ajoute a l arbre pour produire c on donne un algorithme de temps de complexite o e 6r rh 27

approximations in dynamic zero sum games ii we pursue in this paper our study of approximations of values and epsilon saddle point policies in dynamic zero sum games after extending the general theorem for approximation we study zero sum stochastic games with countable state space and unbounded immediate reward we focus on the expected average payoff criterion we use some tools developed in m m tidball and e altman siam j control optim 34 1996 pp 311 328 to obtain the convergence of the values as well as the convergence of the epsilon saddle point policies in various approximation problems we consider several schemes of truncation of the state space e g finite state approximation and approximations of games with discount factor close to one for the game with expected average cost we use the extension of the general theorem for approximation to study approximations in stochastic games with complete information finally we consider the problem of approximating the sets of policies we obtain some general results that we apply to a pursuit evasion differential game

optimality of mixed policies for average continuous time markov decision processes with constraints this article concerns the average criteria for continuous time markov decision processes with n constraints we show the following a every extreme point of the space of performance vectors corresponding to the set of stable measures is generated by a deterministic stationary policy and b there exists a mixed optimal policy where the mixture is over no more than n 1 deterministic stationary policies

constrained undiscounted stochastic dynamic programming in this paper we investigate the computation of optimal policies in constrained discrete stochastic dynamic programming with the average reward as utility function the state space and action sets are assumed to be finite constraints which are linear functions of the state action frequencies are allowed in the general multichain case an optimal policy will be a randomized nonstationary policy an algorithm to compute such an optimal policy is presented furthermore sufficient conditions for optimal policies to be stationary are derived there are many applications for constrained undiscounted stochastic dynamic programming e g in multiple objective markovian decision models

finite state approximations for denumerable multidimensional state discounted markov decision processes abstract a finite state iterative scheme introduced by white in recent developments in markov decision processes r hartley l c thomas and d j white eds academic press new york 1980 to approximate the value function of denumerable state markov decision processes is extended to denumerable multidimensional state space under essentially the same assumptions given in op cit a simpler proof of the convergence theorem is obtained together with convergence rates the iterative scheme is used to determine an asymptotically discount optimal policy which in turn can be used to obtain a discount optimal stationary policy

a finite algorithm for the continuousp center location problem on a graph letg v e be an undirected graph with positive integer edge lengths letm be the number of edges ine and letd be the sum of the edge lengths we prove that the solution value to the continuousp center location problem is a rationalp1 p2 where logp1 o m5 logd m6 logp i 1 2 this result is then used to construct a finite algorithm for the continuousp center problem

approximating ergodic average reward continuous time controlled markov chains we study the approximation of an ergodic average reward continuous time denumerable state markov decision process mdp by means of a sequence of mdps our results include the convergence of the corresponding optimal policies and the optimal gains for a controlled upwardly skip free process we show some computational results to illustrate the convergence theorems

an optimal algorithm for solving the 1 median problem on weighted 4 cactus graphs abstract the median problem has been extensively studied in the last three decades on general graphs the problem can be solved in o n 3 where n is the number of vertices in a graph for tree networks however more efficient algorithms can be devised to find the medians in o n in this paper we study weighted 4 cactus w4c graphs which provide a less restrictive network structure than trees and show that median problem can be solved just as efficiently on w4c graphs as on trees many examples are provided demonstrating the rudiments of our algorithm and a linear time complexity algorithm is developed

constrained discounted dynamic programming this paper deals with constrained optimization of markov decision processes with a countable state space compact action sets continuous transition probabilities and upper semicontinuous reward functions the objective is to maximize the expected total discounted reward for one reward function under several inequality constraints on similar criteria with other reward functions suppose a feasible policy exists for a problem with m constraints we prove two results on the existence and structure of optimal policies first we show that there exists a randomized stationary optimal policy which requires at most m actions more than a nonrandomized stationary one this result is known for several particular cases second we prove that there exists an optimal policy which is i stationary nonrandomized from some step onward ii randomized markov before this step but the total number of actions which are added by randomization is at most m iii the total number of actions that are added by nonstationarity is at most m we also establish pareto optimality of policies from the two classes described above for multi criteria problems we describe an algorithm to compute optimal policies with properties i iii for constrained problems the policies that satisfy properties i iii have the pleasing aesthetic property that the amount of randomization they require over any trajectory is restricted by the number of constraints in contrast a randomized stationary policy may require an infinite number of randomizations over time

a convex analytic approach to markov decision processes this paper develops a new framework for the study of markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced measures on the trajectory space of the joint state and control process this set is shown to be compact convex one then associates with each of the usual cost criteria infinite horizon discounted cost finite horizon control up to an exit time a naturally defined occupation measure such that the cost is an integral of some function with respect to this measure these measures are shown to form a compact convex set whose extreme points are characterized classical results about existence of optimal strategies are recovered from this and several applications to multicriteria and constrained optimization problems are briefly indicated

optimal policies for controlled markov chains with a constraint abstract the time average reward for a discrete time controlled markov process subject to a time average cost constraint is maximized over the class of al causal policies each epoch a reward depending on the state and action is earned and a similarly constituted cost is assessed the time average of the former is maximized subject to a hard limit on the time average of the latter it is assumed that the state space is finite and the action space compact metric an accessibility hypothesis makes it possible to utilize a lagrange multiplier formulation involving the dynamic programming equation thus reducing the optimization problem to an unconstrained optimization parametrized by the multiplier the parametrized dynamic programming equation possesses compactness and convergence properties that lead to the following if the constraint can be satisfied by any causal policy the supremum over time average rewards respective to all causal policies is attained by either a simple or a mixed policy the latter is equivalent to choosing independently at each epoch between two specified simple policies by the throw of a biased coin

constrained average cost markov control processes in borel spaces this paper considers constrained markov control processes in borel spaces with unbounded costs the criterion to be minimized is a long run expected average cost and the constraints can be imposed on similar average costs or on average rewards or discounted costs or rewards we give conditions under which the constrained problem cp is solvable and equivalent to an equality constrained ec linear program furthermore we show that there is no duality gap between ec and the dual program ec and that in fact the strong duality condition holds finally we introduce an explicit procedure to solve cp in some cases which is illustrated with a detailed example

stability and singular perturbations in constrained markov decision problems constrained markov decision problems cmdps with the average cost criterion and a single ergodic chain or the discounted cost with a general multichain structure are considered conditions for stability of the optimal value and control to changes of the parameters of the problem such as immediate costs transition probabilities and the discount factor are established singular constrained problems for which the optimal value and controls exhibit discontinuities are studied

ergodic control of markov chains with constraints the general case the problem of controlling a markov chain on a countable state space with ergodic or long run average cost is studied in the presence of additional constraints requiring finitely many say m other ergodic costs to satisfy prescribed bounds under extremely general conditions it is proved that an optimal stationary randomized strategy can be found that requires at most m randomizations this generalizes a result of ross

markov decision problems and state action frequencies consider a controlled markov chain with countable state and action spaces basic quantities that determine the values of average cost functionals are identified under some regularity conditions these turn out to be a collection of numbers one for each state action pair describing for each state the relative number of uses of each action these conditional frequencies which are defined pathwise are shown to determine the state action frequencies that in the finite case are known to determine the costs this is extended to the countable case allowing for unbounded costs the space of frequencies is shown to be compact and convex and the extreme points are identified with stationary deterministic policies conditions under which the search for optimality in several optimization problems may be restricted to stationary policies are given these problems include the standard markov decision process as well as constrained optimization both in terms of average cost functionals and variability sensitive optimization an application to a queueing problem is given where these results imply the existence and explicit computation of optimal policies in constrained optimization problems the pathwise definition of the conditional frequencies implies that their values can be controlled directly moreover they depend only on the limiting behavior of the control this has immediate application to adaptive control of markov chains including adaptive control under constraints

circumventing the slater conundrum in countably infinite linear programs abstract duality results on countably infinite linear programs are scarce subspaces that admit an interior point which is a sufficient condition for a zero duality gap yield a dual where the constraints cannot be expressed using the ordinary transpose of the primal constraint matrix subspaces that permit a dual with this transpose do not admit an interior point this difficulty has stumped researchers for a few decades it has recently been called the slater conundrum we find a way around this hurdle we propose a pair of primal dual spaces with three properties the series in the primal and dual objective functions converge the series defined by the rows and columns of the primal constraint matrix converge and the order of sums in a particular iterated series of a double sequence defined by the primal constraint matrix can be interchanged so that the dual is defined by the ordinary transpose weak duality and complementary slackness are then immediate instead of using interior point conditions to establish a zero duality gap we call upon the planning horizon method when the series in the primal and dual constraints are continuous we prove that strong duality holds if a sequence of optimal solutions to finite dimensional truncations of the primal and dual cilps has an accumulation point we show by counterexample that the requirement that such an accumulation point exist cannot be relaxed our results are illustrated using several examples and are applied to countable state markov decision processes and to a problem in robust optimization

a class of inexact variable metric proximal point algorithms for the problem of solving maximal monotone inclusions we present a rather general class of algorithms which contains hybrid inexact proximal point methods as a special case and allows for the use of a variable metric in subproblems the global convergence and local linear rate of convergence are established under standard assumptions we demonstrate the advantage of variable metric implementation in the case of solving systems of smooth monotone equations by the proximal newton method

efficient algorithms for center problems in cactus networks efficient algorithms for solving the center problems in weighted cactus networks are presented in particular we have proposed the following algorithms for the weighted cactus networks of size n an o nlogn time algorithm to solve the 1 center problem and an o nlog 3n time algorithm to solve the weighted continuous 2 center problem we have also provided improved solutions to the general p center problems in cactus networks the developed ideas are then applied to solve the obnoxious 1 center problem in weighted cactus networks

algorithm to find a maximum 2 packing set in a cactus abstract in this paper we address the problem of finding a maximum 2 packing set in a cactus let g v g e g be an undirected connected graph then a subset s v g is a 2 packing set if for every pair of vertices u v s the shortest path between them is at least three edges long this type of sets results in a wide range of applications such as the study of molecules network modeling allocation of facilities and frequency assignment the cactus is a planar graph such that any edge belongs to at most one cycle hitherto to the best of the authors knowledge no algorithm finds a maximum 2 packing set in a cactus this problem is np hard in arbitrary graphs our approach to solving this problem was first finding a maximum 2 packing set in a unicyclic graph then we adapted this algorithm to run in a cactus the time complexity of the resulting algorithm is o n 2 where n v g

sample path average optimality for markov control processes the authors consider a markov control process with borel state and actions spaces unbounded costs and under the long run sample path average cost criterion they prove that under very weak assumptions on the transition law and a moment assumption for the one step cost there exists a stationary policy with invariant probability distribution v that is sample path average cost optimal for v almost all initial states in addition every expected average cost optimal stationary policy is in fact liminf sample path average cost optimal and strongly expected average cost optimal

an o n log 2 n algorithm for the kth longest path in a tree with applications to location problems many known algorithms are based on selection in a set whose cardinality is superlinear in terms of the input length it is desirable in these cases to have selection algorithms that run in sublinear time in terms of the cardinality of the set this paper presents a successful development in this direction the methods developed here are applied to improve the previously known upper bounds for the time complexity of various location problems

multiple objective nonatomic markov decision processes with total reward criteria we consider a markov decision process with an uncountable state space and multiple rewards for each policy its performance is evaluated by a vector of total expected rewards under the standard continuity assumptions and the additional assumption that all initial and transition probabilities are nonatomic we prove that the set of performance vectors for all policies is equal to the set of performance vectors for nonrandomized markov policies this result implies the existence of optimal nonrandomized markov policies for nonatomic constrained markov decision processes with total rewards we provide two examples of applications of our results to constrained multiple objective problems in inventory control and finance

a family of projective splitting methods for the sum of two maximal monotone operators a splitting method for two monotone operators a and b is an algorithm that attempts to converge to a zero of the sum a b by solving a sequence of subproblems each of which involves only the operator a or only the operator b prior algorithms of this type can all in essence be categorized into three main classes the douglas peaceman rachford class the forward backward class and the little used double backward class through a certain extended solution set in a product space we construct a fundamentally new class of splitting methods for pairs of general maximal monotone operators in hilbert space our algorithms are essentially standard projection methods using splitting decomposition to construct separators we prove convergence through fejer monotonicity techniques but showing fejer convergence of a different sequence to a different set than in earlier splitting methods our projective algorithms converge under more general conditions than prior splitting methods allowing the proximal parameter to vary from iteration to iteration and even from operator to operator while retaining convergence for essentially arbitrary pairs of operators the new projective splitting class also contains noteworthy preexisting methods either as conventional special cases or excluded boundary cases

exact finite approximations of average cost countable markov decision processes for a countable state markov decision process we introduce an embedding which produces a finite state markov decision process the finite state embedded process has the same optimal cost and moreover it has the same dynamics as the original process when restricting to the approximating set the embedded process can be used as an approximation which being finite is more convenient for computation and implementation