contraction conditions for average and α discount optimality in countable state markov games with unbounded rewards the goal of this paper is to provide a theory of n person markov games with unbounded cost for a countable state space and compact action spaces we investigate both the finite and infinite horizon problems for the latter we consider the discounted cost as well as the expected average cost we present conditions for the infinite horizon problems for which equilibrium policies exist for all players within the stationary policies and show that the costs in equilibrium policies exist for all players within the stationary policies and show that the costs in equilibrium satisfy the optimality equations similar results are obtained for the finite horizon costs for which equilibrium policies are shown to exist for all players within the markov policies as special case of n person games we investigate the zero sum 2 players game for which we establish the convergence of the value iteration algorithm we conclude by studying an application of a zero sum markov game in a queueing model

partially observed semi markov zero sum games with average payoff we study a zero sum partially observed semi markov game with average payoff on a countable state space under certain ergodicity conditions we show that a saddle point equilibrium exists we achieve this by solving the corresponding average cost optimality equation using a span contraction method the average value is shown to be the unique zero of a lipschitz continuous function a value iteration scheme is developed to compute the value

mixed risk neutral minimax control of discrete time finite state markov decision processes addresses the control design problem for discrete time finite state markov decision processes when both risk neutral and minimax objectives are of interest we introduce the mixed risk neutral minimax objective and utilize results from risk neutral and minimax control to derive an information state process and dynamic programming equations for the value function we synthesize optimal control laws both on the finite and the infinite horizons we study the effectiveness of both the mixed risk neutral minimax family and the risk sensitive family of controllers as tools to tradeoff risk neutral and minimax objectives we conclude that the mixed risk neutral minimax family is more effective at the cost of increased controller complexity

perfect information games with upper semicontinuous payoffs it has been shown that every n person perfect information game with no chance moves and bounded lower semicontinuous payoffs has a subgame perfect e equilibrium in pure strategies here the same is proved when the payoffs are bounded and upper semicontinuous

subgame perfection in quitting games with perfect information and differential equations we introduce a new approach to studying subgame perfect equilibrium payoffs in stochastic games the differential equations approach we apply our approach to quitting games with perfect information those are sequential games in which at every stage one ofn players is chosen each player is chosen with probability 1 n the chosen playeri decides whether to quit in which case the game terminates and the terminal payoff is some vectora i rn or whether to continue in which case the game continues to the next stage if no player ever quits the payoff is some vectora rn we define a certain differential inclusion prove that it has at least one solution and prove that every vector on a solution of this differential inclusion is a subgame perfect equilibrium payoff

on the equivalence of two expected average cost criteria for semi markov control processes the two expected average costs used in the theory of semi markov control processes with a borel state space are considered under some stochastic stability conditions we prove that the two criteria are equivalent in the sense that they lead to the same optimality equation

n by n checkers is exptime complete the game of checkers can easily be generalized to be played on an n by n board and the complexity of deciding questions about positions regarded as a function of n this paper considers mainly the question of whether a particular player can force a win from a given position and also the question of what is the best move in a given position each of these problems is shown to be complete in exponential time this means that any algorithm to solve them must take time which rises exponentially with respect to some power of n and moreover that they are amongst the hardest problems with such a time bound for instance if there are any problems solvable in exponential time but not in polynomial space then these two problems are amongst them

subgame perfection in free transition games abstract we prove the existence of a subgame perfect e equilibrium for every e 0 in a class of multi player games with perfect information which we call free transition games the novelty is that a non trivial class of perfect information games is solved for subgame perfection with multiple non terminating actions in which the payoff structure is generally not upper or lower semi continuous due to the lack of semi continuity there is no general rule of comparison between the payoffs that a player can obtain by deviating a large but finite number of times or respectively infinitely many times we introduce new techniques to overcome this difficulty our construction relies on an iterative scheme which is independent of e and terminates in polynomial time with the following output for all possible histories h a pure action a h 1 or in some cases two pure actions a h 2 and b h 2 for the active player at h the subgame perfect e equilibrium then prescribes for every history h that the active player plays a h 1 with probability 1 or respectively plays a h 2 with probability 1 e and b h 2 with probability e here e is arbitrary as long as it is positive and small compared to e so the strategies can be made almost pure

alternation with restrictions on looping alternating turing machines with restrictions preventing them from returning to a previous configuration model games with rules enforcing such a restriction for instance the chinese version of go such restrictions do not affect the time complexity of problems for alternating turing machines but space s on a machine with the restriction is equivalent either to time or to space exponential in s on a normal alternating machine depending on the precise nature of the restriction

stationary strategies for recursive games we study two person zero sum recursive matrix games framing them in the more general context of nonleavable games our aim is to extend a result of orkin orkin m 1972 recursive matrix games j appl probab 9 813 820 by proving that a uniformly e optimal stationary strategy is available to player i player ii in any recursive game such that the set where the value of the game is strictly greater less than its utility is finite the proof exploits some new connections between nonleavable and leavable games in particular we find sufficient conditions for the existence of e optimal stationary strategies for player i in a leavable game and we use these as a basis for constructing uniformly e optimal stationary strategies for the recursive games with which we are concerned

hybrid adaptive wavelet neuro fuzzy system for chaotic time series identification in the paper a five layers architecture of hybrid wavelet neuro fuzzy system which is using the adaptive w neurons as the nodes is proposed w neuron is a neuron which structure is similar to a radial basis functions network but instead of conventional radial basis functions we used multidimensional adaptive wavelet activation membership functions the distinctive feature of the proposed system is usage of the wavelets as membership functions in the antecedent layer and the adaptive multidimensional wavelets as activation functions in the consequent layer that can tune not only the dilation and translation parameters but also its own form during the learning process the learning algorithms for all antecedent and consequent functions parameters that have both following and filtering properties are proposed the experimental results have shown that this wavelet neuro fuzzy system has improved approximation properties and has a higher learning rate in comparison with usual wavelet neuro fuzzy networks the proposed hybrid wavelet neuro fuzzy system can be used to solve tasks of diagnosis forecasting emulation and identification of nonlinear chaotic and stochastic non stationary processes

transcale control for a class of discrete stochastic systems based on wavelet packet decomposition abstract in this paper a wavelet packet decomposition wpd based real time transcale linear quadratic gaussian lqg tracking control algorithm is given for a class of discrete stochastic systems which is developed with the state space models of wavelet packet coefficients at the coarsest scale decomposition layer established by haar wpd the wpd based transcale control algorithm provides a compromise between performance index and computational efficiency compared with the conventional lqg tracking control algorithm and it is an improvement of the wavelet transform based algorithm previously proposed simulation results are presented to demonstrate the effectiveness of the proposed algorithm

perfect information games with lower semicontinuous payoffs we prove that every multiplayer perfect information game with bounded and lower semicontinuous payoffs admits a subgame perfect e equilibrium in pure strategies this result complements example 3 in solan and vieille solan e n vieille 2003 deterministic multi player dynkin games j math econom 39 911 929 which shows that a subgame perfect e equilibrium in pure strategies need not exist when the payoffs are not lower semicontinuous in addition if the range of payoffs is finite we characterize in the form of a folk theorem the set of all plays and payoffs that are induced by subgame perfect 0 equilibria in pure strategies

existence of stationary correlated equilibria with symmetric information for discounted stochastic games the following theorem is proved every nonzero sum discounted stochastic game in countably generated measurable state space with compact metric action spaces admits a stationary correlated equilibrium point with symmetric public information whenever the immediate rewards and transition densities are measurable with respect to the state variable and continuous with respect to joint actions

a new class of wavelet networks for nonlinear system identification a new class of wavelet networks wns is proposed for nonlinear system identification in the new networks the model structure for a high dimensional system is chosen to be a superimposition of a number of functions with fewer variables by expanding each function using truncated wavelet decompositions the multivariate nonlinear networks can be converted into linear in the parameter regressions which can be solved using least squares type methods an efficient model term selection approach based upon a forward orthogonal least squares ols algorithm and the error reduction ratio err is applied to solve the linear in the parameters problem in the present study the main advantage of the new wn is that it exploits the attractive features of multiscale wavelet decompositions and the capability of traditional neural networks by adopting the analysis of variance anova expansion wns can now handle nonlinear identification problems in high dimensions

the optimal reward operator in negative dynamic programming we consider the negative dynamic programming model of strauch 12 and prove that the optimal reward function can be obtained by a transfinite iteration of the optimal reward operator we show that a player loses nothing by restricting himself to measurable policies if the returns from nonmeasurable policies are evaluated by lower integrals

parameter identification of tsk neuro fuzzy models abstract in the framework of the tsk neuro fuzzy model a combination of the two well known identification methods are employed for parameter estimation of the neuro fuzzy inference system namely the series parallel and the parallel configurations the presented paper proposes two new possible configurations for identifying the parameters of the tsk neuro fuzzy model using the combinations of these two existing configurations one of the proposed configurations constitutes the series parallel configuration to the premise part and the parallel configuration to the consequent part of the neuro fuzzy model termed as ps p configuration the second one is composed of the series parallel configuration to the consequent part and the parallel configuration to the premise part of the neuro fuzzy model termed as cs p configuration the presented work mainly deals with a comparative study of the proposed configurations and the existing configurations in the context of parameter identification of the tsk neuro fuzzy model on three different benchmark examples moreover it investigates upper bound of the learning rates using the lyapunov stability theorem to assure the stability and the convergence of the model learning process implementation of the modified mountain clustering mmc and the cluster validity function yields initial models to restrict the upper bound during the learning process it also presents a two phase adaptive learning rate

measurable selection theorems for minimax stochastic optimization problems this paper resolves the measurability questions which arise in the analysis of various minimax stochastic optimization models posed in borel spaces for a large class of such models we provide a set of sufficient conditions which ensure that the questions have positive answers these conditions are essentially weaker than those described in the existing literature moreover they allow us to conclude a fundamental theorem yielding some new results on the existence of universally measurable selections of extrema for both zero sum stochastic games and minimax stochastic optimal control problems in particular a random version of the well known minimax theorem of fan is in this way established the fundamental theorem yields also a descriptive set theoretic fact concerning the projections of borel sets the paper indicates also some counterexamples to possibly more general problems

single hidden layer fuzzy recurrent wavelet neural network when a fuzzy wavelet neural network fwnn has a large number of neurons in its consequent part it may not be able to effectively follow fast variations in the process this paper aims to develop a single hidden layer fuzzy recurrent wavelet neural network slfrwnn for the function approximation and identification of dynamic systems in the proposed framework the consequent part of each fuzzy rule is developed by a single neuron with the capability of storing the past data of the network to guarantee the convergence and to speed up the process of the on line training algorithm the optimal learning rate olr is introduced based on lyapunov stability theory the modifications allow the slfrwnn to be much faster than fwnn and hence it is more appropriate in real time applications the efficiency of the modified model is investigated using three different types of wavelet families in approximating a benchmark piecewise function furthermore two nonlinear dynamic plants are considered to demonstrate the feasibility of the slfrwnn in the system identification the results indicate that slfrwnn achieves higher accuracy and needs a lower number of parameters than the other models

best basis for joint representation the median of marginal best bases for low cost information exchanges in distributed signal representation the paper addresses the selection of the best representations for distributed and or dependent signals given an indexed tree structured library of bases and a semi collaborative distribution scheme associated with minimum information exchange emission and reception of one single index corresponding to a marginal best basis the paper proposes the median basis computed on a set of best marginal bases for joint representation or fusion of distributed dependent signals the paper provides algorithms for computing this median basis with respect to standard tree structured libraries of bases such as wavelet packet bases or cosine trees these algorithms are effective when an additive information cost is under consideration experimental results performed on distributed signal compression confirms worthwhile properties for the median of marginal best bases with respect to the ideal best joint basis the latter being underdetermined in practice except when a full collaboration scheme is under consideration

a stochastic games framework for verification and control of discrete time stochastic hybrid systems we describe a framework for analyzing probabilistic reachability and safety problems for discrete time stochastic hybrid systems within a dynamic games setting in particular we consider finite horizon zero sum stochastic games in which a control has the objective of reaching a target set while avoiding an unsafe set in the hybrid state space and a rational adversary has the opposing objective we derive an algorithm for computing the maximal probability of achieving the control objective subject to the worst case adversary behavior from this algorithm sufficient conditions of optimality are also derived for the synthesis of optimal control policies and worst case disturbance strategies these results are then specialized to the safety problem in which the control objective is to remain within a safe set we illustrate our modeling framework and computational approach using both a tutorial example with jump markov dynamics and a practical application in the domain of air traffic management

the existence of subgame perfect equilibrium in continuous games with almost perfect information a case for public randomization the starting point of this paper is a simple regular dynamic game in which subgame perfect equilibrium fails to exist examination of this example shows that existence would be restored if players were allowed to observe the output of a public randomization device the main result of the paper shows that the introduction of public randomization yields existence not only in the example but also in a large class of dynamic games it is also argued that the introduction of public randomization is the minimal robust extension of subgame perfect equilibrium in this class of games

complexity of path forming games abstract for a number of two player games where players alternately choose the next vertex of a simple or an elementary path in a graph we consider the problem to determine whether for a given game instance there is a winning strategy for the first player we show several of these problems to be pspace complete in some special cases we obtain polynomial time algorithms based on graph rewriting or an intricate form of dynamic programming e g we show generalized geography and some other pspace complete problems to be linear time solvable on graphs with constant bounded treewidth

fuzzy systems as universal approximators an additive fuzzy system can uniformly approximate any real continuous function on a compact domain to any degree of accuracy an additive fuzzy system approximates the function by covering its graph with fuzzy patches in the input output state space and averaging patches that overlap the fuzzy system computes a conditional expectation e y x if we view the fuzzy sets as random sets each fuzzy rule defines a fuzzy patch and connects commonsense knowledge with state space geometry neural or statistical clustering systems can approximate the unknown fuzzy patches from training data these adaptive fuzzy systems approximate a function at two levels at the local level the neural system approximates and tunes the fuzzy rules at the global level the rules or patches approximate the function

zero sum stochastic games in borel spaces average payoff criteria this paper is concerned with two person zero sum dynamic stochastic games in borel spaces with possibly unbounded payoff function and several average or ergodic payoff criteria we give conditions under which the long run expected average payoff criterion the sample path average criterion the existence of solutions to the average payoff shapley equations and a certain martingale condition are all equivalent

minimax control of discrete time stochastic systems this paper gives a unified self contained presentation of minimax control problems for discrete time stochastic systems on borel spaces with possibly unbounded costs the main results include conditions for the existence of minimax strategies for finite horizon problems and infinite horizon discounted and undiscounted average cost criteria the results are specialized to control systems with unknown disturbance distributions also known as games against nature two examples illustrate the theory one of them on the mold level control problem which is a key problem in the steelmaking industry

borel stochastic games with lim sup payoff we consider two person zero sum stochastic games with limit superior payoff function and borel measurable state and action spaces the games are shown to have a value and the value function is calculated by transfinite iteration of an operator and proved to be upper analytic the paper extends results of our earlier article 17 in which the same class of games was considered for countable state spaces and finite action sets

wavelet neural networks for function learning a wavelet based neural network is described the structure of this network is similar to that of the radial basis function rbf network except that in the present paper the radial basis functions are replaced by orthonormal scaling functions that are not necessarily radial symmetric the efficacy of this type of network in function learning and estimation is demonstrated through theoretical analysis and experimental results in particular it has been shown that the wavelet network has universal and l sup 2 approximation properties and is a consistent function estimator convergence rates associated with these properties are obtained for certain function classes where the rates avoid the curse of dimensionality in the experiments the wavelet network performed well and compared favorably to the mlp and rbf networks

existence of value and randomized strategies in zero sum discrete time stochastic dynamic games two players with conflicting objectives are simultaneously controlling a discrete time stochastic system the goal of this paper is to analyze such zero sum discrete time stochastic systems when the two players are allowed to use randomized strategies previous results have been restricted to systems with finite or compact state spaces such restrictions are usually untenable from the point of view of applications since many applications frequently use either the integers or mathbb r n as a state space our results are proved for complete separable metric spaces which are very useful for applications all previously known results emerge as special cases of our results in addition a variety of conjectures and open problems are resolved regarding the existence of a value function its properties such as borel measurability or continuity and the existence for either or both players of optimal or varepsilon optimal stationary strategies

concurrent reachability games we consider concurrent two player games with reachability objectives in such games at each round player 1 and player 2 independently and simultaneously choose moves and the two choices determine the next state of the game the objective of player 1 is to reach a set of target states the objective of player 2 is to prevent this these are zero sum games and the reachability objective is one of the most basic objectives determining the set of states from which player 1 can win the game is a fundamental problem in control theory and system verification there are three types of winning states according to the degree of certainty with which player 1 can reach the target from type 1 states player 1 has a deterministic strategy to always reach the target from type 2 states player 1 has a randomized strategy to reach the target with probability 1 from type 3 states player 1 has for every real e 0 a randomized strategy to reach the target with probability greater than email protected we show that for finite state spaces all three sets of winning states can be computed in polynomial time type 1 states in linear time and type 2 and type 3 states in quadratic time the algorithms to compute the three sets of winning states also enable the construction of the winning and spoiling strategies

zero sum semi markov games this paper deals with borel state and action spaces zero sum semi markov games under the expected long run average payoff criterion the transition probabilities are assumed to satisfy some generalized geometric ergodicity conditions the main result states that the optimality equation has a solution which is approximated by the solutions of some varepsilon perturbed semi markov games as a corollary the existence of value and average optimal strategies for the players is established

recurrence conditions for average and blackwell optimality in denumerable state markov decision chains in a previous paper dekker and hordijk 1988 presented an operator theoretical approach for multichain markov decision processes with a countable state space compact action sets and unbounded rewards conditions were presented guaranteeing the existence of a laurent series expansion for the discounted rewards the existence of average and blackwell optimal policies and the existence of solutions for the average and blackwell optimality equations while these assumptions were operator oriented and formulated as conditions for the deviation matrix we will show in this paper that the same approach can also be carried out under recurrence conditions these new conditions seem easier to check in general and are especially suited for applications in queueing models

stationary markov equilibria conditions are established under which there are equilibria in various settings described by ergodic markov processes with a borel state space s let rho s denote the probability measures on s and let s approaching g s is a subset of rho s be a possibly empty valued correspondence with closed graph a non empty measurable set j is a subset of s is self justified if g s intersects rho j is not empty for all s an element of j a time homogeneous markov equilibrium thme for g is a self justified set j and a measurable selection ii j approaching rho j from the restriction of g to j the paper gives sufficient conditions for existence of compact self justified sets and applies the theorem if g is convex valued and has a compact self justified set then g has an thme with an ergodic measure coauthors are j geanakoplos a mas colell and a mclennan copyright 1994 by the econometric society

zero sum ergodic stochastic games with feller transition probabilities we consider zero sum stochastic games with borel state spaces satisfying a generalized geometric ergodicity condition the main objective of this paper is to establish a minimax theorem for a class of ergodic stochastic games with the feller transition probability function all previous results on ergodic stochastic games are based on a strong continuity of the transition probabilities in the actions of the players

a recurrent self organizing neural fuzzy inference network a recurrent self organizing neural fuzzy inference network rsonfin is proposed the rsonfin is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference and may be considered to be constructed from a series of dynamic fuzzy rules the temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network each weight as well as node in the rsonfin has its own meaning and represents a special element in a fuzzy rule there are no hidden nodes initially in the rsonfin they are created online via concurrent structure identification and parameter identification the structure learning together with the parameter learning forms a fast learning algorithm for building a small yet powerful dynamic neural fuzzy network two major characteristics of the rsonfin can thus be seen 1 the recurrent property of the rsonfin makes it suitable for dealing with temporal problems and 2 no predetermination like the number of hidden nodes must be given since the rsonfin can find its optimal structure and parameters automatically and quickly moreover to reduce the number of fuzzy rules generated a flexible input partition method the aligned clustering based algorithm is proposed various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made efficiency of the rsonfin is verified from these results

a capacitability theorem in measurable gambling theory a player in a measurable gambling house r defined on a polish state space x has available for each x e x the collection t x of possible distributions a for the stochastic process xx xi of future states if the object is to control the process so that it will lie in an analytic subset a of h x x x x then the player s optimal reward is m a x sup rj a e the operator m x is shown to be regular in the sense that m a x mm x oo x where the infimum is over borel stopping times t such that a c x 00 a consequence of this regularity property is that the value of m a x is un changed if as in the gambling theory of dubins and savage the player is allowed to use nonmeasurable strategies this last result is seen to hold for bounded borel measurable payoff functions including that of dubins and savage

a tauberian theorem for nonexpansive operators and applications to zero sum stochastic games we prove a tauberian theorem for nonexpansive operators and apply it to the model of zero sum stochastic game under mild assumptions we prove that the value of the discounted game converges uniformly when goes to zero if and only if the value of the n stage game converges uniformly when n goes to infinity this generalizes the tauberian theorem of lehrer and sorin lehrer e sorin s 1992 a uniform tauberian theorem in dynamic programming math oper res 17 2 303 307 to the two player zero sum case we also provide the first example of a stochastic game with public signals on the state and perfect observation of actions with finite state space signal sets and action sets in which for some initial state known by both players the value of the discounted game and the value of the n stage game starting at that initial state converge to distinct limits

minimax selection theorems on considere un jeu a somme nulle et a 2 joueurs et on donne certaines conditions generales nouvelles qui assurent l existence et la mesurabilite de la fonction valeur du jeu et l existence de strategies mesurables optimales pour les deux joueurs

subgame perfection in positive recursive games with perfect information we consider a class of n player stochastic games with the following properties 1 in every state the transitions are controlled by one player 2 the payoffs are equal to zero in every nonabsorbing state 3 the payoffs are nonnegative in every absorbing state we propose a new iterative method to analyze these games with respect to the expected average reward we prove the existence of a subgame perfect e equilibrium in pure strategies for every e 0 moreover if all transitions are deterministic we obtain a subgame perfect 0 equilibrium in pure strategies

discounted stochastic games we give an alternative proof to a result of mertens and parthasarathy stating that every n player discounted stochastic game with general setup and with a norm continuous transition has a subgame perfect equilibrium

leavable gambling problems with unbounded utilities the optimal return function u of a borel measurable gambling problem with a positive utility function is known to be universally measurable with a negative utility function however u may not be so measurable as shown here the measurability of u for all borel gambling problems with negative utility functions is equivalent to the measurability of all pca sets a property of such sets known to be independent of the usual axioms of set theory if the utility function is further required to satisfy certain uniform integrability conditions or if the gambling problem corresponds to an optimal stopping problem the optimal return function is measurable

advertising models stochastic games and myopic strategies this paper analyzes dynamic advertising models we present conditions that imply that the payoffs under myopic strategies are linear in the state variable which represents the number of customers for each firm these conditions which determine a subclass of stochastic games are satisfied by several dynamic advertising models we give a criterion to check whether this model contains an equilibrium point within the class of myopic strategies that is also an equilibrium point with regard to all strategies

on the relation between recurrence and ergodicity properties in denumerable markov decision chains this paper studies two properties of the set of markov chains induced by the deterministic policies in a markov decision chain these properties are called uniform geometric ergodicity and uniform geometric recurrence uniform ergodicity generalises a quasi compactness condition it can be interpreted as a strong version of stability as it implies that the markov chains generated by the deterministic stationary policies are uniformly stable uniform geometric recurrence can be shown to be equivalent to the simultaneous doeblin condition if is bounded both properties imply the existence of deterministic average and sensitive optimal policies the second key theorem in this paper shows the equivalence of uniform geometric ergodicity and weak uniform geometric recurrence under appropriate continuity conditions in the literature numerous recurrence conditions have been used the first key theorem derives the relation between several of these conditions which interestingly turn out to be equivalent in most cases

pure subgame perfect equilibria in free transition games we consider a class of stochastic games where each state is identified with a player at any moment during play one of the players is called active the active player can terminate the game or he can announce any player who then becomes the active player there is a non negative payoff for each player upon termination of the game which depends only on the player who decided to terminate we give a combinatorial proof of the existence of subgame perfect equilibria in pure strategies for the games in our class

subgame perfect equilibria for stochastic games for an n person stochastic game with borel state space s and compact metric action sets a1 a2 an sufficient conditions are given for the existence of subgame perfect equilibria one result is that such equilibria exist if the law of motion q s a is for fixed s continuous in a a1 a2 an for the total variation norm and the payoff functions f1 f2 fn are bounded borel measurable functions of the sequence of states s1 s2 sn and in addition are continuous when sn is given the product of discrete topologies on s

on model checking for the m calculus and its fragments in this paper we consider the model checking problem for the calculus and show that it is succinctly equivalent to the non emptiness problem of finite state automata on infinite binary trees with the parity acceptance condition we also present efficient model checking algorithms for two rich subclasses of the calculus formulas and relate their expressive power to well known extensions of branching time temporal logics

subgame perfect ϵ equilibria in perfect information games with common preferences at the limit we prove the existence of a pure subgame perfect epsilon equilibrium for every epsilon 0 in multiplayer perfect information games provided that the payoff functions are bounded and exhibit common preferences at the limit if in addition the payoff functions have finite range then there exists a pure subgame perfect 0 equilibrium these results extend and unify recent existence theorems for bounded and semicontinuous payoffs

on markov games with average reward criterion and weakly continuous transition probabilities in this paper we consider two person zero sum stochastic games with the average reward criterion and borel state and action spaces a geometric drift condition is assumed we show that the optimality shapley equation has a unique solution if the transition probability function is weakly continuous the stage reward is lower semicontinuous and the set valued mappings of admissible actions satisfy some semicontinuity assumptions furthermore the minimizing player has an optimal stationary strategy and the maximizing player has an e optimal stationary strategy for every varepsilon 0

a characterization of subgame perfect equilibrium plays in borel games of perfect information we provide a characterization of subgame perfect equilibrium plays in a class of perfect information games where each player s payoff function is borel measurable and has finite range the set of subgame perfect equilibrium plays is obtained through a process of iterative elimination of plays extensions to games with bounded borel measurable payoff functions are discussed as an application of our results we show that if every player s payoff function is bounded and upper semicontinuous then for every positive epsilon the game admits a subgame perfect epsilon equilibrium as we do not assume that the number of players is finite this result generalizes the corresponding result of purves and sudderth 24 purves ra sudderth wd 2011 perfect information games with upper semicontinuous payoffs math oper res 36 3 468 473

on measurable minimax selectors in this note we consider the upper value of a zero sum game with payoff function depending on a state variable we provide a new and much simpler proof of a measurable minimax selection theorem established 25 years ago by the author in nowak 1985 19 a discussion of the basic assumptions and relations with the literature on stochastic games and minimax control models is also included

stopping games with randomized strategies we study stopping games in the setup of neveu we prove the existence of a uniform value in a sense defined below by allowing the players to use randomized strategies in constrast with previous work we make no comparison assumption on the payoff processes moreover we prove that the value is the limit of discounted values and we construct e optimal strategies

a globally convergent adaptive predictor this paper establishes global convergence for an adaptive prediction algorithm key features of the algorithm are i a priori residuals are used in the regression vector ii the input signal need not be persistently exciting iii only an upper bound is required on the system order iv no monitoring or projection procedure is required to guarantee stability and v arbitrary feedback is allowed between output and input

semicontinuous nonstationary stochastic games ii abstract in this paper we introduce a borel space framework for zero sum discrete time stochastic games that is a game theoretic extension of some nonstationary dynamic programming models in the sense of hinderer our game model which allows for all of the primitive data to be nonstationary contains a large class of markov games at the same time it constitutes a considerable generalization of the game model introduced by sengupta to ensure the existence of a value of a nonstationary stochastic game and the existence of universally measurable optimal or e optimal strategies for the players we impose some asymmetric semicontinuity and compactness conditions on the primitive data that are weaker than those described in the existing literature on the subject a certain special case in which the players can restrict themselves to borel measurable strategies is also studied

varepsilon equilibria for stochastic games with uncountable state space and unbounded costs we study a class of noncooperative stochastic games with unbounded cost functions and an uncountable state space it is assumed that the transition law is absolutely continuous with respect to some probability measure on the state space undiscounted stochastic games with expected average costs are considered first it is shown under a uniform geometric ergodicity assumption that there exists a stationary varepsilon equilibrium for each varepsilon 0 the proof is based on recent results on uniform bounds for convergence rates of markov chains s p meyn and r l tweedie ann appl probab 4 1994 pp 981 1011 and on an approximation method similar to that used in a s nowak j optim theory appl 45 1985 pp 591 602 where an varepsilon equilibrium in stationary policies was shown to exist for the bounded discounted costs the stochastic game is approximated by one with a countable state space for which a stationary nash equilibrium exists see e altman a hordijk and f m spieksma math oper res 22 1997 pp 588 618 this equilibrium determines an epsilon equilibrium for the original game finally new results for the existence of stationary varepsilon equilibrium for discounted stochastic games are given

zero sum continuous time markov pure jump game over a fixed duration abstract this paper considers a two person zero sum continuous time markov pure jump game in borel state and action spaces over a fixed finite horizon the main assumption on the model is the existence of a drift function which bounds the reward rate under some regularity conditions we show that the game has a value and both of the players have their optimal policies