lower semicontinuity property of multiparameter optimal stopping value and its application to multiparameter prophet inequalities abstract this paper is concerned with the optimal stopping problem for discrete time multiparameter stochastic processes with the index set n d the optimal stopping value of a discrete time multiparameter integrable stochastic process whose negative part is uniformly integrable is lower semicontinuous for the topology of convergence in distribution the multiparameter version of prophet inequality for the one parameter optimal stopping problem is formulated and the lower semicontinuity property of the optimal stopping value is applied to the multiparameter prophet inequality

on average cost stopping time problems we consider the stopping time problems whose costs are given by time average forms a generalization of the gittins index we show the existence of the optimal solutions and give their approximation to these stopping time problems

a phase transition for the coupled branching process part i the ergodic theory in the range of finite second moments we consider a particular markov process tu on s s n the random variable tu x is interpreted as the number of particles atx at timet the initial distribution of this process is a translation invariant measure withf x d the evolution is as follows at rateb x a particle is born atx but moves instantaneously toy chosen with probabilityq x y all particles at a site die at ratepd withp 0 1 d and individual particles die independently from each other at rate 1 p d every particle moves independently of everything else according to a continuous time random walk

n by n checkers is exptime complete the game of checkers can easily be generalized to be played on an n by n board and the complexity of deciding questions about positions regarded as a function of n this paper considers mainly the question of whether a particular player can force a win from a given position and also the question of what is the best move in a given position each of these problems is shown to be complete in exponential time this means that any algorithm to solve them must take time which rises exponentially with respect to some power of n and moreover that they are amongst the hardest problems with such a time bound for instance if there are any problems solvable in exponential time but not in polynomial space then these two problems are amongst them

rush hour is psapce complete or why you should generously tip parking lot attendants rush hour is a children s game that consists of a grid board several cars that are restricted to move either vertically or horizontally but not both a special target car and a single exit on the perimeter of the grid the goal of the game is to find a sequence of legal moves that allows the target car to exit the grid we consider a slightly generalized version of the game that uses an nn grid and assume that we can place the single exit and target car at any location we choose on initialization of the game in this work we show that deciding if the target car can legally exit the grid is pspace complete our constructive proof uses a lazy form of dual rail reversible logic such that movement of output cars can only occur if logical combinations of input cars can also move emulating this logic only requires three types of devices two switches and one crossover thus our proof technique can be easily generalized to other games and planning problems in which the same three primitive devices can be constructed

optimal strategies for a class of sequential control problems with precedence relations consider the following multi phase project management problem each project is divided into several phases all projects enter the next phase at the same point chosen by the decision maker based on observations up to that point within each phase one can pursue the projects in any order when pursuing the project with one unit of resource the project state changes according to a markov chain the probability distribution of the markov chain is known up to an unknown parameter when pursued the project generates a random reward depending on the phase and the state of the project and the unknown parameter the decision maker faces two problems a how to allocate resources to projects within each phase and b when to enter the next phase so that the total expected reward is as large as possible in this paper we formulate the preceding problem as a stochastic scheduling problem and propose asymptotic optimal strategies which minimize the shortfall from perfect information payoff concrete examples are given to illustrate our method

discrete multi armed bandits and multi parameter processes the general multi armed bandit problem is reformulated and solved as a control problem over a partially ordered set the approach taken provides a technically convenient framework for bandit like problems it also adds insight to the structure of strategies over partially ordered sets

optimal switching among several brownian motions for i 1 cdots d let b s i i be a one dimensional brownian motion on the interval 0 a i with absorption at the endpoints at each instant in time one must decide to run some subset of these d brownian motions while holding the others fixed at their current state the resulting process evolves in the rectangle d 0 a 1 times cdots times 0 a d if at some instant one decides to freeze all of the brownian motions then a reward is received in accordance with this final position two types of reward functions are considered first it is assumed that the reward is zero everywhere in d except along the d edges that correspond to the coordinate axes along these edges it is given by c 3 strictly concave functions gamma i x i which are zero at the endpoints 0 and a i of their domains the optimal control for this problem has a simple description let gamma i left x i right int 0 x i u gamma i u du and put m i left x in d gamma

a stochastic representation theorem with applications to optimization and obstacle problems we study a new type of representation problem for optional processes with connections to singular control optimal stopping and dynamic allocation problems as an application we show how to solve a variant of skorohod s obstacle problem in the context of backward stochastic differential equations

the othello game on an n n board is pspace complete abstract given an arbitrary position of the othello game played on an n n board the problem of determining the winner is shown to be pspace complete it can be reduced from generalized geography played on bipartite graphs with maximum degree 3

optimal multiple stopping of linear diffusions motivated by the analysis of financial instruments with multiple exercise rights of american type and mean reverting underlyers we formulate and solve the optimal multiple stopping problem for a general linear regular diffusion process and a general reward function instead of relying on specific properties of geometric brownian motion and call and put option payoffs as in most of the existing literature we use general theory of optimal stopping for diffusions and we illustrate the resulting optimal exercise policies by concrete examples and constructive recipes

multiscale recursive estimation data fusion and regularization we describe a framework for modeling stochastic phenomena at multiple scales and for their efficient estimation or reconstruction given partial and or noisy measurements which may also be at several scales in particular multiscale signal representations lead naturally to pyramidal or tree like data structures in which each level in the tree corresponds to a particular scale of representation a class of multiscale dynamic models evolving on dyadic trees is introduced the main focus of this paper is on the description analysis and application of an extremely efficient optimal estimation algorithm for this class of models this algorithm consists of a fine to coarse filtering sweep followed by a coarse to fine smoothing step corresponding to the dyadic tree generalization of kalman filtering and rauch tung striebel smoothing the kalman filtering sweep consists of the recursive application of three steps a measurement update step a fine to coarse prediction step and a fusion step we illustrate the use of our methodology for the fusion of multiresolution data and for the efficient solution of fractal regularizations of ill posed signal and image processing problems encountered

hydraulic permeability of un bounded fibrous media using the lattice boltzmann method several articles have been written regarding the hydraulic permeability of ordered and disordered fibrous media here we explore wall effects on hydraulic permeabilities for ordered and disordered media using the lattice boltzmann lb simulation method simulation results are found to be in excellent agreement with the semi analytic result of sangani and acrivos and simulation results for disordered media are in good agreement with the results of jackson and james and higdon and ford s fcc lattice the macroscopic behavior the hydraulic permeability shows a distinct connection with the geometry of the system this connection is explored and elucidated for ordered and disordered media finally hydraulic permeabilities for bounded media at various wall separations are presented for both ordered and disordered media and results are compared with hydraulic permeabilities calculated for the unbounded media and a phenomenological correlation is presented to facilitate rapid prediction of hydraulic permeabilities for both unbounded and bounded fibrous media

applicable stochastic control from theory to practice stochastic control has come of age it is currently applied in a wide variety of fields spanning cancer research and chemotherapy economics and finance engineering management and many other areas of scientific and social investigation both theoretical models formulated to elucidate some characteristics of a problem under study and practical applications which have sought to provide specific answers to well stated problems have been the subjects of intense research for example economists have simultaneously groped with the underlying theoretical constructs justifying the adoption of particular econometric models while at the same time attempting to devise algorithms and optimization approaches to provide economic policy guidelines based on these models in the same spirit t remendous interest in options management which are based on stochastic dynamic models has stimulated much research and numerical analyses not only in universities but also in the back rooms of trading firms the usefulness of stochastic control was recognized once it became clear that it can be used as a unifying framework for treating multi stage or multi period optimization problems subject to stochastic disturbances however just as it is convenient as a formalism to formulate such problems it is becoming increasingly clear that it is practically difficult to solve them there are in fact only few problems one can solve completely and obtain optimal solutions for this reason there is today an increased awareness that a broad range of techniques including stochastic approximations perturbation techniques markov decision processes etc are an essential part of stochastic control kits it is in such a spirit that computer software such as pandore of inria the adaptive control algorithms for econometric models developed by kendrick and many others has been devised providing thereby computer aided expertise for managing stochastic control problems this software is essential even after the great effort applied to solving such problems over the last three decades ever since bellman s principle of optimality there are very few non trivial problems we can solve theoretically these include the well known linear quadratic stochastic control scp problem some version of the multi armed bandit problem and some specific linear optimization problems for example even the adaptive control problems studied over the last few years by kendrick and his co workers and presumed to be convex are believed today to include some unspecified nonconvexities private communication by d kendrick and hans amman in this spirit the intent in this paper is to introduce salient aspects of stochastic control which are useful for applications in management a review of applications can be found in neck 1984 for example as well as in books such as tapiero 1977 1988 bensoussan hurst and naslund 1974 bensoussan kleindorfer and tapiero 1978 1980 sethi and thompson 1981 kamien and schwarz 1981 malliaris and brock 1982 and whittle 1982 further due to an awesome amount of research published in this area appearing under various connotations such a task has become unwieldy rather

phutball is pspace hard we consider the nxn game of phutball it is shown that given an arbitrary position of stones on the board it is a pspace hard problem to determine whether the specified player can win the game regardless of the opponent s choices made during the game

an optimal stopping time problem with time average cost in a bounded interval abstract an optimal stopping time problem for a diffusion process stopped at the boundary of a bounded interval is studied our cost functional is of time average type which is different from standard long term average cost functional but can be considered as a version of the gittins index both characterization of the value function and construction of optimal stopping policy will be presented

predictability and stopping on lattices of sets as a first step in the development of a general theory of set indexed martingales we define predictability on a general space with respect to a filtration indexed by a lattice of sets we prove a characterization of the predictable algebra in terms of adapted and left continuous processes without any form of topology for the index set we then define a stopping set and show that it is a natural generalization of the stopping time in particular the predictable algebra can be characterized by various stochastic intervals generated by stopping sets

continuous multi armed bandits and multiparameter processes a general framework is proposed for continuous time dynamic allocation models of a scarce resource among competing projects the allocation model is formulated as a multi armed bandit model and solved as a control problem of a multiparameter process in contrast to discrete time bandits where only one arm can be pulled at a time the continuous time bandit must allow simultaneous pulls the multiparameter approach allows a strong solution of diffusion type bandits here the main problem is to define precisely how to switch among arms and the solution involves local times

a probabilistic formula for the concave hull of a function let d be a compact convex domain in d dimensional euclidean space and let f be a nonnegative real valued function defined on d the classical optimal stopping problem is to find a stopping time that attains the supremum v x sup e x f b here b is a d dimensional brownian motion with absorption on the boundary of d and the supremum is over all stopping times it is well known that is characterized as the smallest superharmonic majorant of f in this paper we modify this problem by allowing b to be essentially any drift free diffusion with absorption as before on the boundary of d for example it could be a brownian motion diffusing on some lower dimensional affine set in addition one is allowed to switch among these diffusions at any time the problem is to find a stopping time and a switching strategy that together attain the supremum over all stopping times and all switching strategies for this problem we show that is characterized as the smallest concave majorant of f the domain d can be decomposed into a disjoint union of relatively open convex sets on each of which the function is affine furthermore the union of the zero dimensional convex sets is contained in the set on which f an optimal switching strategy is any strategy that at all times diffuses in the affine hull of the current convex set when the diffusion reaches the boundary of the current convex set it will lie on a lower dimensional convex set and must then diffuse on the affine hull of this new set this process continues until the set on which f is reached which is the optimal stopping time

optimal stopping of two parameter processes on nonstandard probability spaces keywords nonstandard analysis optimal stopping problem randomized stopping times nonstandard probability theory loeb spaces conditional independence property reference prob article 1989 001doi 10 2307 2001425 record created on 2008 12 01 modified on 2017 05 12

effects of microstructure on flow properties of fibrous porous media at moderate reynolds number in this study effects of microstructure on the viscous permeability and forchheimer coefficient of monodispersed fibers are investigated the porous material is represented by a unit cell which is assumed to be repeated throughout the medium based on the orientation of the fibers in the space fibrous media are divided into three categories one two and three directional 1d 2d and 3d structures parallel and transverse flow throughsquarearrangements of1dfibers simple2dmats and3dsimplecubicstructuresaresolvednumerically over a wide range of porosity 0 35 e 0 95 and reynolds number 0 01 re 200 the results are used to calculate the permeability and the inertial coefficient of the considered geometries an experimental study is performed the flow coefficients of three different ordered tube banks in the moderate range of reynolds number 0 001 re 15 are determined the numerical results are successfully compared with the present and the existing experimental data in the literature the results suggest that the permeability and forchheimer coefficient are functions of porosity and fiber orientation a comparison of the experimental and numerical results with the ergunequationrevealsthatthisequationisnotsuitableforhighlyporousmaterials assuch accuratecorrelations are proposed for determining the forchheimer coefficient in fibrous porous media

two parameter optimal stopping and bi markov processes in this paper the optimal stopping problem is solved for particular two parameter processes here called bi markov processes a subsequent potential theory is developed with respect to a pair of one parameter semi groups we introduce a new notion of harmonicity for two variable functions and we interpret it in the framework of the theory of bi markov processes

prophet inequalities for finite stage multiparameter optimal stopping problems abstract this paper is concerned with the optimal stopping problem for discrete time multiparameter stochastic processes with the index set n d in the classical optimal stopping problems the comparisons between the expected reward of a player with complete foresight and the expected reward of a player using nonanticipating stop rules known as prophet inequalities have been studied by many authors prophet inequalities in the case of finite stage two parameter optimal stopping problems are extended to the case of finite stage general multiparameter optimal stopping problems

optional sampling of submartingales indexed by partially ordered sets the optional sampling theorem for martingales indexed by a partially ordered set is true if the index set is directed however the corresponding result for submartingales indexed by a partially ordered set is not true in general in this paper we completely characterize the class of stopping times for which the optional sampling theorem is true for all uniformly bounded submartingales indexed by countable partially ordered sets by assuming a conditional independence property we show that when the index set is r 2 the optional sampling theorem is true for all uniformly bounded right continuous submartingales and all stopping times this conditional independence property is satisfied in cases where the submartingales and stopping times are measurable with respect to the two parameter wiener process a counterexample shows that the optional sampling result is false for r n when n 2 even if the conditional independence property is satisfied

hydraulic permeability of ordered and disordered single layer arrays of cylinders the hydraulic permeability of single layer fibrous media is studied through two dimensional 2d and three dimensional navier stokes based flow simulations as simple representations of such materials one dimensional arrays of parallel cylinders as well as two dimensional arrays of perpendicularly crossing cylinders were used the distance between the cylinders was either constant ordered layers or variable disordered layers for both 1d and 2d ordered layers we propose a geometrical scaling rule for the hydraulic permeability as a function of cylinder radius and solid volume fraction porosity which is a modification of a scaling rule previously reported by clague and co workers the proposed modification is based on theoretical considerations and leads to significantly improved correspondence with simulation results the hydraulic permeability of unstructured layers is found to be higher than that of structured layers of equal porosity for both 1d and 2d arrays we propose a single parameter that can be easily determined experimentally to characterize the degree of disorder as well as a generally valid correction factor in the proposed geometrical scaling rule to account for the influence of disorder on the hydraulic permeability

additive comparisons of stopping values and supremum values for finite stage multiparameter stochastic processes abstract this paper concerns the optimal stopping problem for discrete time multiparameter stochastic processes with the index set n d in the classical optimal stopping problems the comparisons between the expected reward of a player with complete foresight and the expected reward of a player using nonanticipating stop rules known as prophet inequalities have been studied by many authors ratio comparisons between these values in the case of multiparameter optimal stopping problems are studied by krengel and sucheston 1981 9 and tanaka 2007 2006 14 15 in this paper an additive comparison in the case of finite stage multiparameter optimal stopping problems is given

optimal stopping problems for multiarmed bandit processes with arms independence abstract this paper deals with the optimal stopping problem for multiarmed bandit processes under the assumption of independence of arms we show that optimal strategies and stopping times are expressed by the dynamic allocation indices for each arm this paper reduces this problem to several independent one parameter optimal stopping problems on the basis of these results we characterize optimal strategies and stopping times moreover this paper also extends those to the case allowing time constraints in the case where arm s state evolve according to markov chains with finite state linear programming calculation of optimal strategies and stopping times is discussed

two armed restless bandits with imperfect information stochastic control and indexability we present a two armed bandit model of decision making under uncertainty where the expected return to investing in the risky arm increases when choosing that arm and decreases when choosing the safe arm these dynamics are natural in applications such as human capital development job search and occupational choice using new insights from stochastic control along with a monotonicity condition on the payoff dynamics we show that optimal strategies in our model are stopping rules that can be characterized by an index which formally coincides with gittins index our result implies the indexability of a new class of restless bandit models

arret optimal sur le plan the optimal stopping problem by stopping points for processes indexed by a partially ordered set 2 is stated in a quite general frame an optimality criterion concerned with conditional pay off is established by studying the maximal elements of the set of stopping points where the snell enveloppe preserves the martingale property the problem is solved on the one point compactification of 2

on the optimal reward function of the continuous time multiarmed bandit problem the optimal reward function associated with the so called multiarmed bandit problem for general markov feller processes is considered it is shown that this optimal reward function has a simple expression product form in terms of individual stopping problems without any smoothness properties of the optimal reward function neither for the global problem nor for the individual stopping problems some results relative to a related problem with switching cost are obtained

a martingale approach to minimal surfaces abstract we provide a probabilistic approach to studying minimal surfaces in r 3 after a discussion of the basic relationship between brownian motion on a surface and minimality of the surface we introduce a way of coupling brownian motions on two minimal surfaces this coupling is then used to study two classes of results in minimal surface theory maximum principle type results such as weak and strong halfspace theorems and the maximum principle at infinity and liouville theorems