uniqueness and stability of optimal policies of finite state markov decision processes in this paper we consider infinite horizon discrete time optimal control of markov decision processes mdps with finite state spaces and compact action sets we restrict attention to unicost mdps which form a class that contains all the weakly communicating mdps the unicost mdps are characterized as those mdps for which there exists a solution to the single optimality equation we address the problem of uniqueness and stability of minimizing markov actions our main result asserts that when we endow the set of unicost mdps with a certain natural metric under which it is complete then the class of mdps with essentially unique and stable minimizing markov actions contains the intersection of countably many open dense sets hence is itself dense thus the property of having essentially unique and stable minimizing markov actions is generic for unicost mdps

eigenfunctions of the laplacian on rotationally symmetric manifolds eigenfunctions of the laplacian on a negatively curved rotationally symmetric manifold m rn ds2 ds2 dr2 f r 2d 2 are constructed explicitly under the assumption that an integral of f r converges this integral is the same one which gives the existence of nonconstant harmonic functions on m

laplace error penalty based m type model detection for a class of high dimensional semiparametric models abstract semiparametric model is a kind of important mathematical modeling method of high dimensional biological big data in human health and disease in this paper we develop a m type variable selection method based on laplace error penalty lep function for a class of high dimensional semiparametric models using a shrinkage idea the proposed procedure can simultaneously select significant covariates with functional coefficients and local significant variables with parametric coefficients the laplace error penalty lep function is constructed as an exponential function with two tuning parameters and is infinitely differentiable everywhere except at the origin so the lep oracle estimator can be easily obtained we also proposed the computational algorithm in order to adapt to our method moreover due to the robustness of the m type loss function to outliers in the finite samples our proposed variable selection method is more robust than the ones based on the least squares criterion finally the method is illustrated with numerical simulations

controllability cost of conservative systems resolvent condition and transmutation abstract this article concerns the exact controllability of unitary groups on hilbert spaces with unbounded control operator it provides a necessary and sufficient condition not involving time which blends a resolvent estimate and an observability inequality by the transmutation of controls in some time l for the corresponding second order conservative system it is proved that the cost of controls in time t for the unitary group grows at most like exp l 2 t as t tends to 0 in the application to the cost of fast controls for the schrodinger equation l is the length of the longest ray of geometric optics which does not intersect the control region this article also provides observability resolvent estimates implying fast smoothing effect controllability at low cost and underscores that the controllability cost of a system is not changed by taking its tensor product with a conservative system

multiplicative markov decision chains previous treatments of multiplicative markov decision chains eg bellman bellman r 1957 dynamic programming princeton university press princeton new jersey mandl mandl p 1967 an iterative method for maximizing the characteristic root of positive matrices rev roumaine math pures appl xii 1317 1322 and howard and matheson howard r a matheson j e 1972 risk sensitive markov decision processes management sci 8 356 369 restricted attention to stationary policies and assumed that all transition matrices are irreducible and aperiodic they also used a first term optimality criterion namely maximizing the spectral radius of the associated transition matrix we give a constructive proof of the existence of optimal policies among all policies under new cumulative average optimality criteria which are more sensitive than the maximization of the spectral radius the algorithm for finding an optimal policy first searches for a stationary policy with a nonnilpotent transition matrix provided such a rule exists otherwise the method still finds an optimal policy though in this case the set of optimal policies usually does not contain a stationary policy if a stationary policy with a nonnilpotent transition matrix exists then we develop a policy improvement algorithm which finds a stationary optimal policy

semiclassical asymptotics gauge fields and quantum chaos a gauge field is given by a connection on a principal bundle p m we consider the semiclassical behavior of a family of schrodinger operators associated with a gauge field in the limit as h 0 we relate the spectral theory of such operators to behavior of the hamiltonian flow on the natural phase space associated to a gauge field examining in particular situations where this flow exhibits chaotic behavior

variational characterizations in markov decision processes abstract most quantities of interest in discounted and undiscounted semi markov decision processes can be obtained by solving a system of functional equations this paper derives bounds and variational characterizations for the solutions of such systems these are useful for at least three reasons 1 in any solution procedure the upper and lower bounds can be used to measure the deviation of the current solution from optimality 2 this in turn may permit elimination of suboptimal actions and 3 the variational characterizations suggest numerical algorithms linear programming policy iteration algorithms successive approximation schemes

discretization of riemannian manifolds applied to the hodge laplacian for kappa geq 0 and r 0 0 let bbb m n kappa r 0 be the set of all connected compact n dimensional riemannian manifolds such that k g leq kappa and inj m g geq r 0 we study the relation between the k rm th positive eigenvalue of the hodge laplacian on differential forms and the k rm th positive eigenvalue of the combinatorial laplacian associated to an open cover acting on v c ech cochains we show that for a fixed sufficiently small varepsilon 0 there exist positive constants c 1 and c 2 depending only on n kappa and varepsilon such that for any m in bbb m n kappa r 0 and for any varepsilon discretization x of m we have c 1 lambda k p x leq lambda k p m leq c 2 lambda k p x for any k leq k k depends on x moreover we find a lower bound for the spectrum of the combinatorial laplacian and a lower bound for the spectrum of the hodge laplacian

on stationary strategies in borel dynamic programming we consider a discrete time markov decision model with borel state and action spaces it is proved that the supremum of the expected total rewards under all randomized stationary strategies is equal to the supremum of these rewards under all nonrandomized stationary strategies

collapsing irreducible 3 manifolds with nontrivial fundamental group let m be a closed orientable irreducible non simply connected 3 manifold we prove that if m admits a sequence of riemannian metrics which volume collapses and whose sectional curvature is locally controlled then m is a graph manifold this is the last step in perelman s proof of thurston s geometrisation conjecture

an algorithm to identify and compute average optimal policies in multichain markov decision processes this paper concerns discrete time finite state multichain mdps with compact action sets the optimality criterion is long run average cost simple examples illustrate that optimal stationary markov policies do not always exist we establish the existence of e optimal policies that are stationary markovian and develop an algorithm that computes these approximate optimal policies we establish a necessary and sufficient condition for the existence of an optimal policy that is stationary markovian and in case that such an optimal policy exists the algorithm computes it

action elimination procedures for modified policy iteration algorithms this paper proposes bounds and action elimination procedures for policy iteration and modified policy iteration procedures to eliminate nonoptimal actions for one iteration and for all subsequent iterations are presented the implementation of these procedures is discussed and encouraging computational results are presented

keller osserman conditions for diffusion type operators on riemannian manifolds abstract in this paper we obtain essentially sharp generalized keller osserman conditions for wide classes of differential inequalities of the form l u b x f u l u and l u b x f u l u g u h u on weighted riemannian manifolds where l is a non linear diffusion type operator prototypical examples of these operators are the p laplacian and the mean curvature operator the geometry of the underlying manifold is reflected via bounds for the modified bakry emery ricci curvature by growth conditions for the functions b and l a weak maximum principle which extends and improves previous results valid for the laplacian is also obtained geometric comparison results valid even in the case of integral bounds for the modified bakry emery ricci tensor are presented

collapsing with a lower bound on the curvature operator cheeger and gromov showed that f structures are related to collapse with a double sided curvature bound we define fibered f structures and extend some of the cheeger gromov results to the setting of collapse with a lower bound on the curvature operator

the average cost of markov chains subject to total variation distance uncertainty abstract this paper addresses the problem of controlling a markov chain so as to minimize the long run expected average cost per unit time when the invariant distribution is unknown but we know it belongs to a given uncertain set the mathematical model used to describe this set is the total variation distance uncertainty we show that the equilibrium control policy which yields higher probability to the states with low cost and lower probability to the states with the high cost is an optimal control policy that minimizes the average cost recognition of such a policy may be of value in practical situations with constraints consistent to those studied here when the invariant distribution is uncertain and deriving online an optimal control policy is required

on the parabolic kernel of the schrodinger operator etude des equations paraboliques du type q x t t u x t 0 sur une variete riemannienne generale introduction estimations de gradients inegalites de harnack majorations et minorations des solutions fondamentales equation de la chaleur et noyau de green operateur de schrodinger

on the upper estimate of the heat kernel of a complete riemannian manifold let m be a complete non compact riemannian manifold whose sectional curvature is bounded between two constants k and k then one expects that the heat diffusion in such a manifold behaves like the heat diffusion in euclidean space the purpose of this paper is to give a justification of such a statement in 5 j cheeger and the third author found a lower estimate of the heat kernel by comparing it with the heat kernel of the space form whose curvature is the lower bound of the curvature of the manifold this lower estimate is sharp if we insist the dependence should be on the lower bound of the ricci curvature alone it remains to give an upper estimate of the heat kernel one does not however expect to have a comparison theorem for the upper bound because it is more sensitive to the geometry of the manifold in fact the heat kernel of the upper hialf space and the heat kernel of the complete hyperbolic manifold with finite volume have quite different behavior this is reflected by the fact that the laplacian has no discrete spectrum in the first case while infinite number of discrete eigenvalues may exist in the latter case what we will prove here is that in any case the heat kernel has to decay in a manner similar to the euclidean heat kernel thus we will prove that for any constant c 4 there exists c1 depending on c t the bound of the curvature of m and x so that for all t e 0 t

gaussian upper bounds for the heat kernels of some second order operators on riemannian manifolds abstract we describe a method of obtaining gaussian upper bounds on heat kernels which unifies and improves recent results for hypoelliptic operators in divergence form and for laplace beltrami operators on complete riemannian manifolds

essential spectrum and heat kernel abstract upper bounds are derived for the large time behavior of the heat kernel on riemannian manifolds whose essential spectrum has a positive lower bound applications include a relative signature theorem and the construction of bounded harmonic functions

multichain markov decision processes with a sample path constraint a decomposition approach we consider finite state finite action markov decision processes which accumulate both a reward and a cost at each decision epoch we study the problem of finding a policy that maximizes the expected long run average reward subject to the constraint that the long run average cost be no greater than a given value with probability one we establish that if there exists a policy that meets the constraint then there exists an e optimal stationary policy furthermore an algorithm is outlined to locate the e optimal stationary policy the proof of the result hinges on a decomposition of the state space into maximal recurrent classes and a set of transient states

denumerable undiscounted semi markov decision processes with unbounded rewards this paper establishes the existence of a solution to the optimality equations in undis counted semi markov decision models with countable state space under conditions generalizing the hitherto obtained results in particular we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy instead of the traditional and restrictive assumption that every stationary policy has a single irreducible set of states a replacement model and an inventory model illustrate why this extension is essential our approach differs fundamentally from classical approaches we convert the optimality equations into a form suitable for the application of a fixed point theorem

existence and non existence of area minimizing hypersurfaces in manifolds of non negative ricci curvature we study minimal hypersurfaces in manifolds of non negative ricci curvature euclidean volume growth and quadratic curvature decay at infinity by comparison with capped spherical cones we identify a precise borderline for the ricci curvature decay above this value no complete area minimizing hypersurfaces exist below this value in contrast we construct examples

solving markovian decision processes by successive elimination of variables abstract the functional equations of stationary infinite horizon markovian decision processes can be solved by eliminating the variables one at a time until only one variable and one equation are left back substitution in the resulting triangular system of transformed functional equations then completes the solution the idea completely parallels the gauss jordan pivoting algorithm for solving a nonsingular system of linear equations this method provides a computational scheme plus proofs of existence of a solution uniqueness of solution is demonstrated in the discounted case in the undiscounted case variables are eliminated successively from a mixed system of equalities and inequalities until the remaining variables may be chosen arbitrarily i e the solution is no longer unique in this case a minimal solution subject to these free choices is exhibited

heat kernel on smooth metric measure spaces with nonnegative curvature we derive a local gaussian upper bound for the f heat kernel on complete smooth metric measure space m g e f dv with nonnegative bakry emery ricci curvature as applications we obtain a sharp l f 1 liouville theorem for f subharmonic functions and an l f 1 uniqueness property for nonnegative solutions of the f heat equation assuming f is of at most quadratic growth in particular any l f 1 integrable f subharmonic function on gradient shrinking and steady ricci solitons must be constant we also provide explicit f heat kernel for gaussian solitons

eigenvalues estimates on complete noncompact riemannian manifolds and applications we obtain some sharp estimates on the first eigenvalues of complete noncom pact riemannian manifolds under assumptions of volume growth using these estimates we study hypersurfaces with constant mean curvature and give some estimates on the mean curvatures

good wannier bases in hilbert modules associated to topological insulators for a large class of physically relevant operators on a manifold with discrete group action we prove general results on the non existence of a basis of well localized wannier functions for their spectral subspaces this turns out to be equivalent to the freeness of a certain hilbert module over the group c algebra canonically associated with the spectral subspace this brings into play k theoretic methods and justifies their importance as invariants of topological insulators in physics

on poincare type inequalities using estimates of the heat kernel we prove a poincare inequality for star shape domains on a complete manifold the method also gives a lower bound for the gap of the first two neumann eigenvalues of a schrodinger operator

constrained undiscounted stochastic dynamic programming in this paper we investigate the computation of optimal policies in constrained discrete stochastic dynamic programming with the average reward as utility function the state space and action sets are assumed to be finite constraints which are linear functions of the state action frequencies are allowed in the general multichain case an optimal policy will be a randomized nonstationary policy an algorithm to compute such an optimal policy is presented furthermore sufficient conditions for optimal policies to be stationary are derived there are many applications for constrained undiscounted stochastic dynamic programming e g in multiple objective markovian decision models

local index theory over etale groupoids we give a superconnection proof of connes index theorem for proper cocompact actions of etale groupoids this includes connes general foliation index theo rem for foliations with hausdorholonomy groupoid

riesz transforms associated to schrodinger operators on weighted hardy spaces let w be some ap weight and enjoy reverse holder inequality and let l v be a schrodinger operator on rn where v lloc1 rn is a non negative function on rn in this article we introduce weighted hardy spaces hl w1 rn associated to l in terms of the area function characterization and prove their atomic characters we show that the riesz transform l 1 2 associated to l is bounded on lwp rn for 1 p 2 and bounded from hl w1 rn to the classical weighted hardy space hw1 rn

adaptive varying coefficient linear models varying coefficient linear models arise from multivariate nonparametric regression nonlinear time series modelling and forecasting functional data analysis longitudinal data analysis and others it has been a common practice to assume that the vary coefficients are functions of a given variable which is often called an index a frequently asked question is which variable should be used as the index in this paper we explore the class of the varying coefficient linear models in which the index is unknown and is estimated as a linear combination of regression and or other variables this will enlarge the modelling capacity substantially we search for the index such that the derived varying coefficient model provides the best approximation to the underlying unknown multi dimensional regression function in the least square sense the search is implemented through the newly proposed hybrid backfitting algorithm the core of the algorithm is the alternative iteration between estimating the index through a one step scheme and estimating coefficient functions through a one dimensional local linear smoothing the generalised cross validation method for choosing bandwidth is efficiently incorporated into the algorithm the locally significant variables are selected in terms of the combined use of t statistic and akaike information criterion we further extend the algorithm for the models with two indices simulation shows that the proposed methodology has appreciable flexibility to model complex multivariate nonlinear structure and is practically feasible with average modern computers the methods are further illustrated through the canadian mink muskrat data in 1925 1994 and the pound dollar exchange rates in 1974 1983

on the solvability of bellman s functional equations for markov renewal programming abstract the functional equations of undiscounted stationary infinite horizon markov renewal programming are shown to possess a solution by an elementary application of the leray schauder fixed point theorem

twistor spaces and the general adiabatic expansions abstract we investigate the behavior of derivatives of the fundamental solution of a parabolic equation for the square of dirac operator on a twistor space when the metric is blown up in the base space direction such a blowing up operation is expected to be an effective method for extracting some intrinsic values from various geometric invariants most of whose cores consist of some derivatives of the fundamental solution

on the second optimality equation for semi markov decision models for a semi markov decision model with average return the validity of the second optimality equation is shown in the nonmodified form where the actions run through the set of all admissible actions rather than through the set of maximum points conserving actions for the first optimal equation as a consequence the existence of a strongly optimal stationary policy is shown the results seem to be known only for finite state finite action models whereas here countable state compact action models with unbounded rewards are considered

a unified approach to markov decision problems and performance sensitivity analysis with discounted and average criteria multichain cases we propose a unified framework to markov decision problems and performance sensitivity analysis for multichain markov processes with both discounted and average cost performance criteria with the fundamental concept of performance potentials we derive both performance gradient and performance difference formulas which play the central role in performance optimization the standard policy iteration algorithms for both discounted and average reward mdps can be established using the performance difference formulas in a simple and intuitive way and the performance gradient formulas together with stochastic approximation may lead to new optimization schemes this sensitivity based point of view of performance optimization provides some insights that link perturbation analysis markov decision processes and reinforcement learning together the research is an extension of the previous work on ergodic markov chains cao automatica 36 2000 771

on single index coefficient regression models abstract in this article we investigate a class of single index coefficient regression models under dependence this includes many existing models such as the smooth transition threshold autoregressive star model of chan and tong the functional coefficient autoregressive far model of chen and tsay and the single index model of ichimura compared to the varying coefficient model of hastie and tibshirani our model can avoid the curse of dimensionality in multivariate nonparametric estimations another advantage of this model is that a threshold variable is chosen automatically an estimation method is proposed and the corresponding estimators are shown to be consistent and asymptotically normal some simulations and applications are also reported

a brouwer fixed point mapping approach to communicating markov decision processes abstract the solvability of the functional equations of communicating markov decision processes is demonstrated by an elementary application of the brouwer fixed point mapping theorem the technical assumptions include finite state space and finite or compact action spaces

successive approximation methods for solving nested functional equations in markov decision problems this paper presents a successive approximation method for solving systems of nested functional equations which arise e g when considering markov renewal programs in which policies that are maximal gain or optimal under more selective discount and average overtaking optimality criteria are to be found in particular a successive approximation method is given to find the optimal bias vector and bias optimal policies applications with respect to a number of additional stochastic control models are pointed out our method is based on systems of simultaneously generated single equation value iteration schemes

sharp geometric inequalities for closed hypersurfaces in manifolds with nonnegative ricci curvature in this paper we consider complete noncompact riemannian manifolds m g with nonnegative ricci curvature and euclidean volume growth of dimension n ge 3 for every bounded open subset omega subset m with smooth boundary we prove that begin aligned int limits partial omega left frac mathrm h n 1 right n 1 mathrm d sigma ge mathrm avr g big mathbb s n 1 big end aligned where mathrm h is the mean curvature of partial omega and mathrm avr g is the asymptotic volume ratio of m g moreover the equality holds true if and only if m setminus omega g is isometric to a truncated cone over partial omega an optimal version of huisken s isoperimetric inequality for 3 manifolds is obtained using this result finally exploiting a natural extension of our techniques to the case of parabolic manifolds we also deduce an enhanced version of kasue s non existence result for closed minimal hypersurfaces in manifolds with nonnegative ricci curvature

relative index theory abstract we develop the framework for the heat equation method in the relative index theory we study pairs of dirac operators on complete manifolds and give an analytic interpretation of the difference of the integrals over their local index densities this yields a generalization of the relative index theorem of gromov and lawson under some assumptions on the curvature we show that supersymmetric scattering theories of borisov muller and schrader arise naturally the scattering index is related to the relative topological index

growth optimality for branching markov decision chains this paper considers a multiplicative process called branching markov decision chains in which the output at the end of the nth period equals the product of n nonnegative matrices chosen at the beginning of periods 1 n respectively times a positive fixed terminal reward vector it is assumed that the above transition matrices are drawn out of a finite set of matrices given in product form i e the rows of the matrices can be selected independently out of finite sets of nonnegative row vectors for each coordinate s we define the geometric and algebraic growth rates respectively of the sth coordinate of the stream of output these growth rates are defined so that the magnitude of the corresponding sequence is of the order nnk where is the geometric growth rate and k is the algebraic growth rate the main result of this paper is the constructive establishment of the existence of a transition matrix whose repeated use will guarantee for each coordinate the achievement of the best geometri

control of markov chains with long run average cost criterion the dynamic programming equations the long run average cost control problem for discrete time markov chains on a countable state space is studied in a very general framework necessary and sufficient conditions for optimality in terms of the dynamic programming equations are given when an optimal stable stationary strategy is known to exist e g for the situations studied in stochastic differential systems stochastic control theory and applications ima vol math app 10 springer verlag new york berlin 1988 pp 57 77 a characterization of the desired solution of the dynamic programming equations is given in a special case also included is a novel convex analytic argument for deducing the existence of an optimal stable stationary strategy when that of a randomized one is known

robust check loss based inference of semiparametric models and its application in environmental data abstract composite quantile regression has been shown to be a more competitive method than the well known mean regression and quantile regression approaches since it can significantly improve the estimation efficiency for a wide class of errors in this paper we propose a profiling composite quantile regression for robust estimation of semiparametric varying coefficient partially linear model vcplm by combining the ideas of local linear composite quantile regression with the profile technique different from the step wise estimation method considered in kai et al 2011 an iterative algorithm is developed for the estimates of parametric and nonparametric functions under some mild regularity conditions we establish the asymptotic normalities of the obtained estimators and further briefly discuss the related asymptotic relative efficiency noteworthy an entirely different way from kai et al 2011 is presented to establish the asymptotic normality of the parametric component which is frequently of the primary interest in vcplm some monte carlo simulations with various errors and an environmental data application are conducted to evaluate the finite sample performance of the proposed methodology

extremal of log sobolev inequality and w entropy on noncompact manifolds abstract let m be a complete connected noncompact manifold with bounded geometry under a condition near infinity we prove that the log sobolev functional 1 1 has an extremal function decaying exponentially near infinity we also prove that an extremal function may not exist if the condition is violated this result has the following consequences 1 it seems to give the first example of connected complete manifolds with bounded geometry where a standard log sobolev inequality does not have an extremal 2 it gives a negative answer to the open question on the existence of extremal of perelman s w entropy in the noncompact case which was stipulated by perelman 2002 22 p 9 3 2 remark 3 it helps to prove in some cases that noncompact shrinking breathers of ricci flow are gradient shrinking solitons

le saut en zero de la fonction de decalage spectral abstract we give a formula for the jump at zero of the spectral shift function associated with schrodinger operator on manifolds with conical ends we show that according to its decay a zero energy resonant state has a non integer contribution

on minimum cost per unit time control of markov chains the minimum cost per unit time control problem is studied for a class of markov chains that though important in applications does not fit the conventional framework for this problem existence of optimal stationary strategies and necessary and sufficient conditions for optimality are established

surfaces with boundary their uniformizations determinants of laplacians and isospectrality let be a compact surface of type g n n 0 obtained by re moving n disjoint disks from a closed surface of genus g assuming 0 while osgood phillips and sarnak o p s3 showed the properness when g 0

plancherel type estimates and sharp spectral multipliers abstract we study general spectral multiplier theorems for self adjoint positive definite operators on l2 x where x is any open subset of a space of homogeneous type we show that the sharp hormander type spectral multiplier theorems follow from the appropriate estimates of the l2 norm of the kernel of spectral multipliers and the gaussian bounds for the corresponding heat kernel the sharp hormander type spectral multiplier theorems are motivated and connected with sharp estimates for the critical exponent for the riesz means summability which we also study here we discuss several examples which include sharp spectral multiplier theorems for a class of scattering operators on r3 and new spectral multiplier theorems for the laguerre and hermite expansions

properties of the probability density function of the non central chi squared distribution abstract in this paper we consider the probability density function pdf of a non central 2 distribution with arbitrary number of degrees of freedom for this function we prove that can be represented as a finite sum and we deduce a partial derivative formula moreover we show that the pdf is log concave when the degrees of freedom is greater or equal than 2 at the end of this paper we present some turan type inequalities for this function and an elegant application of the monotone form of l hospital s rule in probability theory is given

boundedness of spectral multipliers of generalized laplacians on compact manifolds with boundary consider a second order strongly elliptic negative semidefinite differential operator l may be a system on a compact riemannian manifold overline m with smooth boundary where the domain of l is defined by a coercive boundary condition classically known results and also recent work in duong et al j funct anal 196 443 485 2002 and duong and mcintosh rev math iberoam 15 233 265 1999 establish sufficient conditions for l infty text bmo l continuity of varphi sqrt l where varphi in s 0 1 mathbb r and l is a suitable elliptic operator using a variant of the cheeger gromov taylor functional calculus due to mauceri et al math res lett 16 861 879 2009 and short time upper bounds on the integral kernel of e tl due to greiner arch ration mech anal 41 168 218 1971 we prove that a variant of such sufficient conditions holds for our operator l

overtaking and almost sure optimality for infinite horizon markov decision processes we consider infinite horizon optimal control of markov chains on complete metric spaces we employ the overtaking optimality criterion which is either applied to the expected cost flow or to the individual sample paths yielding almost sure optimality results we use the existence of a solution pair to the optimality equation l x to establish and characterize optimal strategies for finite state spaces we derive sufficient as well as necessary conditions for overtaking optimality

on the existence of relative values for undiscounted markovian decision processes with a scalar gain rate abstract the functional equations v max q f gt f p f v f k qv of undiscounted semi markovian decision processes are shown to be solvable if and only if all components of the maximum gain rate vector are equal more generally in the multichain case the functional equations for the value vector possess a solution if and only if there is a policy which achieves the maximal gain vector the method of proof exhibits vectors v such that qv v and qv v

the atomic decomposition of weighted hardy spaces associated to self adjoint operators on product spaces abstract let l be a non negative self adjoint operator acting on l 2 r n satisfying a pointwise gaussian estimate for its heat kernel let w be an a p weight on r n r n 1 p in this article we obtain a weighted q atomic decomposition with q p for the weighted hardy space h l w 1 r n r n associated to l

well posedness and regularity for the elasticity equation with mixed boundary conditions on polyhedral domains and domains with cracks we prove a regularity result for the anisotropic linear elasticity equation p u rm div left boldmath mathsf c cdot nabla u right f with mixed displacement and traction boundary conditions on a curved polyhedral domain omega subset mathbb r 3 in weighted sobolev spaces mathcal k m 1 a 1 omega for which the weight is given by the distance to the set of edges in particular we show that there is no loss of mathcal k m a regularity our curved polyhedral domains are allowed to have cracks we establish a well posedness result when there are no neighboring traction boundary conditions and a 0 that depends on p on the boundary conditions and on the domain our results extend to other strongly elliptic systems and higher dimensions

uniqueness of l 1 solutions for the laplace equation and the heat equation on riemannian manifolds on deduit une hypothese geometrique optimale sur une variete de riemann complete qui assure des proprietes d unicite pour des solutions l 1 de l equation de laplace et de la chaleur

on the convergence of policy iteration in finite state undiscounted markov decision processes the unichain case we study the convergence of policy iteration for the undiscounted finite state discrete time markov decision problem with compact action space and unichain transition structure using a newton method type representation for policy iteration we establish the existence of a solution to the optimality equation we show that to find an average optimal policy it is sufficient to solve the optimality equation on the recurrent set of the maximizing policy under the additional assumption of a unique maximizing policy at each stage of the policy iteration procedure we show that the iterates are convergent and the resulting policy is blackwell optimal