the essentially tame local langlands correspondence i 1 let f be a non archimedean local field we assume throughout that the residue field kp of f is finite of characteristic p let wp be the weil group of f formed relative to some separable algebraic closure of f for each integer n 1 let sn f be the set of equivalence classes of irreducible smooth complex representations of wp of dimension n likewise let an f be the set of equivalence classes of irreducible supercuspidal representations of the group gln f the langlands correspondence gives a bijection

nonconvergence to unstable points in urn models and stochastic approximations a particle in mathbf r d moves in discrete time the size of the n th step is of order 1 n and when the particle is at a position mathbf v the expectation of the next step is in the direction mathbf f mathbf v for some fixed vector function mathbf f of class c 2 it is well known that the only possible points mathbf p where mathbf v n may converge are those satisfying mathbf f mathbf p mathbf 0 this paper proves that convergence to some of these points is in fact impossible as long as the noise the difference between each step and its expectation is sufficiently omnidirectional the points where convergence is impossible are the unstable critical points for the autonomous flow d dt mathbf v t mathbf f v t this generalizes several known results that say convergence is impossible at a repelling node of the flow

supercuspidal characters of sl l over a p adic field l a prime let f be a p adic field of characteristic 0 for odd primeswith certain tameness restric tions techniques due to assem allow us to write supercuspidal characters of sl f in terms of a family of distributions defined by kazhdan following an argument of harish chandra we derive an integral formula for these distributions results of debacker and kazhdan allow us to evaluate this integral formula this in turn gives supercuspidal character formulas for sl f explicit up to the constants appearing in the gl f local character expansion our result also describes the explicit local langlands correspondence for those positive depth supercuspidal representations of gl f which do not restrict irreducibly to sl f 1 introduction in this note using techniques due to assem see 3 we derive formulas for the supercuspidal characters of sl f whereis an odd prime here f is a p adic field of characteristic 0 i e a finite extension of qp we assume p 2 although most of the time we will only need p our results illustrate some facts about supercuspidal characters which are not immediately obvious from previous work in general there is an infinite class of supercuspidal representations of gl f which decompose intoirreducible components upon restriction to sl f the character of each component then blows up near the singular set despite the fact that supercuspidal characters of gl f are bounded on regular elliptic elements near the singular set on the other hand under a certain simple condition on the order of the residue class field of f there is another infinite class of supercuspidal representations of gl f which remain irreducible upon restriction to sl f this means that the charac ters of the restrictions have the same behaviour near the singular set on sl f as on gl f in particular the characters of these supercuspidal representations of sl f are bounded on regular elliptic elements near the singular set this has also been noted in 27 the ultimate effect of this phenomenon on harmonic analysis on sl f is not yet fully understood and will be explored in future work we briefly outline the method we will use here the construction of the supercuspidal representations of gl f is completely understood it is known that all such representations are induced from open compact modulo the centre

on actor critic algorithms in this article we propose and analyze a class of actor critic algorithms these are two time scale algorithms in which the critic uses temporal difference learning with a linearly parameterized approximation architecture and the actor is updated in an approximate gradient direction based on information provided by the critic we show that the features for the critic should ideally span a subspace prescribed by the choice of parameterization of the actor we study actor critic algorithms for markov decision processes with polish state and action spaces we state and prove two results regarding their convergence

adams type inequalities and related elliptic partial differential equations in dimension four abstract motivated by ruf sani s recent work we prove an adams type inequality and a singular adams type inequality in the whole four dimensional euclidean space as applications of those inequalities a class of elliptic partial differential equations are considered existence of nontrivial weak solutions and multiplicity results are obtained via the mountain pass theorem and the ekeland s variational principle this is a continuation of our previous work about singular trudinger moser type inequality

moving target search a real time search for changing goals considers the case of heuristic search where the goal may change during the course of the search for example the goal may be a target that actively avoids the problem solver the authors present a moving target search algorithm mts to solve this problem the authors prove that if the average speed of the target is slower than that of the problem solver then the problem solver is guaranteed to eventually reach the target in a connected problem space the original mts algorithm was constructed with the minimum operations necessary to guarantee its completeness and hence is not very efficient to improve its efficiency the authors introduce ideas from the area of resource bounded planning into mts including 1 commitment to goals and 2 deliberation for selecting plans experimental results demonstrate that the improved mts is 10 to 20 times more efficient than the original mts in uncertain situations

empirical dynamic programming we propose empirical dynamic programming algorithms for markov decision processes in these algorithms the exact expectation in the bellman operator in classical value iteration is replaced by an empirical estimate to get empirical value iteration evi policy evaluation and policy improvement in classical policy iteration are also replaced by simulation to get empirical policy iteration epi thus these empirical dynamic programming algorithms involve iteration of a random operator the empirical bellman operator we introduce notions of probabilistic fixed points for such random monotone operators we develop a stochastic dominance framework for convergence analysis of such operators we then use this to give sample complexity bounds for both evi and epi we then provide various variations and extensions to asynchronous empirical dynamic programming the minimax empirical dynamic program and show how this can also be used to solve the dynamic newsvendor problem preliminary experimental results

abstract critical point theorems and applications to some nonlinear problems with strong resonance at infinity critical point theorems and applications to some nonlinear problems with strong resonance at infinity p bartolo and v benci istituto di matematica applicata via re david x0 bari ital and istituto di analisi matematica palazzo ateneo l ia scolai 2 bari italy received in revised form 20 december 1952

a new value iteration method for the average cost dynamic programming problem we propose a new value iteration method for the classical average cost markovian decision problem under the assumption that all stationary policies are unichain and that furthermore there exists a state that is recurrent under all stationary policies this method is motivated by a relation between the average cost problem and an associated stochastic shortest path problem contrary to the standard relative value iteration our method involves a weighted sup norm contraction and for this reason it admits a gauss seidel implementation computational tests indicate that the gauss seidel version of the new method substantially outperforms the standard method for difficult problems

the borkar meyn theorem for asynchronous stochastic approximations in this paper we give a generalization of a result by borkar and meyn 2000 1 on the stability and convergence of synchronous update stochastic approximation algorithms to the case of asynchronous stochastic approximations with delays we then describe an interesting application of the result to asynchronous distributed temporal difference td learning with function approximation and delays c 2011 elsevier b v all rights reserved

existence of solutions for a higher order kirchhoff type problem with exponential critical growth abstract a higher order kirchhoff type equation r 2 m m u 2 0 m 1 a x u 2 d x m u 0 m 1 1 a x u f x u x h x in r 2 m is considered in this paper we assume that the nonlinearity of the equation has exponential critical growth and prove that for a positive which is small enough there are two distinct nontrivial solutions to the equation when 0 we also prove that the equation has a nontrivial mountain pass type solution

geometric variance reduction in markov chains application to value function and gradient estimation we study a variance reduction technique for monte carlo estimation of functionals in markov chains the method is based on designing sequential control variates using successive approximations of the function of interest v regular monte carlo estimates have a variance of o 1 n where n is the number of sample trajectories of the markov chain here we obtain a geometric variance reduction o n with 1 up to a threshold that depends on the approximation error v av where a is an approximation operator linear in the values thus if v belongs to the right approximation space i e av v the variance decreases geometrically to zero an immediate application is value function estimation in markov chains which may be used for policy evaluation in a policy iteration algorithm for solving markov decision processes another important domain for which variance reduction is highly needed is gradient estimation that is computing the sensitivity v of the performance measure v with respect to some parameter of the transition probabilities for example in policy parametric optimization computing an estimate of the policy gradient is required to perform a gradient optimization method we show that using two approximations for the value function and the gradient a geometric variance reduction is also achieved up to a threshold that depends on the approximation errors of both of those representations

nash q learning for general sum stochastic games we extend q learning to a noncooperative multiagent context using the framework of general sum stochastic games a learning agent maintains q functions over joint actions and performs updates based on assuming nash equilibrium behavior over the current q values this learning protocol provably converges given certain restrictions on the stage games defined by q values that arise during learning experiments with a pair of two player grid games suggest that such restrictions on the game structure are not necessarily required stage games encountered during learning in both grid environments violate the conditions however learning consistently converges in the first grid game which has a unique equilibrium q function but sometimes fails to converge in the second which has three different equilibrium q functions in a comparison of offline learning performance in both games we find agents are more likely to reach a joint optimal path with nash q learning than with a single agent q learning method when at least one agent adopts nash q learning the performance of both agents is better than using single agent q learning we have also implemented an online version of nash q learning that balances exploration with exploitation yielding improved performance

on the local langlands conejcture mathrm gl n the cyclic case let f be a locally compact nonarchimedean field and p its residue characteristic let f be a separable algebraic closure of f wf the weil group of f over f e a character of fx of order n and e the corresponding character of wf then there exists a canonical map from the set of equivalence classes a of irreducible degree n representations of wf such that e a a to the set of equivalence classes 7r of admissible irreducible supercuspidal representations of gl n f verifying odet x 7t this map is shown to be bijective and to preserve e factors for pairs

value function based reinforcement learning in changing markovian environments the paper investigates the possibility of applying value function based reinforcement learning rl methods in cases when the environment may change over time first theorems are presented which show that the optimal value function of a discounted markov decision process mdp lipschitz continuously depends on the immediate cost function and the transition probability function dependence on the discount factor is also analyzed and shown to be non lipschitz afterwards the concept of e mdps is introduced which is a generalization of mdps and e mdps in this model the environment may change over time more precisely the transition function and the cost function may vary from time to time but the changes must be bounded in the limit then learning algorithms in changing environments are analyzed a general relaxed convergence theorem for stochastic iterative algorithms is presented we also demonstrate the results through three classical rl methods asynchronous value iteration q learning and temporal difference learning finally some numerical experiments concerning changing environments are presented

unrefined minimal k types for p adic groups via their restriction to compact open subgroups was begun by mautner shalika and tanaka for groups of type ai in contrast to real reductive groups where the representation theory of a maximal compact subgroup k is given in terms of slight modifications to cartan s theory of the highest weight the representation theory of a compact parahoric subgroup is quite complicated there is still no comprehensive theory for classifying the irreducible representa tions in the case of

real time heuristic search abstract we apply the two player game assumptions of limited search horizon and commitment to moves in constant time to single agent heuristic search problems we present a variation of minimax lookahead search and an analog to alpha beta pruning that significantly improves the efficiency of the algorithm paradoxically the search horizon reachable with this algorithm increases with increasing branching factor in addition we present a new algorithm called real time a for interleaving planning and execution we prove that the algorithm makes locally optimal decisions and is guaranteed to find a solution we also present a learning version of this algorithm that improves its performance over successive problem solving trials by learning more accurate heuristic values and prove that the learned values converge to their exact values along every optimal path these algorithms effectively solve significantly larger problems than have previously been solvable using heuristic evaluation functions

explicit sub optimal linear quadratic regulation with state and input constraints optimal feedback solutions to the infinite horizon lqr problem with state and input constraints based on receding horizon real time quadratic programming are well known in this paper we develop an explicit solution to the same problem eliminating the need for real time optimization it is shown that the resulting feedback controller is piecewise linear this explicit functional structure is exploited for efficient real time implementation a suboptimal strategy based on a suboptimal choice of a finite horizon and imposing additional limitations on the allowed switching between active constraint sets on the horizon is suggested in order to address the computer memory and processing capacity requirements of the explicit solution

the o d e method for convergence of stochastic approximation and reinforcement learning it is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated ode this in turn implies convergence of the algorithm several specific classes of algorithms are considered as applications it is found that the results provide i a simpler derivation of known results for reinforcement learning algorithms ii a proof for the first time that a class of asynchronous stochastic approximation algorithms are convergent without using any a priori assumption of stability iii a proof for the first time that asynchronous adaptive critic and q learning algorithms are convergent for the average cost optimal control problem

concentration compactness principle for trudinger moser inequalities with logarithmic weights and their applications abstract in this paper we establish a sharp concentration compactness principle associated with the trudinger moser inequality on sobolev spaces with logarithmic weights as applications we establish the existence of ground state solutions to the following equation with critical double exponential nonlinearity d i v u n 2 u x f x u in b 1 0 u 0 in b 1 0 u 0 on b 1 0 where x log e x n 1 is the logarithmic weight the nonlinear term f x u is continuous radial in x b 1 0 and has critical double exponential growth which behaves like exp e e n u n

asynchronous stochastic approximations the asymptotic behavior of a distributed asynchronous stochastic approximation scheme is analyzed in terms of a limiting nonautonomous differential equation the relation between the latter and the relative values of suitably rescaled relative frequencies of updates of different components is underscored

the potential structure of sample paths and performance sensitivities of markov systems we study the structure of sample paths of markov systems by using performance potentials as the fundamental units with a sample path based approach we show that performance sensitivity formulas performance gradients and performance differences of markov systems can be constructed intuitively by first principles with performance potentials or equivalently perturbation realization factors as building blocks in particular we derive sensitivity formulas for two markov chains with possibly different state spaces the proposed approach can be used to obtain flexibly the sensitivity formulas for a wide range of problems including those with partial information these formulas are the basis for performance optimization of discrete event dynamic systems including perturbation analysis markov decision processes and reinforcement learning the approach thus provides insight on on line learning and performance optimization and opens up new research directions sample path based algorithms can be developed

supercuspidal characters of reductive p adic groups we compute the characters of many supercuspidal representations of reductive p adic groups specifically we deal with representations that arise via yu s construction from data satisfying a certain compactness condition each character is expressed in terms of a depth zero character of a smaller group the linear characters appearing in yu s construction fourier transforms of orbital integrals and certain signs and cardinalities that are described explicitly in terms of the datum associated to the representation and of the element at which the character is evaluated

actor critic algorithms for hierarchical markov decision processes we consider the problem of control of hierarchical markov decision processes and develop a simulation based two timescale actor critic algorithm in a general framework we also develop certain approximation algorithms that require less computation and satisfy a performance bound one of the approximation algorithms is a three timescale actor critic algorithm while the other is a two timescale algorithm however which operates in two separate stages all our algorithms recursively update randomized policies using the simultaneous perturbation stochastic approximation spsa methodology we briefly present the convergence analysis of our algorithms we then present numerical experiments on a problem of production planning in semiconductor fabs on which we compare the performance of all algorithms together with policy iteration algorithms based on certain hadamard matrix based deterministic perturbations are found to show the best results

adaptive importance sampling technique for markov chains using stochastic approximation for a discrete time finite state markov chain we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states this scheme updates the change of measure at every transition using constant or decreasing step size stochastic approximation the updates are shown to concentrate asymptotically in a neighborhood of the desired zero variance estimator through simulation experiments on simple markovian queues we observe that the proposed technique performs very well in estimating performance measures related to rare events associated with queue lengths exceeding prescribed thresholds we include performance comparisons of the proposed algorithm with existing adaptive importance sampling algorithms on some examples we also discuss the extension of the technique to estimate the infinite horizon expected discounted cost and the expected average cost

lao a heuristic search algorithm that finds solutions with loops classic heuristic search algorithms can find solutions that take the form of a simple path a a tree or an acyclic graph ao in this paper we describe a novel generalization of heuristic search called lao that can find solutions with loops we show that lao can be used to solve markov decision problems and that it shares the advantage heuristic search has over dynamic programming for other classes of problems given a start state it can find an optimal solution without evaluating the entire state space 2001 elsevier science b v all rights reserved

a unified analysis of value function based reinforcement learning algorithms reinforcement learning is the problem of generating optimal behavior in a sequential decision making environment given the opportunity of interacting with it many algorithms for solving reinforcement learning problems work by computing improved estimates of the optimal value function we extend prior analyses of reinforcement learning algorithms and present a powerful new theorem that can provide a unified analysis of such value function based reinforcement learning algorithms the usefulness of the theorem lies in how it allows the convergence of a complex asynchronous reinforcement learning algorithm to be proved by verifying that a simpler synchronous algorithm converges we illustrate the application of the theorem by analyzing the convergence of q learning model based reinforcement learning q learning with multistate updates q learning for markov games and risk sensitive reinforcement learning

quantitatively relating abstractness to the accuracy of admissible heuristics abstract admissible lower bound heuristics are worth discovering because they have desirable properties in various search algorithms one well known source of admissible heuristics is from abstractions of a problem using standard definitions of heuristic accuracy and abstractness we prove that heuristic accuracy decreases inversely with abstractness this is the first quantitative result linking abstractness to the heuristic accuracy using this result it may be possible to predict the accuracy of an abstraction derived heuristic without testing it on a sample set of problems it may also be possible for a heuristic discovery system to use the theory to predict the accuracy of a heuristic thereby better focusing its search

performance of linear space search algorithms abstract search algorithms that use space linear in the search depth are widely employed in practice to solve difficult problems optimally such as planning and scheduling in this paper we study the average case performance of linear space search algorithms including depth first branch and bound dfbnb iterative deepening id and recursive best first search rbfs to facilitate our analyses we use a random tree t b d that has mean branching factor b depth d and node costs that are the sum of the costs of the edges from the root to the nodes we prove that the expected number of nodes expanded by dfbnb on a random tree is no more than bd times the expected number of nodes expanded by best first search bfs on the same tree which usually requires space that is exponential in depth d we also show that dfbnb is asymptotically optimal when bfs runs in exponential time and id and rbfs are asymptotically optimal when the edge costs of t b d are integers if bp 0 is the expected number of children of a node whose costs are the same as that of their parent then the expected number of nodes expanded by these three linear space algorithms is exponential when bp o d 4 when bp 0 1 and at most quadratic when bp 0 1 in addition we study the heuristic branching factor of t b d and the effective branching factor of bfs dfbnb id and rbfs on t b d furthermore we use our analytic results to explain a surprising anomaly in the performance of these algorithms and to predict the existence of a complexity transition in the asymmetric traveling salesman problem

an analysis of stochastic shortest path problems we consider a stochastic version of the classical shortest path problem whereby for each node of a graph we must choose a probability distribution over the set of successor nodes so as to reach a certain destination node with minimum expected cost the costs of transition between successive nodes can be positive as well as negative we prove natural generalizations of the standard results for the deterministic shortest path problem and we extend the corresponding theory for undiscounted finite state markovian decision problems by removing the usual restriction that costs are either all nonnegative or all nonpositive

infinitely many solutions for some nonlinear elliptic problems in symmetrical domains on considere le probleme aux valeurs limites u u u u 2 2 0 u h 0 1 ou r n est un domaine borne n 3 2 2n n 2 est l exposant critique pour le plongement de sobolev et est un parametre positif reel on demontre l existence d une infinite de solutions quand a de bonnes symetries

existence of positive solutions to quasi linear elliptic equations with exponential growth in the whole euclidean space abstract in this paper a quasi linear elliptic equation in the whole euclidean space is considered the nonlinearity of the equation is assumed to have exponential growth or have critical growth in view of trudinger moser type inequality under some assumptions on the potential and the nonlinearity it is proved that there is a nontrivial positive weak solution to this equation also it is shown that there are two distinct positive weak solutions to a perturbation of the equation the method of proving these results is combining trudinger moser type inequality mountain pass theorem and ekeland s variational principle

a geometric approach to multi criterion reinforcement learning we consider the problem of reinforcement learning in a controlled markov environment with multiple objective functions of the long term average reward type the environment is initially unknown and furthermore may be affected by the actions of other agents actions that are observed but cannot be predicted beforehand we capture this situation using a stochastic game model where the learning agent is facing an adversary whose policy is arbitrary and unknown and where the reward function is vector valued state recurrence conditions are imposed throughout in our basic problem formulation a desired target set is specified in the vector reward space and the objective of the learning agent is to approach the target set in the sense that the long term average reward vector will belong to this set we devise appropriate learning algorithms that essentially use multiple reinforcement learning algorithms for the standard scalar reward problem which are combined using the geometric insight from the theory of approachability for vector valued stochastic games we then address the more general and optimization related problem where a nested class of possible target sets is prescribed and the goal of the learning agent is to approach the smallest possible target set which will generally depend on the unknown system parameters a particular case which falls into this framework is that of stochastic games with average reward constraints and further specialization provides a reinforcement learning algorithm for constrained markov decision processes some basic examples are provided to illustrate these results

sensitivity analysis using ito malliavin calculus and martingales and application to stochastic optimal control we consider a multidimensional diffusion process x alpha t 0 leq t leq t whose dynamics depends on a parameter alpha our first purpose is to write as an expectation the sensitivity nabla alpha j alpha for the expected cost j alpha mathbb e f x alpha t in order to evaluate it using monte carlo simulations this issue arises for example from stochastic control problems where the controller is parameterized which reduces the control problem to a parametric optimization one or from model misspecifications in finance previous evaluations of nabla alpha j alpha using simulations were limited to smooth cost functions f or to diffusion coefficients not depending on alpha see yang and kushner siam j control optim 29 1991 pp 1216 1249 in this paper we cover the general case deriving three new approaches to evaluate nabla alpha j alpha which we call the malliavin calculus approach the adjoint approach and the martingale approach to accomplish this we leverage ito calculus malliavin calculus and martingale arguments in the second part of this work we provide discretization procedures to simulate the relevant random variables then we analyze their respective errors this analysis proves that the discretization error is essentially linear with respect to the time step this result which was already known in some specific situations appears to be true in this much wider context finally we provide numerical experiments in random mechanics and finance and compare the different methods in terms of variance complexity computational time and time discretization error

searching for an optimal path in a tree with random costs we consider the problem of finding an optimal path leading from the root of a tree to any of its leaves the tree is known to be uniform binary and of height n and each branch independently may have a cost of 1 or 0 with probability p and 1 p respectively we show that for p 1 2 every algorithm which guarantees finding an exact cheapest path or even a path within a fixed cost ratio of the cheapest must run in exponential average time if however we are willing to accept a near optimal solution almost always then a pruning algorithm exists which finds such a solution in linear expected time the algorithm employs a depth first strategy which stops at regular intervals to appraise its progress and if the progress does not meet a criterion based on domain specific knowledge the current node is irrevocably pruned

multiplicity of solutions for a class of quasilinear equations with exponential critical growth abstract this paper deals with some results of existence and multiplicity of solutions for quasilinear problems involving the n laplacian operator when the nonlinearity has exponential critical growth to obtain our results we utilize ekeland variational principle mountain pass theorem and inequalities of the type trudinger moser bounded and unbounded domains are considered

multiple positive solutions for a quasilinear elliptic equation with critical exponential nonlinearity abstract let be a bounded domain in r n n 2 with c 2 boundary in this work we study the existence of multiple positive solutions of the following problem p div u 2 n 2 1 u 2 n u u 2 n 2 1 u 2 n u u p e u n n 1 u 0 in u 2 n 2 1 u 2 n u u q on where 0 p 2 n 1 q 0 n 1 and is a non negative holder continuous function we show that there exists 0 such that p admits at least two solutions in w 1 n for 0 and no solution for

applications of singular perturbation techniques to control problems this paper discusses typical applications of singular perturbation techniques to control problems in the last fifteen years the first three sections are devoted to the standard model and its time scale stability and controllability properties the next two sections deal with linear quadratic optimal control and one with cheap near singular control then the composite control and trajectory optimization are considered in two sections and stochastic control in one section the last section returns to the problem of modeling this time in the context of large scale systems the bibliography contains more than 250 titles

real time bidirectional search coordinated problem solving in uncertain situations this paper investigates real time bidirectional search rtbs algorithms where two problem solvers starting from the initial and goal states physically move toward each other to evaluate the rtbs performance two kinds of algorithms are proposed and are compared to real time unidirectional search one is called centralized rtbs where a supervisor always selects the best action from all possible moves of the two problem solvers the other is called decoupled rtbs where no supervisor exists and the two problem solvers independently select their next moves experiments on mazes and n puzzles show that 1 in clear situations decoupled rtbs performs better while in uncertain situations centralized rtbs becomes more efficient and 2 rtbs is more efficient than real time unidirectional search for 15 and 24 puzzles but not for randomly generated mazes it is shown that the selection of the problem solving organization is the selection of the problem space which determines the baseline of the organizational efficiency once a difficult problem space is selected the local coordination among problem solvers hardly overcome the deficit

depth first iterative deepening an optimal admissible tree search abstract the complexities of various search algorithms are considered in terms of time space and cost of solution path it is known that breadth first search requires too much space and depth first search can use too much time and doesn t always find a cheapest path a depth first iterative deepening algorithm is shown to be asymptotically optimal along all three dimensions for exponential tree searches the algorithm has been used successfully in chess programs has been effectively combined with bi directional search and has been applied to best first heuristic search as well this heuristic depth first iterative deepening algorithm is the only known algorithm that is capable of finding optimal solutions to randomly generated instances of the fifteen puzzle within practical resource limits

controlling the learning process of real time heuristic search real time search provides an attractive framework for intelligent autonomous agents as it allows us to model an agent s ability to improve its performance through experience however the behavior of real time search agents is far from rational during the learning convergence process in that they fail to balance the efforts to achieve a short term goal i e to safely arrive at a goal state in the present problem solving trial and a long term goal to find better solutions through repeated trials as a remedy we introduce two techniques for controlling the amount of exploration both overall and per trial the weighted real time search reduces the overall amount of exploration and accelerates convergence it sacrifices admissibility but provides a nontrivial bound on the converged solution cost the real time search with upper bounds insures solution quality in each trial when the state space is undirected these techniques result in a convergence process more stable compared with that of the learning real time a algorithm

actor critic type learning algorithms for markov decision processes algorithms for learning the optimal policy of a markov decision process mdp based on simulated transitions are formulated and analyzed these are variants of the well known actor critic or adaptive critic algorithm in the artificial intelligence literature distributed asynchronous implementations are considered the analysis involves two time scale stochastic approximations

agent searching in a tree and the optimality of iterative deepening the agent searching framework models the effort of a search strategy in terms of the distance traversed by an agent while exploring the search space the framework has been found to be useful in modeling search problems where the cost of backtracking and retracing search paths is important in determining search complexity in this paper we show that depth first iterative deepening dfid strategies are optimal for an agent searching in a line in m concurrent rays and in uniform b ary trees in the conventional search model it is known that dfid is asymptotically optimal for uninformed search of uniform b ary trees in this paper we prove the stronger result that for agent searching in uniform b ary trees iterative deepening is optimal up to lower order terms we also discuss the problems involved in optimally performing agent search in a graph

linear space best first search abstract best first search is a general heuristic search algorithm that always expands next a frontier node of lowest cost it includes as special cases breadth first search dijkstra s single source shortest path algorithm and the a algorithm its applicability however is limited by its exponential memory requirement previous approaches to this problem such as iterative deepening do not expand nodes in best first order if the cost function can decrease along a path we present a linear space best first search algorithm rbfs that always explores new nodes in best first order regardless of the cost function and expands fewer nodes than iterative deepening with a nondecreasing cost function on the sliding tile puzzles rbfs with a nonmonotonic weighted evaluation function dramatically reduces computation time with only a small penalty in solution cost in general rbfs reduces the space complexity of best first search from exponential to linear at the cost of only a constant factor in time complexity in our experiments

when is it better not to look ahead in situations where one needs to make a sequence of decisions it is often believed that looking ahead will help produce better decisions however it was shown 30 years ago that there are pathological situations in which looking ahead is counterproductive two long standing open questions are a what combinations of factors have the biggest influence on whether lookahead pathology occurs and b whether it occurs in real world decision making this paper includes simulation results for several synthetic game tree models and experimental results for three well known board games two chess endgames kalah with some modifications to facilitate experimentation and the 8 puzzle the simulations show the interplay between lookahead pathology and several factors that affect it and the experiments confirm the trends predicted by the simulation models the experiments also show that lookahead pathology is more common than has been thought all three games contain situations where it occurs

min max level estimate for a singular quasilinear polyharmonic equation in r2m abstract using the framework first presented by ruf and sani we give a proof of an adams type inequality which can be applied to the functional j u 1 2 r 2 m m u 2 0 m 1 a x u 2 d x r 2 m f x u x d x r 2 m h u d x under two kinds of assumptions on the nonlinearity we estimate the min max level of the functional as an application a multiplicity result for the related singular quasilinear elliptic equation is proved

inconsistent heuristics in theory and practice in the field of heuristic search it is usually assumed that admissible heuristics are consistent implying that consistency is a desirable attribute the term inconsistent heuristic has at times been portrayed negatively as something to be avoided part of this is historical early research discovered that inconsistency can lead to poor performance for a nodes might be re expanded many times however the issue has never been fully investigated and was not re considered after the invention of ida this paper shows that many of the preconceived notions about inconsistent heuristics are outdated the worst case exponential time of inconsistent heuristics is shown to only occur on contrived graphs with edge weights that are exponential in the size of the graph furthermore the paper shows that rather than being something to be avoided inconsistent heuristics often add a diversity of heuristic values into a search which can lead to a reduction in the number of node expansions inconsistent heuristics are easy to create contrary to the common perception in the ai literature to demonstrate this a number of methods for achieving effective inconsistent heuristics are presented pathmax is a way of propagating inconsistent heuristic values in the search from parent to children this technique is generalized into bidirectional pathmax bpmx which propagates values from a parent to a child node and vice versa bpmx can be integrated into ida and a when inconsistent heuristics are used with bpmx experimental results show a large reduction in the search effort required by ida positive results are also presented for a searches

a sensitivity formula for risk sensitive cost and the actor critic algorithm abstract we propose for risk sensitive control of finite markov chains a counterpart of the popular actor critic algorithm for classical markov decision processes the algorithm is based on a sensitivity formula for the risk sensitive cost and is shown to converge with probability one to the desired solution the proof technique is an adaptation of the ordinary differential equations approach for the analysis of two time scale stochastic approximation algorithms

state space search nogood learning online refinement of critical path dead end detectors in planning abstract conflict directed learning is ubiquitous in constraint satisfaction problems like sat but has been elusive for state space search on reachability problems like classical planning almost all existing approaches learn nogoods relative to a fixed solution length bound in which case planning reachability reduces to a constraint satisfaction problem here we introduce an approach to learning more powerful nogoods that are sound regardless of solution length i e that identify dead end states for which no solution exists the key technique we build on are critical path heuristics h c relative to a set c of conjunctions these recognize a dead end state s returning h c s if s has no solution even when allowing to break up conjunctive subgoals into the elements of c our key idea is to learn c during search whenever forward search has identified an unrecognized dead end s where h c s we analyze the situation at s and add new conjunctions into c in a way guaranteeing to obtain h c s thus we learn to recognize s as well as similar dead ends search may encounter in the future we furthermore learn clauses where s implies h c s to avoid the overhead of computing h c on every search state arranging these techniques in a depth first search we obtain an algorithm approaching the elegance of nogood learning in constraint satisfaction learning to refute search subtrees we run comprehensive experiments on solvable and unsolvable planning benchmarks in cases where forward search can identify dead ends and where h c dead end detection is effective our techniques reduce the depth first search space size by several orders of magnitude and often result in state of the art performance

iterative state space reduction for flexible computation abstract flexible computation is a general framework for decision making under limited computational resources it enables an agent to allocate limited computational resources to maximize its overall performance or utility in this paper we present a strategy for flexible computation which we call iterative state space reduction the main ideas are to reduce a problem space that is difficult to search to one that is relatively easy to explore to use the optimal solution from the reduced space as an approximate solution to the original problem and to iteratively apply multiple reductions to progressively find better solutions the basic operation for state space reduction is heuristic node pruning which excludes the nodes that do not seem to lead to high quality solutions from further examination iterative state space reduction can be combined with a state space search such as best first search or depth first branch and bound dfbnb the resulting algorithm can run as an anytime algorithm which provides a tradeoff between solution quality and computation based on an analytical modal we analyze the probability that one iteration of the iterative process finds a solution furthermore by combining it within dfbnb we apply iterative state space reduction to three combinatorial problems the maximum boolean satisfiability the symmetric tsp and asymmetric tsp our experimental results show that iterative state space reduction is effective and efficient making dfbnb find better solutions with less computation

adaptive aggregation methods for infinite horizon dynamic programming a class of iterative aggregation algorithms for solving infinite horizon dynamic programming problems is proposed the idea is to interject aggregation iterations in the course of the usual successive approximation method an important feature that sets this method apart from earlier ones is that the aggregate groups of states change adaptively from one aggregation iteration to the next depending on the progress of the computation this allows acceleration of convergence in difficult problems involving multiple ergodic classes for which methods using fixed groups of aggregate states are ineffective no knowledge of special problem structure is utilized by the algorithms