asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays part ii markovian rewards at each instant of time we are required to sample a fixed number m geq 1 out of n i i d processes whose distributions belong to a family suitably parameterized by a real number theta the objective is to maximize the long run total expected value of the samples following lai and robbins the learning loss of a sampling scheme corresponding to a configuration of parameters c theta 1 theta n is quantified by the regret r n c this is the difference between the maximum expected reward at time n that could be achieved if c were known and the expected reward actually obtained by the sampling scheme we provide a lower bound for the regret associated with any uniformly good scheme and construct a scheme which attains the lower bound for every configuration c the lower bound is given explicitly in terms of the kullback liebler number between pairs of distributions part ii of this paper considers the same problem when the reward processes are markovian

on saint venant s principle in finite anti plane shear an energy approach some time ago a version of saint venant s principle was formulated and established for finite elastostatics 1 as was discussed in 1 the issues of concern in connection with saint venant s principle in the nonlinear theory of elasticity are considerably more involved then those arising in the linear theory for a survey of results on saint venant s principle primarily for linear theories see e g 2 5 one difficulty is that the appropriate saint venant solutions need to be carefully characterized see e g 6 13 and the references cited therein secondly in the absence of superposition consideration of self equilibrated end loads is no longer sufficient furthermore instabilities may have to be taken into account also the decay rate for end effects even if exponential might depend on the overall loading as well as on geometry and material characteristics several of these issues have been considered in recent studies in the nonlinear elasticity context 1 14 23 as well as in investigations of spatial decay of solutions of nonlinear elliptic partial differential equations 24 32

a phragmen lindelof alternative result for the navier stokes equations for steady compressible viscous flow abstract in this paper the authors consider the navier stokes equations for steady compressible viscous flow in three dimensional cylindrical domain a differential inequality for appropriate energy associated with the solutions of the navier stokes isentropic flow in semi infinite pipe is derived from which the authors show a phragmen lindelof alternative result i e the solutions for steady compressible viscous n s flow problem either grow or decay exponentially as the distance from the entry section tends to infinity in the decay case the authors indicate how to bound explicitly the total energy in terms of data

the irrevocable multiarmed bandit problem this paper considers the multiarmed bandit problem with multiple simultaneous arm pulls and the additional restriction that we do not allow recourse to arms that were pulled at some point in the past but then discarded this additional restriction is highly desirable from an operational perspective and we refer to this problem as the irrevocable multiarmed bandit problem we observe that natural modifications to well known heuristics for multiarmed bandit problems that satisfy this irrevocability constraint have unsatisfactory performance and thus motivated introduce a new heuristic the packing heuristic we establish through numerical experiments that the packing heuristic offers excellent performance even relative to heuristics that are not constrained to be irrevocable we also provide a theoretical analysis that studies the price of irrevocability i e the performance loss incurred in imposing the constraint we propose on the multiarmed bandit model we show that this performance loss is uniformly bounded for a general class of multiarmed bandit problems and indicate its dependence on various problem parameters finally we obtain a computationally fast algorithm to implement the packing heuristic the algorithm renders the packing heuristic computationally cheaper than methods that rely on the computation of gittins indices

existence of global weak solutions for a 2d viscous shallow water equations and convergence to the quasi geostrophic model we consider a two dimensional viscous shallow water model with friction term existence of global weak solutions is obtained and convergence to the strong solution of the viscous quasi geostrophic equation with free surface term is proven in the well prepared case the ill prepared data case is also discussed

on the spatial decay of the energy for some quasilinear boundary value problems in semi infinite cylinders dans un cylindre semi infini a 3 dimensions r on considere le probleme p x u u u j j 0 avec des donnees aux limites de type dirichlet ou neumann on deduit une borne d energie

a survey of some results in stochastic adaptive control some results in discrete time stochastic adaptive control are surveyed the survey divides itself into two parts bayesian and non bayesian adaptive control in the former area the problems of converting an incompletely observed system into a completely observed one multi armed bandit processes bayesian adaptive control of markov chains and bayesian adaptive control of linear systems are exposed and surveyed in the latter area non bayesian adaptive control of markov chains and the self tuning regulator are dealt with proofs are given where appropriate to illustrate the methods involved

a new conceptual framework for analog computation in this paper we show how to explore the classical theory of computability using the tools of analysis a differential scheme is substituted for the classical recurrence scheme and a limit operator is substituted for the classical minimization we show that most relevant problems of computability over the non negative integers can be dealt with over the reals elementary functions are computable turing machines can be simulated the hierarchy of non computable functions can be represented the classical halting problem being solvable at some level the most typical concepts in analysis become natural in this framework the most relevant question is posed can we solve open problems of classical computability and computational complexity using as popper says the toolbox of analysis

improved decay estimates in time dependent stokes flow abstract this paper investigates the time dependent stokes flow of a viscous fluid in a channel with nonzero net entry flow assuming the fluid to be initially at rest with entry flow at the finite end of a semi infinite channel energy bounds for the flow are derived it is shown that the flow decays exponentially in energy norm to a transient poiseuille flow as the distance from the finite end tends to infinity the problem was previously investigated by lin saacm 2 1992 249 264 for the case in which the net entry flow was zero our methods are patterned after those of lin but a somewhat better choice of arbitrary constants yields an improved decay rate for lin s problem

on the universality of axon p systems axon p systems are computing models with a linear structure in the sense that all nodes i e computing units are arranged one by one along the axon such models have a good biological motivation an axon in a nervous system is a complex information processor of impulse signals because the structure of axon p systems is linear the computational power of such systems has been proved to be greatly restricted in particular axon p systems are not universal as language generators it remains open whether axon p systems are universal as number generators in this paper we prove that axon p systems are universal as both number generators and function computing devices and investigate the number of nodes needed to construct a universal axon p system it is proved that four nodes respectively nine nodes are enough for axon p systems to achieve universality as number generators respectively function computing devices these results illustrate that the simple linear structure is enough for axon p systems to achieve a desired computational power

small fast universal turing machines we present deterministic polynomial time universal turing machines utms with state symbol pairs of 3 11 5 7 6 6 7 5 and 8 4 these are the smallest known utms that simulate turing machines in polynomial time

on inequalities of korn friedrichs and babuska aziz korn s inequalities for integrals of quadratic functionals subject to certain side conditions have played a fundamental role in the development of elasticity theory see e g i 14 the korn inequalities are essential in establishing coerciveness of the differential operators of linear elastostatics and thus form the basis for existence results in that theory 9 among other areas in elasticity where consideration of korn s inequalities arise we cite for example fundamental studies on the mathematical foundations of finite elements 15 17 stability theory 11 and qualitative analyses of solution behavior such as those involved in the investigation of saint venant s principle 18 20 in the latter two areas of application in particular information on the optimal constants appearing in the inequalities the korn constants is of importance thus several investigations have been concerned with evaluating the korn constants for specific domains and with obtaining bounds for these constants 4 6 8 12 13 our purpose in this paper is to draw attention to an equivalence for two dimensional simply connected domains between korn s inequality in the second case see w 2 here and two other inequalities for quadratic functionals which have received comparatively little attention in the literature the first of these is an inequality due apparently to k o frmorichs 2 between the real and imaginary parts of an analytic function or equivalently between a harmonic function and its conjugate defined on a two dimensional domain r thus if w z f ig z xl ix2 is analytic on r friedrichs inequality reads

small universal turing machines abstract let utm m n be the class of universal turing machine with m states and n symbols universal turing machines are proved to exist in the following classes utm 24 2 utm 10 3 utm 7 4 utm 5 5 utm 4 6 utm 3 10 and utm 2 18

approximating the stochastic knapsack problem the benefit of adaptivity we consider a stochastic variant of the np hard 0 1 knapsack problem in which item values are deterministic and item sizes are independent random variables with known arbitrary distributions items are placed in the knapsack sequentially and the act of placing an item in the knapsack instantiates its size our goal is to compute a solution policy that maximizes the expected value of items successfully placed in the knapsack where the final overflowing item contributes no value we consider both nonadaptive policies that designate a priori a fixed sequence of items to insert and adaptive policies that can make dynamic choices based on the instantiated sizes of items placed in the knapsack thus far an important facet of our work lies in characterizing the benefit of adaptivity for this purpose we advocate the use of a measure called the adaptivity gap the ratio of the expected value obtained by an optimal adaptive policy to that obtained by an optimal nonadaptive policy we bound the adaptivity

saint venant s principle in nonlinear plane elasticity with sufficiently small strains a homogeneous isotropic cylinder in an equilibrium state of plane strain whose cross section is a rectangle r 0 y1 2l 0 y2 h with h l 1 is considered there are no body forces and the long sides are stress free at y1 0 and y1 2l there are arbitrary loadings each statically equivalent to a uniformly distributed tensile or compressive stress c within the theory of nonlinear elasticity and with the strains and strain gradients assumed to be sufficiently small but with no such assumptions on the displacement gradients it is proved that if 1 2 represents the cauchy stress tensor and the kronecker delta then c 1 1 decays exponentially to zero in r with distance from the nearer end and the decay constant depends only upon the material but is independent of l

spiking neural p systems with rules on synapses spiking neural p systems sn p systems for short are a class of membrane systems inspired from the way the neurons process information and communicate by means of spikes in this paper we introduce and investigate a new class of sn p systems with spiking rules placed on synapses the computational completeness is first proved then two small universal sn p systems with rules on synapses for computing functions are constructed specifically when using standard spiking rules we obtain a universal system with 39 neurons while when using extended spiking rules on synapses a universal sn p system with 30 neurons is constructed

a learning approach for interactive marketing to a customer segment when a marketer in an interactive environment decides which messages to send to her customers she may send messages currently thought to be most promising exploitation or use poorly understood messages for the purpose of information gathering exploration we assume that customers are already clustered into homogeneous segments and we consider the adaptive learning of message effectiveness within a customer segment we present a bayesian formulation of the problem in which decisions are made for batches of customers simultaneously although decisions may vary within a batch this extends the classical multiarmed bandit problem for sampling one by one from a set of reward populations our solution methods include a lagrangian decomposition based approximate dynamic programming approach and a heuristic based on a known asymptotic approximation to the multiarmed bandit solution computational results show that our methods clearly outperform approaches that ignore the effects of information gain

the learning component of dynamic allocation indices this article is free to read on the publisher s website for a multiarmed bandit problem with exponential discounting the optimal allocation rule is defined by a dynamic allocation index defined for each arm on its space the index for an arm is equal to the expected immediate reward from the arm with an upward adjustment reflecting any uncertainty about the prospects of obtaining rewards from the arm and the possibilities of resolving those uncertainties by selecting that arm thus the learning component of the index is defined to be the difference between the index and the expected immediate reward for two arms with the same expected immediate reward the learning component should be larger for the arm for which the reward rate is more uncertain this is shown to be true for arms based on independent samples from a fixed distribution with an unknown parameter in the cases of bernoulli and normal distributions and similar results are obtained in other cases

restless bandits linear programming relaxations and a primal dual index heuristic we develop a mathematical programming approach for the classicalpspace hard restless bandit problem in stochastic optimization we introduce a hierarchy ofn wheren is the number of bandits increasingly stronger linear programming relaxations the last of which is exact and corresponds to the exponential size formulation of the problem as a markov decision chain while the other relaxations provide bounds and are efficiently computed we also propose a priority index heuristic scheduling policy from the solution to the firstorder relaxation where the indices are defined in terms of optimal dual variables in this way we propose a policy and a suboptimality guarantee we report results of computational experiments that suggest that the proposed heuristic policy is nearly optimal moreover the second order relaxation is found to provide strong bounds on the optimal value

spatial decay bounds for the channel flow of the boussinesq equations in this paper we consider the boussinesq equations in a semi infinite channel we show that under appropriate restrictions on the data if the fluid velocity initially is small the solution will tend exponentially to a transient laminar flow as the distance from the entry section tends to infinity we also derive the explicit decay bounds

phragmen lindelof alternative results for the shallow water equations for transient compressible viscous flow abstract in this paper the spatial behavior of the viscous shallow water equations in a two dimensional channel domain is studied a differential inequality for suitable energy associated with the solutions of the viscous shallow water equations in a semi infinite channel is derived from which we show the phragmen lindelof alternative results the main tool used is the energy method our results can be viewed as a version of saint venant s principle to the transient compressible viscous flow

closed form analytic maps in one and two dimensions can simulate universal turing machines we show closed form analytic functions consisting of a finite number of trigonometric terms can simulate turing machines with exponential slowdown in one dimension or in real time in two or more

on the value of learning for bernoulli bandits with unknown parameters investigates the multiarmed bandit problem where each arm generates an infinite sequence of bernoulli distributed rewards the parameters of these bernoulli distributions are unknown and initially assumed to be beta distributed every time a bandit is selected its beta distribution is updated to new information in a bayesian way the objective is to maximize the long term discounted rewards we study the relationship between the necessity of acquiring additional information and the reward this is done by considering two extreme situations which occur when a bandit has been played n times the situation where the decision maker stops learning and the situation where the decision maker acquires full information about that bandit we show that the difference in reward between this lower and upper bound goes to zero as n grows large

thompson sampling for stochastic control the finite parameter case in this paper we apply thompson sampling to a class of average reward stochastic control problems with parameter uncertainty specifically we study an average reward stochastic control problem over an infinite horizon in which both the reward and state transition distributions are parameterized by an unknown parameter taking values in a finite space the main result of this paper is a proof showing that thompson sampling achieves a worst case average per period regret of o t 1 which is asymptotically optimal

inaccessibility and undecidability in computation geometry and dynamical systems non self similar sets defined by a decision procedure are numerically investigated by introducing the notion of inaccessibility to ideal decision procedure that is connected with undecidability a halting set of a universal turing machine utm the mandelbrot set and a riddled basin are mainly investigated as non self similar sets with a decision procedure by encoding a symbol sequence to a point in a euclidean space a halting set of a utm is shown to be geometrically represented as a non self similar set having different patterns and different fine structures on arbitrarily small scales the boundary dimension of this set is shown to be equal to the space dimension implying that the ideal decision procedure is inaccessible in the presence of error this property is shown to be invariant under application of fractal code transformations thus a characterization of undecidability is given by the inaccessibility to the ideal decision procedure and its invariance against the code transformations it is also shown that the distribution of halting time of the utm decays with a power law or slower and that this characteristic is also unchanged under code transformation the mandelbrot set is shown to have these features including the invariance against the code transformation in common and is connected with undecidable sets in contrast although a riddled basin as a geometric representation of a certain context free language has the boundary dimension equal to the space dimension and a power law halting time distribution these properties are not invariant against the code transformation thus the riddled basin is ranked as middle between an ordinary fractal and a halting set of a utm or the mandelbrot set last we propose undecidability for analog computation and discuss the appropriate condition for coding 2001 elsevier science

on small reduced and fast universal accepting networks of splicing processors in this paper we show that accepting networks of splicing processors ansps of size 2 are computationally complete since by definition an ansp needs at least two nodes to perform non trivial computations this completely settles the question of designing complete ansps of minimal size also we derive from this result the fact that all the languages in pspace can be accepted by ansps of size 2 having polynomial length complexity the ansp complexity measure for the space used in a computation however the construction that we propose although efficient from the descriptional complexity and space complexity points of view does not seem to have good properties from the time complexity point of view in this respect we prove that ansps of size three can decide all languages in np in polynomial time the previous lower bound on size for both completeness and efficient acceptance of np languages was seven we also consider ansps with restricted features proving the following normal forms for any ansp there exists an equivalent ansp without input filters and one without output filters finally we show how to construct a small universal ansp and make several considerations on the computational efficiency of universal ansps

phragmen lindelof type results for harmonic functions with nonlinear boundary conditions this paper is concerned with investigating the asymptotic behavior of harmonic functions defined on a three dimensional semi infinite cylinder where homogeneous nonlinear boundary conditions are imposed on the lateral surface of the cylinder such problems arise in the theory of steady state heat conduction the classical phragmen lindelof theorem states that harmonic functions which vanish on the lateral surface of the cylinder must either grow exponentially or decay exponentially with distance from the finite end of the cylinder here we show that the results are significantly different when the homogeneous dirichlet boundary condition is replaced by the nonlinear heatloss or heat gain type boundary condition we show that polynomial growth or decay or exponential growth or decay may occur depending on the form of the nonlinearity explicit estimates for the growth or decay rates are obtained

on the asymptotic behaviour of solutions of some nonlinear elliptic and parabolic equations in this paper we investigate the spatial asymptotic behaviour of some nonlinear elliptic and parabolic equations related with the process of diffusion convection we obtain results of the kind of the phragmen lindelof alternative the main idea of the paper is to obtain a nonlinear ordinary differential inequality which is satisfied by a certain measure of the solutions this ordinary inequality was studied by horgan and payne in a recent paper we obtain our results with the help of estimates known for this inequality when the solutions tend to zero we also obtain an estimate for the amplitude term

normal hyperbolicity of center manifolds and saint venant s principle the concept of normal hyperbolicity of center manifolds is generalized to infinite dimensional differential equations in particular to elliptic problems in cylindrical domains it is shown that all solutions u staying close to the center manifold for t l l satisfy an estimate of the form left u t tilde u t right leqslant ce alpha l t where c and are independent of l and is a solution on the center manifold these results are applied to saint venant s principle for the static deformation of nonlinearly elastic prismatic bodies the use of the center manifold permits the effective treatment of the general case of non zero resultant forces and moments acting on each cross section

global existence for the dirichlet problem for the viscous shallow water equations abstract a global existence and uniqueness theorem of strong solutions for the initial boundary value problem with dirichlet boundary conditions is established for small forcing and small initial data an exponentialc0decay rate is also established and the solution is shown to be classical fort 0 the positivity of the fluid heighthfort 0 is also proved

phragmen lindelof decay theorems for classes of nonlinear dirichlet problems in a circular cylinder abstract classes of nonlinear elliptic equations in a long circular cylinder of radius one are considered the equations are of the form 2u s u u u t u u 2 where u u x1 x2 x3 and u u represent general partial derivatives of the indicated order homogeneous dirichlet data are prescribed on the long sides of the cylinder and throughout the cylinder u is a priori assumed to be sufficiently small while u and for some classes also u is assumed to be bounded in absolute value by one with the above assumptions it is proved that every solution u decays exponentially with distance from the nearer end with a decay constant k which depends on the smoothness properties of s and t but is independent of the length of the cylinder

decay estimates for a class of second order quasilinear equations in three dimensions in a recent paper 1 the authors made use of differential inequality techniques to derive exponential decay estimates for a class of second order quasilinear equations not necessarily elliptic in the plane specifically they were concerned with solutions of equations in divergence form defined in a semi infinite strip with non zero boundary data on one end only decay estimates in various norms were derived

networks of splicing processors with evaluation sets as optimization problems solvers a new version of networks of splicing processors a generalization of the protocol of cooperation in networks of polarized evolutionary processors a linear time solution to the 0 1 knapsack problem based on nspes a succinct solution to the 0 1 knapsack problem based on nspes in this paper we consider a new highly parallel and distributed bio inspired computational model having its underlying structure an undirected graph whose nodes are processors that use the splicing operation on the data they are processing they communicate with each other via a protocol based on the compatibility between their accepting values with respect to some predefined evaluation sets and the values of the data computed by a valuation mapping here we use the valuation mapping from a quantitative perspective hence this model might be more suitable to solve some specific hard optimization problems in a more efficient and succinct way in the aim of supporting this possibility we propose an algorithm based on these networks able to efficiently solve an np hard combinatorial problem namely the 0 1 knapsack problem we prove that the computation time of the network solving an instance of the problem is linear surprisingly enough the size of the network is very small only four nodes are sufficient and the topology of the network may be chosen as chain ring or complete graph furthermore the evaluation sets and the sets of compatible values associated with the nodes as well as the valuation mapping remain unchanged for all instances of the same size this result suggests that this model is suitable to address optimization problems where quantitative conditions have a relevant role

accepting networks of splicing processors complexity results in this paper we consider a new bio inspired computing model the accepting network of splicing processors we define two computational complexity classes based on this model and show how they are related to the classical ones defined for turing machines namely np and pspace furthermore we approach the topic of problem solving using these newly defined devices in this context a linear time solution for one of the most interesting np complete problems the sat problem is presented the results presented here suggest once more that nondeterminism might be approached in a deterministic way by means of multiplicities

an automaton group with undecidable order and engel problems abstract for every turing machine we construct an automaton group that simulates it precisely starting from an initial configuration of the turing machine we explicitly construct an element of the group such that the turing machine stops if and only if this element is of finite order if the turing machine is universal the corresponding automaton group has an undecidable order problem this solves a problem raised by grigorchuk the above group also has an undecidable engel problem there is no algorithm that given g h in the group decides whether there exists an integer n such that the n iterated commutator g h h h is the identity or not this solves a problem raised by bartholdi

three small universal spiking neural p systems in this work we give three small spiking neural p systems we begin by constructing a universal spiking neural p system with extended rules and only 4 neurons this is the smallest possible number of neurons for a universal system of its kind we prove this by showing that the set of problems solved by spiking neural p systems with 3 neurons is bounded above by nlnl and so there exists no such universal system with 3 neurons if we generalise the output technique we immediately find a universal spiking neural p system with extended rules that has only 3 neurons this is also the smallest possible number of neurons for a universal system of its kind finally we give a universal spiking neural p system with standard rules and only 7 neurons in addition to giving a significant improvement in terms of reducing the number of neurons our systems also offer an exponential improvement on the time and space overheads of the small universal spiking neural p systems of other authors

spatial decay bounds in time dependent pipe flow of an incompressible viscous fluid in this paper the authors investigate the flow of an incompressible viscous fluid in a semi infinite cylindrical pipe if the net entrance flow is nonzero then the fluid velocity will not tend to zero as the distance from the entrance end tends to infinity when the fluid adheres at the cylinder wall and the fluid is initially at rest assuming that the entrance velocity data are small enough and that the fluid flow converges to a laminar flow as the distance down the pipe tends to infinity it is shown that the convergence in energy measure is at least exponential

computational bounds on polynomial differential equations in this paper we study from a computational perspective some properties of the solutions of polynomial ordinary differential equations we consider elementary in the sense of analysis discrete time dynamical systems satisfying certain criteria of robustness we show that those systems can be simulated with elementary and robust continuous time dynamical systems which can be expanded into fully polynomial ordinary differential equations in q p this sets a computational lower bound on polynomial odes since the former class is large enough to include the dynamics of arbitrary turing machines we also apply the previous methods to show that the problem of determining whether the maximal interval of definition of an initial value problem defined with polynomial odes is bounded or not is in general undecidable even if the parameters of the system are computable and comparable and if the degree of the corresponding polynomial is at most 56 combined with earlier results on the computability of solutions of polynomial odes one can conclude that there is from a computational point of view a close connection between these systems and turing machines

small turing machines and generalized busy beaver competition let tm k l be the set of one tape turing machines with k states and l symbols it is known that the halting problem is decidable for machines in tm 2 3 and tm 3 2 we prove that the decidability of machines in tm 2 4 and tm 3 3 will be difficult to settle by giving machines in these sets for which the halting problem depends on an open problem in number theory a machine in tm 5 2 with the same result is already known and moreover this machine is the record holder for the busy beaver competitions this is the machine in tm 5 2 which halts when starting from a blank tape making the greatest number of steps and leaving the greatest number of non blank symbols we give potential winners for similar generalized busy beaver competitions in tm 2 3 tm 2 4 and tm 3 3

exponential decay estimates for solutions of the von karman equations on a semi infinite strip this paper is concerned with the analysis of saint venant edge effects for nonlinear elastic plates the model used is based on the von karman plate equations a coupled system of two nonlinear elliptic partial differential equations with the biharmonic operator as the principal part energy methods are used to establish a nonlinear integro differential inequality for a quadratic functional arguments based on comparison theorems are then used to establish exponential decay of end effects

improvements and generalizations of stochastic knapsack and markovian bandits approximation algorithms we study the multi armed bandit problem with arms which are markov chains with rewards in the finite horizon setting the celebrated gittins indices do not apply and the exact solution is intractable we provide approximation algorithms for the general model of markov decision processes with nonunit transition times when preemption is allowed we provide a 1 2 e approximation along with an example showing this is tight when preemption isn t allowed we provide a 1 12 approximation which improves to a 4 27 approximation when transition times are unity our model captures the markovian bandits model of gupta et al the stochastic knapsack model of dean et al and the budgeted learning model of guha and munagala our algorithms improve existing results in all three areas in our analysis we encounter and overcome to our knowledge a new obstacle an algorithm that provably exists via analytical arguments but cannot be found in polynomial time