interior point methods for full information and bandit online learning we study the problem of predicting individual sequences with linear loss with full and partial or bandit feed back our main contribution is the first efficient algorithm for the problem of online linear optimization in the bandit setting which achieves the optimal o t regret in addition for the full information setting we give a novel regret minimization algorithm these results are made possible by the introduction of interior point methods for convex optimization to online learning

the continuum armed bandit problem in this paper we consider the multiarmed bandit problem where the arms are chosen from a subset of the real line and the mean rewards are assumed to be a continuous function of the arms the problem with an infinite number of arms is much more difficult than the usual one with a finite number of arms because the built in learning task is now infinite dimensional we devise a kernel estimator based learning scheme for the mean reward as a function of the arms using this learning scheme we construct a class of certainty equivalence control with forcing schemes and derive asymptotic upper bounds on their learning loss to the best of our knowledge these bounds are the strongest rates yet available moreover they are stronger than the o n required for optimality with respect to the average cost per unit time criterion

regret bounds for restless markov bandits we consider the restless markov bandit problem in which the state of each arm evolves according to a markov process independently of the learner s actions we suggest an algorithm that first represents the setting as an mdp which exhibits some special structural properties in order to grasp this information we introduce the notion of e structured mdps which are a generalization of concepts like approximate state aggregation and mdp homomorphisms we propose a general algorithm for learning e structured mdps and show regret bounds that demonstrate that additional structural information enhances learning applied to the restless bandit setting this algorithm achieves after any t steps regret of order o t with respect to the best policy that knows the distributions of all arms we make no assumptions on the markov chains underlying each arm except that they are irreducible in addition we show that index based policies are necessarily suboptimal for the considered problem

finding numerical derivatives for unstructured and noisy data by multiscale kernels the recently developed multiscale kernel of r opfer adv comput math 25 2006 pp 357 380 is applied to approximate numerical derivatives the proposed method is truly mesh free and can handle unstructured data with noise in any dimension the method of tikhonov and the method of l curve are employed for regularization no information about the noise level is required an error analysis is provided in a general setting for all dimensions numerical comparisons are given in two dimensions which show competitive results with recently published thin plate spline methods

kernel estimation and model combination in a bandit problem with covariates multi armed bandit problem is an important optimization game that requires an exploration exploitation tradeoff to achieve optimal total reward motivated from industrial applications such as online advertising and clinical research we consider a setting where the rewards of bandit machines are associated with covariates and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules under a exible problem setup we establish asymptotic strong consistency and perform a finite time regret analysis for a sequential randomized allocation strategy based on kernel estimation in addition since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable we propose a model combining allocation strategy for adaptive performance simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy

moving least squares via orthogonal polynomials a method for moving least squares interpolation and differentiation is presented in the framework of orthogonal polynomials on discrete points this yields a robust and efficient method which can avoid singularities and breakdowns in the moving least squares method caused by particular configurations of nodes in the system the method is tested by applying it to the estimation of first and second derivatives of test functions on random point distributions in two and three dimensions and by examining in detail the evaluation of second derivatives on one selected configuration the accuracy and convergence of the method are examined with respect to length scale point separation and the number of points used the method is found to be robust accurate and convergent

the approximation power of moving least squares a general method for near best approximations to functionals on r d using scattered data information is discussed the method is actually the moving least squares method presented by the backus gilbert approach it is shown that the method works very well for interpolation smoothing and derivatives approximations for the interpolation problem this approach gives mclain s method the method is near best in the sense that the local error is bounded in terms of the error of a local best polynomial approximation the interpolation approximation in r d is shown to be a c function and an approximation order result is proven for quasi uniform sets of data points

non asymptotic analysis of a new bandit algorithm for semi bounded rewards in this paper we consider a stochastic multiarmed bandit problem it is known in this problem that deterministic minimum empirical divergence dmed policy achieves the asymptotic theoretical bound for the model where each reward distribution is supported in a known bounded interval say 0 1 however the regret bound of dmed is described in an asymptotic form and the performance in finite time has been unknown we modify this policy and derive a finite time regret bound for the new policy indexed minimum empirical divergence imed by refining large deviation probabilities to a simple nonasymptotic form further the refined analysis reveals that the finite time regret bound is valid even in the case that the reward is not bounded from below therefore our finite time result applies to the case that the minimum reward that is the maximum loss is unknown or unbounded we also present some simulation results which shows that imed much improves dmed and performs competitively to other state of the art policies

prediction using unsupervised learning abstract the manner and efficiency with which neural networks use competitive learning to discover long term regularities in the dynamics of nonlinear dissipative systems is analysed the network creates a coarse grained dynamics which depends both on the connectivity of the network and the intrinsic properties of the time series the learning process is analysed using a master equation approach and the scaling behaviour of the information entropy while the prediction error after learning is examined as a function of network size and the nature of dynamical clustering in phase space scaling arguments are deduced and compared with simulations of discrete mapping

learning to optimize via posterior sampling this paper considers the use of a simple posterior sampling algorithm to balance between exploration and exploitation when learning to optimize actions such as in multiarmed bandit problems the algorithm also known as thompson sampling and as probability matching offers significant advantages over the popular upper confidence bound ucb approach and can be applied to problems with finite or infinite action spaces and complicated relationships among action rewards we make two theoretical contributions the first establishes a connection between posterior sampling and ucb algorithms this result lets us convert regret bounds developed for ucb algorithms into bayesian regret bounds for posterior sampling our second theoretical contribution is a bayesian regret bound for posterior sampling that applies broadly and can be specialized to many model classes this bound depends on a new notion we refer to as the eluder dimension which measures the degree of dependence among action rewards compared to ucb

a simple adaptive procedure leading to correlated equilibrium we propose a new and simple adaptive procedure for playing a game regret matching in this procedure players may depart from their current play with probabilities that are proportional to measures of regret for not having used other strategies in the past it is shown that our adaptive procedure guarantees that with probability one the empirical distributions of play converge to the set of correlated equilibria of the game

on derivative estimation and the solution of least squares problems surface interpolation finds application in many aspects of science and technology two specific areas of interest are surface reconstruction techniques for plant architecture and approximating cell face fluxes in the finite volume discretisation strategy for solving partial differential equations numerically an important requirement of both applications is accurate local gradient estimation in surface reconstruction this gradient information is used to increase the accuracy of the local interpolant while in the finite volume framework accurate gradient information is essential to ensure second order spatial accuracy of the discretisation in this work two different least squares strategies for approximating these local gradients are investigated and the errors associated with each analysed it is shown that although the two strategies appear different they produce the same least squares error some carefully chosen case studies are used to elucidate this finding

cover tree bayesian reinforcement learning this paper proposes an online tree based bayesian approach for reinforcement learning for inference we employ a generalised context tree model this defines a distribution on multivariate gaussian piecewise linear models which can be updated in closed form the tree structure itself is constructed using the cover tree method which remains efficient in high dimensional spaces we combine the model with thompson sampling and approximate dynamic programming to obtain effective exploration policies in unknown environments the flexibility and computational simplicity of the model render it suitable for many reinforcement learning problems in continuous state spaces we demonstrate this in an experimental comparison with a gaussian process model a linear model and simple least squares policy iteration

surfaces generated by moving least squares methods an analysis of moving least squares m l s methods for smoothing and interpolating scattered data is presented in particular theorems are proved concerning the smoothness of interpolants and the description of m l s processes as projection methods some properties of compositions of the m l s projector with projectors associated with finiteelement schemes are also considered the analysis is accompanied by examples of univariate and bivariate problems

regret minimization under partial monitoring we consider repeated games in which the player instead of observing the action chosen by the opponent in each game round receives a feedback generated by the combined choice of the two players we study hannan consistent players for these games that is randomized playing strategies whose per round regret vanishes with probability one as the number n of game rounds goes to infinity we prove a general lower bound of n 1 3 for the convergence rate of the regret and exhibit a specific strategy that attains this rate for any game for which a hannan consistent player exists

truthful mechanisms with implicit payment computation it is widely believed that computing payments needed to induce truthful bidding is somehow harder than simply computing the allocation we show that the opposite is true creating a randomized truthful mechanism is essentially as easy as a single call to a monotone allocation rule our main result is a general procedure to take a monotone allocation rule for a single parameter domain and transform it via a black box reduction into a randomized mechanism that is truthful in expectation and individually rational for every realization the mechanism implements the same outcome as the original allocation rule with probability arbitrarily close to 1 and requires evaluating that allocation rule only once we also provide an extension of this result to multiparameter domains and cycle monotone allocation rules under mild star convexity and nonnegativity hypotheses on the type space and allocation rule respectively because our reduction is simple versatile and general it has many applications to mechanism design problems in which re evaluating the allocation rule is either burdensome or informationally impossible applying our result to the multiarmed bandit problem we obtain truthful randomized mechanisms whose regret matches the information theoretic lower bound up to logarithmic factors even though prior work showed this is impossible for truthful deterministic mechanisms we also present applications to offline mechanism design showing that randomization can circumvent a communication complexity lower bound for deterministic payments computation and that it can also be used to create truthful shortest path auctions that approximate the welfare of the vcg allocation arbitrarily well while having the same running time complexity as dijkstra s algorithm

toward a classification of finite partial monitoring games partial monitoring games constitute a mathematical framework for sequential decision making problems with imperfect feedback the learner repeatedly chooses an action the opponent responds with an outcome and then the learner suffers a loss and receives a feedback signal both of which are fixed functions of the action and the outcome the goal of the learner is to minimize his total cumulative loss we make progress toward the classification of these games based on their minimax expected regret namely we classify almost all games with two outcomes and a finite number of actions we show that their minimax expected regret is either zero email protected t q t 2 3 or q t and we give a simple and efficiently computable classification of these four classes of games our hope is that the result can serve as a stepping stone toward classifying all finite partial monitoring games

the nonstochastic multiarmed bandit problem in the multiarmed bandit problem a gambler must decide which arm of k nonidentical slot machines to play in a sequence of trials so as to maximize his reward this classical problem has received much attention because of the simple model it provides of the trade off between exploration trying out each arm to find the best one and exploitation playing the arm believed to give the best payoff past solutions for the bandit problem have almost always relied on assumptions about the statistics of the slot machines in this work we make no statistical assumptions whatsoever about the nature of the process generating the payoffs of the slot machines we give a solution to the bandit problem in which an adversary rather than a well behaved stochastic process has complete control over the payoffs in a sequence of t plays we prove that the per round payoff of our algorithm approaches that of the best arm at the rate o t 1 2 we show by a matching lower bound that this is the best possible we also prove that our algorithm approaches the per round payoff of any set of strategies at a similar rate if the best strategy is chosen from a pool of n strategies then our algorithm approaches the per round payoff of the strategy at the rate o log n1 2 t 1 2 finally we apply our results to the problem of playing an unknown repeated matrix game we show that our algorithm approaches the minimax payoff of the unknown game at the rate o t 1 2

dimensions and entropies of strange attractors from a fluctuating dynamics approach abstract it is shown that the fluctuations in the divergence of near by trajectories on strictly deterministic strange attractors can be modelled by stochastic concepts in particular we propose kramers moyal type equations for correlation functions between points on the attractor the drift terms are the lyapunov exponents the diffusion terms depend on the above fluctuations from this we obtain bounds on generalized dimensions and entropies numerical results show that in nearly all studied cases henon map zaslavskii map mackey glass eq the attractors are fractal measures in the sense of farmer information dimension hausdorff dimension metric entropy topological entropy

new bounds on the price of bandit feedback for mistake bounded online multiclass learning abstract this paper is about two generalizations of the mistake bound model to online multiclass classification in the standard model the learner receives the correct classification at the end of each round and in the bandit model the learner only finds out whether its prediction was correct or not for a set f of multiclass classifiers let opt std f and opt bandit f be the optimal bounds for learning f according to these two models we show that an opt bandit f 1 o 1 y ln y opt std f bound is the best possible up to the leading constant closing a log y factor gap

the weighted majority algorithm we study the construction of prediction algorithms in a situation in which a learner faces a sequence of trials with a prediction to be made in each and the goal of the learner is to make few mistakes we are interested in the case where the learner has reason to believe that one of some pool of known algorithms will perform well but the learner does not know which one a simple and effective method based on weighted voting is introduced for constructing a compound algorithm in such a circumstance we call this method the weighted majority algorithm we show that this algorithm is robust in the presence of errors in the data we discuss various versions of the weighted majority algorithm and prove mistake bounds for them that are closely related to the mistake bounds of the best algorithms of the pool for example given a sequence of trials if there is an algorithm in the pool a that makes at most m mistakes then the weighted majority algorithm will make at most c log a m mistakes on that sequence where c is fixed constant

state space reconstruction in the presence of noise takens theorem demonstrates that in the absence of noise a multidimensional state space can be reconstructed from a scalar time series this theorem gives little guidance however about practical considerations for reconstructing a good state space we extend takens treatment applying statistical methods to incorporate the effects of observational noise and estimation error we define the distortion matrix which is proportional to the conditional covariance of a state given a series of noisy measurements and the noise amplification which is proportional to root mean square time series prediction errors with an ideal model we derive explicit formulae for these quantities and we prove that in the low noise limit minimizing the distortion is equivalent to minimizing the noise amplification we identify several different scaling regimes for distortion and noise amplification and derive asymptotic scaling laws when the dimension and lyapunov exponents are sufficiently large these scaling laws show that no matter how the state space is reconstructed there is an explosion in the noise amplification from a practical point of view determinism is lost and the time series is effectively a random process in the low noise large data limit we show that the technique of local singular value decomposition is an optimal coordinate transformation in the sense that it achieves the minimum distortion in a state space of the lowest possible dimension however in numerical experiments we find that estimation error complicates this issue for local approximation methods we analyze the effect of reconstruction on estimation error derive a scaling law and suggest an algorithm for reducing estimation errors

learning where to attend with deep architectures for image tracking we discuss an attentional model for simultaneous object tracking and recognition that is driven by gaze data motivated by theories of perception the model consists of two interacting pathways identity and control intended to mirror the what and where pathways in neuroscience models the identity pathway models object appearance and performs classification using deep factored restricted boltzmann machines at each point in time the observations consist of foveated images with decaying resolution toward the periphery of the gaze the control pathway models the location orientation scale and speed of the attended object the posterior distribution of these states is estimated with particle filtering deeper in the control pathway we encounter an attentional mechanism that learns to select gazes so as to minimize tracking uncertainty unlike in our previous work we introduce gaze selection strategies that operate in the presence of partial information and on a continuous action space we show that a straightforward extension of the existing approach to the partial information setting results in poor performance and we propose an alternative method based on modeling the reward surface as a gaussian process this approach gives good performance in the presence of partial information and allows us to expand the action space from a small discrete set of fixation points to a continuous domain

combinatorial multi armed bandit and its extension to probabilistically triggered arms we define a general framework for a large class of combinatorial multi armed bandit cmab problems where subsets of base arms with unknown distributions form super arms in each round a super arm is played and the base arms contained in the super arm are played and their outcomes are observed we further consider the extension in which more base arms could be probabilistically triggered based on the outcomes of already triggered arms the reward of the super arm depends on the outcomes of all played arms and it only needs to satisfy two mild assumptions which allow a large class of nonlinear reward instances we assume the availability of an offline approximation oracle that takes the means of the outcome distributions of arms and outputs a super arm that with probability generates an fraction of the optimal expected reward the objective of an online learning algorithm for cmab is to minimize approximation regret which is the difference in total expected reward between the fraction of expected reward when always playing the optimal super arm and the expected reward of playing super arms according to the algorithm we provide cucb algorithm that achieves o log n distribution dependent regret where n is the number of rounds played and we further provide distribution independent bounds for a large class of reward functions our regret analysis is tight in that it matches the bound of ucb1 algorithm up to a constant factor for the classical mab problem and it significantly improves the regret bound in an earlier paper on combinatorial bandits with linear rewards we apply our cmab framework to two new applications probabilistic maximum coverage pmc for online advertising and social in uence maximization for viral marketing both having nonlinear reward structures in particular application to social in uence maximization requires our extension on probabilistically triggered arms

contextual bandits with similarity information in a multi armed bandit mab problem an online algorithm makes a sequence of choices in each round it chooses from a time invariant set of alternatives and receives the payoff associated with this alternative while the case of small strategy sets is by now well understood a lot of recent work has focused on mab problems with exponentially or infinitely large strategy sets where one needs to assume extra structure in order to make the problem tractable in particular recent literature considered information on similarity between arms we consider similarity information in the setting of contextual bandits a natural extension of the basic mab problem where before each round an algorithm is given the context a hint about the payoffs in this round contextual bandits are directly motivated by placing advertisements on web pages one of the crucial problems in sponsored search a particularly simple way to represent similarity information in the contextual bandit setting is via a similarity distance between the context arm pairs which bounds from above the difference between the respective expected payoffs prior work on contextual bandits with similarity uses uniform partitions of the similarity space so that each context arm pair is approximated by the closest pair in the partition algorithms based on uniform partitions disregard the structure of the payoffs and the context arrivals which is potentially wasteful we present algorithms that are based on adaptive partitions and take advantage of benign payoffs and context arrivals without sacrificing the worst case performance the central idea is to maintain a finer partition in high payoff regions of the similarity space and in popular regions of the context space our results apply to several other settings e g mab with constrained temporal change slivkins and upfal 2008 and sleeping bandits kleinberg et al 2008a

universal communication over arbitrarily varying channels consider the problem of universally communicating over an arbitrarily varying channel i e a channel comprised of an unknown arbitrary sequence of memoryless channels it is shown that there is a communication system using feedback and common randomness that asymptotically attains with high probability the capacity of the time averaged channel universally for every sequence of channels this attainable rate is optimal under certain conditions while no prior knowledge of the channel sequence is assumed the capacity of the time averaged channel meets or exceeds the traditional arbitrarily varying channel avc capacity for every memoryless avc defined over the same alphabets and therefore the system universally attains the random code avc capacity without knowledge of the avc parameters the presented system combines rateless coding with a universal prediction scheme for the input prior distribution from which the codebook is randomly drawn because at each point in time the future of the channel sequence is unknown to the communicators the adaptation of the input behavior by universally predicting the prior plays a major role in the result

regret minimization for reserve prices in second price auctions we show a regret minimization algorithm for setting the reserve price in a sequence of second price auctions under the assumption that all bids are independently drawn from the same unknown and arbitrary distribution our algorithm is computationally efficient and achieves a regret of o t in a sequence of t auctions this holds even when the number of bidders is stochastic with a known distribution

regret and convergence bounds for a class of continuum armed bandit problems we consider a class of multi armed bandit problems where the set of available actions can be mapped to a convex compact region of ropfd sometimes denoted as the ldquocontinuum armed banditrdquo problem the paper establishes bounds on the efficiency of any arm selection procedure under certain conditions on the class of possible underlying reward functions both finite time lower bounds on the growth rate of the regret as well as asymptotic upper bounds on the rates of convergence of the selected control values to the optimum are derived we explicitly characterize the dependence of these convergence rates on the minimal rate of variation of the mean reward function in a neighborhood of the optimal control the bounds can be used to demonstrate the asymptotic optimality of the kiefer wolfowitz method of stochastic approximation with regard to a large class of possible mean reward functions

a model reference adaptive search method for global optimization model reference adaptive search mras for solving global optimization problems works with a parameterized probabilistic model on the solution space and generates at each iteration a group of candidate solutions these candidate solutions are then used to update the parameters associated with the probabilistic model in such a way that the future search will be biased toward the region containing high quality solutions the parameter updating procedure in mras is guided by a sequence of implicit probabilistic models we call reference models we provide a particular algorithm instantiation of the mras method where the sequence of reference models can be viewed as the generalized probability distribution models for estimation of distribution algorithms edas with proportional selection scheme in addition we show that the model reference framework can also be used to describe the recently proposed cross entropy ce method for optimization and to study its properties hence this paper can also be seen as a study on the effectiveness of combining ce and edas we prove global convergence of the proposed algorithm in both continuous and combinatorial domains and we carry out numerical studies to illustrate the performance of the algorithm

x armed bandits we consider a generalization of stochastic bandits where the set of arms x is allowed to be a generic measurable space and the mean payoff function is locally lipschitz with respect to a dissimilarity function that is known to the decision maker under this condition we construct an arm selection policy called hoo hierarchical optimistic optimization with improved regret bounds compared to previous results for a large class of problems in particular our results imply that if x is the unit hypercube in a euclidean space and the mean payoff function has a finite number of global maxima around which the behavior of the function is locally continuous with a known smoothness degree then the expected regret of hoo is bounded up to a logarithmic factor by n that is the rate of growth of the regret is independent of the dimension of the space we also prove the minimax optimality of our algorithm when the dissimilarity is a metric our basic strategy has quadratic computational complexity as a function of the number of time steps and does not rely on the doubling trick we also introduce a modified strategy which relies on the doubling trick but runs in linearithmic time both results are improvements with respect to previous approaches

adaptive adversarial multi armed bandit approach to two person zero sum markov games this technical note presents a recursive sampling based algorithm for finite horizon two person zero sum markov games mgs based on the exp3 algorithm developed by auer et al for adaptive adversarial multi armed bandit problems we provide a finite iteration bound to the equilibrium value of the induced sample average approximation game of a given mg and prove asymptotic convergence to the equilibrium value of the given mg the time and space complexities of the algorithm are independent of the state space of the game

a geometric approach to multi criterion reinforcement learning we consider the problem of reinforcement learning in a controlled markov environment with multiple objective functions of the long term average reward type the environment is initially unknown and furthermore may be affected by the actions of other agents actions that are observed but cannot be predicted beforehand we capture this situation using a stochastic game model where the learning agent is facing an adversary whose policy is arbitrary and unknown and where the reward function is vector valued state recurrence conditions are imposed throughout in our basic problem formulation a desired target set is specified in the vector reward space and the objective of the learning agent is to approach the target set in the sense that the long term average reward vector will belong to this set we devise appropriate learning algorithms that essentially use multiple reinforcement learning algorithms for the standard scalar reward problem which are combined using the geometric insight from the theory of approachability for vector valued stochastic games we then address the more general and optimization related problem where a nested class of possible target sets is prescribed and the goal of the learning agent is to approach the smallest possible target set which will generally depend on the unknown system parameters a particular case which falls into this framework is that of stochastic games with average reward constraints and further specialization provides a reinforcement learning algorithm for constrained markov decision processes some basic examples are provided to illustrate these results

pure exploration in finitely armed and continuous armed bandits we consider the framework of stochastic multi armed bandit problems and study the possibilities and limitations of forecasters that perform an on line exploration of the arms these forecasters are assessed in terms of their simple regret a regret notion that captures the fact that exploration is only constrained by the number of available rounds not necessarily known in advance in contrast to the case when the cumulative regret is considered and when exploitation needs to be performed at the same time we believe that this performance criterion is suited to situations when the cost of pulling an arm is expressed in terms of resources rather than rewards we discuss the links between the simple and the cumulative regret one of the main results in the case of a finite number of arms is a general lower bound on the simple regret of a forecaster in terms of its cumulative regret the smaller the latter the larger the former keeping this result in mind we then exhibit upper bounds on the simple regret of some forecasters the paper ends with a study devoted to continuous armed bandit problems we show that the simple regret can be minimized with respect to a family of probability distributions if and only if the cumulative regret can be minimized for it based on this equivalence we are able to prove that the separable metric spaces are exactly the metric spaces on which these regrets can be minimized with respect to the family of all probability distributions with continuous mean payoff functions

strategies for prediction under imperfect monitoring we propose simple randomized strategies for sequential decision or prediction under imperfect monitoring that is when the decision maker forecaster does not have access to the past outcomes but rather to a feedback signal the proposed strategies are consistent in the sense that they achieve asymptotically the best possible average reward among all fixed actions it was rustichini rustichini a 1999 minimizing regret the general case games econom behav 29 224 243 who first proved the existence of such consistent predictors the forecasters presented here offer the first constructive proof of consistency moreover the proposed algorithms are computationally efficient we also establish upper bounds for the rates of convergence in the case of deterministic feedback signals these rates are optimal up to logarithmic terms

regret bounds and minimax policies under partial monitoring this work deals with four classical prediction settings namely full information bandit label efficient and bandit label efficient as well as four different notions of regret pseudo regret expected regret high probability regret and tracking the best expert regret we introduce a new forecaster inf implicitly normalized forecaster based on an arbitrary function for which we propose a unified analysis of its pseudo regret in the four games we consider in particular for x exp x k inf reduces to the classical exponentially weighted average forecaster and our analysis of the pseudo regret recovers known results while for the expected regret we slightly tighten the bounds on the other hand with x x q k which defines a new forecaster we are able to remove the extraneous logarithmic factor in the pseudo regret bounds for bandits games and thus fill in a long open gap in the characterization of the minimax rate for the pseudo regret in the bandit game we also provide high probability bounds depending on the cumulative reward of the optimal action finally we consider the stochastic bandit game and prove that an appropriate modification of the upper confidence bound policy ucb1 auer et al 2002a achieves the distribution free optimal rate while still having a distribution dependent rate logarithmic in the number of plays

image coding using vector quantization a review a review of vector quantization techniques used for encoding digital images is presented first the concept of vector quantization is introduced then its application to digital images is explained spatial predictive transform hybrid binary and subband vector quantizers are reviewed the emphasis is on the usefulness of the vector quantization when it is combined with conventional image coding techniques or when it is used in different domains

how to use expert advice we analyze algorithms that predict a binary value by combining the predictions of several prediction strategies called experts our analysis is for worst case situations i e we make no assumptions about the way the sequence of bits to be predicted is generated we measure the performance of the algorithm by the difference between the expected number of mistakes it makes on the bit sequence and the expected number of mistakes made by the best expert on this sequence where the expectation is taken with respect to the randomization in the predictins we show that the minimum achievable difference is on the order of the square root of the number of mistakes of the best expert and we give efficient algorithms that achieve this our upper and lower bounds have matching leading constants in most cases we then show how this leads to certain kinds of pattern recognition learning algorithms with performance bounds that improve on the best results currently know in this context we also compare our analysis to the case in which log loss is used instead of the expected number of mistakes

ranked bandits in metric spaces learning diverse rankings over large document collections most learning to rank research has assumed that the utility of different documents is independent which results in learned ranking functions that return redundant results the few approaches that avoid this have rather unsatisfyingly lacked theoretical foundations or do not scale we present a learning to rank formulation that optimizes the fraction of satisfied users with several scalable algorithms that explicitly takes document similarity and ranking context into account our formulation is a non trivial common generalization of two multi armed bandit models from the literature ranked bandits radlinski et al 2008 and lipschitz bandits kleinberg et al 2008b we present theoretical justifications for this approach as well as a near optimal algorithm our evaluation adds optimizations that improve empirical performance and shows that our algorithms learn orders of magnitude more quickly than previous approaches

regret in online combinatorial optimization we address online linear optimization problems when the possible actions of the decision maker are represented by binary vectors the regret of the decision maker is the difference between her realized loss and the minimal loss she would have achieved by picking in hindsight the best possible action our goal is to understand the magnitude of the best possible minimax regret we study the problem under three different assumptions for the feedback the decision maker receives full information and the partial information models of the so called semi bandit and bandit problems in the full information case we show that the standard exponentially weighted average forecaster is a provably suboptimal strategy for the semi bandit model by combining the mirror descent algorithm and the inf implicitely normalized forecaster strategy we are able to prove the first optimal bounds finally in the bandit case we discuss existing results in light of a new lower bound and suggest a conjecture on the optimal regret in that case

characterization of strange attractors a new measure of strange attractors is introduced which offers a practical algorithm to determine their character from the time series of a single observable the relation of this new measure to fractal dimension and information theoretic entropy is discussed