optimal tracking control of nonlinear dynamical systems this paper presents a simple methodology for obtaining the entire set of continuous controllers that cause a nonlinear dynamical system to exactly track a given trajectory the trajectory is provid

the liapunov dimension of strange attractors abstract many papers have been published recently on studies of dynamical processes in which the attracting sets appear quite strange in this paper the question of estimating the dimension of the attractor is addressed while more general conjectures are made here particular attention is paid to the idea that if the jacobian determinant of a map is greater than one and a ball is mapped into itself then generically the attractor will have positive two dimensional measure and most of this paper is devoted to presenting cases with such jacobians for which the attractors are proved to have non empty interior

minimum cost spanning tree games we consider the problem of cost allocation among users of a minimum cost spanning tree network it is formulated as a cooperative game in characteristic function form referred to as a minimum cost spanning tree m c s t game we show that the core of a m c s t game is never empty in fact a point in the core can be read directly from any minimum cost spanning tree graph associated with the problem for m c s t games with efficient coalition structures we define and construct m c s t games on the components of the structure we show that the core and the nucleolus of the original game are the cartesian products of the cores and the nucleoli respectively of the induced games on the components of the efficient coalition structure

the continuum armed bandit problem in this paper we consider the multiarmed bandit problem where the arms are chosen from a subset of the real line and the mean rewards are assumed to be a continuous function of the arms the problem with an infinite number of arms is much more difficult than the usual one with a finite number of arms because the built in learning task is now infinite dimensional we devise a kernel estimator based learning scheme for the mean reward as a function of the arms using this learning scheme we construct a class of certainty equivalence control with forcing schemes and derive asymptotic upper bounds on their learning loss to the best of our knowledge these bounds are the strongest rates yet available moreover they are stronger than the o n required for optimality with respect to the average cost per unit time criterion

robust synchronization of chaotic horizontal platform systems with phase difference this paper studies a robust synchronization of non autonomous chaotic systems with parameter mismatch in the synchronization scheme a linear state error feedback control is used to couple the master and slave horizontal platform systems excited by harmonic external forces between which there exists a phase mismatch a new definition of global synchronization with error bound is introduced using lyapunov s stability theory the sufficient synchronization criteria for the scheme are proven and the corresponding synchronization error bound is estimated the synchronization criteria are further optimized by optimally designing a quadratic lyapunov function to more precisely estimate the synchronization error bound the illustrative simulations verify the effectiveness of these criteria the estimated synchronization error bound is compared with numerical one in the examples

chaos and chaos synchronization of a symmetric gyro with linear plus cubic damping the dynamic behavior of a symmetric gyro with linear plus cubic damping which is subjected to a harmonic excitation is studied in this paper the liapunov direct method has been used to obtain the sufficient conditions of the stability of the equilibrium points of the system by applying numerical results time history phase diagrams poincare maps liapunov exponents and liapunov dimensions are presented to observe periodic and chaotic motions besides several control methods the delayed feedback control the addition of constant motor torque the addition of period force and adaptive control algorithm aca have been used to control chaos effectively finally attention is shifted to the synchronization of chaos in the two identical chaotic motions of symmetric gyros the results show that one can make two identical chaotic systems to synchronize through applying four different kinds of one way coupling furthermore the synchronization time is also examined

chaos synchronization of nonlinear gyros in presence of stochastic excitation via sliding mode control abstract in this paper the problem of synchronization between two chaotic gyros with stochastic base excitation based on the mater slave scheme is studied the stochastic excitation is modeled by applying a gaussian white noise to the deterministic model of gyro the white noise is derived from the wiener process so the resulted system is a stochastic differential equation and is modeled by an ito differential form using a modified sliding mode control a markov synchronizing control law is designed and the convergence of error states to the sliding surface in the mean square norm is analytically proved simulation results show the high performance of the proposed controller in stochastic chaos synchronization of gyro systems

chaos synchronization and parameter estimation of single degree of freedom oscillators via adaptive control this paper addresses the problem of synchronizing a class of single degree of freedom oscillators with uncertain parameters a modified adaptive control scheme is proposed to achieve globally asymptotic stable synchronization between the master and slave oscillators with arbitrary different initial conditions based on the barbalat s lemma one of the advantages of the method is that it requires only a scalar driving signal which may be easily designed in practical applications on the other hand the method can ensure that the unknown parameters in the slave oscillator would be completely estimated with an adaptive updating law numerical simulations are performed on two examples to verify the analytical results

a strongly polynomial algorithm to solve combinatorial linear programs khachiyan and recently karmarkar gave polynomial algorithms to solve the linear programming problem these algorithms have a small theoretical drawback namely the number of arithmetic steps depends on the size of the input numbers we present a polynomial linear programming algorithm whose number of arithmetic steps depends only on the size of the numbers in the constraint matrix but is independent of the size of the numbers in the right hand side and objective vectors in particular it gives a polynomial algorithm for the minimum cost flow and multicommodity flow problems in which the number of arithmetic steps is independent of the size of the costs and capacities the algorithm makes use of an existing polynomial linear programming algorithm the problem of whether any algorithm has a running time that is independent even of the size of the numbers in the constraint matrix remains open

master slave chaos synchronization criteria for the horizontal platform systems via linear state error feedback control global chaos synchronization of two identical non autonomous horizontal platform systems coupled by linear state error feedback controller is investigated the sufficient criteria for global chaos synchronization are deduced based on the stability theory of linear time varied systems and lyapunov s direct method of which the criteria related to general coupling matrix are first proved and applied to derive the ones related to some special coupling matrices in the examples the coupling strengths are designed by the obtained criteria and the appearances of chaos synchronization are verified it is analytically and numerically examined that the synchronization criteria based on lyapunov s direct method are sharper than the criteria based on the stability theory of linear time varied systems

optional polya tree and bayesian inference we introduce an extension of the polya tree approach for constructing distributions on the space of probability measures by using optional stopping and optional choice of splitting variables the construction gives rise to random measures that are absolutely continuous with piecewise smooth densities on partitions that can adapt to fit the data the resulting optional polya tree distribution has large support in total variation topology and yields posterior distributions that are also optional polya trees with computable parameter values

cover tree bayesian reinforcement learning this paper proposes an online tree based bayesian approach for reinforcement learning for inference we employ a generalised context tree model this defines a distribution on multivariate gaussian piecewise linear models which can be updated in closed form the tree structure itself is constructed using the cover tree method which remains efficient in high dimensional spaces we combine the model with thompson sampling and approximate dynamic programming to obtain effective exploration policies in unknown environments the flexibility and computational simplicity of the model render it suitable for many reinforcement learning problems in continuous state spaces we demonstrate this in an experimental comparison with a gaussian process model a linear model and simple least squares policy iteration

anticontrol of chaos of the fractional order modified van der pol systems anticontrol of chaos of fractional order modified van der pol systems is studied addition of a constant term and addition of k x sinx term where x is a state of the system are used to anticontrol the system effectively by applying numerical results phase portrait poincare maps and bifurcation diagrams a variety of the phenomena of the chaotic motion can be presented finally it can be find that chaos under these procedures exists in the fractional order systems of a modified van der pol system

new techniques for cost sharing in combinatorial optimization games combinatorial optimization games form an important subclass of cooperative games in recent years increased attention has been given to the issue of finding good cost shares for such games in this paper we define a very general class of games called integer minimization games which includes the combinatorial optimization games in the literature as special cases we then present new techniques based on row and column generation for computing good cost shares for these games to illustrate the power of these techniques we apply them to traveling salesman and vehicle routing games our results generalize and unify several results in the literature the main underlying idea is that suitable valid inequalities for the associated combinatorial optimization problems can be used to derive improved cost shares

the nonstochastic multiarmed bandit problem in the multiarmed bandit problem a gambler must decide which arm of k nonidentical slot machines to play in a sequence of trials so as to maximize his reward this classical problem has received much attention because of the simple model it provides of the trade off between exploration trying out each arm to find the best one and exploitation playing the arm believed to give the best payoff past solutions for the bandit problem have almost always relied on assumptions about the statistics of the slot machines in this work we make no statistical assumptions whatsoever about the nature of the process generating the payoffs of the slot machines we give a solution to the bandit problem in which an adversary rather than a well behaved stochastic process has complete control over the payoffs in a sequence of t plays we prove that the per round payoff of our algorithm approaches that of the best arm at the rate o t 1 2 we show by a matching lower bound that this is the best possible we also prove that our algorithm approaches the per round payoff of any set of strategies at a similar rate if the best strategy is chosen from a pool of n strategies then our algorithm approaches the per round payoff of the strategy at the rate o log n1 2 t 1 2 finally we apply our results to the problem of playing an unknown repeated matrix game we show that our algorithm approaches the minimax payoff of the unknown game at the rate o t 1 2

universal piecewise linear prediction via context trees this paper considers the problem of piecewise linear prediction from a competitive algorithm approach in prior work prediction algorithms have been developed that are universal with respect to the class of all linear predictors such that they perform nearly as well in terms of total squared prediction error as the best linear predictor that is able to observe the entire sequence in advance in this paper we introduce the use of a context tree to compete against a doubly exponential number of piecewise linear affine models we use the context tree to achieve the total squared prediction error performance of the best piecewise linear model that can choose both its partitioning of the regressor space and its real valued prediction parameters within each region of the partition based on observing the entire sequence in advance uniformly for every bounded individual sequence this performance is achieved with a prediction algorithm whose complexity is only linear in the depth of the context tree per prediction upper bounds on the regret with respect to the best piecewise linear predictor are given for both the scalar and higher order case and lower bounds on the regret are given for the scalar case an explicit algorithmic description and examples demonstrating the performance of the algorithm are given

universal reinforcement learning we consider an agent interacting with an unmodeled environment at each time the agent makes an observation takes an action and incurs a cost its actions can influence future observations and costs the goal is to minimize the long term average cost we propose a novel algorithm known as the active lz algorithm for optimal control based on ideas from the lempel ziv scheme for universal data compression and prediction we establish that under the active lz algorithm if there exists an integer k such that the future is conditionally independent of the past given a window of k consecutive actions and observations then the average cost converges to the optimum experimental results involving the game of rock paper scissors illustrate merits of the algorithm

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

on the core of network synthesis games we use polynomial formulations to show that several rational and discrete network synthesis games including the minimum cost spanning tree game satisfy the assumptions of owen s linear production game model we also discuss computational issues related to finding and recognizing core points for these classes of games

the regular and chaotic motions of a symmetric heavy gyroscope with harmonic excitation abstract the non linear motion of a symmetric gyro mounted on a vibrating base is investigated with particular emphasis on its long term dynamic behavior for a wide range of parameters a single modal equation is used to analyze the qualitative behaviour of the system external disturbance appears as vertical harmonic motion of the support point and linear damping is assumed the complete equation of motion is a non linear non autonomous one the stability of the system has been studied by damped mathieu equation theory and the liapunov direct method as the system is subjected to external disturbance the melnikov method is used to show the existence of chaotic motion finally the bifurcation of the parameter dependent system is studied numerically the time evolutions of the non linear dynamical system responses are described in phase portraits via the poincare map technique the occurrence and the nature of chaotic attractors are verified by evaluating liapunov exponents and average power spectra the effect of the gyroscope s spinning speed is also studied and it is shown that the gyroscope s spin velocity zhas a significant effect on the dynamic behaviour of the motion

regret and convergence bounds for a class of continuum armed bandit problems we consider a class of multi armed bandit problems where the set of available actions can be mapped to a convex compact region of ropfd sometimes denoted as the ldquocontinuum armed banditrdquo problem the paper establishes bounds on the efficiency of any arm selection procedure under certain conditions on the class of possible underlying reward functions both finite time lower bounds on the growth rate of the regret as well as asymptotic upper bounds on the rates of convergence of the selected control values to the optimum are derived we explicitly characterize the dependence of these convergence rates on the minimal rate of variation of the mean reward function in a neighborhood of the optimal control the bounds can be used to demonstrate the asymptotic optimality of the kiefer wolfowitz method of stochastic approximation with regard to a large class of possible mean reward functions

x armed bandits we consider a generalization of stochastic bandits where the set of arms x is allowed to be a generic measurable space and the mean payoff function is locally lipschitz with respect to a dissimilarity function that is known to the decision maker under this condition we construct an arm selection policy called hoo hierarchical optimistic optimization with improved regret bounds compared to previous results for a large class of problems in particular our results imply that if x is the unit hypercube in a euclidean space and the mean payoff function has a finite number of global maxima around which the behavior of the function is locally continuous with a known smoothness degree then the expected regret of hoo is bounded up to a logarithmic factor by n that is the rate of growth of the regret is independent of the dimension of the space we also prove the minimax optimality of our algorithm when the dissimilarity is a metric our basic strategy has quadratic computational complexity as a function of the number of time steps and does not rely on the doubling trick we also introduce a modified strategy which relies on the doubling trick but runs in linearithmic time both results are improvements with respect to previous approaches

a cooperative location game based on the 1 center location problem abstract in this paper we introduce and analyze new classes of cooperative games related to facility location models defined on general metric spaces the players are the customers demand points in the location problem and the characteristic value of a coalition is the cost of serving its members specifically the cost in our games is the service radius of the coalition we call these games the minimum radius location games mrlg we study the existence of core allocations and the existence of polynomial representations of the cores of these games focusing on network spaces i e finite metric spaces induced by undirected graphs and positive edge lengths and on the lp metric spaces defined over r d

pure exploration in finitely armed and continuous armed bandits we consider the framework of stochastic multi armed bandit problems and study the possibilities and limitations of forecasters that perform an on line exploration of the arms these forecasters are assessed in terms of their simple regret a regret notion that captures the fact that exploration is only constrained by the number of available rounds not necessarily known in advance in contrast to the case when the cumulative regret is considered and when exploitation needs to be performed at the same time we believe that this performance criterion is suited to situations when the cost of pulling an arm is expressed in terms of resources rather than rewards we discuss the links between the simple and the cumulative regret one of the main results in the case of a finite number of arms is a general lower bound on the simple regret of a forecaster in terms of its cumulative regret the smaller the latter the larger the former keeping this result in mind we then exhibit upper bounds on the simple regret of some forecasters the paper ends with a study devoted to continuous armed bandit problems we show that the simple regret can be minimized with respect to a family of probability distributions if and only if the cumulative regret can be minimized for it based on this equivalence we are able to prove that the separable metric spaces are exactly the metric spaces on which these regrets can be minimized with respect to the family of all probability distributions with continuous mean payoff functions

on the core and nucleolus of minimum cost spanning tree games we develop two efficient procedures for generating cost allocation vectors in the core of a minimum cost spanning tree m c s t game the first procedure requires o n 2 elementary operations to obtain each additional point in the core wheren is the number of users the efficiency of the second procedure which is a natural strengthening of the first procedure stems from the special structure of minimum excess coalitions in the core of an m c s t game this special structure is later used i to ease the computational difficulty in computing the nucleolus of an m c s t game and ii to provide a geometric characterization for the nucleolus of an m c s t game this geometric characterization implies that in an m c s t game the nucleolus is the unique point in the intersection of the core and the kernel we further develop an efficient procedure for generating fair cost allocations which in some instances coincide with the nucleolus finally we show that by employing sterns transfer scheme we can generate a sequence of cost vectors which converges to the nucleolus

generalized synchronization of chaotic systems by pure error dynamics and elaborate lyapunov function abstract the generalized synchronization is studied by applying pure error dynamics and elaborate lyapunov function in this paper generalized synchronization can be obtained by pure error dynamics without auxiliary numerical simulation instead of current mixed error dynamics in which master state variables and slave state variables are presented the elaborate lyapunov function is applied rather than the current plain square sum lyapunov function deeply weakening the power of lyapunov direct method the scheme is successfully applied to both autonomous and nonautonomous double mathieu systems with numerical simulations

integral control for synchronization of complex dynamical networks with unknown non identical nodes this paper proposes a high gain integral controller for synchronization of complex dynamical networks with unknown non identical nodes the integral controller converts the complex dynamical system into a singular perturbation form then a sufficient condition for global synchronization is derived finally the numerical simulation is presented to illustrate the effectiveness of the proposed method

least squares policy iteration we propose a new approach to reinforcement learning for control problems which combines value function approximation with linear architectures and approximate policy iteration this new approach is motivated by the least squares temporal difference learning algorithm lstd for prediction problems which is known for its efficient use of sample experiences compared to pure temporal difference algorithms heretofore lstd has not had a straightforward application to control problems mainly because lstd learns the state value function of a fixed policy which cannot be used for action selection and control without a model of the underlying process our new algorithm least squares policy iteration lspi learns the state action value function which allows for action selection without a model and for incremental policy improvement within a policy iteration framework lspi is a model free off policy method which can use efficiently and reuse in each iteration sample experiences collected in any manner by separating the sample collection method the choice of the linear approximation architecture and the solution method lspi allows for focused attention on the distinct elements that contribute to practical reinforcement learning lspi is tested on the simple task of balancing an inverted pendulum and the harder task of balancing and riding a bicycle to a target location in both cases lspi learns to control the pendulum or the bicycle by merely observing a relatively small number of trials where actions are selected randomly lspi is also compared against q learning both with and without experience replay using the same value function architecture while lspi achieves good performance fairly consistently on the difficult bicycle task q learning variants were rarely able to balance for more than a small fraction of the time needed to reach the target location

chaos synchronization of double duffing systems with parameters excited by a chaotic signal chaos synchronization by driving parameter for two uncoupled identical chaotic double duffing systems is presented replacing two corresponding parameters of the identical systems by the same function of chaotic state variables of a third chaotic system the synchronization or anti synchronization of two uncoupled systems can be obtained numerical simulations are illustrated for either synchronization or anti synchronization of which the occurrence depends significantly on initial conditions and on driving strength alternative complete synchronization and anti synchronization is also discovered