a generalization of the borkar meyn theorem for stochastic recursive inclusions in this paper the stability theorem of borkar and meyn is extended to include the case when the mean field is a set valued map two different sets of sufficient conditions are presented that guarantee the stability and convergence of stochastic recursive inclusions our work builds on the works of benaim hofbauer and sorin as well as borkar and meyn as a corollary to one of the main theorems a natural generalization of the borkar and meyn theorem follows in addition the original theorem of borkar and meyn is shown to hold under slightly relaxed assumptions as an application to one of the main theorems we discuss a solution to the approximate drift problem finally we analyze the stochastic gradient algorithm with constant error gradient estimators as yet another application of our main result

on the convergence of piecewise linear strategic interaction dynamics on networks we prove that the piecewise linear best response dynamical systems of strategic interactions are asymptotically convergent to their set of equilibria on any weighted undirected graph we study various features of these dynamical systems including the uniqueness and abundance properties of the set of equilibria and the emergence of unstable equilibria we also introduce the novel notions of social equivalence and social dominance on directed graphs and demonstrate some of their interesting implications including their correspondence to consensus and chromatic number of partite graphs examples illustrate our results

reinforcement learning with restrictions on the action set consider a two player normal form game repeated over time we introduce an adaptive learning procedure where the players only observe their own realized payoff at each stage we assume that agents do not know their own payoff function and have no information on the other player furthermore we assume that they have restrictions on their own actions such that at each stage their choice is limited to a subset of their action set we prove that the empirical distributions of play converge to the set of nash equilibria for zero sum and potential games and games where one player has two actions

on lipschitz optimization based on gray box piecewise linearization we address the problem of minimizing objectives from the class of piecewise differentiable functions whose nonsmoothness can be encapsulated in the absolute value function they possess local piecewise linear approximations with a discrepancy that can be bounded by a quadratic proximal term this overestimating local model is continuous but generally nonconvex it can be generated in its abs normal form by a minor extension of standard algorithmic differentiation tools here we demonstrate how the local model can be minimized by a bundle type method which benefits from the availability of additional gray box information via the abs normal form in the convex case our algorithm realizes the consistent steepest descent trajectory for which finite convergence was established earlier specifically covering counterexamples where steepest descent with exact line search famously fails the analysis of the abs normal representation and the design of the optimization algorithm are geared toward the general case whereas the convergence proof so far covers only the convex case

a simple adaptive procedure leading to correlated equilibrium we propose a new and simple adaptive procedure for playing a game regret matching in this procedure players may depart from their current play with probabilities that are proportional to measures of regret for not having used other strategies in the past it is shown that our adaptive procedure guarantees that with probability one the empirical distributions of play converge to the set of correlated equilibria of the game

on the convergence rate for stochastic approximation in the nonsmooth setting we consider a stochastic approximation sa method for finding the minimizer of a function f which is convex but nondifferentiable at the minimizer due to the nondifferentiability at the minimizer f is allowed to increase at a positive rate in a neighborhood of the minimizer from this property we show that the nth estimate for the minimizer generated by the sa procedure converges at a rate of 1 n in the mean which is significantly faster than the classical convergence rates for differentiable functions f we also discuss an example from an inventory control system that exhibits a convex but nondifferentiable cost function

numerical solution of the higher order linear fredholm integro differential difference equation with variable coefficients the main aim of this paper is to apply the legendre polynomials for the solution of the linear fredholm integro differential difference equation of high order this equation is usually difficult to solve analytically our approach consists of reducing the problem to a set of linear equations by expanding the approximate solution in terms of shifted legendre polynomials with unknown coefficients the operational matrices of delay and derivative together with the tau method are then utilized to evaluate the unknown coefficients of shifted legendre polynomials illustrative examples are included to demonstrate the validity and applicability of the presented technique and a comparison is made with existing results

lexicographic differentiation of nonsmooth functions we present a survey on the results related to the theory of lexicographic differentiation this theory ensures an efficient computation of generalized lexicographic derivative of a nonsmooth function belonging to a special class of lexicographically smooth functions this class is a linear space which contains all differentiable functions all convex functions and which is closed with respect to component wise composition of the members in order to define lexicographic derivative in a unique way it is enough to fix a basis in the space of variables lexicographic derivatives can be used in black box optimization methods we give some examples of applications of these derivatives in analysis of nonsmooth functions it is shown that the system of lexicographic derivatives along a fixed basis correctly represents corresponding nonsmooth function newton leibnitz formula we present nonsmooth versions of standard theorems on potentiality of nonlinear operators on differentiation of parametric integrals and on differentiation of functional sequences finally we show that an appropriately defined lexicographic subdifferential ensures a more rigorous selection of a candidate optimal solution than the subdifferential of clarke

relaxing kink qualifications and proving convergence rates in piecewise smooth optimization in the paper optim methods softw 31 2016 pp 904 930 we derived first order kkt and second order second order sufficiency condition sosc optimality conditions for functions defined b

consistency of vanishingly smooth fictitious play we discuss consistency of vanishingly smooth fictitious play a strategy in the context of game theory which can be regarded as a smooth fictitious play procedure where the smoothing parameter is time dependent and asymptotically vanishes this answers a question initially raised by drew fudenberg and satoru takahashi

on the choice of weighting matrices in the minimum variance controller abstract the application of minimum variance control in general does not result in an asymptotically stable closed loop system in this paper we show that for injective controllable linear time invariant systems a suitable choice of weighting matrix yields a minimum variance controller which stabilizes the system moreover a weighting matrix is constructed which yields a minimum variance controller which drives any initial state to zero in finite time by means of a simulation study the influence of the choice of the weighting matrix on the controller and the closed loop behaviour is illustrated

the combined reproducing kernel method and taylor series for handling nonlinear volterra integro differential equations with derivative type kernel abstract the reproducing kernel method is applied to volterra nonlinear integro differential equations in this technique the nonlinear term is replaced by its taylor series the exact solution is represented in the form of series in the reproducing hilbert kernel space the approximation solution is expressed by n term summation of reproducing kernel functions some numerical examples are solved in two different spaces and parameters of n measurements of the experimental data is an indications of stability and convergence on the reproducing kernel

two time scale stochastic approximation with controlled markov noise and off policy temporal difference learning we present for the first time an asymptotic convergence analysis of two time scale stochastic approximation driven by controlled markov noise in particular the faster and slower recursions have nonadditive controlled markov noise components in addition to martingale difference noise we analyze the asymptotic behavior of our framework by relating it to limiting differential inclusions in both time scales that are defined in terms of the ergodic occupation measures associated with the controlled markov processes finally we present a solution to the off policy convergence problem for temporal difference learning with linear function approximation using our results

a taylor polynomial approach for solving differential difference equations the purpose of this study is to give a taylor polynomial approximation for the solution of mth order linear differential difference equations with variable coefficients under the mixed conditions about any point for this purpose taylor matrix method is introduced this method is based on first taking the truncated taylor expansions of the functions in the differential difference equations and then substituting their matrix forms into the equation hence the result matrix equation can be solved and the unknown taylor coefficients can be found approximately in addition examples that illustrate the pertinent features of the method are presented and the results of study are discussed also we have discussed the accuracy of the method we use the symbolic algebra program maple to prove our results

equilibrium routing under uncertainty we provide a brief introduction to the basic models used to describe traffic on congested networks both in urban transport and telecommunications we discuss traffic equilibrium models covering atomic and non atomic routing games with emphasis on situations where the travel times are subject to random fluctuations we use convex optimization to present the models in a unified framework that stresses the common underlying structures as a prototypical example of traffic equilibrium with elastic demands we discuss some models for routing and congestion control in telecommunications we also describe a class of stochastic dynamics that model the adaptive behavior of agents and which provides a plausible micro foundation for the equilibrium finally we present some recent ideas on how risk averse behavior might be incorporated in the equilibrium models

stochastic approximations with constant step size and differential inclusions we consider stochastic approximation processes with constant step size whose associated deterministic system is an upper semicontinuous differential inclusion we prove that over any finite time span the sample paths of the stochastic process are closely approximated by a solution of the differential inclusion with high probability we then analyze infinite horizon behavior showing that if the process is markov its stationary measures must become concentrated on the birkhoff center of the deterministic system our results extend those of benaim for settings in which the deterministic system is lipschitz continuous and build on the work of benaim hofbauer and sorin for the case of decreasing step sizes we apply our results to models of population dynamics in games obtaining new conclusions about the medium and long run behavior of myopic optimizing agents

a derivative for complex lipschitz maps with generalised cauchy riemann equations a r t i c l e i n f o a b s t r a c t we introduce the lipschitz derivative or the l derivative of a locally lipschitz complex map it is a scott continuous compact and convex set valued map that extends the classical derivative to the bigger class of locally lipschitz maps and allows an extension of the fundamental theorem of calculus and a new generalisation of cauchy riemann equations to these maps which form a continuous scott domain we show that a complex lipschitz map is analytic in an open set if and only if its l derivative is a singleton at all points in the open set the calculus of the l derivative for sum product and composition of maps is derived the notion of contour integration is extended to scott continuous non empty compact convex valued functions on the complex plane and by using the l derivative the fundamental theorem of contour integration is extended to these functions 2014 the author published by elsevier b v this is an open access article under the cc by license http creativecommons org licenses by 3 0

rotund norms clarke subdifferentials and extensions of lipschitz functions we show that a certain condition regarding the separation of points by lipschitz functions is useful in extending a given lipschitz function from a subspace of a separable banach space to the whole space so that the extension function has a maximal clarke subdifferential we then establish connections between this separation property and the rotundity properties of the norm on the banach space

the diagonalizability of quadratic functions and the arbitrariness of shadow prices abstract the possibility of tailoring the quadratic objective function to generate optimal policies which are acceptable to the policy maker is explored with two alternative algorithms one of these is for objective functions with diagonal hessians and uses updates of the desired values the second algorithm proceeds by updating nondiagonal hessians while keeping the desired values fixed the equivalence of both algorithms are established the arbitrariness of shadow prices are established for all quadratic equality constrained optimization problems starting initially with linear constraints it is shown that either one of the algorithms can be used to alter the shadow prices by arbitrarily specified amounts without altering the primal optimal solution in one step the algorithms are able to change the shadow prices while keeping the optimal policy fixed it is shown that this is also true for nonlinear problems

stochastic approximation with controlled markov noise abstract stochastic approximation algorithms with additional noise that can be modelled as a controlled markov process are analyzed and shown to track the solutions of a differential inclusion defined in terms of the ergodic occupation measures associated with the controlled markov process

integer programming duality price functions and sensitivity analysis recently a duality theory for integer programming has been developed here we examine some of the economic implications of this theory in particular the necessity of using price functions in place of prices and the possibility of carrying out sensitivity analysis of optimal solutions in addition we consider the form of price functions that are generated by known algorithms for integer programming

repeated games and qualitative differential games approachability and comparison of strategies we study the notion of approachability in a repeated game with vector payoffs from a new point of view using techniques recently developed for qualitative differential games namely we relate the sufficient condition for approachability b set to the notion of discriminating domain for a suitably chosen differential game the other goal of the present article is to obtain a new precise link between the strategies in the differential game and in the repeated game preserving approachability properties

a fast multiscale galerkin method for solving second order linear fredholm integro differential equation with dirichlet boundary conditions abstract in this paper a fast multiscale galerkin method is developed for solving second order linear fredholm integro differential equation with dirichlet boundary conditions the method is based on a matrix truncation strategy which leads to generating coefficient matrix rapidly we prove that the method is stable and has an optimal convergence order and nearly linear computational complexity up to a logarithmic factor numerical examples are presented to illustrate its computational efficiency approximation accuracy and theoretical results and to compare the computed results with those of the original multiscale galerkin method proposed recently by the same authors

a computational model for multi variable differential calculus we develop a domain theoretic computational model for multi variable differential calculus which for the first time gives rise to data types for piecewise differentiable or more generally lipschitz functions by constructing an effectively given continuous scott domain for real valued lipschitz functions on finite dimensional euclidean spaces the model for real valued lipschitz functions of n variables is built as a sub domain of the product of two domains by tupling together consistent information about locally lipschitz functions and their differential properties as given by their l derivative or equivalently clarke gradient which has values given by non empty convex and compact subsets of r n to obtain a computationally practical framework the derivative information is approximated by the best fit compact hyper rectangles in r n in this case we show that consistency of the function and derivative information can be decided by reducing it to a linear programming problem this provides an algorithm to check consistency on the rational basis elements of the domain implying that the domain can be equipped with an effective structure and giving a computable framework for multi variable differential calculus we also develop a domain theoretic interval valued notion of line integral and show that if a scott continuous function representing a non empty convex and compact valued vector field is integrable then its interval valued integral over any closed piecewise c 1 path contains zero in the case that the derivative information is given in terms of compact hyper rectangles we use techniques from the theory of minimal surfaces to deduce the converse result a hyper rectangular valued vector field is integrable if its interval valued line integral over any piecewise c 1 path contains zero this gives a domain theoretic extension of the fundamental theorem of path integration finally we construct the least and the greatest piecewise linear functions obtained from a pair of function and hyper rectangular derivative information when the pair is consistent this provides the least and greatest maps to witness consistency

polynomial solution of the most general linear fredholm volterra integrodifferential difference equations by means of taylor collocation method abstract in this paper a taylor collocation method has been presented for numerically solving the high order general linear fredholm volterra integrodifferential difference equations with variable coefficients under the mixed conditions in terms of taylor polynomials using the taylor collocation points the method transforms the equation and conditions into the matrix equations with the unknown taylor coefficients and hence by finding the taylor coefficients the polynomial approach is obtained also to illustrate the accuracy of the method numerical examples are presented and the results are discussed

a class of self interacting processes with applications to games and reinforced random walks this paper studies a class of non markovian and nonhomogeneous stochastic processes on a finite state space relying on a recent paper by benaim hofbauer and sorin siam j control optim 44 2005 pp 328 348 it is shown that under certain assumptions the asymptotic behavior of occupation measures can be described in terms of a certain set valued deterministic dynamical system this provides a unified approach to simulated annealing type processes and permits the study of new models of vertex reinforced random walks and new models of learning in games such as markovian fictitious play

null sets and essentially smooth lipschitz functions in this paper we extend the notion of a lebesgue null set to a notion which is valid in any completely metrizable abelian topological group we then use this definition to introduce and study the class of essentially smooth functions these are roughly speaking those lipschitz functions which are smooth in each direction almost everywhere

a dynamical system approach to stochastic approximations it is known that some problems of almost sure convergence for stochastic approximation processes can be analyzed via an ordinary differential equation ode obtained by suitable averaging the goal of this paper is to show that the asymptotic behavior of such a process can be related to the asymptotic behavior of the ode without any particular assumption concerning the dynamics of this ode the main results are as follows a the limit sets of trajectory solutions to the stochastic approximation recursion are under classical assumptions almost surely nonempty compact connected sets invariant under the flow of the ode and contained in its set of chain recurrence b if the gain parameter goes to zero at a suitable rate depending on the expansion rate of the ode any trajectory solution to the recursion is almost surely asymptotic to a forward trajectory solution to the ode

on nesterov s nonsmooth chebyshev rosenbrock functions abstract we discuss two nonsmooth functions on r n introduced by nesterov we show that the first variant is partly smooth in the sense of lewis and that its only stationary point is the global minimizer in contrast we show that the second variant has 2 n 1 clarke stationary points none of them local minimizers except the global minimizer but also that its only mordukhovich stationary point is the global minimizer nonsmooth optimization algorithms from multiple starting points generate iterates that approximate all 2 n 1 clarke stationary points not only the global minimizer but it remains an open question as to whether the nonminimizing clarke stationary points are actually points of attraction for optimization algorithms

stochastic approximations and differential inclusions the dynamical systems approach to stochastic approximation is generalized to the case where the mean differential equation is replaced by a differential inclusion the limit set theorem of benaim and hirsch is extended to this situation internally chain transitive sets and attractors are studied in detail for set valued dynamical systems applications to game theory are given in particular to blackwell s approachability theorem and the convergence of fictitious play

adaptation coordination and local interactions via distributed approachability abstract this paper investigates the relation between cooperation competition and local interactions in large distributed multi agent systems the main contribution is the game theoretic problem formulation and solution approach based on the new framework of distributed approachability and the study of the convergence properties of the resulting game model approachability theory is the theory of two player repeated games with vector payoffs and distributed approachability is here presented for the first time as an extension to the case where we have a team of agents cooperating against a team of adversaries under local information and interaction structure the game model turns into a nonlinear differential inclusion which after a proper design of the control and disturbance policies presents a consensus term and an exogenous adversarial input local interactions enter into the model through a graph topology and the corresponding graph laplacian matrix given the above model we turn the original questions on cooperation competition and local interactions into convergence properties of the differential inclusion in particular we prove convergence and exponential convergence conditions around zero under general markovian strategies we illustrate our results in the case of decentralized organizations with multiple decision makers

a smooth variational principle with applications to subdifferentiability and to differentiability of convex functions we show that typically lower semicontinuous functions on a banach space densely inherit lower subderivatives of the same degree of smoothness as the norm in particular every continuous convex function on a space with a g teaux weak hadamard frechet smooth renorm is densely g teaux weak hadamard frechet differentiable our technique relies on a more powerful analogue of ekeland s variational principle in which the function is perturbed by a quadratic like function this smooth variational principle has very broad applicability in problems of nonsmooth analysis

the nonstochastic multiarmed bandit problem in the multiarmed bandit problem a gambler must decide which arm of k nonidentical slot machines to play in a sequence of trials so as to maximize his reward this classical problem has received much attention because of the simple model it provides of the trade off between exploration trying out each arm to find the best one and exploitation playing the arm believed to give the best payoff past solutions for the bandit problem have almost always relied on assumptions about the statistics of the slot machines in this work we make no statistical assumptions whatsoever about the nature of the process generating the payoffs of the slot machines we give a solution to the bandit problem in which an adversary rather than a well behaved stochastic process has complete control over the payoffs in a sequence of t plays we prove that the per round payoff of our algorithm approaches that of the best arm at the rate o t 1 2 we show by a matching lower bound that this is the best possible we also prove that our algorithm approaches the per round payoff of any set of strategies at a similar rate if the best strategy is chosen from a pool of n strategies then our algorithm approaches the per round payoff of the strategy at the rate o log n1 2 t 1 2 finally we apply our results to the problem of playing an unknown repeated matrix game we show that our algorithm approaches the minimax payoff of the unknown game at the rate o t 1 2

legendre spectral collocation method for fredholm integro differential difference equation with variable coefficients and mixed conditions in this article the legendre spectral collocation method has been applied to solve fredholm integro differential difference equations with variable coefficients the proposed method is based on the gauss legendre points with the basis functions of lagrange polynomials usually this type of integral equations are very difficult to solve analytically as well as numerically the presented method applied to the integral equation reduces to solve the system of algebraic equations also the numerical results obtained by legendre spectral collocation method have been compared with the results obtained by existing methods illustrative examples have been discussed to demonstrate the validity and applicability of the presented method

the differentiability of real functions on normed linear space using generalized subgradients abstract the modification of the clarke generalized subdifferential due to michel and penot is a useful tool in determining differentiability properties for certain classes of real functions on a normed linear space the g teaux differentiability of any real function can be deduced from the g teaux differentiability of the norm if the function has a directional derivative which attains a constant related to its generalized directional derivative for any distance function on a space with uniformly g teaux differentiable norm the clarke and michel penot generalized subdifferentials at points off the set reduce to the same object and this generates a continuity characterization for g teaux differentiability however on a banach space with rotund dual the frechet differentiability of a distance function implies that it is a convex function a mean value theorem for the modified generalized subdifferential has implications for g teaux differentiability

a reliable algorithm for solving boundary value problems for higher order integro differential equations the aim of this paper is to present an efficient analytical and numerical procedure for solving boundary value problems for higher order integro differential equations the modified form of adomian decomposition method is found to be fast and accurate the analysis is accompanied by numerical examples the results demonstrate reliability and efficiency of the proposed algorithm

oscillation of the bounded solutions of impulsive differential difference equations of second order the present paper is devoted to a class of impulsive differential difference equations of second order the moments of impulse effect are fixed ones sufficient conditions are found for unboundedness of the solutions and oscillation of the bounded solutions

a new polynomial approach for solving difference and fredholm integro difference equations with mixed argument the purpose of this study is to give a taylor polynomial approximation for the solution of high order linear fredholm difference equations with variable coefficients and mixed argument under the mixed conditions about any point for this purpose a taylor matrix method is introduced this method is based on first taking the truncated taylor expansions of the functions in the difference equation and then substituting their matrix forms into the given equation hence the result matrix equation can be solved and the unknown taylor coefficients can be found approximately also examples that illustrate the pertinent features of the method are presented and the results are discussed

generalized subdifferentials a baire categorical approach we use baire categorical arguments to construct dramatically pathological locally lipschitz functions the origins of this approach can be traced back to banach and mazurkiewicz 1931 who independently used similar categorical arguments to show that almost every continuous real valued function defined on 0 1 is nowhere differentiable as with the results of banach and mazurkiewicz it appears that it is easier to show that almost every function possesses a certain property than to construct a single concrete example among the most striking results contained in this paper are almost every 1 lipschitz function defined on a banach space has a clarke subdifferential mapping that is identically equal to the dual ball if t 1 t 2 ldots t n is a family of maximal cyclically monotone operators defined on a banach space x then there exists a real valued locally lipschitz function g such that partial 0 g x mbox co t 1 x t 2 x ldots t n x for each x in x in a separable banach space each non empty weak compact convex subset in the dual space is identically equal to the approximate subdifferential mapping of some lipschitz function and for locally lipschitz functions defined on separable spaces the notions of strong and weak integrability coincide

self triggered best response dynamics for continuous games motivated by the development of novel and practically implementable coordination algorithms for multi agent systems here we investigate the adaptation of classical best response dynamics by means of self triggered communications first we show that if the best response dynamics is defined for a continuous action space potential game convergence towards the nash equilibria set is guaranteed under continuity on utilities and component wise pseudo concavity of the potential function then we modify the best response dynamics to account for more economic self triggered communication strategies while ensuring convergence to the equilibrium set the proposed algorithm is then analyzed in the framework of hybrid systems theory

two methods for solving integral equations the objective of this work is to introduce two efficient strategies that produce the accurate solutions to linear and nonlinear volterra and fredholm integral equations of the second kind for illustration several linear nonlinear and integro differential examples have been tested and the results showed improvements over existing techniques

nonsmooth optimization via quasi newton methods we investigate the behavior of quasi newton algorithms applied to minimize a nonsmooth function f not necessarily convex we introduce an inex act line search that generates a sequence of nested intervals containing a set of points of nonzero measure that satisfy the armijo and wolfe conditions if f is absolutely continuous along the line furthermore the line search is guaranteed to terminate if f is semi algebraic it seems quite difficult to establish a convergence theorem for quasi newton methods applied to such general classes of functions so we give a care ful analysis of a special but illuminating case the euclidean norm in one variable using the inexact line search and in two variables assuming that the line search is exact in practice we find that when f is locally lipschitz and semi algebraic with bounded sublevel sets the bfgs broyden fletcher goldfarb shanno method with the inexact line search almost always generates sequences whose cluster points are clarke stationary and with function values converging r linearly to a clarke station ary value we give references documenting the successful use of bfgs in a variety of nonsmooth applications particularly the design of low order controllers for linear dynamical systems we conclude with a challenging open question

a continuity property related to kuratowski s index of non compactness its relevance to the drop property and its implications for differentiability theory abstract we define and study the properties of upper semi continuity a new continuity property for set valued mappings from a topological space into subsets of a metric space expressed in terms of kuratowski s index of non compactness this upper semi continuity is related closely to the usual upper semi continuity but significantly upper semi continuous minimal weak cuscos from a baire space into subsets of the dual of any banach space are generically single valued kuratowski s index of non compactness has been used to study the drop property and upper semi continuity is dual to property studied there uniform upper semi continuity of the duality mapping is dual to nearly uniform rotundity properties importantly upper semi continuity has application in differentiability theory providing another characterisation for asplund spaces an examination of an associated concept denting point for sets yields still further advances concerning the differentiability of convex functions on a large class of banach spaces

differentiability of lipschitz functions on banach spaces abstract the main result of this note says that if the norm of a banach space e is differentiable frechet gateaux or in some intermediate sense away from the origin then every locally lipschitz function defined on an open subset g of e is differentiable in the same sense at every point of some subset of g dense in g as the main corollary we prove that every lipschitz function on an asplund space is frechet differentiable on a dense set moreover in all these results the mean value theorem for the points of differentiability which is equivalent to the possibility of computing the clarke subdifferential from the points of differentiability is shown to hold

a chain rule for essentially smooth lipschitz functions in this paper we introduce a new class of real valued locally lipschitz functions that are similar in nature and definition to valadier s saine functions which we call arcwise essentially smooth and we show that if g rm rightarrow r is arcwise essentially smooth on rm and each function f j r n rightarrow r 1 leq j leq m is strictly differentiable almost everywhere in rn then g circ f is strictly differentiable almost everywhere in rn where f equiv f 1 f 2 ldots f m we also show that all the semismooth and all the pseudoregular functions are arcwise essentially smooth thus we provide a large and robust lattice algebra of lipschitz functions whose generalized derivatives are well behaved

on the global convergence of stochastic fictitious play we establish global convergence results for stochastic fictitious play for four classes of games games with an interior ess zero sum games potential games and supermodular games we do so by appealing to techniques from stochastic approximation theory which relate the limit behavior of a stochastic process to the limit behavior of a differential equation defined by the expected motion of the process the key result in our analysis of supermodular games is that the relevant differential equation defines a strongly monotone dynamical system our analyses of the other cases combine lyapunov function arguments with a discrete choice theory result that the choice probabilities generated by any additive random utility model can be derived from a deterministic model based on payoff perturbations that depend nonlinearly on the vector of choice probabilities

stochastic approximations and differential inclusions part ii applications we apply the theoretical results on stochastic approximations and differential inclusions developed in benaim et al m benaim j hofbauer s sorin 2005 stochastic approximations and differential inclusions siam j control optim 44 328 348 to several adaptive processes used in game theory including classical and generalized approachability no regret potential procedures hart and mas colell s hart a mas colell 2003 regret based continuous time dynamics games econom behav 45 375 394 and smooth fictitious play d fudenberg d k levine 1995 consistency and cautious fictitious play j econom dynam control19 1065 1089

an algorithm for nonsmooth optimization by successive piecewise linearization we present an optimization method for lipschitz continuous piecewise smooth ps objective functions based on successive piecewise linearization since in many realistic cases nondifferentiabilities are caused by the occurrence of abs max and min we concentrate on these nonsmooth elemental functions the method s idea is to locate an optimum of a ps objective function by explicitly handling the kink structure at the level of piecewise linear models this piecewise linearization can be generated in its abs normal form by minor extension of standard algorithmic or automatic differentiation tools in this paper it is shown that the new method when started from within a compact level set generates a sequence of iterates whose cluster points are all clarke stationary numerical results including comparisons with other nonsmooth optimization methods then illustrate the capabilities of the proposed approach

stochastic and robust control of nonlinear economic systems abstract in this paper we use a robust policy concept that is an extension to nonlinear systems of the mean variance approach for linear models in the linear case the approach reduces to the minimization of a convex combination of the mean and variance of the stochastic problem we also discuss a method for computing expectations of nonlinear systems based on evaluating the bias from certainty equivalence using monte carlo simulations the method computes robust strategies if desired these can be in the form of simple decision rules as aggregate parametrisations of full feedback strategies the parameters are determined to minimize a convex combination of the original objective function and a sensitivity variance expression the latter can either be chosen as the sensitivity variance of the original objective function or the sensitivity variance of the individual endogenous variables

a multiscale galerkin method for second order boundary value problems of fredholm integro differential equation in this paper multiscale galerkin method is presented to approximate the solutions of second order boundary value problems of fredholm integro differential equation the method is based on traditional galerkin method and uses the multiscale orthonormal bases to discretize the equations the proposed method is proved to be stable and have the optimal convergence order numerical examples are presented to confirm the theoretical results and show that the method is computationally stable valid and accurate

numerical solution of the nonlinear fredholm integral equations by positive definite functions a numerical method for solving the nonlinear fredholom integral equations is presented the method is based on interpolation by radial basis functions rbf to approximate the solution of the fredholm nonlinear integral equations several examples are given and numerical examples are presented to demonstrate the validity and applicability of the method

variable metric bundle methods from conceptual to implementable forms to minimize a convex function we combine moreau yosida regularizations quasi newton matrices and bundling mechanisms first we develop conceptual forms using reversal quasi newton formulae and we state their global and local convergence then to produce implementable versions we incorporate a bundle strategy together with a curve search no convergence results are given for the implementable versions however some numerical illustrations show their good behaviour even for large scale problems

theoremes de convergence presque sure pour une classe d algorithmes stochastiques a pas decroissant the paper studies the pathwise asymptotic behaviour of stochastic algorithms of the following general form theta n 1 theta n gamma n 1 f theta n y n 1 the hypotheses allowing discontinuities on the adaptation termf the process y n n 0 is a markov chain controlled by n for each fixed the markov chain y n n 0 is essentially of positive recurrent type

analysis of gradient descent methods with nondiminishing bounded errors the main aim of this paper is to provide an analysis of gradient descent text gd algorithms with gradient errors that do not necessarily vanish asymptotically in particular sufficient conditions are presented for both stability almost sure boundedness of the iterates and convergence of text gd with bounded possibly nondiminishing gradient errors in addition to ensuring stability such an algorithm is shown to converge to a small neighborhood of the minimum set which depends on the gradient errors it is worth noting that the main result of this paper can be used to show that text gd with asymptotically vanishing errors indeed converges to the minimum set the results presented herein are not only more general when compared to previous results but our analysis of text gd with errors is new to the literature to the best of our knowledge our work extends the contributions of mangasarian and solodov bertsekas and tsitsiklis and tadic and doucet using our framework a simple yet effective implementation of text gd using simultaneous perturbation stochastic approximations with constant sensitivity parameters is presented another important improvement over many previous results is that there are no additional restrictions imposed on the step sizes in machine learning applications where step sizes are related to learning rates our assumptions unlike those of other papers do not affect these learning rates finally we present experimental results to validate our theory

bessel polynomial solutions of high order linear volterra integro differential equations abstract in this study a practical matrix method which is based on collocation points is presented to find approximate solutions of high order linear volterra integro differential equations vides under the mixed conditions in terms of bessel polynomials numerical examples are included to demonstrate the validity and applicability of the technique and comparisons are made with the existing results the results show the efficiency and accuracy of the present work all of the numerical computations have been performed on the computer using a program written in matlab v7 6 0 r2008a

the maximal normal operator space and integration of subdifferentials on nonconvex functions on definit des operateurs normaux maximaux et des fonctions primales sur des ensembles normaux on trouve que la normalite maximale est conservee dans plusieurs operations et que les operateurs normaux maximaux ont de bonnes proprietes de fermeture et de limitation

lipschitz functions with prescribed derivatives and subderivatives in general it is difficult to construct lipschitz functions which are not directly built up from either convex or distance functions one impediment to such constructions is that outside of the real line it is difficult to find anti derivatives the main result of this paper provides under suitable circumstances a technique for constructing such anti derivatives more precisely we show that if f 1 f 2 dots f n are continuously differentiable real valued locally lipschitz functions defined on a non empty open subset a of a separable banach space x then there exists a real valued locally lipschitz function g defined on a such that at each point x in a the clarke subgradient of g at x equals co nabla f 1 x nabla f 2 x dots nabla f n x this same construction also shows that for any finite family t 1 t 2 dots t n of maximal cyclically monotone mappings from a into non empty subsets of x there exists a real valued locally lipschitz function g defined on a such that at each point x in a the clarke subgradient of g at x equals co t 1 x t 2 x dots t n x moreover we show that g is convex if and only if t 1 t 2 cdots t n

chain recurrence in dynamical systems without uniqueness on presente la theorie des systemes dynamiques disperses on considere les attracteurs et leurs repulseurs complementaires on introduit la notion de point recurrent en chaine pour un systeme dynamique disperse

a tau method for the one dimensional parabolic inverse problem subject to temperature overspecification in this work an approximational technique based on shifted legendre tau ideas is presented for the one dimensional parabolic inverse problem with a control parameter the method consists of expanding the required approximate solution as the elements of a shifted legendre polynomial using the operational matrices we reduce the problem to a set of algebraic equations a numerical example is included to demonstrate the validity and applicability of the technique and a comparison is made with existing results the method is easy to implement and produces very accurate results

time average replicator and best reply dynamics using an explicit representation in terms of the logit map we show in a unilateral framework that the time average of the replicator dynamics is a perturbed solution of the best reply dynamics

stochastic approximations of set valued dynamical systems convergence with positive probability to an attractor a successful method to describe the asymptotic behavior of a discrete time stochastic process governed by some recursive formula is to relate it to the limit sets of a well chosen mean differential equation under an attainability condition benaim proved that convergence to a given attractor of the flow induced by this dynamical system occurs with positive probability for a class of robbins monro algorithms benaim hofbauer and sorin generalised this approach for stochastic approximation algorithms whose average behavior is related to a differential inclusion instead we pursue the analogy by extending to this setting the result of convergence with positive probability to an attractor

the spectral methods for parabolic volterra integro differential equations in this paper we study the numerical solutions to parabolic volterra integro differential equations in one dimensional bounded and unbounded spatial domains in a bounded domain the given parabolic volterra integro differential equation is converted to two equivalent equations then a legendre collocation method is used to solve them and finally a linear algebraic system is obtained for an unbounded case we use the algebraic mapping to transfer the problem on a bounded domain and then apply the same presented approach for the bounded domain in both cases some numerical examples are presented to illustrate the efficiency and accuracy of the proposed method

density flow in dynamical networks via mean field games current distributed routing control algorithms for dynamic networks model networks using the time evolution of density at network edges while the routing control algorithm ensures edge density to converge to a wardrop equilibrium which was characterized by an equal traffic density on all used paths we rearrange the density model to recast the problem within the framework of mean field games in doing that we illustrate an extended state space solution approach and we study the stochastic case where the density evolution is driven by a brownian motion further we investigate the case where the density evolution is perturbed by a bounded adversarial disturbance for both the stochastic and the worst case scenarios we provide conditions for the density to converge to a pre assigned set moreover we analyze such conditions from two different perspectives repeated games with vector payoffs and inclusion theory

applying some modern developments to choosing your own lagrange multipliers this paper has primarily been written for those applying lagrange multipliers in practice in it it is shown how these multipliers can be used for solving optimization problems in a much more general fashion than when using the classical method in general it is even possible to apply the generalized method in more than only one way depending either on which variation is the most attractive from a computational point of view or on the kind of equilibrium prices or the kind of sensitivity analysis one is interested in sensitivity analysis is closely related to shadow prices and shadow costs which terms are also used for equilibrium prices the paper discusses the difference between the two and in this connection argues that lagrange multipliers should not be interpreted as something like imputed costs the paper stresses the fact that the basic ideas regarding lagrange multipliers can be developed without using differentiability convexity or continuity duality too is not a necessary ingredient y

a simplex algorithm for piecewise linear programming i derivation and proof the simplex method for linear programming can be extended to permit the minimization of any convex separable piecewise linear objective subject to linear constraints this three part paper develops and analyzes a general computationally practical simplex algorithm for piecewiselinear programming

efficient chebyshev collocation methods for solving optimal control problems governed by volterra integral equations the main purpose of this work is to provide efficient chebyshev collocation methods for solving optimal control problems ocps governed by volterra integral equations the basic principle of our approach is to approximate the state and control using the chebyshev polynomials and collocate the dynamic constraints at the chebyshev type points furthermore we present an exact efficient and stable approach for computing the associated chebyshev integration matrices numerical results on benchmark ocps demonstrate the spectral rate of convergence for the proposed methods

integration of subdifferentials of lower semicontinuous functions on banach spaces abstract this paper studies the lower semicontinuous functions on banach spaces whose subdifferentials are not disassociated from several conditions are given to ensure that two lower semicontinuous functions with the same subdifferential are equal up to an additive constant

general existence theorems for hamilton jacobi equations in the scalar and vectorial cases keywords dirichlet problem hamilton jacobi equation existence of solutions reference caa article 1997 002doi 10 1007 bf02392708view record in web of science record created on 2008 11 25 modified on 2017 05 12

essentially smooth lipschitz functions in this paper we address some of the most fundamental questions regarding the differentiability structure of locally lipschitz functions defined on separable banach spaces for example we examine the relationship between integrability d representability and strict differentiability in addition to this we show that on any separable banach space there is a significant family of locally lipschitz functions that are integrable d representable and possess desirable differentiability properties we also present some striking applications of our results to distance functions

exponential weight algorithm in continuous time the exponential weight algorithm has been introduced in the framework of discrete time on line problems given an observed process x m m 1 2 ldots the input at stage m 1 is an exponential function of the sum s m sum ell 1 m x ell we define the analog algorithm for a continuous time process x t and prove similar properties in terms of external or internal consistency we then deduce results for discrete time from their counterpart in continuous time finally we compare this approach to another continuous time approximation of a discrete time exponential algorithm based on the average sum s m m

analysis of stochastic approximation schemes with set valued maps in the absence of a stability guarantee and their stabilization in this paper we analyze the behavior of stochastic approximation schemes with set valued maps in the absence of a stability guarantee we prove that after a large number of iterations if the stochastic approximation process enters the domain of attraction of an attracting set it gets locked into the attracting set with high probability we demonstrate that the above mentioned result is an effective instrument for analyzing stochastic approximation schemes in the absence of a stability guarantee by using it to obtain an alternate criterion for convergence in the presence of a locally attracting set for the mean field and by using it to show that a feedback mechanism which involves resetting the iterates at regular time intervals stabilizes the scheme when the mean field possesses a globally attracting set thereby guaranteeing convergence the results in this paper build on the works of borkar andrieu et al and chen et al by allowing for the presence of set valued drift functions

a direct approach using the shifted legendre series expansion for near optimum control of linear time varying systems with multiple state and control delays a computationally efficient method which uses finite orthogonal expansions to approximate state and control variables is developed to obtain suboptimal control for time varying systems with multiple state and control delays and with quadratic cost shifted legendre polynomials form the basis of expansions as a consequence of their useful properties the galerkin method is used to reduce the problem to one of minimizing a quadratic form with linear algebraic constraints the solution of a coupled two point boundary value problem with both delayed and advanced terms which is always required in applying the pontryagin s maximum principle to the optimization of delay systems is thus avoided

adaptive learning vector quantization for online parametric estimation this paper addresses the problem of parameter estimation in a quantized and online setting a sensing unit collects random vector valued samples from the environment these samples are quantized and transmitted to a central processor which generates an online estimate of the unknown parameter this paper provides a closed form expression of the excess mean square error mse caused by quantization in the high rate regime i e when the number of quantization levels is supposed to be large next we determine the quantizers which mitigate the excess mse the optimal quantization rule unfortunately depends on the unknown parameter to circumvent this issue we introduce a novel adaptive learning vector quantization scheme which allows to simultaneously estimate the parameter of interest and select an efficient quantizer

proof of the gradient conjecture of r thom let x t be a trajectory of the gradient of a real analytic function and suppose that x0 is a limit point of x t we prove the gradient conjecture of r thom which states that the secants of x t a tx0 have a limit actually we show a stronger statement the radial projection of x t from x0 onto the unit sphere has flnite length

learning in games via reinforcement and regularization we investigate a class of reinforcement learning dynamics where players adjust their strategies based on their actions cumulative payoffs over time specifically by playing mixed strategies that maximize their expected cumulative payoff minus a regularization term a widely studied example is exponential reinforcement learning a process induced by an entropic regularization term which leads mixed strategies to evolve according to the replicator dynamics however in contrast to the class of regularization functions used to define smooth best responses in models of stochastic fictitious play the functions used in this paper need not be infinitely steep at the boundary of the simplex in fact dropping this requirement gives rise to an important dichotomy between steep and nonsteep cases in this general framework we extend several properties of exponential learning including the elimination of dominated strategies the asymptotic stability of strict nash equilibria and the convergence of time averaged trajectories in zero sum games with an interior nash equilibrium

numerical solution for the weakly singular fredholm integro differential equations using legendre multiwavelets an effective method based upon legendre multiwavelets is proposed for the solution of fredholm weakly singular integro differential equations the properties of legendre multiwavelets are first given and their operational matrices of integral are constructed these wavelets are utilized to reduce the solution of the given integro differential equation to the solution of a sparse linear system of algebraic equations in order to save memory requirement and computational time a threshold procedure is applied to obtain the solution to this system of algebraic equations through numerical examples performance of the present method is investigated concerning the convergence and the sparseness of the resulted matrix equation

coupling general penalty schemes for convex programming with the steepest descent and the proximal point algorithm we study the asymptotic behavior of the continuous flow generated by coupling the steepest descent method with a general class of penalty schemes for convex programming we establish convergence of the trajectories towards primal optimal solutions as well as convergence of some naturally associated dual paths the results are extended to the sequences generated by an implicit discretization scheme which corresponds to the coupling of an inexact proximal point iteration with the penalty schemes

solving piecewise linear systems in abs normal form abstract with the ultimate goal of iteratively solving piecewise smooth ps systems we consider the solution of piecewise linear pl equations as shown in 7 pl models can be derived in the fashion of automatic or algorithmic differentiation as local approximations of ps functions with a second order error in the distance to a given reference point the resulting pl functions are obtained quite naturally in what we call the abs normal form a variant of the state representation proposed by bokhoven in his dissertation 27 apart from the tradition of pl modelling by electrical engineers which dates back to the master thesis of thomas stern 26 in 1956 we take into account more recent results on linear complementarity problems and semi smooth equations originating in the optimization community 3 25 5 we analyze simultaneously the original pl problem opl in abs normal form and a corresponding complementary system cpl which is closely related to the absolute value equation ave studied by mangasarian and meyer 14 and a corresponding linear complementarity problem lcp we show that the cpl like kkt conditions and other simply switched systems cannot be open without being injective hence some of the intriguing pl structure described by scholtes in 25 is lost in the transformation from opl to cpl to both problems one may apply newton variants with appropriate generalized jacobians directly computable from the abs normal representation alternatively the cpl can be solved by bokhoven s modulus method and related fixed point iterations we compile the properties of the various schemes and highlight the connection to the properties of the schur complement matrix in particular its partial contractivity as analyzed by rohn and rump 23 numerical experiments and suitable combinations of the fixed point solvers and stabilized generalized newton variants remain to be realized