an lp approach to dynamic programming principles for stochastic control problems with state constraints abstract we study a class of nonlinear stochastic control problems with semicontinuous cost and state constraints using a linear programming lp approach first we provide a primal linearized problem stated on an appropriate space of probability measures with support contained in the set of constraints this space is completely characterized by the coefficients of the control system second we prove a semigroup property for this set of probability measures appearing in the definition of the primal value function this leads to dynamic programming principles for control problems under state constraints with general bounded costs a further linearized dpp is obtained for lower semicontinuous costs

computing moments of the exit time distribution for markov processes by linear programming we provide a new approach to the numerical computation of moments of the exit time distribution of markov processes the method relies on a linear programming formulation of a process exiting from a bounded domain the lp formulation characterizes the evolution of the process through the moments of the induced occupation measure and naturally provides upper and lower bounds for the exact values of the moments the conditions the moments have to satisfy are derived directly from the generator of the markov process and are not based on some approximation of the process excellent software is readily available because the computations involve finite dimensional linear programs

occupation measures for controlled markov processes characterization and optimality for controlled markov processes taking values in a polish space control problems with ergodic cost infinite horizon discounted cost and finite horizon cost are studied each is posed as a convex optimization problem wherein one tries to minimize a linear functional on a closed convex set of appropriately defined occupation measures for the problem these are characterized as solutions of a linear equation asssociated with the problem this characterization is used to establish the existence of optimal markov controls the dual convex optimization problem is also studied

a survey of the maximum principles for optimal control problems with state constraints this paper gives a survey of the various forms of pontryagin s maximum principle for optimal control problems with state variable inequality constraints the relations between the different sets of optimality conditions arising in these forms are shown furthermore the application of these maximum principle conditions is demonstrated by solving some illustrative examples

modal occupation measures and lmi relaxations for nonlinear switched systems control this paper presents a linear programming approach for the optimal control of nonlinear switched systems where the control is the switching sequence this is done by introducing modal occupation measures which allow to relax the problem as a primal linear programming lp problem its dual linear program of hamilton jacobi bellman inequalities is also characterized the lps are then solved numerically with a converging hierarchy of primal dual moment sum of squares sos linear matrix inequalities lmi because of the special structure of switched systems we obtain a much more efficient method than could be achieved by applying standard moment sos lmi hierarchies for general optimal control problems

ℋ filter design for nonlinear polynomial systems the problem of filter design for continuous time nonlinear polynomial systems is addressed in this paper the aim is to design a full order dynamic filter that depends polynomially on the filter states the strategy relies on the use of a quadratic lyapunov function and an inequality condition that assures an performance bound for the augmented polynomial system composed by the original system and the filter to be designed in a regional local context then by using finsler s lemma an enlarged parameter space is created where the lyapunov matrix appears separated from the system matrices in the conditions imposing structural constraints to the decision variables and fixing some values for a scalar parameter design conditions for the filter can be obtained in terms of linear matrix inequalities as illustrated by numerical experiments the proposed conditions can improve the performance provided by standard linear filtering by including the polynomial terms in the filter dynamics

reachability analysis using dissipation inequalities for uncertain nonlinear systems abstract we propose a method to outer bound forward reachable sets on finite horizons for uncertain nonlinear systems with polynomial dynamics this method makes use of time dependent polynomial storage functions that satisfy appropriate dissipation inequalities that account for time varying uncertain parameters l 2 disturbances and perturbations characterized by integral quadratic constraints iqcs with both hard and soft factorizations the use of iqcs in forward reachability analysis allows for various types of uncertainty including unmodeled dynamics the generalized s procedure and sum of squares techniques are used to derive algorithms with the goal of finding the tightest outer bound with a desired shape both pedagogical and practically motivated examples are presented including a 7 state f 18 aircraft model

output feedback stabilization and estimation of the region of attraction for nonlinear systems a vector control lyapunov function perspective this technical note deals with local asymptotic stabilization of uncertain nonlinear systems by output feedback control our study provides a new design technique from a vector control lyapunov function perspective a comparison principle is developed and as a major consequence an effective estimation method of the region of attraction roa is established application to a class of minimum phase strict feedback nonlinear systems gives rise to a constructive stabilization design as well as an estimate of the resulting roa

hamilton jacobi equations with state constraints in the present paper we consider hamilton jacobi equations of the form h x u vu 0 x e q where q is a bounded open subset of rn h is a given continuous real valued function of x s p e q x r x rn and vu is the gradient of the unknown function u we are interested in particular solutions of the above equation which are required to be supersolutions in a suitable weak sense of the same equation up to the boundary of q this requirement plays the role of a boundary condition the main motivation for this kind of solution comes from deterministic optimal control and differential games problems with constraints on the state of the system as well from related questions in constrained geodesics

convex computation of the maximum controlled invariant set for polynomial control systems we characterize the maximum controlled invariant mci set for discrete as well as continuous time nonlinear dynamical systems as the solution of an infinite dimensional linear programming problem for systems with polynomial dynamics and compact semialgebraic state and control constraints we describe a hierarchy of finite dimensional linear matrix inequality lmi relaxations whose optimal values converge to the volume of the mci set dual to these lmi relaxations are sum of squares sos problems providing a converging sequence of outer approximations to the mci set the approach is simple and readily applicable in the sense that the approximations are the outcome of a single semidefinite program with no additional input apart from the problem description a number of numerical examples illustrate the approach

inner approximating reachable sets for polynomial systems with time varying uncertainties in this paper we propose a convex programming based method to address a long standing problem of inner approximating backward reachable sets of state constrained polynomial systems subject to time varying uncertainties the backward reachable set is a set of states from which all trajectories starting will surely enter a target region at the end of a given time horizon without violating a set of state constraints in spite of the actions of uncertainties it is equal to the zero sublevel set of the unique lipschitz viscosity solution to a hamilton jacobi partial differential equation hje we show that inner approximations of the backward reachable set can be formed by zero sublevel sets of its viscosity supersolutions consequently we reduce the inner approximation problem to a problem of synthesizing polynomial viscosity supersolutions to this hje such a polynomial solution in our method is synthesized by solving a single semidefinite program we also prove that polynomial solutions to the formulated semidefinite program exist and can produce a convergent sequence of inner approximations to the interior of the backward reachable set in measure under appropriate assumptions this is the main contribution of this paper several illustrative examples demonstrate the merits of our approach

existence of markov controls and characterization of optimal markov controls given a solution of a controlled martingale problem it is shown under general conditions that there exists a solution having markov controls which has the same cost as the original solution this result is then used to show that the original stochastic control problem is equivalent to a linear program over a space of measures under a variety of optimality criteria existence and characterization of optimal markov controls then follows an extension of echeverria s theorem characterizing stationary distributions for uncontrolled markov processes is obtained as a corollary in particular this extension covers diffusion processes with discontinuous drift and diffusion coefficients

on maximal robust positively invariant sets in constrained nonlinear systems abstract in this technical communique we study the maximal robust positively invariant set for state constrained continuous time nonlinear systems subjected to a bounded disturbance extending results from the theory of barriers we show that this set is closed and that its boundary consists of two complementary parts one of which we name the invariance barrier which consists of trajectories that satisfy the maximum principle

convex duality and nonlinear optimal control problems in nonlinear optimal control can be reformulated as convex optimization problems over a vector space of linear functionals in this way methods of convex analysis can be brought to bear on the task of characterizing solutions to such problems the result is a necessary and sufficient condition of optimality that generalizes well known sufficient conditions referred to as verification theorems in dynamic programming as a byproduct we obtain a representation of the minimum cost in terms of the upper envelope of subsolutions to the hamilton jacobi equation it is a striking illustration of the wide range of problems to which convex analysis and in particular convex duality is applicable the approach applied to parametric problems in the calculus of variations was pioneered by l c young lectures on the calculus of variations and optimal control theory w b saunders philadelphia pa 1969 as recent work has shown however it is equally fruitful when applied in optimal control thi

asymptotic controllability and exponential stabilization of nonlinear control systems at singular points we discuss the relation between exponential stabilization and asymptotic controllability of nonlinear control systems with constrained control range at singular points using a discounted optimal control approach we construct discrete feedback laws minimizing the lyapunov exponent of the linearization thus we obtain an equivalence result between uniform exponential controllability and uniform exponential stabilizability by means of a discrete feedback law

the problem of optimal control with reflection studied through a linear optimization problem stated on occupational measures abstract we obtain a linear programming characterization for the minimum cost associated with finite dimensional reflected optimal control problems in order to describe the value functions we employ an infinite dimensional dual formulation instead of using the characterization via hamilton jacobi partial differential equations in this paper we consider control problems with both infinite and finite horizons the reflection is given by the normal cone to a proximal retract set

liouville theorems for the navier stokes equations and applications we study bounded ancient solutions of the navier stokes equations these are solutions with bounded velocity defined in rn 1 0 in two space dimensions we prove that such solutions are either constant or of the form u x t b t depending on the exact definition of admissible solutions the general 3 dimensional problem seems to be out of reach of existing techniques but partial results can be obtained in the case of axisymmetric solutions we apply these results to some scenarios of potential singularity formation for axi symmetric solutions and obtain extensions of results in a recent paper by chen strain tsai and yau 4

stability analysis and control of rigid body systems with impacts and friction many critical tasks in robotics such as locomotion or manipulation involve collisions between a rigid body and the environment or between multiple bodies methods based on sums of squares sos for numerical computation of lyapunov certificates are a powerful tool for analyzing the stability of continuous nonlinear systems and can additionally be used to automatically synthesize stabilizing feedback controllers here we present a method for applying sums of squares verification to rigid bodies with coulomb friction undergoing discontinuous inelastic impact events the proposed algorithm explicitly generates lyapunov certificates for stability positive invariance and safety over admissible non penetrating states and contact forces we leverage the complementarity formulation of contact which naturally generates the semialgebraic constraints that define this admissible region the approach is demonstrated on multiple robotics examples including simple models of a walking robot a perching aircraft and control design of a balancing robot

on the liouville theorem for the stationary navier stokes equations in a critical space abstract in this paper we prove the liouville type theorem for the stationary navier stokes equations on r 3 more specifically if a solution u h 1 r 3 to the stationary navier stokes system satisfies additional conditions characterized by the decays near infinity and by the oscillation then we show that u 0

a moment and sum of squares extension of dual dynamic programming with application to nonlinear energy storage problems abstract we present a finite horizon optimization algorithm that extends the established concept of dual dynamic programming ddp in two ways first in contrast to the linear costs dynamics and constraints of standard ddp we consider problems in which all of these can be polynomial functions second we allow the state trajectory to be described by probability distributions rather than point values and return approximate value functions fitted to these the algorithm is in part an adaptation of sum of squares techniques used in the approximate dynamic programming literature it alternates between a forward simulation through the horizon in which the moments of the state distribution are propagated through a succession of single stage problems and a backward recursion in which a new polynomial function is derived for each stage using the moments of the state as fixed data the value function approximation returned for a given stage is the point wise maximum of all polynomials derived for that stage this contrasts with the piecewise affine functions derived in conventional ddp we prove key convergence properties of the new algorithm and validate it in simulation on two case studies related to the optimal operation of energy storage devices with nonlinear characteristics the first is a small borehole storage problem for which multiple value function approximations can be compared the second is a larger problem for which conventional discretized dynamic programming is intractable

on a representation of the limit occupational measures set of a control system with applications to singularly perturbed control systems a representation of the limit occupational measures set of a control system in terms of the vector function defining the system s dynamics is established applications in averaging of singularly perturbed control systems are demonstrated

stability analysis for a class of partial differential equations via semidefinite programming this paper studies scalar integral inequalities in one dimensional bounded domains with polynomial integrands we propose conditions to verify the integral inequalities in terms of differential matrix inequalities these conditions allow for the verification of the inequalities in subspaces defined by boundary values of the dependent variables the results are applied to solve integral inequalities arising from the lyapunov stability analysis of partial differential equations examples illustrate the results

inner approximations for polynomial matrix inequalities and robust stability regions following a polynomial approach many robust fixed order controller design problems can be formulated as optimization problems whose set of feasible solutions is modeled by parametrized polynomial matrix inequalities pmis these feasibility sets are typically nonconvex given a parametrized pmi set we provide a hierarchy of linear matrix inequality lmi problems whose optimal solutions generate inner approximations modeled by a single polynomial superlevel set those inner approximations converge in a well defined analytic sense to the nonconvex original feasible set with asymptotically vanishing conservatism one may also impose the hierarchy of inner approximations to be nested or convex in the latter case they do not converge any more to the feasible set but they can be used in a convex optimization framework at the price of some conservatism finally we show that the specific geometry of nonconvex polynomial stability regions can be exploited to improve convergence of the hierarchy of inner approximations

robust region of attraction estimation we propose a method to compute invariant subsets of the region of attraction for asymptotically stable equilibrium points of polynomial dynamical systems with bounded parametric uncertainty parameter independent lyapunov functions are used to characterize invariant subsets of the robust region of attraction a branch and bound type refinement procedure reduces the conservatism we demonstrate the method on an example from the literature and uncertain controlled short period aircraft dynamics

liouville type theorems for the 3d stationary navier stokes mhd and hall mhd equations abstract in this paper we consider the liouville type problem of the three dimensional stationary incompressible navier stokes equations and establish an improved liouville type result only under the condition that the l p norm of a velocity u is finite moreover these results are extended to the mhd and hall mhd equations

characterizing bad semidefinite programs normal forms and short proofs semidefinite programs sdps some of the most useful and versatile optimization problems of the last few decades are often pathological the optimal values of the primal and dual problems may d

on some theorems on circulant matrices abstract in 2 some theorems on n n circulant matrices were introduced under the hypothesis n a prime number we extend these theorems to the case n 2 r where r is a prime greater then 2

on sets of occupational measures generated by a deterministic control system on an infinite time horizon abstract we give a representation for the closed convex hull of the set of discounted occupational measures generated by control state trajectories of a deterministic control system we also investigate the limit behavior of the latter when the discount factor tends to zero and compare it with the limit behavior of the long run time average occupational measures set the novelty of our results is in that we allow the control set dependence on the state variables that make the results to be applicable to differential inclusions

filippov s and filippov wazewski s theorems on closed domains abstract the celebrated filippov s theorem implies that given a trajectory x1 0 r n of a differential inclusion x f t x with the set valued map f measurable in t and k lipschitz in x for any initial condition x2 0 r n there exists a trajectory x2 starting from x2 0 such that x1 t x2 t ekt x1 0 x2 0 filippov wazewski s theorem establishes the possibility of approximating any trajectory of the convexified differential inclusion x co f t x by a trajectory of the original inclusion x f t x starting from the same initial condition in the present paper we extend both theorems to the case when the state variable x is constrained to the closure of an open subset r n the latter is allowed to be non smooth we impose a generalized soner type condition on f and yielding extensions of the above classical results to infinite horizon constrained problems applications to the study of regularity of value functions of optimal control problems with state constraints are discussed as well

liouville type theorems for the forced euler equations and the navier stokes equations in this paper we study the liouville type properties for solutions to the steady incompressible euler equations with forces in mathbb r n if we assume single signedness condition on the force then we can show that a c 1 mathbb r n solution v p with v 2 p in l frac q 2 mathbb r n q in frac 3n n 1 infty is trivial v 0 for the solution of the steady navier stokes equations satisfying v x to 0 as x to infty the condition int mathbb r 3 delta v frac 6 5 dx infty which is stronger than the important d condition int mathbb r 3 nabla v 2 dx infty but both having the same scaling property implies that v 0 in the appendix we reprove theorem 1 1 chae commun math phys 273 203 215 2007 using the self similar euler equations directly

paper maximal lyapunov functions and domains of attraction for autonomous nonlinear systems in this paper we present various theoretical and computational methods for estimating the domain of attraction of an autonomous nonlinear system these methods are based on the concept of a maximal lyapunov function which is introduced in this paper a partial differential equation characterizing a maximal lyapunov function is derived and the relationships of this equation as compared to zubov s partial differential equation are discussed an iterative procedure is given for solving the new partial differential equation this procedure yields lyapunov function candidates that are rational functions rather than polynomials the method is applied to four two dimensional examples and one three dimensional example and it is shown that the estimates obtained using this method are in many cases substantially better than those obtained using known methods

convex duality approach to the optimal control of diffusions an alternative to the usual dynamic programming approach to the optimal control of markov processes is considered it is based on duality of convex analysis the control problem is embedded in a convex mathematical programming problem on a space of measures the dual problem is to find the supremum of smooth subsolutions to the hamilton jacobi bellman equation

characterization of the optimal trajectories for the averaged dynamics associated to singularly perturbed control systems abstract the aim of this paper is to study singularly perturbed control systems firstly we provide linearized formulation version for the calculus of the value function associated with the averaged dynamics secondly we obtain necessary and sufficient conditions in order to identify the optimal trajectory of the averaged system

bracketing backward reach sets of a dynamical system in this paper we present a new method for bracketing i e characterising from inside and from outside backward reach set of the target region t of a continuous time dynamical system the princip

global optimization with polynomials and the problem of moments we consider the problem of finding the unconstrained global minimum of a real valued polynomial p x mathbb r n to mathbb r as well as the global minimum of p x in a compact set k defined by polynomial inequalities it is shown that this problem reduces to solving an often finite sequence of convex linear matrix inequality lmi problems a notion of karush kuhn tucker polynomials is introduced in a global optimality condition some illustrative examples are provided

lmi techniques for optimization over polynomials in control a survey numerous tasks in control systems involve optimization problems over polynomials and unfortunately these problems are in general nonconvex in order to cope with this difficulty linear matrix inequality lmi techniques have been introduced because they allow one to obtain bounds to the sought solution by solving convex optimization problems and because the conservatism of these bounds can be decreased in general by suitably increasing the size of the problems this survey aims to provide the reader with a significant overview of the lmi techniques that are used in control systems for tackling optimization problems over polynomials describing approaches such as decomposition in sum of squares positivstellensatz theory of moments polya s theorem and matrix dilation moreover it aims to provide a collection of the essential problems in control systems where these lmi techniques are used such as stability and performance investigations in nonlinear systems uncertain systems time delay systems and genetic regulatory networks it is expected that this survey may be a concise useful reference for all readers

lyapunov measure for almost everywhere stability this paper is concerned with the analysis and computational methods for verifying global stability of an attractor set of a nonlinear dynamical system based upon a stochastic representation of deterministic dynamics a lyapunov measure is proposed for these purposes this measure is shown to be a stochastic counterpart of stability transience just as an invariant measure is a counterpart of the attractor recurrence it is a dual of the lyapunov function and is useful for the study of more general weaker and set wise notions of stability in addition to the theoretical framework constructive methods for computing approximations to the lyapunov measures are presented these methods are based upon set oriented numerical approaches several equivalent descriptions including a series formula and a system of linear inequalities are provided for computational purposes these descriptions allow one to carry over the intuition from the linear case with stable equilibrium to nonlinear systems with globally stable attractor sets finally in certain cases the exact relationship between lyapunov functions and lyapunov measures is also given

region of attraction analysis with integral quadratic constraints abstract a general framework is presented to estimate the region of attraction of attracting equilibrium points the system is described by a feedback connection of a nonlinear polynomial system and a bounded operator the input output behavior of the operator is characterized using an integral quadratic constraint this allows to analyze generic problems including for example hard nonlinearities and different classes of uncertainties adding to the state of practice in the field which is typically limited to polynomial vector fields the iqc description is also nonrestrictive with the main result given for both hard and soft factorizations optimization algorithms based on sum of squares techniques are then proposed with the aim to enlarge the inner estimates of the roa numerical examples are provided to show the applicability of the approaches these include a saturated plant where bounds on the states are exploited to refine the sector description and a case study with parametric uncertainties for which the conservativeness of the results is reduced by using soft iqcs

on liouville type theorems for the steady navier stokes equations in r3 abstract in this paper we prove three different liouville type theorems for the steady navier stokes equations in r 3 in the first theorem we improve logarithmically the well known l 9 2 r 3 result in the second theorem we present a sufficient condition for the trivially of the solution v 0 in terms of the head pressure q 1 2 v 2 p the imposed integrability condition here has the same scaling property as the dirichlet integral in the last theorem we present fubini type condition which guarantee v 0

convergence rates of moment sum of squares hierarchies for optimal control problems we study the convergence rate of moment sum of squares hierarchies of semidefinite programs for optimal control problems with polynomial data it is known that these hierarchies generate polynomial under approximations to the value function of the optimal control problem and that these under approximations converge in the l1 norm to the value function as their degree d tends to infinity we show that the rate of this convergence is o 1 log log d we treat in detail the continuous time infinite horizon discounted problem and describe in brief how the same rate can be obtained for the finite horizon continuous time problem and for the discrete time counterparts of both problems

stochastic optimal control via forward and backward stochastic differential equations and importance sampling abstract the aim of this work is to present a novel sampling based numerical scheme designed to solve a certain class of stochastic optimal control problems utilizing forward and backward stochastic differential equations fbsdes by means of a nonlinear version of the feynman kac lemma we obtain a probabilistic representation of the solution to the nonlinear hamilton jacobi bellman equation expressed in the form of a system of decoupled fbsdes this system of fbsdes can be solved by employing linear regression techniques the proposed framework relaxes some of the restrictive conditions present in recent sampling based methods within the linearly solvable optimal control framework and furthermore addresses problems in which the time horizon is not prespecified to enhance the efficiency of the proposed scheme when treating more complex nonlinear systems we then derive an iterative algorithm based on girsanov s theorem on the change of measure which features importance sampling this scheme is shown to be capable of learning the optimal control without requiring an initial guess

structure of algebras of commutative matrices commutative spaces of matrices a nk 1 akjk are studied where jk is a set of 0 1 matrices with prescribed sum s and the ak s are complex parameters the great number of commutative spaces a obeying simple prescriptions on the first two rows of s can all be described in terms of two basic algebras of matrices denoted by tn and n a relationship between the structure of spaces a and the multiplicative complexity of the bilinear forms defined by the jk s is discussed

linear systems with state and control constraints the theory and application of maximal output admissible sets the initial state of an unforced linear system is output admissible with respect to a constraint set y if the resulting output function satisfies the pointwise in time condition y t in y t or 0 the set of all possible such initial conditions is the maximal output admissible set o sub infinity the properties of o sub infinity and its characterization are investigated in the discrete time case it is generally possible to represent o sub infinity or a close approximation of it by a finite number of functional inequalities practical algorithms for generating the functions are described in the continuous time case simple representations of the maximal output admissible set are not available however it is shown that the discrete time results may be used to obtain approximate representations

measures and lmis for impulsive nonlinear optimal control this note shows how to use semi definite programming to find lower bounds on a large class of nonlinear optimal control problems with polynomial dynamics and convex semialgebraic state constraints and an affine dependence on the control this is done by relaxing an optimal control problem into a linear programming problem on measures also known as a generalized moment problem the handling of measures by their moments reduces the problem to a convergent series of standard linear matrix inequality relaxations when the optimal control consists of a finite number of impulses we can recover simultaneously the actual impulse times and amplitudes by simple linear algebra finally our approach can be readily implemented with standard software as illustrated by a numerical example

an iterative procedure for constructing subsolutions of discrete time optimal control problems an iterative procedure for constructing subsolutions of deterministic or stochastic optimal control problems in discrete time with continuous state space is introduced the procedure generates a nondecreasing sequence of subsolutions giving true lower bounds on the minimal costs convergence of the values at any fixed initial state is shown

linear programming approach to deterministic long run average problems of optimal control we establish that deterministic long run average problems of optimal control are asymptotically equivalent to infinite dimensional linear programming problems lpps and we establish that these lpps can be approximated by finite dimensional lpps the solutions of which can be used for construction of the optimal controls general results are illustrated with numerical examples

time average control of martingale problems existence of a stationary solution one studies the average cost for controlled systems via the martingale problem for their generator the stationary distributions for these processes on the product of the state and control spaces are characterized in terms of integration of the generator and existence of a stationary solution is established the gives existence of a stationary control

approximate volume and integration for basic semialgebraic sets given a basic compact semialgebraic set mathbf k subset mathbb r n we introduce a methodology that generates a sequence converging to the volume of mathbf k this sequence is obtained from optimal values of a hierarchy of either semidefinite or linear programs not only the volume but also every finite vector of moments of the probability measure that is uniformly distributed on mathbf k can be approximated as closely as desired which permits the approximation of the integral on mathbf k of any given polynomial the extension to integration against some weight functions is also provided finally some numerical issues associated with the algorithms involved are briefly discussed

survey paper a partial history of the early development of continuous time nonlinear stochastic systems theory this article is a survey of the early development of selected areas in nonlinear continuous time stochastic control key developments in optimal control and the dynamic programming principle existence of optimal controls under complete and partial observations nonlinear filtering stochastic stability the stochastic maximum principle and ergodic control are discussed issues concerning wide bandwidth noise for stability modeling filtering and ergodic control are dealt with the focus is on the earlier work but many important topics are omitted for lack of space

the moment problem on compact semi algebraic sets in this chapter we begin the study of the multidimensional moment problem the passage to dimensions d 2 brings new difficulties and unexpected phenomena in sect 3 2 we derived solvability criteria of the moment problem on intervals in terms of positivity conditions it seems to be natural to look for similar characterizations in higher dimensions as well this leads us immediately into the realm of real algebraic geometry and to descriptions of positive polynomials on semi algebraic sets in this chapter we treat this approach for basic closed compact semi algebraic subsets of mathbb r d it turns out that for such sets there is a close interaction between the moment problem and positivstellensatze for strictly positive polynomials

permutation matrix groups with positive sum abstract explicit forms are given for all commutative sets of permutation matrices which sum to a positive matrix and for nonabelian groups of permutation matrices of order twice an odd prime which sum to the matrix of all ones a relationship to circulants of level k is indicated

optimal control problems with oscillations concentrations and discontinuities optimal control problems with oscillations chattering controls and concentrations impulsive controls can have integral performance criteria such that concentration of the control signal occurs at a discontinuity of the state signal techniques from functional analysis anisotropic parametrized measures are applied to give a precise meaning of the integral cost and to allow for the sound application of numerical methods we show how this can be combined with the lasserre hierarchy of semidefinite programming relaxations

linear programming approach to deterministic infinite horizon optimal control problems with discounting we investigate relationships between the deterministic infinite time horizon optimal control problem with discounting in which the state trajectories remain in a given compact set y and a certain infinite dimensional linear programming idlp problem we introduce the problem dual with respect to this idlp problem and obtain some duality results we construct necessary and sufficient optimality conditions for the optimal control problem under consideration and we give a characterization of the viability kernel of y we also indicate how one can use finite dimensional approximations of the idlp problem and its dual for construction of near optimal feedback controls the construction is illustrated with a numerical example

duality in linear programming problems related to deterministic long run average problems of optimal control it has been established recently that under mild conditions deterministic long run average problems of optimal control are asymptotically equivalent to infinite dimensional linear programming problems lpps and that these lpps can be approximated by finite dimensional lpps in this paper we introduce the corresponding infinite and finite dimensional dual problems and study duality relationships we also investigate the possibility of using solutions of finite dimensional lpps and their duals for numerical construction of the optimal controls in periodic optimization problems the construction is illustrated with a numerical example

on the relation between discounted and average optimal value functions abstract we investigate the relation between discounted and average deterministic optimal control problems for nonlinear control systems in particular we are interested in the corresponding optimal value functions using the concepts of viability chain controllability and controllability a global convergence result for vanishing discount rate is obtained basic ingredients for the analysis are an abelian type theorem controllability properties of the system and the morse decomposition of the corresponding control flow

survey paper set invariance in control the properties of positively invariant sets are involved in many different problems in control theory such as constrained control robustness analysis synthesis and optimization in this paper we provide an overview of the literature concerning positively invariant sets and their application to the analysis and synthesis of control systems

on the existence of a limit value in some nonexpansive optimal control problems we investigate the limit of the average value of an optimal control problem when the horizon converges to infinity for this aim we suppose suitable nonexpansive like assumptions which does not imply that the limit is independent of the initial state as is usually done in the literature

approximation schemes for infinite linear programs this paper presents approximation schemes for an infinite linear program in particular it is shown that under suitable assumptions the program s optimum value can be approximated by the values of finite dimensional linear programs and that in addition every accumulation point of a sequence of optimal solutions for the approximating programs is an optimal solution for the original problem

time average control of martingale problems a linear programming formulation one studies the average cost for controlled systems given as solutions of the martingale problem for their generator the control problem is reformulated as a linear programming problem and conditions are given for the existence of an optimal solution it is further shown that the optimal control can be taken to depend only on the history of the system and that this cost remains optimal for systems with different initial distributions

a dual to lyapunov s stability theorem lyapunov s second theorem is a standard tool for stability analysis of ordinary differential equations here we introduce a theorem which can be viewed as a dual to lyapunov s result from existence of a scalar function satisfying certain inequalities it follows that almost all trajectories of the system tend to zero the scalar function has a physical interpretation as the stationary density of a substance that is generated in all points of the state space and flows along the system trajectories if the stationary density is bounded everywhere except at a singularity in the origin then almost all trajectories tend towards the origin the weaker notion of stability allows for applications also in situations where lyapunov s theorem cannot be used moreover the new criterion has a striking convexity property related to control synthesis

convex computation of the region of attraction of polynomial control systems we address the long standing problem of computing the region of attraction roa of a target set e g a neighborhood of an equilibrium point of a controlled nonlinear system with polynomial dynamics and semialgebraic state and input constraints we show that the roa can be computed by solving an infinite dimensional convex linear programming lp problem over the space of measures in turn this problem can be solved approximately via a classical converging hierarchy of convex finite dimensional linear matrix inequalities lmis our approach is genuinely primal in the sense that convexity of the problem of computing the roa is an outcome of optimizing directly over system trajectories the dual infinite dimensional lp on nonnegative continuous functions approximated by polynomial sum of squares allows us to generate a hierarchy of semialgebraic outer approximations of the roa at the price of solving a sequence of lmi problems with asymptotically vanishing conservatism this sharply contrasts with the existing literature which follows an exclusively dual lyapunov approach yielding either nonconvex bilinear matrix inequalities or conservative lmi conditions the approach is simple and readily applicable as the outer approximations are the outcome of a single semidefinite program with no additional data required besides the problem description the approach is demonstrated on several numerical examples

vanishing discount limit and nonexpansive optimal control and differential games a classical problem in ergodic control consists of studying the limit behavior of the optimal value v lambda of a discounted cost functional with infinite horizon as the discount factor lambda tends to zero in the literature this problem has been addressed under various conditions ensuring that the rescaled value function lambda v lambda converges uniformly to a constant limit the main goal of this paper is to study this problem without such conditions so that the aforementioned limit need not be constant so under a nonexpansivity assumption we derive lipschitz bounds which yield compactness of lambda v lambda for both control systems and differential games then we study the convergence of solutions to hamilton jacobi equations under the hypothesis that the hamiltonian is radially nondecreasing hence allowing for the existence noncoercivity directions using pde methods we show that the convergence is monotone and we characterize the limit as the maximal subsolution of a cert

nonlinear optimal control via occupation measures and lmi relaxations we consider the class of nonlinear optimal control problems ocps with polynomial data i e the differential equation state and control constraints and cost are all described by polynomials and more generally for ocps with smooth data in addition state constraints as well as state and or action constraints are allowed we provide a simple hierarchy of lmi linear matrix inequality relaxations whose optimal values form a nondecreasing sequence of lower bounds on the optimal value under some convexity assumptions the sequence converges to the optimal value of the ocp preliminary results show that good approximations are obtained with few moments

on the estimation of asymptotic stability regions state of the art and new proposals this paper deals with the problem of the estimation of regions of asymptotic stability for continuous autonomous nonlinear systems the first part of the work provides a comprehensive survey of the existing methods and of their applications in engineering fields in the second part certain topological considerations are first developed and the trajectory reversing method is then presented together with a theorem on which it is based in the final part several examples of application are reported showing the efficiency of the proposed technique for low order second and third systems

on representation formulas for long run averaging optimal control problem we investigate an optimal control problem with an averaging cost the asymp totic behavior of the values is a classical problem in ergodic control to study the long run averaging we consider bothce aro and abel means a main result of the paper says that there is at most one possible accumulation point in the uniform convergence topology of the values when the time horizon of thece aro means converges to infinity or the discount factor of the abel means converges to zero this unique accumulation point is explicitly described by representation formulas involving probability measures on the state and control spaces as a byproduct we obtain the existence of a limit value whenever thece aro or abel values are equicon tinuous our approach allows to generalise several results in ergodic control and in particular it allows to cope with cases where the limit value is not constant with respect to the initial condition

underapproximating backward reachable sets by semialgebraic sets underapproximations uas of backward reachable sets play an important role in controller synthesis and trajectory analysis for constrained nonlinear dynamical systems but there are few methods available to compute them given a nonlinear system a target region of simply connected compact type and a time duration we present a method using boundary analysis to compute an ua of the backward reachable set the ua is represented as a semialgebraic set formed by what we term polynomial mathtt level set functions the polynomial mathtt level set function is a semidefinite positive function with one real root such that the interior and closure of a semialgebraic set formed by it are both simply connected and have the same boundary the function can be computed by solving a convex program which is constructed based on sum of squares decomposition and linear interval inequalities we test our method on several examples and compare them with existing methods the results show that our method can obtain better estimations more efficiently in terms of time for these special examples

linear programming formulation for optimal stopping problems optimal stopping problems for continuous time markov processes are shown to be equivalent to infinite dimensional linear programs over a space of pairs of measures under very general conditions the measures involved represent the joint distribution of the stopping time and stopping location and the occupation measure of the process until it is stopped these measures satisfy an identity for each function in the domain of the generator which is sufficient to characterize the stochastic process finite dimensional linear programs obtained using markov chain approximations are solved in two examples to illustrate the numerical accuracy of the linear programming formulation

safe nonlinear control design for input constrained polynomial systems using sum of squares programming we study the feedback stabilisation problem for input affine polynomial systems subject to polytopic input constraints first we characterise a subset of the state space which we refer to as the

viscosity solutions of fully nonlinear parabolic path dependent pdes part ii the main objective of this paper and the accompanying one viscosity solutions of fully nonlinear parabolic path dependent pdes part ii 2012 preprint is to provide a notion of viscosity solutions for fully nonlinear parabolic path dependent pdes our definition extends our previous work ann probab 2014 42 204 236 focused on the semilinear case and is crucially based on the nonlinear optimal stopping problem analyzed in stochastic process appl 2014 124 3277 3311 we prove that our notion of viscosity solutions is consistent with the corresponding notion of classical solutions and satisfies a stability property and a partial comparison result the latter is a key step for the well posedness results established in viscosity solutions of fully nonlinear parabolic path dependent pdes part ii 2012 preprint we also show that the value processes of path dependent stochastic control problems are viscosity solutions of the corresponding path dependent dynamic programming equations

hamilton jacobi formulation for reach avoid differential games a new framework for formulating reachability problems with competing inputs nonlinear dynamics and state constraints as optimal control problems is developed such reach avoid problems arise in among others the study of safety problems in hybrid systems earlier approaches to reach avoid computations are either restricted to linear systems or face numerical difficulties due to possible discontinuities in the hamiltonian of the optimal control problem the main advantage of the approach proposed in this paper is that it can be applied to a general class of target hitting continuous dynamic games with nonlinear dynamics and has very good properties in terms of its numerical solution since the value function and the hamiltonian of the system are both continuous the performance of the proposed method is demonstrated by applying it to a case study which involves the target hitting problem of an underactuated underwater vehicle in the presence of obstacles

optimal control with state space constraint i optimal control of piecewise deterministic processes with state space constraint is studied under appropriate assumptions it is shown that the optimal value function is the only viscosity solution on the open domain which is also a supersolution on the closed domain finally the uniform continuity of the value function is obtained under a condition on the deterministic drift

controller design and value function approximation for nonlinear dynamical systems this work considers the infinite time discounted optimal control problem for continuous time input affine polynomial dynamical systems subject to polynomial state and box input constraints we propose a sequence of sum of squares sos approximations of this problem obtained by first lifting the original problem into the space of measures with continuous densities and then restricting these densities to polynomials these approximations are tightenings rather than relaxations of the original problem and provide a sequence of rational controllers with value functions associated to these controllers converging under some technical assumptions to the value function of the original problem in addition we describe a method to obtain polynomial approximations from above and from below to the value function of the extracted rational controllers and a method to obtain approximations from below to the optimal value function of the original problem thereby obtaining a sequence of asymptotically optimal rational controllers with explicit estimates of suboptimality numerical examples demonstrate the approach

discontinuous control problems with state constraints linear formulations and dynamic programming principles abstract this paper aims at studying a class of discontinuous deterministic control problems under state constraints using a linear programming approach as for classical control problems gaitsgory and quincampoix 2009 16 goreac and serea 2011 19 the primal linear problem is stated on some appropriate space of probability measures naturally the support of these measures is contained in the set of constraints this linearized value function and its dual can alternatively be seen as the limit of standard penalized problems second we provide a semigroup property for this set of probability measures leading to dynamic programming principles for control problems under state constraints an abstract principle is provided for general bounded costs linearized versions are obtained under further semi continuity assumptions

global optimization in hilbert space we propose a complete search algorithm for solving a class of non convex possibly infinite dimensional optimization problems to global optimality we assume that the optimization variables are in a bounded subset of a hilbert space and we determine worst case run time bounds for the algorithm under certain regularity conditions of the cost functional and the constraint set because these run time bounds are independent of the number of optimization variables and in particular are valid for optimization problems with infinitely many optimization variables we prove that the algorithm converges to an varepsilon suboptimal global solution within finite run time for any given termination tolerance varepsilon 0 finally we illustrate these results for a problem of calculus of variations

ergodic control for constrained diffusions characterization using hjb equations recently in a budhiraja siam j control optim 42 2003 pp 532 558 an ergodic control problem for a class of diffusion processes constrained to take values in a polyhedral cone was considered the main result of that paper was that under appropriate conditions on the model there is a markov control for which the infimum of the cost function is attained in the current work we characterize the value of the ergodic control problem via a suitable hamilton jacobi bellman hjb equation the theory of existence and uniqueness of classical solutions for pdes in domains with corners and reflection fields which are oblique discontinuous and multivalued on corners is not available we show that the natural hjb equation for the ergodic control problem admits a unique continuous viscosity solution which enables us to characterize the value function of the control problem the existence of a solution to this hjb equation is established via the classical vanishing discount argument the key step is proving the precompactness of the family of suitably renormalized discounted value functions in this regard we use a recent technique introduced in v s borkar stochastic process appl 103 2003 pp 293 310 of using the athreya ney nummelin pseudoatom construction for obtaining a coupling of a pair of embedded discrete time controlled markov chains

region of attraction estimation using invariant sets and rational lyapunov functions this work addresses the problem of estimating the region of attraction ra of equilibrium points of nonlinear dynamical systems the estimates we provide are given by positively invariant sets which are not necessarily defined by level sets of a lyapunov function moreover we present conditions for the existence of lyapunov functions linked to the positively invariant set formulation we propose connections to fundamental results on estimates of the ra are presented and support the search of lyapunov functions of a rational nature we then restrict our attention to systems governed by polynomial vector fields and provide an algorithm that is guaranteed to enlarge the estimate of the ra at each iteration

lagrange lemma and the optimal control of diffusions ii nonlinear lagrange functionals abstract in kosmol 1991 kosmol and pavon 1993 a new elementary approach to optimal control problems relying on the lagrange lemma was described which appears to be technically and conceptually much simpler than existing methods and furthermore provides a unified variational approach in kosmol and pavon 1992 this method was further clarified and developed for linear lagrange functionals we devote this second paper to nonlinear lagrange functionals the power of this approach is here clearly demonstrated in particular it is shown that the nonlinear lagrange functional induced by the value function of the problem is just one of the many functionals which may effectively be employed to solve control problems see examples 1 and 2 in section 3 moreover in our approach we can deal in the same framework with problems with state constraints or nonsmooth problems hence the lagrange functionals approach provides new tools to solve control problems which may be readily applied even when existing approaches fail