recursive nonparametric identification of nonlinear systems with adaptive binary sensors in this paper the problem of identifying nonlinear systems under adaptive binary valued output measurements is considered we follow a nonparametric approach which directly estimates the value of the nonlinear function representing the system at any fixed point with the help of a recursive kernel based stochastic approximation algorithm with expanding truncations saawet the thresholds of the binary sensor are adaptively designed to achieve a sufficient richness of information in the output observations the constructed estimates are proved to converge to the true values with probability one two numerical examples are given showing that the simulation results are consistent with the theoretical analysis

dynamic recomputation cannot extend the optimality range of priority indices sequencing rules that rely on priority indices are particularly attractive because they are simple to calculate and easy to implement it has been established that there are exactly two classes of delay cost functions for which policies that are determined by time invariant priority indices are capable of producing optimal sequences linear delay costs and discounted linear delay costs we consider index based policies that allow dynamic recalculation of priority indices and show that this class does not enlarge the set of delay cost functions for which index based rules can produce optimal policies our analysis relies on the argument that index based rules must induce a transitive ranking of the tasks we show that the classes of linear and discounted linear functions are the only ones that can be associated with such rankings

hammerstein systems identification in presence of hard nonlinearities of preload and dead zone type hammerstein system identification is considered in presence of preload and dead zone nonlinearities the discontinuous feature of these nonlinearities makes it difficult to get a single system parameterization involving linearly all unknown parameters those of the linear subsystem and those of the nonlinearity therefore system identification has generally been dealt with using multiple stage schemes including different parametrizations and several data acquisition experiences however the consistency issue has only been solved under restrictive assumptions regarding the identified system in this paper a new identification scheme is designed and shown to be consistent under mild assumptions

existence and properties of optimal production and inventory policies it is an important and fundamental question whether an optimal inventory policy exists for a general deterministic multi item multistage gdmm production and inventory model we show that an inventory model with a nonsubadditive ordering cost function could have no optimal policy over a finite horizon however when the ordering cost is a subadditive function an optimal policy exists for a gdmm production and inventory model over both a finite horizon and an infinite horizon properties of optimal inventory policies are crucial to investigating an inventory problem we present four properties of optimal policies 1 nonpositive inventory ordering 2 last minute ordering 3 extended last minute ordering and 4 nonnegative filling properties they have been explicitly or implicitly used in analyzing many different inventory models the last two properties have been used but not proved before

recursive identification for nonlinear arx systems based on stochastic approximation algorithm the nonparametric identification for nonlinear autoregressive systems with exogenous inputs narx described by yk 1 f yk yk 1 n0 uk uk 1 n0 ek 1 is considered first a condition on f is introduced to guarantee ergodicity and stationarity of yk then the kernel function based stochastic approximation algorithm with expanding truncations saawet is proposed to recursively estimate the value of f at any given y 1 y n0 u 1 u n0 r2n0 it is shown that the estimate converges to the true value with probability one in establishing the strong consistency of the estimate the properties of the markov chain associated with the narx system play an important role numerical examples are given which show that the simulation results are consistent with the theoretical analysis the intention of the paper is not only to present a concrete solution to the problem under consideration but also to profile a new analysis method for nonlinear systems the proposed method consisting in combining the markov chain properties with stochastic approximation algorithms may be of future potential although a restrictive condition has to be imposed on f that is the growth rate of f x should not be faster than linear with coefficient less than 1 as x tends to infinity

optimal risk probability for first passage models in semi markov decision processes this paper studies the risk minimization problem in semi markov decision processes with denumerable states the criterion to be optimized is the risk probability or risk function that a first passage time to some target set doesn t exceed a threshold value we first characterize such risk functions and the corresponding optimal value function and prove that the optimal value function satisfies the optimality equation by using a successive approximation technique then we present some properties of optimal policies and further give conditions for the existence of optimal policies in addition a value iteration algorithm and a policy improvement method for obtaining respectively the optimal value function and optimal policies are developed finally two examples are given to illustrate the value iteration procedure and essential characterization of the risk function

a max plus based fundamental solution for a class of discrete time linear regulator problems abstract efficient riccati equation based techniques for the approximate solution of discrete time linear regulator problems are restricted in their application to problems with quadratic terminal payoffs where non quadratic terminal payoffs are required these techniques fail due to the attendant non quadratic value functions involved in order to compute these non quadratic value functions it is often necessary to appeal directly to dynamic programming in the form of grid or element based iterations for the value function these iterations suffer from poor scalability with respect to problem dimension and time horizon in this paper a new max plus based method is developed for the approximate solution of discrete time linear regulator problems with non quadratic payoffs this new method is underpinned by the development of new fundamental solutions to such linear regulator problems via max plus duality in comparison with a typical grid based approach a substantial reduction in computational effort is observed in applying this new max plus method a number of simple examples are presented that illustrate this and other observations

robust h filtering for nonlinear systems abstract a robust or h approach to filtering for nonlinear systems is considered a bound on the estimate error as a function of the disturbance energy is obtained the corresponding dynamic programming equation is a first order pde this has computational ramifications the case where the measurements are discrete time is considered also a numerical method is discussed

the slater conundrum duality and pricing in infinite dimensional optimization duality theory is pervasive in finite dimensional optimization there is growing interest in solving infinite dimensional optimization problems and hence a corresponding interest in duality theory in infinite dimensions unfortunately many of the intuitions and interpretations common to finite dimensions do not extend to infinite dimensions in finite dimensions a dual solution is represented by a vector of dual prices that index the primal constraints and have a natural economic interpretation in infinite dimensions we show that this simple dual structure and its associated economic interpretation may fail to hold for a broad class of problems with constraint vector spaces that are riesz spaces ordered vector spaces with a lattice structure that either are sigma order complete or satisfy the projection property in these spaces we show that the existence of interior points required by common constraint qualifications for zero duality gap such as slater s condition implies the existence of

optimality of quasi open loop policies for discounted semi markov decision processes quasi open loop policies consist of sequences of markovian decision rules that are insensitive to one component of the state space given a semi markov decision process smdp we distinguish between exogenous and endogenous state components as follows i the decision maker s actions do not impact the evolution of an exogenous state component and ii between consecutive decision epochs the exogenous and endogenous state components are conditionally independent given the decision maker s latest action for simplicity we consider an smdp with one exogenous and one endogenous state component when transition times between epochs are conditionally independent of the exogenous state given the most recent action and the exogenous component is a multiplicative compound poisson process we provide an almost everywhere condition on the reward function sufficient for the optimality of a quasi open loop policy after adjusting the discount factor to account for the statistical properties of the exogenous state process obtaining this policy amounts to solving a reduced smdp in which the exogenous state is static depending on the relationship between the structure of the exogenous state process and the shape of the reward function we can replace the almost everywhere condition with one that applies only in expectation quasi open loop optimality holds even if the times between decision epochs depend on the poisson process underlying the exogenous state component and or the poisson process is replaced with a generic counting process

the joint replenishment problem with general joint cost structures we consider inventory systems with several distinct items demands occur at constant item specific rates the items are interdependent because of jointly incurred fixed procurement costs the joint cost structure reflects general economies of scale merely assuming a monotonicity and concavity submodularity property under a power of two policy each item is replenished with constant reorder intervals which are power of two multiples of some fixed or variable base planning period our main results include a proof that depending upon whether the base planning period is fixed or variable the best among all power of two policies has an average cost which comes within either 6 or 2 of an easily computable lower bound for the minimum cost value we also derive two efficient algorithms to compute an optimal power of two policy the proposed algorithms generate as a by product a specific cost allocation of the joint cost structure to the individual items with this specific allocation the problem with separable costs is in fact equivalent to the original problem with nonseparable joint costs in the sense that the two problems share the same sets of optimal power of two policies with identical associated long run average costs

a partial differential inequality for dissipative nonlinear systems abstract in this paper we show that storage functions for dissipative systems can be characterised by a partial differential inequality in the weak or viscosity sense the storage functions need not be differentiable this characterisation is stable with respect to perturbations we formulate a version of the positive real lemma for passive systems using the weak sense framework

cal h _ infty control of nonlinear systems differential games and viscosity solutions dealing with disturbances is one of the most important questions for controlled systems cal h infty optimal control theory is a deterministic way to tackle the problem in the presence of unfavorable disturbances the theory of differential games and the study of the associated hamilton jacobi isaacs equation appear to be basic tools of the theory we consider a general nonlinear system and prove that the existence of a continuous local viscosity supersolution of the isaacs equation corresponding to the cal h infty control problem is sufficient for its solvability we also show that the existence of a lower semicontinuous viscosity supersolution is necessary

average cost optimal policies for markov control processes with borel state space and unbounded costs abstract we show the existence of average cost optimal stationary policies for markov control processes with borel state space and unbounded costs per stage under a set of assumptions recently introduced by l i sennott 1989 for control processes with countable state space and finite control sets

finite horizon semi markov decision processes with application to maintenance systems abstract this paper investigates finite horizon semi markov decision processes with denumerable states the optimality is over the class of all randomized history dependent policies which include states and also planning horizons and the cost rate function is assumed to be bounded below under suitable conditions we show that the value function is a minimum nonnegative solution to the optimality equation and there exists an optimal policy moreover we develop an effective algorithm for computing optimal policies derive some properties of optimal policies and in addition illustrate our main results with a maintenance system

average optimality in markov control processes via discounted costproblems and linear programming this paper shows the existence of solutions to the average cost problem for markov control processes on borel spaces with possibly unbounded costs and noncompact control constraint sets this is done via a combination of the well known vanishing discount approach and recent results on the linear programming formulation for both discounted and average cost markov control problems

stochastic approximation in nonparametric identification of hammerstein systems derived from the idea of stochastic approximation recursive algorithms to identify a hammerstein system are presented two of them recover the characteristic of the nonlinear memoryless subsystem while the third one estimates the impulse response of the linear dynamic part the a priori information about both subsystems is nonparametric consistency in quadratic mean is shown and the convergence rate is examined results of numerical simulation are also presented

l sub 2 gain analysis of nonlinear systems and nonlinear state feedback h sub infinity control previously obtained results on l2 gain analysis of smooth nonlinear systems are unified and extended using an approach based on hamilton jacobi equations and inequalities and their relation to invariant manifolds of an associated hamiltonian vector field on the basis of these results a nonlinear analog is obtained of the simplest part of a state space approach to linear h sub infinity control namely the state feedback h sub infinity optimal control problem furthermore the relation with h sub infinity control of the linearized system is dealt with

linear programming and average optimality of markov control processes on borel spaces unbounded costs this paper is concerned with the linear programming formulation of markov control processes with borel state and action spaces and the average cost ac criterion the one stage cost function may be unbounded a linear program ep and its dual ep are introduced their values inf ep and sup ep bound the value say inf ac of the ac problem i e sup ep leq inf ac leq inf ep conditions are provided for the existence of no duality gap viz sup ep inf ep and also for strong duality so that both ep and ep are solvable and their optimal values satisfy max ep min ep the latter implies i the existence of an ac optimal control policy and that ii the ac optimality equation holds almost everywhere these results are applied to a general vector valued additive noise system with quadratic costs

a general framework for nonparametric identification of nonlinear stochastic systems in this paper nonparametric identification of nonlinear autoregressive systems with exogenous inputs narx is considered a general criterion function is introduced for estimating the value of the nonlinear function within the system at any fixed point the criterion function is constructed using a kernel together with a convex objective function not only does this framework include the classical kernel based weighted least squares estimator but also the kernel based l q q geq1 criteria as special cases first we prove that the minimizer of the general criterion function converges to the true function value with probability one second recursive algorithms are proposed to find the estimates which minimize the criterion function and it is shown that these estimates also converge to the true function value with probability one numerical examples are given justifying that the framework guarantees the strong consistency of the estimates and exhibits the robustness against outliers in the observations

recursive prediction error identification and scaling of non linear state space models using a restricted black box parameterization a recursive prediction error algorithm for identification of systems described by non linear ordinary differential equation ode models is presented the model is an ode model parameterized with coefficients of a multi variable polynomial that describes one component of the right hand side function of the ode this avoids over parameterization problems the selected model can also handle systems with more complicated right hand side structure by identification of a local input output equivalent system in the coordinate system of the selected state variables a novel technique based on scaling of the sampling period is proposed the technique can improve the conditioning of the identification problem thereby enhancing the chances of convergence to the correct minimum of the criterion the algorithm is applied to live data from a system consisting of two cascaded tanks with promising results a matlab software package which implements the proposed algorithm and a set of support scripts can be freely downloaded from http www it uu se research reports

a dynamic traveling salesman problem with stochastic arc costs we propose a dynamic traveling salesman problem tsp with stochastic arc costs motivated by applications such as dynamic vehicle routing in which the cost of a decision is known only probabilistically beforehand but is revealed dynamically before the decision is executed we formulate this as a dynamic program dp and compare it to static counterparts to demonstrate the advantage of the dynamic paradigm over an a priori approach we then apply approximate linear programming alp to overcome the dp s curse of dimensionality obtain a semi infinite linear programming lower bound and discuss its tractability we also analyze a rollout version of the price directed policy implied by our alp and derive worst case guarantees for its performance our computational study demonstrates the quality of a heuristically modified rollout policy using a computationally effective a posteriori bound

hard frequency domain model error bounds from least squares like identification techniques the problem of deriving so called hard error bounds for estimated transfer functions is addressed a hard bound is one that is sure to be satisfied i e the true system nyquist plot will be confined with certainty to a given region provided that the underlying assumptions are satisfied by blending a priori knowledge and information obtained from measured data it is shown how the uncertainty of transfer function estimates can be quantified the emphasis is on errors due to model mismatch the effects of unmodeled dynamics can be considered as bounded disturbances hence techniques from set membership identification can be applied to this problem the approach taken corresponds to weighted least squares estimation and provides hard frequency domain transfer function error bounds the main assumptions used in the current contribution are that the measurement errors are bounded that the true system is indeed linear with a certain degree of stability and that there is some knowledge about the shape of the true frequency response

a 98 effective lot sizing rule for a multi product multi stage production inventory system we study a multi product multi stage production inventory system in continuous time in process and finished goods are referred to as products and inventories of a single item held at different locations are treated as different products external demand can occur for any or all of the products at a constant product dependent rate when an order is placed for a product it is instantly delivered the set up cost is a function of the set of products being ordered at that point in time the goal is to schedule orders for each of the products over an infinite horizon so as to minimize the long run average cost we define a new class of policies in which each product uses a stationary interval of time between successive orders and the ratio of the order intervals of any two products is an integer power of two we show that there is always a policy in this class that is within 2 of optimal this extends a similar result for the one warehouse multi retailer system for assembly systems we show how to compute such a policy in 0n log n time for general systems a sequence of max flow min cut computations is required the algorithm is efficient for very large systems

the max plus finite element method for solving deterministic optimal control problems basic properties and convergence analysis we introduce a max plus analogue of the petrov galerkin finite element method to solve finite horizon deterministic optimal control problems the method relies on a max plus variational formulation we show that the error in the sup norm can be bounded from the difference between the value function and its projections on max plus and min plus semimodules when the max plus analogue of the stiffness matrix is exactly known in general the stiffness matrix must be approximated this requires approximating the operation of the lax oleinik semigroup on finite elements we consider two approximations relying on the hamiltonian we derive a convergence result in arbitrary dimension showing that for a class of problems the error estimate is of order delta delta x delta 1 or sqrt delta delta x delta 1 depending on the choice of the approximation where delta and delta x are respectively the time and space discretization steps we compare our method with another max plus based discretization method previously introduced by fleming and mceneaney we give numerical examples in dimensions 1 and 2

optimal toll design a lower bound framework for the asymmetric traveling salesman problem we propose a framework of lower bounds for the asymmetric traveling salesman problem tsp based on approximating the dynamic programming formulation with different basis vector sets we discuss how several well known tsp lower bounds correspond to intuitive basis vector choices and give an economic interpretation wherein the salesman must pay tolls as he travels between cities we then introduce an exact reformulation that generates a family of successively tighter lower bounds all solvable in polynomial time we show that the base member of this family yields a bound greater than or equal to the well known held karp bound obtained by solving the linear programming relaxation of the tsp s integer programming arc based formulation

existence of optimal policies for semi markov decision processes using duality for infinite linear programming semi markov decision processes on borel spaces with deterministic kernels have many practical applications particularly in inventory theory most of the results from general semi markov decision processes do not carry over to a deterministic kernel since such a kernel does not provide smoothness we develop infinite dimensional linear programming theory for a general stochastic semi markov decision process we give conditions general enough to allow deterministic kernels for solvability and strong duality of the resulting linear programs by using the developed linear programming theory we give conditions for the existence of a stationary deterministic policy for deterministic kernels which is optimal among all possible policies

a max plus based algorithm for a hamilton jacobi bellman equation of nonlinear filtering the hamilton jacobi bellman hjb equation associated with the robust hinfty filter as well as the mortensen filter is considered these filters employ a model where the disturbances have finite power the hjb equation for the filter information state is a first order equation with a term that is quadratic in the gradient yet the solution operator is linear in the max plus algebra this property is exploited by the development of a numerical algorithm where the effect of the solution operator on a set of basis functions is computed off line the precomputed solutions are stored as vectors of coefficients of the basis functions these coefficients are then used directly in the real time computations

learning theory estimates with observations from general stationary stochastic processes this letter investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes here by general we mean that many stationary stochastic processes can be included we show that when the stochastic processes satisfy a generalized bernstein type inequality a unified treatment on analyzing the learning schemes with various mixing processes can be conducted and a sharp oracle inequality for generic regularized empirical risk minimization schemes can be established the obtained oracle inequality is then applied to derive convergence rates for several learning schemes such as empirical risk minimization erm least squares support vector machines ls svms using given generic kernels and svms using gaussian kernels for both least squares and quantile regression it turns out that for independent and identically distributed i i d processes our learning rates for erm recover the optimal rates for non i i d processes including geometrically mixing markov processes geometrically mixing processes with restricted decay mixing processes and time reversed geometrically mixing processes our learning rates for svms with gaussian kernels match up to some arbitrarily small extra term in the exponent the optimal rates for the remaining cases our rates are at least close to the optimal rates as a by product the assumed generalized bernstein type inequality also provides an interpretation of the so called effective number of observations for various mixing processes

performance bounds for nonlinear systems with a nonlinear ℒ 2 gain property nonlinear 2 gain is a finite gain concept that generalises the notion of conventional linear finite 2 gain to admit the application of 2 gain analysis tools of a broader class of nonlinear systems the computation of tight comparison function bounds for this nonlinear 2 gain property is important in applications such as small gain design this article presents an approximation framework for these comparison function bounds through the formulation and solution of an optimal control problem key to the solution of this problem is the lifting of an 2 norm input constraint which is facilitated via the introduction of an energy saturation operator this admits the solution of the optimal control problem of interest via dynamic programming and associated numerical methods leading to the computation of the proposed bounds two examples are presented to demonstrate this approach

system identification of nonlinear state space models this paper is concerned with the parameter estimation of a general class of nonlinear dynamic systems in state space form more specifically a maximum likelihood ml framework is employed and an expectation maximisation em algorithm is derived to compute these ml estimates the expectation e step involves solving a nonlinear state estimation problem where the smoothed estimates of the states are required this problem lends itself perfectly to the particle smoother which provides arbitrarily good estimates the maximisation m step is solved using standard techniques from numerical optimisation theory simulation examples demonstrate the efficacy of our proposed solution

dual based methods for solving infinite horizon nonstationary deterministic dynamic programs we develop novel dual ascent and primal dual methods to solve infinite horizon nonstationary deterministic dynamic programs these methods are finitely implementable and converge in value to optimality moreover the dual ascent method produces a sequence of improving dual solutions that pointwise converge to an optimal dual solution while the primal dual algorithm provides a sequence of primal basic feasible solutions with value error bounds from optimality that converge to zero our dual based methods work on a more general class of infinite network flow problems that include the shortest path formulation of dynamic programs as a special case to our knowledge these are the first dual based methods proposed in the literature to solve infinite horizon nonstationary deterministic dynamic programs

an infinite dimensional linear programming algorithm for deterministic semi markov decision processes on borel spaces we devise an algorithm for solving the infinite dimensional linear programs that arise from general deterministic semi markov decision processes on borel spaces the algorithm constructs a sequence of approximate primal dual solutions that converge to an optimal one the innovative idea is to approximate the dual solution with continuous piecewise linear ridge functions that naturally represent functions defined on a high dimensional domain as linear combinations of functions defined on only a single dimension this approximation gives rise to a primal dual pair of semi infinite programs for which we show strong duality in addition we prove various properties of the underlying ridge functions

exact computation of optimal inventory policies over an unbounded horizon an inventory scheduling model with forbidden time intervals is analyzed the objective is to minimize the long term average cost per time unit unlike most of the literature on inventory theory no restrictive assumptions are made a priori about the nature of optimal solutions rather it is proved that optimal policies exist and that some of them are cyclic with cycles of a particular structure it is then shown that such optimal policies can be computed and an algorithm is given

a shadow simplex method for infinite linear programs we present a simplex type algorithm that is an algorithm that moves from one extreme point of the infinite dimensional feasible region to another not necessarily adjacent extreme point for solving a class of linear programs with countably infinite variables and constraints each iteration of this method can be implemented in finite time whereas the solution values converge to the optimal value as the number of iterations increases this simplex type algorithm moves to an adjacent extreme point and hence reduces to a true infinite dimensional simplex method for the important special cases of nonstationary infinite horizon deterministic and stochastic dynamic programs

establishing consistent and realistic reorder intervals in production distribution systems the objective of this paper is to present a model and algorithm that can be used to find consistent and realistic reorder intervals for each item in large scale production distribution systems we assume such systems can be represented by directed acyclic graphs demand for each end item is assumed to occur at a constant and continuous rate production is instantaneous and no backorders are allowed both fixed setup costs and echelon holding costs are charged at each stage we limit our attention to nested and stationary policies furthermore we restrict the reorder interval for each stage to be a power of 2 times a base planning period the model that results from these assumptions is an integer nonlinear programming problem the optimal solution can be found using the proposed algorithm which is a polynomial time algorithm a real world example is given to illustrate the procedure

convergence of the optimal feedback policies in a numerical method for a class of deterministic optimal control problems we consider a markov chain based numerical approximation method for a class of deterministic nonlinear optimal control problems it is known that methods of this type yield convergent approximations to the value function on the entire domain these results do not easily extend to the optimal control which need not be uniquely defined on the entire domain there are however regions of strong regularity on which the optimal control is well defined and smooth typically the union of these regions is open and dense in the domain using probabilistic methods we prove that on the regions of strong regularity the markov chain method yields a convergent sequence of approximations to the optimal feedback control the result is illustrated with several examples

robustness and risk sensitive filtering this paper gives a precise meaning to the robustness of risk sensitive filters for problems in which one is uncertain as to the exact value of the probability model it is shown that risk sensitive estimators including filters enjoy an error bound which is the sum of two terms the first of which coincides with an upper bound on the error that one would obtain if one knew exactly the underlying probability model while the second term is a measure of the distance between the true and design probability models the paper includes a discussion of several approaches to estimation including h sub spl infin filtering

an approximate dynamic programming approach to the joint replenishment problem the main contribution of this paper is to propose a new dynamic programming approach that e approximates the joint replenishment problem with stationary demands and holding costs in its discrete time finite horizon setting our first and foremost objective is to show that the computation time of classical dynamic programming algorithms can be improved on by orders of magnitude when one is willing to lose an e factor in optimality based on synthesizing ideas such as commodity aggregation approximate dynamic programming and a few guessing tricks we show that one can attain any required degree of accuracy in near polynomial time

zero sum average semi markov games fixed point solutions of the shapley equation this paper deals with zero sum average semi markov games with borel state and action spaces unbounded payoffs and mean holding times a solution to the shapley equation is obtained via the banach fixed point theorem assuming that the model satisfies a lyapunov like condition a growth hypothesis on the payoff function and the mean holding times besides standard continuity and compactness requirements

max plus eigenvector representations for solution of nonlinear h sub spl infin problems basic concepts the h sub spl infin problem for a nonlinear system is considered the corresponding dynamic programming equation is a fully nonlinear first order steady state partial differential equation pde possessing a term which is quadratic in the gradient the solutions are typically nonsmooth and further there is nonuniqueness among the class of viscosity solutions in the case where one tests a feedback control to see if it yields an h sub spl infin controller the pde is a hamilton jacobi bellman equation in the case where the optimal feedback control is being determined as well the problem takes the form of a differential game and the pde is in general an isaacs equation the computation of the solution of a nonlinear steady state first order pde is typically quite difficult in this paper we develop an entirely new class of methods for obtaining the correct solution of such pdes these methods are based on the linearity of the associated semigroup over the max plus or in some cases min plus algebra in particular solution of the pde is reduced to solution of a max plus or min plus eigenvector problem for known unique eigenvalue 0 the max plus multiplicative identity it is demonstrated that the eigenvector is unique and that the power method converges to it an example is included

a flexible state space model for learning nonlinear dynamical systems we consider a nonlinear state space model with the state transition and observation functions expressed as basis function expansions the coefficients in the basis function expansions are learned f

markov chain approximations for deterministic control problems with affine dynamics and quadratic cost in the control we consider the construction of markov chain approximations for an important class of deterministic control problems the emphasis is on the construction of schemes that can be easily implemented and which possess a number of highly desirable qualitative properties the class of problems covered is that for which the control is affine in the dynamics and with quadratic running cost this class covers a number of interesting areas of application including problems that arise in large deviations risk sensitive and robust control robust filtering and certain problems in computer vision examples are given as well as a proof of convergence

duality and existence of optimal policies in generalized joint replenishment we establish a duality theory for a broad class of deterministic inventory control problems on continuous spaces that includes the classical joint replenishment problem and inventory routing using this theory we establish the existence of an optimal policy which has been an open question we show how a primal dual pair of infinite dimensional linear programs encode both cyclic and noncyclic schedules and provide various results regarding cyclic schedules including an example showing that they need not be optimal

max plus summation of fenchel transformed semigroups for solution of nonlinear bellman equations abstract max plus methods have been explored for solution of first order nonlinear hamilton jacobi bellman partial differential equations hjb pdes and corresponding nonlinear control problems these methods exploit the max plus linearity of the associated semigroups in particular although the problems are nonlinear the semigroups are linear in the max plus sense these methods have been used successfully to compute solutions although they provide certain advantages they still generally suffer from the curse of dimensionality the natural analog to the laplace transform in ordinary spaces is the fenchel transform over max plus spaces the range space being referred to as the dual space one can transform the semigroup operators into operators on the dual space there are natural operations on the transformed operators which may be used to construct solutions of nonlinear control problems natural building blocks correspond to transforms of operators for linear quadratic problems in this paper a method for exploiting operations in the fenchel transform space is used to develop a method for certain problems such that the computational growth in the most time consuming portion of the computations can be hugely reduced although the curse of dimensionality is not entirely avoided the computational cost reductions are very high for some classes of problems

infinite linear programming and multichain markov control processes in uncountable spaces in this paper we use infinite linear programming to study markov control processes in borel spaces and the average cost criterion in the unichain and multichain cases under appropriate assumptions we show that in both cases the associated linear programs are solvable and that there is no duality gap moreover conditions are given for minimizing respectively maximizing sequences for the primal respectively dual programs to converge to optimal solutions

representations of non linear systems the narmax model abstract input output representations of non linear discrete time systems are discussed it is shown that the narmax non linear autoregressive moving average with exogenous inputs model is a general and natural representation of non linear systems and contains as special cases several existing non linear models the problem of approximating non linear input output systems is also addressed and several properties of non linear models are highlighted using simple examples