control of singularly perturbed hybrid stochastic systems we study a class of optimal stochastic control problems involving two different time scales the fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process under a fundamental ergodicity property for a class of infinitesimal control systems associated with the fast mode we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system both the finite and infinite discounted horizon cases are considered we show how an approximate optimal control law can be constructed from the solution of the limit control problem in the particular case where the infinitesimal control systems possess the so called turnpike property i e characterized by the existence of global attractors the limit control problem can be given an interpretation related to a decomposition approach

causal inference through a witness protection program one of the most fundamental problems in causal inference is the estimation of a causal effect when treatment and outcome are confounded this is difficult in an observational study because one has no direct evidence that all confounders have been adjusted for we introduce a novel approach for estimating causal effects that exploits observational conditional independencies to suggest weak paths in an unknown causal graph the widely used faithfulness condition of spirtes et al is relaxed to allow for varying degrees of path cancellations that imply conditional independencies but do not rule out the existence of confounding causal paths the output is a posterior distribution over bounds on the average causal effect via a linear programming approach and bayesian inference we claim this approach should be used in regular practice as a complement to other tools in observational studies

spatial prediction of soil salinity using electromagnetic induction techniques 1 statistical prediction models a comparison of multiple linear regression and cokriging we describe a regression based statistical methodology suitable for predicting field scale spatial salinity ec conditions from rapidly acquired electromagnetic induction ec data this technique uses multiple linear regression mlr models to estimate soil salinity from ec survey data the mlr models incorporate multiple ec measurements and trend surface parameters to increase the prediction accuracy and can be fitted from limited amounts of ec calibration data this estimation technique is compared to some commonly recommended cokriging techniques with respect to statistical modeling assumptions calibration sample size requirements and prediction capabilities we show that mlr models are theoretically equivalent to and cost effective relative to cokriging for estimating a spatially distributed random variable when the residuals from the regression model are spatially uncorrelated mlr modeling and prediction techniques are demonstrated with data from three salinity surveys

numerical solutions for stochastic differential games with regime switching this paper is concerned with numerical methods for stochastic differential games of regime switching diffusions numerical methods using markov chain approximation techniques are developed a new proof of the existence of a saddle point for the stochastic differential game is provided this new proof enables us to treat certain systems with nonseparable in controls structure convergence of the algorithms is derived by means of weak convergence methods in addition examples are also provided for demonstration purposes

hierarchical production control in a stochastic n machine flowshop with long run average cost abstract this paper presents an asymptotic analysis of stochastic manufacturing systems consisting of machines in tandem subject to breakdown and repair and facing a constant demand as the rates of change of the machine states approach infinity this situation gives rise to a limiting problem in which the stochastic machine availability is replaced by its equilibrium mean availability the long run average cost for the original problem converges to the long run average cost of the limiting problem a method of shrinking and entire lifting is introduced in order to construct the near optimal controls for the original problem by using near optimal controls of the limiting problem the convergence rate of the long run average cost for the original problem to that of the limiting problem is established this helps in providing an error estimate for the constructed open loop asymptotic optimal control

controlled diffusions with constraints controlled diffusion problems with classical cost structures ergodic discounted etc are considered when finitely many additional cost functionals of the same type are required to lie in prescribed intervals existence of an optimal markov control is proved by convex analytic methods

adaptive fault tolerant stabilization for nonlinear systems with markovian jumping actuator failures and stochastic noises in this paper we consider the fault tolerant stabilization problem for a class of nonlinear systems with uncertain parameters uncertainties caused by markovian jumping actuator failures and stochastic noises are also taken into consideration different from most existing results the number of actuator failures may be infinite and stochastic functions related to multi markovian variables have been introduced to denote the failure scaling factors for the actuators which is practical but challenging three main difficulties arise first is how to establish fundamentals for systems involving multi markovian variables and stochastic noises including the joint transition probability the infinitesimal generator the existence and uniqueness of the solution and so on second is how to handle the extra transition rate related terms appearing in the infinitesimal generator of the lyapunov function last is how to cope with the involved higher order hessian term in the ito stochastic differentiation by proposing a new adaptive fault tolerant control scheme the boundedness in probability of all the closed loop signals has been ensured an example of altitude fault tolerant control for a generic hypersonic air vehicle is presented to show the effectiveness of the proposed scheme

model predictive control of discrete hybrid stochastic automata this paper focuses on optimal and receding horizon control of a class of hybrid dynamical systems called discrete hybrid stochastic automata dhsa whose discrete state transitions depend on both deterministic and stochastic events a finite time optimal control approach optimistically determines the trajectory that provides the best tradeoff between tracking performance and the probability of the trajectory to actually execute under possible chance constraints the approach is also robustified less optimistically to ensure that the system satisfies a set of constraints for all possible realizations of the stochastic events or alternatively for those having enough probability to realize sufficient conditions for asymptotic convergence in probability are given for the receding horizon implementation of the optimal control solution the effectiveness of the suggested stochastic hybrid control techniques is shown on a case study in supply chain management

addendum on the scoring of gaussian directed acyclic graphical models we provide a correction to the expression for scoring gaussian directed acyclic graphical models derived in geiger and heckerman 2002 annals of statistics 20 1414 1440 and discuss how to evaluate the score efficiently

on reachability and stabilization of switched linear systems reachability and stabilization issues for switched linear control systems are addressed a necessary and a sufficient condition are presented for reachability under mild assumptions we prove that the switched linear control systems are stabilizable in addition we show that an element driven switching stabilization strategy can be explicitly constructed

optimal control of stochastic hybrid system with jumps a numerical approximation the generalized class of stochastic hybrid systems consists of models with regime changes including the occurrence of impulsive behavior in this paper the stochastic hybrid processes with jumps are approximated by locally consistent markov decision processes that preserve local mean and covariance we further apply a randomized switching policy for approximating the dynamics on the switching boundaries to investigate the validity of the approximation we study a stochastic optimal control problem on the basis of the discrete approximation we solve the optimal control problem and show that the solution obtained from the discretized problem converges to the solution of the original optimal control problem

a new criterion for confounder selection we propose a new criterion for confounder selection when the underlying causal structure is unknown and only limited knowledge is available we assume all covariates being considered are pretreatment variables and that for each covariate it is known i whether the covariate is a cause of treatment and ii whether the covariate is a cause of the outcome the causal relationships the covariates have with one another is assumed unknown we propose that control be made for any covariate that is either a cause of treatment or of the outcome or both we show that irrespective of the actual underlying causal structure if any subset of the observed covariates suffices to control for confounding then the set of covariates chosen by our criterion will also suffice we show that other commonly used criteria for confounding control do not have this property we use formal theory concerning causal diagrams to prove our result but the application of the result does not rely on familiarity with causal diagrams an investigator simply need ask is the covariate a cause of the treatment and is the covariate a cause of the outcome if the answer to either question is yes then the covariate is included for confounder control we discuss some additional covariate selection results that preserve unconfoundedness and that may be of interest when used with our criterion

observer design for nonlinear systems with discrete time measurements this paper focuses on the development of asymptotic observers for nonlinear discrete time systems it is argued that instead of trying to imitate the linear observer theory the problem of constructing a nonlinear observer can be more fruitfully studied in the context of solving simultaneous nonlinear equations in particular it is shown that the discrete newton method properly interpreted yields an asymptotic observer for a large class of discrete time systems while the continuous newton method may be employed to obtain a global observer furthermore it is analyzed how the use of broyden s method in the observer structure affects the observer s performance and its computational complexity an example illustrates some aspects of the proposed methods moreover it serves to show that these methods apply equally well to discrete time systems and to continuous time systems with sampled outputs

pc algorithm for nonparanormal graphical models the pc algorithm uses conditional independence tests for model selection in graphical modeling with acyclic directed graphs in gaussian models tests of conditional independence are typically based on pearson correlations and high dimensional consistency results have been obtained for the pc algorithm in this setting analyzing the error propagation from marginal to partial correlations we prove that high dimensional consistency carries over to a broader class of gaussian copula or nonparanormal models when using rank based measures of correlation for graph sequences with bounded degree our consistency result is as strong as prior gaussian results in simulations the rank pc algorithm works as well as the pearson pc algorithm for normal data and considerably better for non normal data all the while incurring a negligible increase of computation time while our interest is in the pc algorithm the presented analysis of error propagation could be applied to other algorithms that test the vanishing of low order partial correlations

uniform consistency in causal inference there is a long tradition of representing causal relationships by directed acyclic graphs wright 1934 spirtes 1994 spirtes et al 1993 and pearl verma 1991 describe procedures for inferring the presence or absence of causal arrows in the graph even if there might be unobserved confounding variables and or an unknown time order and that under weak conditions for certain combinations of directed acyclic graphs and probability distributions are asymptotically in sample size consistent these results are surprising since they seem to contradict the standard statistical wisdom that consistent estimators of causal effects do not exist for nonrandomised studies if there are potentially unobserved confounding variables we resolve the apparent incompatibility of these views by closely examining the asymptotic properties of these causal inference procedures we show that the asymptotically consistent procedures are pointwise consistent but uniformly consistent tests do not exist thus no finite sample size can ever be guaranteed to approximate the asymptotic results we also show the nonexistence of valid consistent confidence intervals for causal effects and the nonexistence of uniformly consistent point estimators our results make no assumption about the form of the tests or estimators in particular the tests could be classical independence tests they could be bayes tests or they could be tests based on scoring methods such as bic or aic the implications of our results for observational studies are controversial and are discussed briefly in the last section of the paper the results hinge on the following fact it is possible to find for each sample size n distributions p and q such that p and q are empirically indistinguishable and yet p and q correspond to different causal effects

differential algebraic approach in non linear dynamical compensator design for d c to d c power converters abstract a non linear dynamical compensator design based on fliess generalized controller canonical form for non linear dynamical systems is proposed for the asymptotic stabilization of output load voltage or current in pulse width modulation controlled d c to d c power supplies the technique rests on the recently introduced differential algebraic approach to the study of controlled dynamical systems

bounds on treatment effects from studies with imperfect compliance abstract this article establishes nonparametric formulas that can be used to bound the average treatment effect in experimental studies in which treatment assignment is random but subject compliance is imperfect the bounds provided are the tightest possible given the distribution of assignments treatments and responses the formulas show that even with high rates of noncompliance experimental data can yield useful and sometimes accurate information on the average effect of a treatment on the population

spectral bayesian estimation for general stochastic hybrid systems abstract general stochastic hybrid systems gshs have been formulated to represent various types of uncertainties in hybrid dynamical systems in this paper we propose computational techniques for bayesian estimation of gshs in particular the fokker planck equation that describes the evolution of uncertainty distributions along gshs is solved by spectral techniques where an arbitrary form of probability density of the hybrid state is represented by a mixture of fourier series the method is based on splitting the fokker planck equation represented by an integro partial differential equation into the partial differentiation part for continuous diffusion and the integral part for discrete transition and integrating the solution of each part the propagated density function is used with a likelihood function in the bayes formula to estimate the hybrid state for given sensor measurements the unique feature of the proposed technique is that the probability density describing complete stochastic properties of the hybrid state is constructed without relying on the common gaussian assumption in contrast to other methods that compute certain expected values such as mean or variance we apply the proposed method to the estimation of two examples the bouncing ball model and the dubins vehicle model we show that the proposed technique yields propagated densities consistent with monte carlo simulations more accurate estimates over the gaussian based approach and some computational benefits over a particle filter

stochastic production planning with production constraints this paper considers an infinite horizon stochastic production planning problem with the constraint that production rate must be nonnegative it is shown that an optimal feedback solution exists for the problem moreover this solution is characterized and is then compared with the solution of the unconstrained problem also obtained by using a policy iteration procedure are computational solutions to the related problems with upper bounds on the production rate which determines production rates over time to minimize an integral representing a discounted quadratic loss function the loss function is defined in terms of the deviations of production and inventory levels from their rated or factory optimal values the model is solved both with and without nonnegative production constraints a stochastic extension of this paper involving a white noise process 1 is analyzed by sethi and thompson 9 10 closed form solutions for optimal feedback production policy for both finite and infinite horizon versions of the model without production constraints are obtained in particular the model must allow negative production rates or disposals in this paper we consider the stochastic production planning problem with the constraint that production rates must be nonnegative only the infinite horizon problem is treated it is shown that an optimal feedback solution exists for the problem this solution is characterized also obtained by using a policy iteration procedure are computational solutions to the related problems with upper bounds on the production rate

a two factor stochastic production model with two time scales a two factor production model where one factor is characterized by a continuous state variable and corresponds to the fast mode via a controlled diffusion process whereas the second factor is characterized by a discrete variable and corresponds to the slow mode via a controlled jump process

transition time optimization for switched mode dynamical systems this note considers the problem of determining optimal switching times at which mode transitions should occur in multimodal hybrid systems it derives a simple formula for the gradient of the cost functional with respect to the switching times and uses it in a gradient descent algorithm much of the analysis is carried out in the setting of optimization problems involving fixed switching mode sequences but a possible extension is pointed out for the case where the switching mode sequence is a part of the variable numerical examples testify to the viability of the proposed approach

hierarchical trajectory optimization for a class of hybrid dynamical systems we discuss trajectory optimization for a class of hybrid systems with a natural hierarchical separation of discrete and continuous dynamics where the continuous dynamics do not change with the discrete state the trajectory optimization problem considered requires that a discrete state sequence and a continuous state trajectory must be both determined to minimize a single cost function such that the discrete state sequence also solves a symbolic planning problem we model this symbolic planning problem as a search on a planning graph and we introduce a family of graphs called lifted planning graphs parametrized by an integer h we define a family of continuous state trajectory optimization problems and associate them with edge costs in the lifted planning graphs next we present an algorithm for finding an optimal solution to the hybrid trajectory optimization problem which includes mapping paths in the lifted planning graphs to discrete state sequences and continuous state trajectories we show that the cost of optimal hybrid trajectories is a nonincreasing function of h and that there exists a finite h for which this cost attains a minimum we illustrate the proposed algorithm with numerical simulation results for two application examples an autonomous mobile vehicle and an autonomous robotic manipulator

bounds on direct effects in the presence of confounded intermediate variables this article considers the problem of estimating the average controlled direct effect acde of a treatment on an outcome in the presence of unmeasured confounders between an intermediate variable and the outcome such confounders render the direct effect unidentifiable even in cases where the total effect is unconfounded hence identifiable kaufman et al 2005 statistics in medicine 24 1683 1702 applied a linear programming software to find the minimum and maximum possible values of the acde for specific numerical data in this article we apply the symbolic balke pearl 1997 journal of the american statistical association 92 1171 1176 linear programming method to derive closed form formulas for the upper and lower bounds on the acde under various assumptions of monotonicity these universal bounds enable clinical experimenters to assess the direct effect of treatment from observed data with minimum computational effort and they further shed light on the sign of the direct effect and the accuracy of the assessments

modeling subdiffusion using reaction diffusion systems in many types of media particularly within living cells or within their membranes diffusing species do not follow fick s laws but instead show transient subdiffusive behavior formulating spatiotemporal models that take this behavior into account is a delicate matter as one is faced with the choice of resorting either to fractional calculus or to microscopic descriptions in this article we provide an alternative designed to be easier to handle analytically and numerically than the existing approaches specifically starting from the continuous time random walk model we construct linear reaction diffusion systems that can be used as components within such a model and which capture the defining properties of subdiffusion we show how to impose physically relevant parameters and prove stability and mass conservation while applications to cellular biology are our main motivation our approach is abstract and should thus be applicable to any situation where anomalous subdiffusion is observed

stability of degenerate diffusions with state dependent switching this paper deals with the ergodic properties of hybrid systems modelled by diffusion processes with state dependent switching we investigate the sufficient conditions expressed in terms of the parameters of the underlying process which would ensure the existence of a unique invariant probability and stability in distribution of the flow it turns out that the conditions would depend on certain averaging mechanisms over the states of the discrete component of the hybrid system

discrete mode observability of structured switching descriptor linear systems a graph theoretic approach abstract the main result of the paper consists in necessary and sufficient graphical conditions which ensure the generic discrete mode observability of structured switching descriptor systems the methods used in the previous studies on the observability of switching linear systems on standard form are not applicable to switching descriptor systems so we develop a new approach starting from bipartite representations of these systems and then building a new kind of digraph dedicated to the discrete mode observability study the proposed method assumes only the knowledge of the system s structure and is applicable to a large class of descriptor systems including regular and non regular systems even if they are square or under determined the provided conditions can be implemented by the classical graph theory algorithms

the lagrange and the vanishing discount techniques to controlled diffusions with cost constraints abstract in this paper we introduce two useful methods to compute optimal control policies for either the discounted or the average payoff criterion with cost constraints when the dynamic system evolves as a n dimensional diffusion processes as for the attribute cost constraints we mean the coexistence of a given cost function that in general is dominated above by another function in particular by a constant playing the role of an extra constraint in the control model to deduce optimality results for the discounted case we employ the lagrange multipliers technique along with dynamic programming arguments then the vanishing discount method is applied to easily obtain average optimal policies we support our theory by providing an example on pollution accumulation problem

on invariant probability measures of regime switching diffusion processes with singular drifts abstract under integrability conditions of the coefficients the existence uniqueness and regularity estimates are derived for the invariant probability measure of regime switching diffusion processes moreover we study the l 1 uniqueness of the semigroup generated by regime switching diffusion processes as an application the l 1 uniqueness of the extension of weakly coupled elliptic systems is obtained

disjunctive kriging 1 overview of estimation and conditional probability the disjunctive kriging method described in this paper produces a nonlinear unbiased estimator with the characteristic minimum variance of errors disjunctive kriging is as good or otherwise better than linear estimators in the sense of reduced kriging variance and exactness of estimation it does not suffer from the difficulties associated with computing the conditional expectation and can be thought of as its estimator disjunctive kriging also provides an estimate of the conditional probability that a random variable located at a point or averaged over a block in two dimensional space is above some specified cutoff or tolerance level and this can be written in terms of the probability distribution or the density function the method has important implications in aiding management decisions by providing a quantitative input which is not readily obtained from the linear kriging estimators based on the available data which is the best nonlinear unbiased estimator short of the conditional expectation a major disadvantage in using disjunctive kriging is the increased computational time this however is mitigated by increased information about the estimate

robust stability and stabilization of linear stochastic systems with markovian switching and uncertain transition rates abstract this paper is concerned with the robust stabilization problem for a class of linear uncertain stochastic systems with markovian switching the uncertain stochastic system with markovian switching under consideration involves parameter uncertainties both in the system matrices and in the mode transition rates matrix new criteria for testing the robust stability of such systems are established in terms of bi linear matrix inequalities blmis and sufficient conditions are proposed for the design of robust state feedback controllers a numerical example is given to illustrate the effectiveness of our results

on line optimization of switched mode dynamical systems this paper considers an optimization problem in the setting of switched mode hybrid dynamical systems where the control variable independent variable consists of the mode switching times and the performance criterion is a cost functional defined on the system s state trajectory the system is deterministic nonlinear and autonomous and its state variable cannot be measured and hence it has to be estimated we propose an on line newton like optimization algorithm that recomputes the control variable by attempting to optimize the cost to go at equally spaced epochs the main result concerns the algorithm s convergence rate which can vary from sublinear to quadratic depending on its computing rate and the state estimation error

stochastic stabilization of hybrid differential equations this paper aims to determine whether or not a stochastic state feedback control can stabilize a given linear or nonlinear hybrid system new methods are developed and sufficient conditions on the stability for hybrid stochastic differential equations are provided these results are then used to examine stochastic stabilization by stochastic feedback controls two types of structure controls namely state feedback and output injection are discussed our stabilization criteria are in terms of linear matrix inequalities lmis whence the feedback controls can be designed more easily in practice a couple of examples and computer simulations are worked out to illustrate our theory

disjunctive kriging 2 examples the disjunctive kriging dk method described in detail in the previous paper of this series is illustrated by two examples the first example is for the electrical conductivity ec which is approximately lognormally distributed in the second example the natural logarithm of the ec is used to produce an approximately normally distributed data set the data set was chosen since the hermite coefficients for a normal and lognormal distribution could be determined analytically as well as to facilitate a comparison between ordinary and disjunctive kriging the capability of dk to be a nonlinear estimator and an estimator of the conditional probability is explored the results indicate that dk is a better estimator than linear estimators i e ordinary kriging in the sense of reduced variance of errors and average kriging variance even when the data are approximately normally or lognormally distributed also using dk one can obtain an estimate of the conditional probability

stability of a random diffusion with linear drift we consider a linear system with markovian switching which is perturbed by gaussian type noise if the linear system is mean square stable then we show that under certain conditions the perturbed system is also stable we also show that under certain conditions the linear system with markovian switching can be stabilized by such noisy perturbation

disjunctive kriging 3 cokriging the disjunctive kriging dk method described in the first paper of this series is extended to account for more than one random function in the derivation contained herein two random functions are considered but this is easily generalized to any number an example is presented using disjunctive cokriging dck where the surface gravimetric moisture content is estimated using the bare soil temperature as an auxiliary random function the results indicate that the dck procedure produces a better estimator than ordinary cokriging in terms of reduced variance of errors and exactness of estimation also using dck an estimate of the conditional probability that the level of a property is greater than a known cutoff value can be obtained in general this conditional probability is better than the dk probability by virtue of the additional information contained in the second auxiliary random function

on strong feller recurrence and weak stabilization of regime switching diffusions focusing on two component markov processes having continuous dynamics and discrete events this work develops asymptotic properties of regime switching diffusions first strong feller property is established next classifying the underlying processes as recurrence positive recurrence or null recurrence or transience sufficient conditions for such processes are obtained in addition to providing general criteria for recurrent and transient switching diffusions for processes that are linearizable with respect to the continuous component easily verifiable conditions are established also provided are conditions for null recurrent processes furthermore conditions for controlled switching diffusions in which the feedback controls ensure weak stability or ergodicity are given finally simple examples are used for demonstration

an asymptotic analysis of hierarchical control of manufacturing systems under uncertainty this paper presents an asymptotic analysis of a hierarchical manufacturing system with machines subject to breakdown and repair the rate of change in machine states is much larger than the rate of fluctuation in demand and the rate of discounting of costs and this gives rise to a limiting problem in which the stochastic machine availability is replaced by the equilibrium mean availability the value function for the original problem converges to the value function of the limiting problem moreover the control for the original problem can be constructed from the optimal controls of the limiting problem in a way which guarantees asymptotic optimality of the value function the limiting problem is computationally more tractable and sometimes has a closed form solution

brief digital optimal reduced order control of pulse width modulated switched linear systems in this paper digital optimal control of pulse width modulated switched linear systems is considered the equivalent discrete time system and its deviation around a stationary solution plays an important role an optimal linear discrete time deviation controller is developed the controller is chosen to be one step ahead predictive which is a natural choice for control of digital pulse modulated systems the controller is optimal i e a cost criterion is minimized the theory is generally applicable and gives in principle a tool to design a digital optimal controller for an arbitrary pulse width modulated switched linear system the theory is illustrated by examples from the field of switched electrical networks

a framework for worst case and stochastic safety verification using barrier certificates this paper presents a methodology for safety verification of continuous and hybrid systems in the worst case and stochastic settings in the worst case setting a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do not enter an unsafe region no explicit computation of reachable sets is required in the construction of barrier certificates which makes it possible to handle nonlinearity uncertainty and constraints directly within this framework in the stochastic setting our method computes an upper bound on the probability that a trajectory of the system reaches the unsafe set a bound whose validity is proven by the existence of a barrier certificate for polynomial systems barrier certificates can be constructed using convex optimization and hence the method is computationally tractable some examples are provided to illustrate the use of the method

optimal control of switched systems via non linear optimization based on direct differentiations of value functions this paper presents an approach for solving optimal control problems of switched systems in general in such problems one needs to find both optimal continuous inputs and optimal switching sequences since the system dynamics vary before and after every switching instant after formulating a general optimal control problem we propose a two stage optimization methodology since many practical problems only concern optimization where the number of switchings and the sequence of active subsystems are given we concentrate on such problems and propose a method which uses nonlinear optimization and is based on direct differentiations of value functions the method is then applied to general switched linear quadratic gslq problems examples illustrate the results

second order switching time optimization for nonlinear time varying dynamic systems this technical note gives a method for calculating the first and second derivatives of a cost function with respect to switching times for systems with piecewise second differentiable dynamics differential equations governing the linear and bilinear operators required for calculating the derivatives are presented example optimizations of linear and nonlinear systems are presented as evidence for the value of second order optimization methods one example converges in 33 iterations using second order methods whereas the first order algorithm requires over 30 000 iterations

stabilization of regime switching processes by feedback control based on discrete time observations this work aims to extend x r mao s work automatica j ifac 49 2013 pp 3677 3681 on stabilization of hybrid stochastic differential equations by discrete time feedback control in x r mao s work the feedback control depends on discrete time observation of the state process but on continuous time observation of the switching process while in this work we study the feedback control depending on discrete time observations of the state process and the switching process our criteria depend explicitly on the regular conditions of the coefficients of the stochastic differential equation and on the stationary distribution of the switching process the sharpness of our criteria is shown through studying the stability of linear systems which also shows explicitly that the stability of hybrid stochastic differential equations depends essentially on the long time behavior of the switching process

optimal control of production rate in a failure prone manufacturing system we address the problem of controlling the production rate of a failure prone manufacturing system so as to minimize the discounted inventory cost where certain cost rates are specified for both positive and negative inventories and there is a constant demand rate for the commodity produced the underlying theoretical problem is the optimal control of a continuous time system with jump markov disturbances with an infinite horizon discounted cost criterion we use two complementary approaches first proceeding informally and using a combination of stochastic coupling linear system arguments stable and unstable eigenspaces renewal theory parametric optimization etc we arrive at a conjecture for the optimal policy then we address the previously ignored mathematical difficulties associated with differential equations with discontinuous right hand sides singularity of the optimal control problem smoothness and validity of the dynamic programming equation etc to give a rigorous proof of optimality of the conjectured policy it is hoped that both approaches will find uses in other such problems also we obtain the complete solution and show that the optimal solution is simply characterized by a certain critical number which we call the optimal inventory level if the current inventory level exceeds the optimal one should not produce at all if less one should produce at the maximum rate while if exactly equal one should produce exactly enough to meet demand we also give a simple explicit formula for the optimal inventory level

robust stability and controllability of stochastic differential delay equations with markovian switching in this paper we investigate the almost surely asymptotic stability for the nonlinear stochastic differential delay equations with markovian switching some sufficient criteria on the controllability and robust stability are also established for linear stochastic differential delay equations with markovian switching

technical communique h model reduction for uncertain switched linear discrete time systems in this paper the problem of h model reduction for switched linear discrete time systems with polytopic uncertainties is investigated a reduced order switched model is constructed for a given robustly stable switched system which has the same structural polytopic uncertainties as the original system such that the resulting error system is robustly asymptotically stable and an h error performance is guaranteed a sufficient condition for the existence of the desired reduced order model is derived and formulated in terms of a set of linear matrix inequalities by solving the corresponding convex optimization problem in such existence condition the vertex system of reduced order model can be obtained which also provides an h gain for the error system between the original system and the reduced order model a numerical example is given to show the effectiveness and the potential of the proposed techniques

controlled diffusion processes with markovian switchings for modeling dynamical engineering systems abstract in this paper the intrinsic complex nature of engineering systems under control is treated by introducing an approach based on controlled stochastic differential equations with markovian switchings in short csdems technical conditions for the existence and uniqueness of the solutions of the csdems are provided in this context it is not unusual to deal with non linear csdems that cannot be solved analytically therefore we develop a new two step predictor corrector method for finding numerical approximations to solutions of csdems this method utilizes the euler maruyama method an illustrative application to the biochemical engineering area is presented to highlight the usefulness of our approach as a simulation tool

production control of a manufacturing system with multiple machine states the production control of a single product manufacturing system with arbitrary number of machine states failure modes is discussed the objective is to find a production policy that would meet the demand for the product with minimum average inventory or backlog cost the optimal production policy has a special structure and is called a hedging point policy if the hedging points are known the optimal production rate is readily specified assuming a set of tentative hedging points the simple structure of the optimal policy is utilized to find the steady state probability distribution of the surplus inventory or backlog once this function is determined the average surplus cost is easily calculated in terms of the values of the hedging points the average cost is then minimized to find the optimum hedging points

passivity based controllers for the stabilization of dc to dc power converters abstract passivity based feedback controllers are derived for the indirect stabilization of the average output voltage in pulse width modulation pwm controlled dc to dc power converters of the boost buck boost and buck types the controller design is carried out on the basis of well known average pwm models of such circuits the average models are first shown to be euler lagrange systems corresponding to a suitable set of average euler lagrange parameters the proposed regulators are based on an energy shaping plus damping injection scheme achievable through nonlinear dynamical feedback the performance of the proposed passivity based duty ratio synthesis policies is tested for the boost converter case the regulating feedback law derived on the basis of a boost model composed of ideal switches and ideal circuit components is assessed via computer simulations on a realistic stochastically perturbed switched converter model including parasitic resistances and parasitic voltage sources

adaptive tracking for stochastic nonlinear systems with markovian switching the problem of the adaptive tracking for a class of stochastic nonlinear systems with stationary markovian switching is considered in this note an ito formula is proposed for stochastic integral equations with an integral about martingale measure an adaptive backstepping controller is designed such that the closed loop system has a unique solution that is globally bounded in probability and l4 norm of the tracking error converges to an arbitrarily small neighborhood of zero a simulation example demonstrates the efficiency of the proposed scheme

optimal control of a brownian production inventory system with average cost criterion in this paper we study a stochastic production inventory system with finite production capacity and random demand the cumulative production and demand are modeled by a two dimensional brownian motion process there is a setup cost for switching on the production and a convex holding and shortage cost and our objective is to find the optimal production inventory control that minimizes the average cost both lost sales and backlog cases are studied for the lost sales model we show that within a large class of policies the optimal production strategy is either to produce according to an s s policy or never turn on the machine at all thus it is optimal for the firm to not enter the business whereas for the backlog model we prove that the optimal production policy is always of the s s type our approach first develops a lower bound for the average cost among a large class of nonanticipating policies and then shows that the value function of the desired policy reaches the lower bound the results offer insights on the structure of the optimal control policies as well as the interplays between system parameters

μ dependent model reduction for uncertain discrete time switched linear systems with average dwell time in this article the model reduction problem for a class of discrete time polytopic uncertain switched linear systems with average dwell time switching is investigated the stability criterion for general discrete time switched systems is first explored and a dependent approach is then introduced for the considered systems to the model reduction solution a reduced order model is constructed and its corresponding existence conditions are derived via lmi formulation the admissible switching signals and the desired reduced model matrices are accordingly obtained from such conditions such that the resulting model error system is robustly exponentially stable and has an exponential h performance a numerical example is presented to demonstrate the potential and effectiveness of the developed theoretical results

quadratic optimal control of switched linear stochastic systems this paper studies a quadratic optimal control problem for discrete time switched linear stochastic systems with nonautonomous subsystems perturbed by gaussian random noises the goal is to jointly design a deterministic switching sequence and a continuous feedback law to minimize the expectation of a finite horizon quadratic cost function both the value function and the optimal control strategy are characterized analytically a numerical relaxation framework is developed to efficiently compute a control strategy with a guaranteed performance upper bound it is also proved that by choosing the relaxation parameter sufficiently small the performance of the resulting control strategy can be made arbitrarily close to the optimal one

a unified framework for hybrid control model and optimal control theory we propose a very general framework that systematizes the notion of a hybrid system combining differential equations and automata governed by a hybrid controller that issues continuous variable commands and makes logical decisions we first identify the phenomena that arise in real world hybrid systems then we introduce a mathematical model of hybrid systems as interacting collections of dynamical systems evolving on continuous variable state spaces and subject to continuous controls and discrete transitions the model captures the identified phenomena subsumes previous models yet retains enough structure to pose and solve meaningful control problems we develop a theory for synthesizing hybrid controllers for hybrid plants in all optimal control framework in particular we demonstrate the existence of optimal relaxed and near optimal precise controls and derive generalized quasi variational inequalities that the associated value function satisfies we summarize algorithms for solving these inequalities based on a generalized bellman equation impulse control and linear programming

likelihood analysis of the binary instrumental variable model instrumental variables are widely used for the identification of the causal effect of one random variable on another under unobserved confounding the distribution of the observable variables for a discrete instrumental variable model satisfies certain inequalities but no conditional independence relations such models are usually tested by checking whether the relative frequency estimators of the parameters satisfy the constraints this ignores sampling uncertainty in the data using the observable constraints for the instrumental variable model a likelihood analysis is conducted a significance test for its validity is developed and a bootstrap algorithm for computing confidence intervals for the causal effect is proposed applications are given to illustrate the advantage of the suggested approach copyright 2011 oxford university press

on the design and synthesis of limit cycles using switching linear systems this article deals with the design and synthesis of limit cycles for a class of switched linear systems this work is motivated by an application in the context of electrical networks but the methodologies can be applied in other engineering fields as well several methods for the design and synthesis of limit cycles are presented based on a monodromy matrix associated with a periodically switched system a design method and two feedback strategies are developed the first strategy is based on linear feedback design using pole placement the second strategy is based on the observation that certain state dependent switching strategies can be implemented by means of a simple nonlinear output feedback controller advantages of this latter strategy are not only the ease with which the switching strategy can be implemented but also the fact that classical techniques may also be used to ascertain the stability of the resulting limit cycle our next contribution is the development of a novel frequency domain

technical communique near optimal control for a class of stochastic hybrid systems to address a computationally intractable optimal control problem for a class of stochastic hybrid systems this paper proposes a near optimal state feedback control scheme which is constructed by using a statistical prediction method based on approximate numerical solution that samples over the entire state space a numerical example illustrates the potential of the approach

strong solutions and strong feller properties for regime switching diffusion processes in an infinite state space we establish the existence and pathwise uniqueness of regime switching diffusion processes in an infinite state space which could be time inhomogeneous and state dependent then the strong feller properties of these processes are investigated by using the theory of parabolic differential equations and dimensional free harnack inequalities

event based pulse modulated control of linear stochastic systems in this technical note event based pulse modulated control is studied for linear stochastic systems with an average quadratic performance over an infinite horizon from the probabilistic representation of solutions to differential equations an analytic expression of the quadratic performance is provided for stable and critically stable first order systems the schemes for seeking optimal parameters are given in the unstable case the stabilization problem is discussed unstable systems first order or higher order with white noises cannot be stabilized with probability 1 except that the noises are known to be bounded

approximation of infinite dimensional linear programming problems which arise in stochastic control we study a general approximation scheme for infinite dimensional linear programming lp problems which arise naturally in stochastic control we prove that the optimal value of the approximating problems converges to the value of the original lp problem for the controls we show that if the approximating optimal controls converge the limiting control is an optimal control for the original lp problem as an application of this theory we present numerical approximations to the lp formulation of stochastic control problems in continuous time we study long term average and discounted control problems for the example for which the theoretical solution is known our approximation results are very accurate

optimal control of switching time in switched stochastic systems with multi switching times and different costs abstractin this paper we solve an optimal control problem for a class of time invariant switched stochastic systems with multi switching times where the objective is to minimise a cost functional with different costs defined on the states in particular we focus on problems in which a pre specified sequence of active subsystems is given and the switching times are the only control variables based on the calculus of variation we derive the gradient of the cost functional with respect to the switching times on an especially simple form which can be directly used in gradient descent algorithms to locate the optimal switching instants finally a numerical example is given highlighting the validity of the proposed methodology

stability and stabilizability of switched linear systems a survey of recent results during the past several years there have been increasing research activities in the field of stability analysis and switching stabilization for switched systems this paper aims to briefly survey recent results in this field first the stability analysis for switched systems is reviewed we focus on the stability analysis for switched linear systems under arbitrary switching and we highlight necessary and sufficient conditions for asymptotic stability after a brief review of the stability analysis under restricted switching and the multiple lyapunov function theory the switching stabilization problem is studied and a variety of switching stabilization methods found in the literature are outlined then the switching stabilizability problem is investigated that is under what condition it is possible to stabilize a switched system by properly designing switching control laws note that the switching stabilizability problem has been one of the most elusive problems in the switched systems literature a necessary and sufficient condition for asymptotic stabilizability of switched linear systems is described here

stabilization and destabilization of hybrid systems of stochastic differential equations this paper aims to determine whether or not a stochastic feedback control can stabilize or destabilize a given nonlinear hybrid system new methods are developed and sufficient conditions on the stability and instability for hybrid stochastic differential equations are provided these results are then used to examine stochastic stabilization and destabilization

on the exponential stability of switching diffusion processes in this note we investigate almost sure exponential stability for a class of switching diffusion processes lyapunov type sufficient conditions to ensure this type of stability for nonlinear switching diffusions are derived the conditions are an improvement over related results in the literature since they do not rely on the moment stability of the system these conditions are then specialized to case of linear switching diffusion processes to provide easily checkable sufficient criteria for exponential stabilization and robust stability

a note on an lqg regulator with markovian switching and pathwise average cost we study a linear system with a markovian switching parameter perturbed by white noise the cost function is quadratic under certain conditions we find a linear feedback control which is almost surely optimal for the pathwise average cost over the infinite planning horizon

optimal control of switching diffusions with application to flexible manufacturing systems a controlled switching diffusion model is developed to study the hierarchical control of flexible manufacturing systems the existence of a homogeneous markov nonrandomized optimal policy is established by a convex analytic method using the existence of such a policy the existence of a unique solution in a certain class to the associated hamilton jacobi bellman equations is established and the optimal policy is characterized as a minimizing selector of an appropriate hamiltonian

continuous dependence estimates and homogenization of quasi monotone systems of fully nonlinear second order parabolic equations abstract the aim of this paper is to extend the continuous dependence estimates proved by jakobsen and karlsen 2002 3 to quasi monotone systems of fully nonlinear second order parabolic equations as a by product of these estimates we get an holder estimate for bounded solutions of systems and a rate of convergence estimate for the vanishing viscosity approximation in the second part of the paper we employ similar techniques to study the periodic homogenization of quasi monotone systems of fully nonlinear second order uniformly parabolic equations finally some examples are discussed

transportation cost inequalities for diffusions with jumps and its application to regime switching processes abstract by using the malliavin calculus we establish the t 1 transportation cost inequalities for stochastic differential equations with jumps which generalizes the results in 14 as an application we establish the t 1 transportation cost inequalities for regime switching diffusion processes

causal bounds and observable constraints for non deterministic models conditional independence relations involving latent variables do not necessarily imply observable independences they may imply inequality constraints on observable parameters and causal bounds which can be used for falsification and identification the literature on computing such constraints often involve a deterministic underlying data generating process in a counterfactual framework if an analyst is ignorant of the nature of the underlying mechanisms then they may wish to use a model which allows the underlying mechanisms to be probabilistic a method of computation for a weaker model without any determinism is given here and demonstrated for the instrumental variable model though applicable to other models the approach is based on the analysis of mappings with convex polytopes in a decision theoretic framework and can be implemented in readily available polyhedral computation software well known constraints and bounds are replicated in a probabilistic model and novel ones are computed for instrumental variable models without non deterministic versions of the randomization exclusion restriction and monotonicity assumptions respectively