strong planning under partial observability rarely planning domains are fully observable for this reason the ability to deal with partial observability is one of the most important challenges in planning in this paper we tackle the problem of strong planning under partial observability in nondeterministic domains find a conditional plan that will result in a successful state regardless of multiple initial states nondeterministic action effects and partial observability we make the following contributions first we formally define the problem of strong planning within a general framework for modeling partially observable planning domains second we propose an effective planning algorithm based on and or search in the space of beliefs we prove that our algorithm always terminates and is correct and complete in order to achieve additional effectiveness we leverage on a symbolic bdd based representation for the domain and propose several search strategies we provide a thorough experimental evaluation of our approach based on a wide selection of benchmarks we compare the performance of the proposed search strategies and identify a uniform winner that combines heuristic distance measures with mechanisms that reduce runtime uncertainty then we compare our planner mbp with other state of the art systems mbp is able to outperform its competitor systems often by orders of magnitude

convergence rates of the truncated euler maruyama method for stochastic differential equations influenced by higham et al 2002 several numerical methods have been developed to study the strong convergence of the numerical solutions to stochastic differential equations sdes under the local lipschitz condition these numerical methods include the tamed euler maruyama em method the tamed milstein method the stopped em the backward em the backward forward em etc recently we developed a new explicit method in mao 2015 called the truncated em method for the nonlinear sde d x t f x t d t g x t d b t and established the strong convergence theory under the local lipschitz condition plus the khasminskii type condition x t f x p 1 2 g x 2 k 1 x 2 however due to the page limit there we did not study the convergence rates for the method which is the aim of this paper we will under some additional conditions discuss the rates of l q convergence of the truncated em method for 2 q p and show that the order of l q convergence can be arbitrarily close to q 2

higher order implicit strong numerical schemes for stochastic differential equations higher order implicit numerical methods which are suitable for stiff stochastic differential equations are proposed these are based on a stochastic taylor expansion and converge strongly to the corresponding solution of the stochastic differential equation as the time step size converges to zero the regions of absolute stability of these implicit and related explicit methods are also examined

finite time fault detection filter design for discrete time interconnected systems with average dwell time abstract in this paper we present an approach to finite time robust fault detection filter design for interconnected systems with disconnected interconnections among these subsystems taking various disconnected interconnections into account a switched scheme is proposed to analysis these complex cases by modeling these cases of disconnections into independent subsystems a switching signal is introduced to specify which subsystem is activated in the aid of the average dwell time method sufficient conditions on existence of the desired filter are derived in term of linear matrices inequalities which guarantee the augmented system is finite time boundedness and achieve the h performance a numerical example is used to illustrate the effectiveness of the proposed approach

reasoning about actions with imprecise and incomplete state descriptions this article is a first step in the direction of extending possibilistic planning to account for incomplete and imprecise knowledge of the world state fundamental definitions are given and the possibilistic planning problem is recast in this new setting finally it is shown that under certain conditions possibilistic planning with imprecise and incomplete state descriptions is no harder than possibilistic planning with crisp and complete information

solving langevin equation with the bicolour rooted tree method national basic research program of china 973 program 2007cb814800 national natural science foundation of china 10475067

universal property of the housekeeping entropy production the entropy production of a nonequilibrium system with broken detailed balance is a random variable whose mean value is nonnegative the housekeeping entropy production which is a part of total entropy production is associated with the heat dissipation in maintaining a nonequilibrium steady state we derive a langevin type stochastic differential equation for the housekeeping entropy production the equation allows us to define a housekeeping entropic time ensuremath tau remarkably it turns out that the probability distribution of the housekeeping entropy production at a fixed value of ensuremath tau is given by the gaussian distribution regardless of system details the gaussian distribution is universal for any systems whether in the steady state or in the transient state and whether they are driven by time independent or time dependent driving forces we demonstrate the universal distribution numerically for model systems

weak second order conditions for stochastic runge kutta methods a general procedure to construct weak methods for the numerical solution of stochastic differential systems is presented as in the deterministic case the procedure consists of comparing the stochastic expansion of the approximation with the corresponding taylor scheme in this way the authors obtain the order conditions that a stochastic runge kutta method must satisfy to have weak order two explicit examples of generalizations of the classical family of second order two stage explicit runge kutta methods are shown also numerical examples are presented

synchrony and variability induced by spatially correlated additive and multiplicative noise in the coupled langevin model the synchrony and variability of the coupled langevin model subjected to spatially correlated additive and multiplicative noise are discussed we have employed numerical simulations and the analytical augmented moment method which is the second order moment method for local and global variables h hasegawa phys rev e 67 041903 2003 it has been shown that the synchrony of an ensemble is increased decreased by a positive negative spatial correlation in both additive and multiplicative noise although the variability for local fluctuations is almost insensitive to spatial correlations that for global fluctuations is increased decreased by positive negative correlations when a pulse input is applied the synchrony is increased for the correlated multiplicative noise whereas it may be decreased for correlated additive noise coexisting with uncorrelated multiplicative noise an application of our study to neuron ensembles has demonstrated the possibility that information is conveyed by the variance and synchrony in input signals which accounts for some neuronal experiments

the direct richardson p th order drp schemes a new class of time integration schemes for stochastic differential equations we describe a new family of weak p th order accurate sde time integration schemes called the direct richardson p th order accurate drp schemes the drp schemes use the idea of richardson extrapolation on euler time steps performed by way of an acceptance rejection algorithm previous applications of richardson extrapolation to the euler scheme are applicable only when the objective is to estimate a functional of the final distribution of the process in contrast provided that the diffusion matrix is strictly positive definite the drp class of schemes can be used in all applications which require a weak sde time integration scheme numerical results have been obtained and a comparison is made between the second and third order accurate drp schemes and other modern sde time integration schemes indicating that the drp schemes incur less error than standard algorithms based on ito taylor expansions and have similar computational efficiency finally we provide a proof of the convergence properties of the drp schemes

finite time h_ infty filtering for t s fuzzy discrete time systems with time varying delay and norm bounded uncertainties in this paper we investigate the filtering problem of discrete time takagi sugeno t s fuzzy uncertain systems subject to time varying delays a reduced order filter is designed with the augmentation technique a filtering error system with delayed states is obtained in order to deal with time delays in system states the filtering error system is first transformed into two interconnected subsystems by using a two term approximation for the time varying delay sufficient delay dependent conditions of finite time boundedness and h infty performance of the filtering error system are derived with the lyapunov function based on these conditions the filter design methods are proposed and the filter gain matrices can be obtained by calculating a set of linear matrix inequalities a numerical example is used to illustrate the effectiveness of the proposed approaches

population rate codes carried by mean fluctuation and synchrony of neuronal firings abstract a population of firing neurons is expected to carry information not only by the mean firing rate but also by fluctuation and synchrony among neurons in order to examine this possibility we have studied responses of neuronal ensembles to three kinds of inputs mean fluctuation and synchrony driven inputs the generalized rate code model including additive and multiplicative noise h hasegawa phys rev e 75 2007 051904 has been studied by direct simulations dss and the augmented moment method amm in which equations of motion for mean firing rate fluctuation and synchrony are derived results calculated by the amm are in good agreement with those by dss the independent component analysis ica of our results has shown that mean firing rate fluctuation or variability and synchrony may carry independent information in the population rate code model the input output relation of mean firing rates is shown to have higher sensitivity for larger multiplicative noise as recently observed in prefrontal cortex a comparison is made between results obtained by the integrate and fire if model and our rate code model

strong convergence of the stopped euler maruyama method for nonlinear stochastic differential equations in this paper numerical methods for the nonlinear stochastic differential equations sdes with non global lipschitz drift coefficient are discussed the existing known results have only so far shown that the classical explicit euler maruyama em approximate solutions converge to the true solution in probability 22 23 more recently the authors in 16 proved that the classical em method will diverge in l 2 sense for the underlying sdes in this paper and those sdes with superlinearly growing coefficients these strongly indicate that the classical em method is not good enough for the highly nonlinear sdes however in this paper we introduce a modified em method using stopping time and show successfully that the discrete version of the modified em approximate solution converges to the true solution in the strong sense namely in l 2 with a order arbitrarily close to a half

specific volume coupling and convergence properties in hybrid particle finite volume algorithms for turbulent reactive flows we investigate the coupling between the two components of a large eddy simulation probability density function les pdf algorithm for the simulation of turbulent reacting flows in such an algorithm the large eddy simulation les component provides a solution to the hydrodynamic equations whereas the lagrangian monte carlo probability density function pdf component solves for the pdf of chemical compositions special attention is paid to the transfer of specific volume information from the pdf to the les code the specific volume field contains probabilistic noise due to the nature of the monte carlo pdf solution and thus the use of the specific volume field in the les pressure solver needs careful treatment using a test flow based on the sandia sydney bluff body flame we determine the optimal strategy for specific volume feedback then the overall second order convergence of the entire les pdf procedure is verified using a simple vortex ring test case with special attention being given to bias errors due to the number of particles per les finite volume fv cell

stability analysis of numerical schemes for stochastic differential equations stochastic differential equations sdes represent physical phenomena dominated by stochastic processes as for deterministic ordinary differential equations odes various numerical schemes are proposed for sdes in this paper we study the stability of numerical schemes for scalar sdes with respect to the mean square norm which we call ms stability we will show some figures of the ms stability domain or regions for some numerical schemes and present numerical results which confirm it this notion is an extension of absolute stability in numerical methods for odes

numerical implementation of mixing and molecular transport in les pdf studies of turbulent reacting flows probability density function pdf methods in combination with large eddy simulations les are a powerful tool for studying turbulent reacting flow problems and we are interested in the implementation of mixing and molecular transport in les pdf methods the numerical methodology used for solution procedure is the hybrid particle mesh method and a fractional step scheme is used to solve for transport reaction and mixing sequentially mixing is modeled using the interaction by exchange with the mean iem model and the effects of molecular transport are incorporated as a mean drift term in the mixing step this methodology avoids spurious production of scalar variance and also allows direct incorporation of differential diffusion effects in this study various numerical implementations of mixing and molecular transport are presented and evaluated using the method of manufactured solutions mms for 1 accuracy 2 detailed conservation 3 realizability and 4 stability moreover the methodology is shown to be successful in capturing the effects of differential diffusion accurately with the additional property of ensuring realizability of species mass fractions finally and most importantly we describe a new variance reduction technique by way of an implicit smoothing methodology this smoothing scheme is shown to satisfy conservation boundedness and regularity criteria moreover for an appropriate choice of the smoothing length scale significant improvements in accuracy can be achieved for an incremental increase in computational cost also it is shown that with smoothing on a length scale greater than the grid size the bias and statistical errors due to there being a finite number of particles in the lagrangian monte carlo simulations scale as n t o t 1 and n t o t 1 2 respectively where n t o t is the total number of particles in the computational domain whereas without smoothing these errors scale as n p c 1 and n p c 1 2 where n p c is the much smaller number of particles in a computational cell

numerical integration of stochastic differential equations several numerical methods for treating stochastic differential equations are considered both the convergence in the mean square limit and the convergence of the moments is discussed and the generation of appropriate random numbers is treated the necessity of simulations at various time steps with an extrapolation to time step zero is emphasized and demonstrated by a simple example

a random map implementation of implicit filters implicit particle filters for data assimilation generate high probability samples by representing each particle location as a separate function of a common reference variable this representation requires that a certain underdetermined equation be solved for each particle and at each time an observation becomes available we present a new implementation of implicit filters in which we find the solution of the equation via a random map as examples we assimilate data for a stochastically driven lorenz system with sparse observations and for a stochastic kuramoto sivashinsky equation with observations that are sparse in both space and time

an algorithm for probabilistic planning abstract we define the probabilistic planning problem in terms of a probability distribution over initial world states a boolean combination of propositions representing the goal a probability threshold and actions whose effects depend on the execution time state of the world and on random chance adopting a probabilistic model complicates the definition of plan success instead of demanding a plan that provably achieves the goal we seek plans whose probability of success exceeds the threshold in this paper we present buridan an implemented least commitment planner that solves problems of this form we prove that the algorithm is both sound and complete we then explore buridan s efficiency by contrasting four algorithms for plan evaluation using a combination of analytic methods and empirical experiments we also describe the interplay between generating plans and evaluating them and discuss the role of search control in probabilistic planning

strong predictor corrector euler maruyama methods for stochastic differential equations with markovian switching in this paper numerical methods for solving stochastic differential equations with markovian switching sdewmss are developed by pathwise approximation the proposed family of strong predictor corrector euler maruyama methods is designed to overcome the propagation of errors during the simulation of an approximate path this paper not only shows the strong convergence of the numerical solution to the exact solution but also reveals the order of the error under some conditions on the coefficient functions a natural analogue of the p stability criterion is studied numerical examples are given to illustrate the computational efficiency of the new predictor corrector euler maruyama approximation

weak second order splitting schemes for lagrangian monte carlo particle methods for the composition pdf fdf transport equations we study a class of methods for the numerical solution of the system of stochastic differential equations sdes that arises in the modeling of turbulent combustion specifically in the monte carlo particle method for the solution of the model equations for the composition probability density function pdf and the filtered density function fdf this system consists of an sde for particle position and a random differential equation for particle composition the numerical methods considered advance the solution in time with weak second order accuracy with respect to the time step size the four primary contributions of the paper are i establishing that the coefficients in the particle equations can be frozen at the mid time while preserving second order accuracy ii examining the performance of three existing schemes for integrating the sdes iii developing and evaluating different splitting schemes which treat particle motion reaction and mixing on different sub steps and iv developing the method of manufactured solutions mms to assess the convergence of monte carlo particle methods tests using mms confirm the second order accuracy of the schemes in general the use of frozen coefficients reduces the numerical errors otherwise no significant differences are observed in the performance of the different sde schemes and splitting schemes

stochastic resonance in bistable systems with nonlinear dissipation and multiplicative noise a microscopic approach abstract we have studied the stochastic resonance sr of bistable systems coupled to a bath with a nonlinear system bath interaction by using the microscopic generalized caldeira leggett cl model the adopted cl model yields the non markovian langevin equation with nonlinear dissipation and state dependent multiplicative diffusion which preserve the fluctuation dissipation relation fdr results of our simulations are given as follows 1 the spectral power amplification spa exhibits sr not only for a and b but also for while the stationary probability distribution function is independent of them where a and b denote magnitudes of multiplicative and additive noises respectively and expresses the relaxation time of ornstein uhlenbeck ou colored noise 2 the spa for coexisting additive and multiplicative noises has a single peak but two peak structure as functions of a b and or results 1 and 2 are qualitatively different from previous ones obtained by phenomenological langevin models where the fdr is not held or indefinite these show an importance of the fdr in a study on sr of open bistable systems

the parabolic edge reconstruction method perm for lagrangian particle advection we describe a lagrangian particle advection scheme which is intended for use in hybrid finite volume fv large eddy simulation filtered density function les fdf methods for low mach flows but which may also be applicable to unsteady probability density function pdf methods direct numerical simulation dns or any other situation where tracking fluid particles is of concern a key ingredient of the scheme is a subgrid reconstruction of the filtered velocity field with desirable divergence properties which is necessary for accurate evolution of the particle number density we develop reconstructions for 2d and 3d cartesian staggered non uniform grids the reconstructed velocity field is continuous and piecewise parabolic in the velocity component direction in the direction normal to the velocity component the reconstruction is piecewise linear the divergence of the reconstructed field is bilinear in 2d trilinear in 3d within a given cell and consistent with the discrete divergence given by the staggered grid velocities though the reconstructed divergence field may be discontinuous from cell to cell the norm of the differences between the vertex values of the reconstructed divergence for neighboring cells is minimized as a consequence the divergence is everywhere zero for the constant density case a two stage runge kutta scheme is employed for advancement of the particle positions to assess the performance of the scheme we utilize a set of non trivial velocity test functions which are designed to mimic realistic flow fields we show that an advection scheme based on the new velocity reconstruction method is effective at maintaining an accurate particle number density in the particle tracking limit

finite time control of discrete time linear systems in this note we consider the finite time stabilization of discrete time linear systems subject to disturbances generated by an exosystem finite time stability can be used in all those applications where large values of the state should not be attained for instance in the presence of saturations the main result provided in the note is a sufficient condition for finite time stabilization via state feedback this result is then used to find some sufficient conditions for the existence of an output feedback controller guaranteeing finite time stability all the conditions are then reduced to feasibility problems involving linear matrix inequalities lmis some numerical examples are presented to illustrate the proposed methodology

runge kutta methods for numerical solution of stochastic differential equations the way to obtain deterministic runge kutta methods from taylor approximations is generalized for stochastic differential equations now by means of stochastic truncated expansions about a point for sufficiently smooth functions of an ito process a class of explicit runge kutta schemes of second order in the weak sense for systems of stochastic differential equations with multiplicative noise is developed also two runge kutta schemes of third order have been obtained for scalar equations with constant diffusion coefficients numerical examples that compare the proposed schemes to standard ones are presented

modeling a dynamic and uncertain world i symbolic and probabilistic reasoning about change abstract intelligent agency requires some ability to predict the future an agent must ask itself what is presently its best course of action given what it now knows about what the world will be like when it intends to act this paper presents a system that uses a probabilistic model to reason about the effects of an agent s proposed actions on a dynamic and uncertain world computing the probability that relevant propositions will hold at a specified point in time the model allows for incomplete information about the world the occurrence of exogenous unplanned events unreliable sensors and the possibility of an imperfect causal theory the system provides an application program with answers to questions of the form is the probability that will hold in the world at time t greater than it is unique among algorithms for probabilistic temporal reasoning in that it tries to limit its inference according to the proposition time and probability threshold provided by the application the system will also notify the application if subsequent evidence invalidates its answer to a query

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

numerical procedures for sample structures on stochastic differential equations procedures are discussed for the numerical construction of sample processes on stochastic differential equations structures of a relevant class of gaussian white noise functionals are analyzed to some closed forms and new numerical algorithms for single noise problems are presented up to the accuracy of o h3 or within errors of o h7 2 in the sense of the mean square norm

numerical integration of multiplicative noise stochastic differential equations a two stage runge kutta algorithm for vector ito and by transform also stratonovich stochastic differential equations with multiplicative noise has been developed the method is second order accurate but for vanishing drift the algorithm yields a martingale independent of step size several examples are included to illustrate our method a discussion of errors shows that sample size can be as important as truncation

particle method for turbulent flows integration of stochastic model equations abstract a numerical method is developed to integrate the stochastic differential equations that arise in a particle method for modelling turbulent flows these equations present several challenges the foremost being the presence of multiple time scales the smallest of which can be significantly less than an acceptable time step size t the essence of the approach adopted is to transform and decompose the equations so that the stochastic components which contain the small time scales appear as strictly linear stochastic differential equations analytic solutions to these equations with frozen coefficients are then exploited to produce a stable and accurate scheme when the method is used to advance the properties of n particles the resulting numerical error can be decomposed into three contributions statistical error bias and time stepping error comprehensive tests to study these errors are reported for two test cases a novel variance reduction technique is described that significantly reduces the statistical error which scales as n 1 2 in general the bias is smaller and scales as n 1 in accord with a simple analysis the time stepping error is less than 1 for a non dimensional time step of 1 32 which may be several times larger than the smallest time scale over the range of time step size investigated the dominant time stepping error varies as t 3 2 the method has the requisite stability accuracy and efficiency for incorporation in multi dimensional particle methods

a matrix factorization solution to the h h fault detection problem in this paper we give a matrix factorization solution to the h h fault detection fd problem for linear time invariant dynamic systems an h h fd filter minimizes the sensitivity of the residual signal to disturbances while maintaining a minimum level of sensitivity to faults more specifically it minimizes the largest singular value over the extended imaginary axis of the transfer matrix from the disturbance to the residual signal vectors subject to the constraint that the singular values of the transfer matrix from the fault to the residual vectors are larger than or equal to 1 we show through the use of matrix function factorizations and completions that the problem reduces to a nonstandard model matching problem which is then solved under a mild assumption concerning the rank of a matrix function we also give an illustrative example

a logic of time chance and action for representing plans abstract this paper integrates logical and probabilistic approaches to the representation of planning problems by developing a first order logic of time chance and action we start by making explicit and precise commonsense notions about time chance and action central to the planning problem we then develop a logic the semantics of which incorporates these intuitive properties the logical language integrates both modal and probabilistic constructs and allows quantification over time points probability values and domain individuals probability is treated as a sentential operator in the language so it can be arbitrarily nested and combined with other logical operators the language can represent the chance that facts hold and events occur at various times it can represent the chance that actions and other events affect the future the model of action distinguishes between action feasibility executability and effects we present a proof theory for the logic and show how the logic can be used to describe actions in such a way that the action descriptions can be composed to infer properties of plans via the proof theory

planning for conjunctive goals abstract the problem of achieving conjunctive goals has been central to domain independent planning research the nonlinear constraint posting approach has been most successful previous planners of this type have been complicated heuristic and ill defined i have combined and distilled the state of the art into a simple precise implemented algorithm tweak which i have proved correct and complete i analyze previous work on domain independent conjunctive planning in retrospect it becomes clear that all conjunctive planners linear and nonlinear work the same way the efficiency and correctness of these planners depends on the traditional add delete list representation for actions which drastically limits their usefulness i present theorems that suggest that efficient general purpose planning with more expressive action representations is impossible and suggest ways to avoid this problem

a haar like construction for the ornstein uhlenbeck process the classical haar construction of brownian motion uses a binary tree of triangular wedge shaped functions this basis has compactness properties which make it especially suited for certain classes of numerical algorithms we present a similar basis for the ornstein uhlenbeck process in which the basis elements approach asymptotically the haar functions as the index increases and preserve the following properties of the haar basis all basis elements have compact support on an open interval with dyadic rational endpoints these intervals are nested and become smaller for larger indices of the basis element and for any dyadic rational only a finite number of basis elements is nonzero at that number thus the expansion in our basis when evaluated at a dyadic rational terminates in a finite number of steps we prove the covariance formulae for our expansion and discuss its statistical interpretation

a fokker planck approach for probability distributions of species concentrations transported in heterogeneous media we identify sufficient conditions under which evolution equations for probability density functions pdf of random concentrations are equivalent to fokker planck equations the novelty of our approach is that it allows consistent pdf approximations by densities of computational particles governed by ito processes in concentration position spaces accurate numerical solutions are obtained with a global random walk grw algorithm stable free of numerical diffusion and insensitive to the increase of the total number of computational particles the system of ito equations is specified by drift and diffusion coefficients describing the pdf transport in the physical space provided by up scaling procedures as well as by drift and mixing coefficients describing the pdf transport in concentration spaces mixing models can be obtained similarly to classical pdf approaches or alternatively from measured or simulated concentration time series we compare their performance for a grw pdf numerical solution to a problem of contaminant transport in heterogeneous groundwater systems

brief paper optimal fault detection for linear discrete time varying systems this paper deals with the problem of observer based fault detection for linear discrete time varying ldtv systems a problem formulation is first proposed to address the optimization of the fault detection filter fdf design which is expressed in terms of maximizing a finite horizon h h or h h performance index this formulation can be applied to fdf design of ldtv systems subject to l 2 norm bounded unknown inputs or stochastic noise sequences it is shown that a unified optimal solution to the fdf can be obtained by solving the discrete time riccati equation and the optimal fdf is not unique a numerical example is given to illustrate the proposed method

accurate monte carlo tests of the stochastic ginzburg landau model with multiplicative colored noise an accurate and fast monte carlo algorithm is proposed for solving the ginzburg landau equation with multiplicative colored noise the stable cases of solution for choosing time steps and trajectory numbers are discussed

exponential timestepping with boundary test for stochastic differential equations we present new numerical methods for scalar stochastic differential equations successive time increments are independent random variables with an exponential distribution we perform numerical experiments using a double well potential exponential timestepping algorithms are efficient for escape time problems because a simple boundary test can be performed at the end of each step

complexity of finite horizon markov decision process problems controlled stochastic systems occur in science engineering manufacturing social sciences and many other cntexts if the systems is modeled as a markov decision process mdp and will run ad infinitum the optimal control policy can be computed in polynomial time using linear programming the problems considered here assume that the time that the process will run is finite and based on the size of the input there are mny factors that compound the complexity of computing the optimal policy for instance there are many factors that compound the complexity of this computation for instance if the controller does not have complete information about the state of the system or if the system is represented in some very succint manner the optimal policy is provably not computable in time polynomial in the size of the input we analyze the computational complexity of evaluating policies and of determining whether a sufficiently good policy exists for a mdp based on a number of confounding factors including the observability of the system state the succinctness of the representation the type of policy even the number of actions relative to the number of states in almost every case we show that the decision problem is complete for some known complexity class some of these results are familiar from work by papadimitriou and tsitsiklis and others but some such as our pl completeness proofs are surprising we include proofs of completeness for natural problems in the as yet little studied classes np pp

stochastic dynamic programming with factored representations markov decision processes mdps have proven to be popular models for decision theoretic planning but standard dynamic programming algorithms for solving mdps rely on explicit state based specifications and computations to alleviate the combinatorial problems associated with such methods we propose new representational and computational techniques for mdps that exploit certain types of problem structure we use dynamic bayesian networks with decision trees representing the local families of conditional probability distributions to represent stochastic actions in an mdp together with a decision tree representation of rewards based on this representation we develop versions of standard dynamic programming algorithms that directly manipulate decision tree representations of policies and value functions this generally obviates the need for state by state computation aggregating states at the leaves of these trees and requiring computations only for each aggregate state the key to these algorithms is a decision theoretic generalization of classic regression analysis in which we determine the features relevant to predicting expected value we demonstrate the method empirically on several planning problems showing significant savings for certain types of domains we also identify certain classes of problems for which this technique fails to perform well and suggest extensions and related ideas that may prove useful in such circumstances we also briefly describe an approximation scheme based on this approach

robust finite time stability and stabilization of uncertain markovian jump systems with time varying delay this paper deals with the robust finite time stability and stabilization problems of uncertain stochastic delayed jump systems where the uncertainty is in the form of additive perturbations and exists in the drift and diffusion sections simultaneously though perturbation time varying delay and brownian motion existing at the same time two conditions checking its robust finite time stability are proposed by a mode dependent parameter approach which are different from some existing methods based on the proposed results sufficient conditions for the existence of the state feedback controller are provided with lmis which could be solved directly it is seen that all the features of the underlying system such as time varying delay perturbation diffusion mode dependent parameters and uncertain transition rate matrix play important roles in the system analysis and synthesis of finite time stability finally numerical examples are used to demonstrate the effectiveness and superiority of the proposed methods

fault detection for discrete time linear systems based on descriptor observer approach this paper investigates the problem of actuator fault detection filter design for discrete time linear systems with output disturbance by using the descriptor observer method an h fault detection filter is designed to guarantee the residual system is admissible and satisfies the h performance by utilizing lyapunov function approach a sufficient condition for the admissible of the residual system is obtained in the form of linear matrix inequality lmi the desired fault detection filter can be designed by solving a set of lmis finally a numerical example is proposed to illustrate the effectiveness of the developed method

optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages decision processes with incomplete state feedback have been traditionally modelled as partially observable markov decision processes in this article we present an alternative formulation based on probabilistic regular languages the proposed approach generalises the recently reported work on language measure theoretic optimal control for perfectly observable situations and shows that such a framework is far more computationally tractable to the classical alternative in particular we show that the infinite horizon decision problem under partial observation modelled in the proposed framework is approximable and in general is not harder to solve compared to the fully observable case the approach is illustrated via two simple examples

balanced implicit methods for stiff stochastic systems this paper introduces some implicitness in stochastic terms of numerical methods for solving stiff stochastic differential equations and especially a class of fully implicit methods the balanced methods their order of strong convergence is proved numerical experiments compare the stability properties of these schemes with explicit ones

reasoning about partially ordered events abstract this paper describes a class of temporal reasoning problems involving events whose order is not completely known we examine the complexity of such problems and show that for all but trivial cases these problems are likely to be intractable as an alternative to a complete but potentially exponential time decision procedure we provide a partial decision procedure that reports useful results and runs in polynomial time

stability of weak numerical schemes for stochastic differential equations abstract this paper considers numerical stability and convergence of weak schemes solving stochastic differential equations a relatively strong notion of stability for a special type of test equations is proposed these are stochastic differential equations with multiplicative noise for different explicit and implicit schemes the regions of stability are also examined

planning and acting in partially observable stochastic domains in this paper we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains we begin by introducing the theory of markov decision processes mdps and partially observable mdps pomdps we then outline a novel algorithm for solving pomdps off line and show how in some cases a finite memory controller can be extracted from the solution to a pomdp we conclude with a discussion of how our approach relates to previous work the complexity of finding exact solutions to pomdps and of some possibilities for finding approximate solutions

decentralized adaptive output feedback controller design for stochastic nonlinear interconnected systems in this paper the controller design for a class of stochastic interconnected systems with both parametric uncertainties and unknown nonlinear interactions is presented the diffusion terms considered are dependent on the outputs of local subsystems and allowed to be unbounded first by employing decentralized state observers totally decentralized adaptive tracking controllers with suitable parameter adaptive laws are designed to ensure the boundedness in probability of all the signals in the closed loop system it is shown that the tracking errors converge to small residual sets around the origin then for systems with relaxed diffusion vector fields a decentralized adaptive stabilizing scheme is proposed to ensure the boundedness in probability of all signals in the closed loop system

statistical mechanics of kinks for quasi exactly solvable potentials we study the thermodynamics of kinks in the presence of quasi exactly solvable qes potentials using a langevin code implemented on a massively parallel computer specifically we study the potential v 2 1 2 for which certain exact solutions of the schrodinger equation are known provided the parameters and satisfy certain constraints for 1 this potential has two degenerate minima otherwise it has only one minimum thus a system with this potential is capable of describing a second order phase transition a la the 4 model however no exact solutions of the schrodinger equation for this potential exist we have obtained an exact solution for the kink in this model in addition we have calculated in a 1d model such quantities as the probability density function pdf field configuration and field field correlation functions both above and below the transition temperature tc for several temperatures these quantities help us understand the contribution to the specific heat from coherent structures such as domain walls kinks as opposed to the contribution from lattice vibrations we have calibrated our results against known exact solutions for limiting cases with very high accuracy

contingent planning under uncertainty via stochastic satisfiability we describe a new planning technique that efficiently solves probabilistic propositional contingent planning problems by converting them into instances of stochastic satisfiability ssat and solving these problems instead we make fundamental contributions in two areas the solution of ssat problems and the solution of stochastic planning problems this is the first work extending the planning as satisfiability paradigm to stochastic domains our planner zander can solve arbitrary goal oriented finite horizon partially observable markov decision processes pomdps an empirical study comparing zander to seven other leading planners shows that its performance is competitive on a range of problems

predictor corrector methods of runge kutta type for stochastic differential equations in this paper we construct predictor corrector pc methods based on the trivial predictor and stochastic implicit runge kutta rk correctors for solving stochastic differential equations using the colored rooted tree theory and stochastic b series the order condition theorem is derived for constructing stochastic rk methods based on pc implementations we also present detailed order conditions of the pc methods using stochastic implicit rk correctors with strong global order 1 0 and 1 5 a two stage implicit rk method with strong global order 1 0 and a four stage implicit rk method with strong global order 1 5 used as the correctors are constructed in this paper the mean square stability properties and numerical results of the pc methods based on these two implicit rk correctors are reported

numerical treatment of stochastic differential equations we define general runge kutta approximations for the solution of stochastic differential equations sde these approximations are proved to converge in quadratic mean to the solution of an sde with a corrected drift the explicit form of the correction term is given concerning the order of convergence we show that in general it is impossible for the quadratic mean of the one step error to be of an order greater than o h 3 this order is attained e g by the stochastic analogue of heun s method in the n dimensional case the highest order of convergence is in general only o h 2 attained by euler s method the order o h 3 can only be reached if nabla x sigma r sigma s nabla x sigma s sigma r for the diffusion matrix

computer simulations of multiplicative stochastic differential equations a class of robust algorithms for the computer simulation of stochastic differential equations with multiplicative noise is investigated excellent agreement is obtained with the known analytic behaviour of the kubo oscillator in the white noise limit the algorithms include a known first order one dimensional explicit method as well as implicit methods of increased stability a distinction is drawn between classes of stochastic differential equations depending on the type of spatial variation or curvature defined by the diffusion tensor this allows greatly simplified numerical implementations of the new algorithms in certain cases the results of different techniques are compared for the case of the kubo oscillator where a semi implicit technique gives the greatest accuracy

the truncated euler maruyama method for stochastic differential equations influenced by higham et al 2003 several numerical methods have been developed to study the strong convergence of the numerical solutions to stochastic differential equations sdes under the local lipschitz condition these numerical methods include the tamed euler maruyama em method the tamed milstein method the stopped em the backward em the backward forward em etc in this paper we will develop a new explicit method called the truncated em method for the nonlinear sde d x t f x t d t g x t d b t and establish the strong convergence theory under the local lipschitz condition plus the khasminskii type condition x t f x p 1 2 g x 2 k 1 x 2 the type of convergence specifically addressed in this paper is strong l q convergence for 2 q p and p is a parameter in the khasminskii type condition

statistical mechanics of double sinh gordon kinks abstract we study the classical thermodynamics of the double sinh gordon dshg theory in 1 1 dimensions this model theory has a double well potential v cosh2 n 2 when n thus allowing for the existence of kinks and antikinks though it is non integrable the dshg model is remarkably amenable to analysis below we obtain exact single kink and kink lattice solutions as well as the asymptotic kink antikink interaction in the continuum limit finding the classical partition function is equivalent to solving for the ground state of a schrodinger like equation obtained via the transfer integral method for the dshg model this equation turns out to be quasi exactly solvable we exploit this property to obtain exact energy eigenvalues and wavefunctions for several temperatures both above and below the symmetry breaking transition temperature provided n 1 2 6 the availability of exact results provides an excellent testing ground for large scale langevin simulations the probability distribution function pdf calculated from langevin dynamics is found to be in striking agreement with the exact pdf obtained from the ground state wavefunction this validation points to the utility of a pdf based computation of thermodynamics utilizing langevin methods in addition to the pdf field field and field fluctuation correlation functions were computed and also found to be in excellent agreement with the exact results

weighted fuzzy observer based fault detection approach for discrete time nonlinear systems via piecewise fuzzy lyapunov functions the main focus of this paper is on the analysis and integrated design of mathcal l 2 observer based fault detection fd systems for discrete time nonlinear industrial processes to gain a deeper insight into this fd framework the existence condition is introduced first then an integrated design of mathcal l 2 observer based fd approach is realized by solving the proposed existence condition with the aid of takagi sugeno fuzzy dynamic modeling technique and piecewise fuzzy lyapunov functions most importantly a weighted piecewise fuzzy observer based residual generator is proposed aiming at achieving an optimal integration of residual evaluation and threshold computation into fd systems the core of this approach is to make use of the knowledge provided by fuzzy models of each local region and then to weight the local residual signal by means of different weighting factors in comparison with the standard norm based fuzzy observer based fd methods the proposed scheme may lead to a significant improvement of the fd performance in the end the effectiveness of the proposed method is verified by a numerical example and a case study on the laboratory setup of continuous stirred tank heater plant

simultaneous fault detection and control for switched linear systems with mode dependent average dwell time this paper investigates the problem of the simultaneous fault detection and control sfdc for switched linear systems to meet the control and detection objectives the time dependent detection filters and dynamic output feedback controllers are presented in sfdc under a mixed h h framework a mode dependent average dwell time mdadt approach which means that each subsystem has its own average dwell time is adopted in this paper to reduce the conservativeness of the average dwell time method and the discretized lyapunov function dlf technique is first used to relax the mdadt constraints in sfdc some sufficient conditions for designing filters controllers which satisfy the h h performance are given in terms of linear matrix inequalities lmis what s more a two step algorithm to solve the sfdc problem is proposed the effectiveness of the proposed method is illustrated through two simulation examples

on the undecidability of probabilistic planning and related stochastic optimization problems automated planning the problem of how an agent achieves a goal given a repertoire of actions is one of the foundational and most widely studied problems in the ai literature the original formulation of the problem makes strong assumptions regarding the agent s knowledge and control over the world namely that its information is complete and correct and that the results of its actions are deterministic and known recent research in planning under uncertainty has endeavored te relax these assumptions providing formal and computation models wherein the agent has incomplete or noisy information about the world and has noisy sensors and effectors this research has mainly taken one of two approaches extend the classical planning paradigm to a semantics that admits uncertainty or adopt another framework for approaching the problem most commonly the markov decision process mdp model this paper presents a complexity analysis of planning under uncertainty it begins with the probabilistic classical planning problem showing that problem to be formally undecidable this fundamental result is then applied to a broad class of stochastic optimization problems in brief any problem statement where the agent a operates over an infinite or indefinite time horizon and b has available only probabilistic information about the system s state undecidability is established for policy existence problems for partially observable infinite horizon markov decision processes under discounted and undiscounted total reward models average reward models and state avoidance models the results also apply to corresponding approximation problems with undiscounted objective functions the paper answers a significant open question raised by papadimitriou and tsitsiklis math oper res 12 3 1987 441 450 about the complexity of infinite horizon pomdps

numerical integration of stochastic differential equations weak second order mid point scheme for application in the composition pdf method in this paper a weak second order accurate mid point scheme for the stochastic differential equations sdes arising in the composition pdf method for turbulent reactive flows is proposed and tested the results are compared with two other schemes which are commonly used in the composition pdf method in contrast to most higher order schemes for sdes the present scheme uses a mid point which makes it especially suitable for the implementation of the position advance fractional step in the composition pdf method the scheme can also be applied to the pdf method used in conjunction with large eddy simulation les since the sdes considered in this paper include explicit time dependence of the drift and diffusion coefficients test calculations are reported including a 2d unsteady case to demonstrate the weak second order accuracy of the scheme and to compare its performance to that of two first order schemes a new methodology for developing higher order scheme for sdes by comparing the moments of the increments of the numerical approximation with those of the exact increments is used to develop this scheme

equivalence notions and model minimization in markov decision processes many stochastic planning problems can be represented using markov decision processes mdps a difficulty with using these mdp representations is that the common algorithms for solving them run in time polynomial in the size of the state space where this size is extremely large for most real world planning problems of interest recent ai research has addressed this problem by representing the mdp in a factored form factored mdps however are not amenable to traditional solution methods that call for an explicit enumeration of the state space one familiar way to solve mdp problems with very large state spaces is to form a reduced or aggregated mdp with the same properties as the original mdp by combining equivalent states in this paper we discuss applying this approach to solving factored mdp problems we avoid enumerating the state space by describing large blocks of equivalent states in factored form with the block descriptions being inferred directly from the original factored representation the resulting reduced mdp may have exponentially fewer states than the original factored mdp and can then be solved using traditional methods the reduced mdp found depends on the notion of equivalence between states used in the aggregation the notion of equivalence chosen will be fundamental in designing and analyzing algorithms for reducing mdps optimally these algorithms will be able to find the smallest possible reduced mdp for any given input mdp and notion of equivalence i e find the minimal model for the input mdp unfortunately the classic notion of state equivalence from non deterministic finite state machines generalized to mdps does not prove useful we present here a notion of equivalence that is based upon the notion of bisimulation from the literature on concurrent processes our generalization of bisimulation to stochastic processes yields a non trivial notion of state equivalence that guarantees the optimal policy for the reduced model immediately induces a corresponding optimal policy for the original model with this notion of state equivalence we design and analyze an algorithm that minimizes arbitrary factored mdps and compare this method analytically to previous algorithms for solving factored mdps we show that previous approaches implicitly derive equivalence relations that we define here

weak order stochastic runge kutta methods for commutative stochastic differential equations a new explicit stochastic runge kutta scheme of weak order 2 is proposed under a commutativity condition which is derivative free and which attains order 4 for ordinary differential equations the weak order conditions are derived by utilizing multi colored rooted tree analysis and a solution is found in a transparent way the scheme is compared with other derivative free and weak second order schemes in numerical experiments

numerical solution of the kardar parisi zhang equation in one two and three dimensions abstract on a course grained level a family of microscopic growth processes may be described by a stochastic differential equation which is solved numerically for surface dimensions d 1 2 and 3 dimensional analysis shows that the spatial discretization parameter has the meaning of an effective coupling constant the numerical stability of the euler integration scheme is discussed for the strong coupling exponents defined by surface width time the following effective values were obtained d 1 0 330 0 004 and d 2 0 24 0 005 considering the width and its ensemble fluctuations at constant dimensionless time the transition between strong and weak coupling phases is located in d 3 for the largest coupling for which reliable data are available we obtain an effective exponent close to the best estimates on discrete models d 3 0 17