disordered one dimensional contact process new theoretical and numerical analysis of the one dimensional contact process with quenched disorder are presented we derive new scaling relations different from their counterparts in the pure model which are valid not only at the critical point but also away from it due to the presence of generic scale invariance all the proposed scaling laws are verified in numerical simulations in addition we map the disordered contact process into a non markovian contact process by using the so called run time statistic and write down the associated field theory this turns out to be in the same universality class as one derived by janssen for the quenched system with a gaussian distribution of impurities our findings here support the lack of universality suggested by the field theoretical analysis generic power law behaviors are obtained evidence is shown of the absence of a characteristic time away from the critical point and the absence of universality is put forward the intermediate sublinear regime predicted by bramsom et al is also found

crossover from directed percolation to compact directed percolation we study critical spreading in a surface modified directed percolation model in which the left and right most sites have different occupation probabilities than in the bulk as we vary the probability for growth at an edge the critical exponents switch from the compact directed percolation class to ordinary directed percolation we conclude that the nonuniversality observed in models with multiple absorbing configurations cannot be explained as a simple surface effect

applications of partial differential approximants test functions and dimensional crossover in the ising model the method of partial differential approximants pda has been introduced for approximating functions of two or more variables given a finite number of coefficients of their power series it is supposed to be effective close to the multicritical points where the thermodynamic functions are expected to behave according to the scaling hypothesis in order to assess the performance of the method the authors have undertaken the construction of pdas for several test functions of two variables for some simple functions which are represented exactly by pdas and which exhibit an analogue of a multicritical point the numerical estimates yielded very good results even at lower orders for other functions the estimates tended to improve as the order of the approximants is increased for the best pdas they constructed flow diagrams and estimated the scaling functions also they have analysed the dimensional crossover from d 3 to d 2 of the sign 1 2 axial anisotropic ising model on the fcc lattice they considered series expansions for the direct susceptibility and also for the sum of the direct and the staggered susceptibilities

on actor critic algorithms in this article we propose and analyze a class of actor critic algorithms these are two time scale algorithms in which the critic uses temporal difference learning with a linearly parameterized approximation architecture and the actor is updated in an approximate gradient direction based on information provided by the critic we show that the features for the critic should ideally span a subspace prescribed by the choice of parameterization of the actor we study actor critic algorithms for markov decision processes with polish state and action spaces we state and prove two results regarding their convergence

absorbing state phase transitions with quenched disorder quenched disorder in the sense of the harris criterion is generally a relevant perturbation at an absorbing state phase transition point here using a strong disorder renormalization group framework and effective numerical methods we study the properties of random fixed points for systems in the directed percolation universality class for strong enough disorder the critical behavior is found to be controlled by a strong disorder fixed point which is isomorph with the fixed point of random quantum ising systems in this fixed point dynamical correlations are logarithmically slow and the static critical exponents are conjecturedly exact for one dimensional systems the renormalization group scenario is confronted with numerical results on the random contact process in one and two dimensions and satisfactory agreement is found for weaker disorder the numerical results indicate static critical exponents which vary with the strength of disorder whereas the dynamical correlations are compatible with two possible scenarios either they follow a power law decay with a varying dynamical exponent like in random quantum systems or the dynamical correlations are logarithmically slow even for a weak disorder for models in the parity conserving universality class there is no strong disorder fixed point according to our renormalization group analysis

time dependent perturbation theory for nonequilibrium lattice models we develop a time dependent perturbation theory for nonequilibrium interacting particle systems we focus on models such as the contact process which evolve via destruction and autocatalytic creation of particles at a critical value of the destruction rate there is a continuous phase transition between an active steady state and the vacuum state which is absorbing we present several methods for deriving series for the evolution starting from a single seed particle including expansions for the ultimate survival probability in the super and subcritical regions expansions for the average number of particles in the subcritical region and short time expansions algorithms for computer generation of the various expansions are presented rather long series 24 terms or more and precise estimates of critical parameters are presented

two neighbour stochastic cellular automata and their planar lattice duals two neighbour stochastic cellular automata sca are the set of one dimensional discrete time interacting particle systems with two parameters which show non equilibrium phase transitions from the extinction phase to the survival phase the phase diagram was first studied by kinzel using a numerical method called transfer matrix scaling for some parameter region the processes can be defined as directed percolation models on the spatio temporal plane and the bond and site directed percolation models are included as special cases extending the argument of dhar barma and phani 1981 originally given for bond directed percolation we introduce diode resistor percolation models which are the planar lattice duals of the sca and give rigorous lower bounds for the critical line in special cases our results give 0 6885 or alpha c and 0 6261 or beta c where alpha c and beta c denote the critical probabilities of the site and bond directed percolation models on the square lattice respectively combining the upper bound for the critical line recently proved by liggett 1994 we summarize the rigorous results for the phase diagram of the systems results of computer simulation are also shown

low density series expansions for directed percolation i a new efficient algorithm with applications to the square lattice a new algorithm for the derivation of low density series for percolation on directed lattices is introduced and applied to the square lattice bond and site problems numerical evidence shows that the computational complexity grows exponentially but with a growth factor which is much smaller than the growth factor of the previous best algorithm for bond site percolation on the directed square lattice the series has been extended to order 171 158 analysis of the series yields sharper estimates of the critical points and exponents

stochastic approximation for nonexpansive maps application to q learning algorithms we discuss synchronous and asynchronous iterations of the form xk 1 x k gamma k h x k w k where h is a suitable map and wk is a deterministic or stochastic sequence satisfying suitable conditions in particular in the stochastic case these are stochastic approximation iterations that can be analyzed using the ode approach based either on kushner and clark s lemma for the synchronous case or on borkar s theorem for the asynchronous case however the analysis requires that the iterates xk be bounded a fact which is usually hard to prove we develop a novel framework for proving boundedness in the deterministic framework which is also applicable to the stochastic case when the deterministic hypotheses can be verified in the almost sure sense this is based on scaling ideas and on the properties of lyapunov functions we then combine the boundedness property with borkar s stability analysis of odes involving nonexpansive mappings to prove convergence with probability 1 in the stochastic case we also apply our convergence analysis to q learning algorithms for stochastic shortest path problems and are able to relax some of the assumptions of the currently available results

stability of stochastic approximation under verifiable conditions in this paper we address the problem of the stability and convergence of the stochastic approximation procedure theta n 1 theta n gamma n 1 h theta n xi n 1 the stability of such sequences theta n is known to heavily rely on the behavior of the mean field h at the boundary of the parameter set and the magnitude of the stepsizes used the conditions typically required to ensure convergence and in particular the boundedness or stability of theta n are either too difficult to check in practice or not satisfied at all this is the case even for very simple models the most popular technique for circumventing the stability problem consists of constraining theta n to a compact subset mathcal k in the parameter space this is obviously not a satisfactory solution as the choice of mathcal k is a delicate one in this paper we first prove a deterministic stability result which relies on simple conditions on the sequences xi n and gamma n we then propose and analyze an algorithm based on projections on adaptive truncation sets which ensures that the aforementioned conditions required for stability are satisfied we focus in particular on the case where xi n is a so called markov state dependent noise we establish both the stability and convergence with probability 1 w p 1 of the algorithm under a set of simple and verifiable assumptions we illustrate our results with an example related to adaptive markov chain monte carlo algorithms

robust em kernel based methods for linear system identification recent developments in system identification have brought attention to regularized kernel based methods this type of approach has been proven to compare favorably with classic parametric methods however current formulations are not robust with respect to outliers in this paper we introduce a novel method to robustify kernel based system identification methods to this end we model the output measurement noise using random variables with heavy tailed probability density functions pdfs focusing on the laplacian and the student s t distributions exploiting the representation of these pdfs as scale mixtures of gaussians we cast our system identification problem into a gaussian process regression framework which requires estimating a number of hyperparameters of the data size order to overcome this difficulty we design a new maximum a posteriori map estimator of the hyperparameters and solve the related optimization problem with a novel iterative scheme based on the expectation maximization em method in the presence of outliers tests on simulated data and on a real system show a substantial performance improvement compared to currently used kernel based methods for linear system identification

low density series expansions for directed percolation iv temporal disorder we introduce a model for temporally disordered directed percolation in which the probability of spreading from a vertex t x where t is the time and x is the spatial coordinate is independent of x but depends on t using a very efficient algorithm we calculate low density series for bond percolation on the directed square lattice analysis of the series yields estimates for the critical point pc and various critical exponents which are consistent with a continuous change of the critical parameters as the strength of the disorder is increased

critical exponents of directed percolation measured in spatiotemporal intermittency an experimental system showing a transition to spatiotemporal intermittency is presented it consists of a ring of hundred oscillating ferrofluidic spikes four of five of the measured critical exponents of the system agree with those obtained from a theoretical model of directed percolation

robust adapative kalman filtering for systems with unknown step inputs and non gaussian measurement errors target tracking with kalman filters is hampered by target maneuvering and unknown process and measurement noises we show that moving data windows may be used to analyze state and measurement error sequences determining robust estimates of bias and covariance for steps in the system forcing functions and non gaussian measurement errors the robust estimators yield improvements over linear bias and covariance estimators extensive simulations compare conventional linear adaptive and robust adaptive average step responses of a first order system filter quantities examined are state estimate state error process and measurement covariance estimates kalman gain and input step estimate

sparse robust estimation and kalman smoothing with nonsmooth log concave densities modeling computation and theory we introduce a new class of quadratic support qs functions many of which already play a crucial role in a variety of applications including machine learning robust statistical inference sparsity promotion and inverse problems such as kalman smoothing well known examples of qs penalties include the l2 huber l1 and vapnik losses we build on a dual representation for qs functions using it to characterize conditions necessary to interpret these functions as negative logs of true probability densities this interpretation establishes the foundation for statistical modeling with both known and new qs loss functions and enables construction of non smooth multivariate distributions with specified means and variances from simple scalar building blocks the main contribution of this paper is a flexible statistical modeling framework for a variety of learning applications together with a toolbox of efficient numerical methods for estimation in particular a broad subclass of qs loss functions known as piecewise linear quadratic plq penalties has a dual representation that can be exploited to design interior point ip methods ip methods solve nonsmooth optimization problems by working directly with smooth systems of equations characterizing their optimality we provide several numerical examples along with a code that can be used to solve general plq problems the efficiency of the ip approach depends on the structure of particular applications we consider the class of dynamic inverse problems using kalman smoothing this class comprises a wide variety of applications where the aim is to reconstruct the state of a dynamical system with known process and measurement models starting from noisy output samples in the classical case gaussian errors are assumed both in the process and measurement models for such problems we show that the extended framework allows arbitrary plq densities to be used and that the proposed ip approach solves the generalized kalman smoothing problem while maintaining the linear complexity in the size of the time series just as in the gaussian case this extends the computational efficiency of the mayne fraser and rauch tung striebel algorithms to a much broader nonsmooth setting and includes many recently proposed robust and sparse smoothers as special cases

hyperscaling in the domany kinzel cellular automaton an apparent violation of hyperscaling at the endpoint of the critical line in the domany kinzel stochastic cellular automaton finds an elementary resolution upon noting that the order parameter is discontinuous at this point we derive a hyperscaling relation for such transitions and discuss applications to related examples

monte carlo smoothing for nonlinear time series we develop methods for performing smoothing computations in general state space models the methods rely on a particle representation of the filtering distributions and their evolution through time using sequential importance sampling and resampling ideas in particular novel techniques are presented for generation of sample realizations of historical state sequences this is carried out in a forward filtering backward smoothing procedure that can be viewed as the nonlinear non gaussian counterpart of standard kalman filter based simulation smoothers in the linear gaussian case convergence in the mean squared error sense of the smoothed trajectories is proved showing the validity of our proposed method the methods are tested in a substantial application for the processing of speech signals represented by a time varying autoregression and parameterized in terms of time varying partial correlation coefficients comparing the results of our algorithm with those from a simple smoother based on the filte

variational properties of value functions regularization plays a key role in a variety of optimization formulations of inverse problems a recurring theme in regularization approaches is the selection of regularization parameters and their effect on the solution and on the optimal value of the optimization problem the sensitivity of the value function to the regularization parameter can be linked directly to the lagrange multipliers this paper characterizes the variational properties of the value functions for a broad class of convex formulations which are not all covered by standard lagrange multiplier theory an inverse function theorem is given that links the value functions of different regularization formulations not necessarily convex these results have implications for the selection of regularization parameters and the development of specialized algorithms numerical examples illustrate the theoretical results

robust l1 principal component analysis and its bayesian variational inference we introduce a robust probabilistic l1 pca model in which the conventional gaussian distribution for the noise in the observed data was replaced by the laplacian distribution or l1 distribution due to the heavy tail characteristics of the l1 distribution the proposed model is supposed to be more robust against data outliers in this letter we demonstrate how a variational approximation scheme enables effective inference of key parameters in the probabilistic l1 pca model as the l1 density can be expanded as a superposition of infinite number of gaussian densities we express the l1 pca model as a marginalized model over the superpositions by doing so a tractable bayesian inference can be achieved based on the variational expectation maximization type algorithm

monte carlo maximum likelihood estimation for non gaussian state space models state space models are considered for observations which have non gaussian distributions we obtain accurate approximations to the loglikelihood for such models by monte carlo simulation devices are introduced which improve the accuracy of the approximations and which increase computational efficiency the loglikelihood function is maximised numerically to obtain estimates of the unknown hyperparameters standard errors of the estimates due to simulation are calculated details are given for the important special cases where the observations come from an exponential family distribution and where the observation equation is linear but the observation errors are non gaussian the techniques are illustrated with a series for which the observations have a poisson distribution and a series for which the observation errors have a t distribution

stabilization of stochastic approximation by step size adaptation a scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed this scheme leads to the same limiting differential equation as the original scheme and therefore has the same limiting behavior while avoiding the difficulties associated with projection schemes the proof technique requires only that the limiting o d e descend a certain lyapunov function outside an arbitrarily large bounded set c 2012 elsevier b v all rights reserved

maximum a posteriori state path estimation discretization limits and their interpretation continuous discrete models with dynamics described by stochastic differential equations are used in a wide variety of applications for these systems the maximum a posteriori map state path can be defined as the curves around which lie the infinitesimal tubes with greatest posterior probability which can be found by maximizing a merit function built upon the onsager machlup functional a common approach used in the engineering literature to obtain the map state path is to discretize the dynamics and obtain the map state path for the discretized system in this paper we prove that if the trapezoidal scheme is used for discretization then the discretized map state path estimation converges hypographically to the continuous discrete map state path estimation as the discretization gets finer however if the stochastic euler scheme is used instead then the discretized estimation converges to the minimum energy estimation the minimum energy estimates are in turn proved to be the state paths associated with the map noise paths which in some cases differ from the map state paths therefore the discretized map state paths can have different interpretations depending on the discretization scheme used

violation of scaling in the contact process with quenched disorder we study the two dimensional contact process cp with quenched disorder dcp and determine the static critical exponents beta and nu perp the dynamic behavior is incompatible with scaling as applied to models such as the pure cp that have a continuous phase transition to an absorbing state we find that the survival probability starting with all sites occupied for a finite size system at critical decays according to a power law as does the off critical density autocorrelation function thus the critical exponent nu parallle which governs the relaxation time is undefined since the characteristic relaxation time is itself undefined the logarithmic time dependence found in recent simulations of the critical dcp moreira and dickman phys rev e54 r3090 1996 is further evidence of violation of scaling a simple argument based on percolation cluster statistics yields a similar logarithmic evolution

the o d e method for convergence of stochastic approximation and reinforcement learning it is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated ode this in turn implies convergence of the algorithm several specific classes of algorithms are considered as applications it is found that the results provide i a simpler derivation of known results for reinforcement learning algorithms ii a proof for the first time that a class of asynchronous stochastic approximation algorithms are convergent without using any a priori assumption of stability iii a proof for the first time that asynchronous adaptive critic and q learning algorithms are convergent for the average cost optimal control problem

contact process on a voronoi triangulation we study the continuous absorbing state phase transition in the contact process on the voronoi delaunay lattice the voronoi construction is a natural way to introduce quenched coordination disorder in lattice models we simulate the disordered system using the quasistationary simulation method and determine its critical exponents and moment ratios our results suggest that the critical behavior of the disordered system is unchanged with respect to that on a regular lattice i e that of directed percolation

a comparative study for the pair creation contact process using series expansions a comparative study between two distinct perturbative series expansions for the pair creation contact process is presented in contrast to the ordinary contact process whose supercritical series expansions provide accurate estimates for its critical behaviour the supercritical approach does not work properly when applied to the pair creation process to circumvent this problem a procedure is introduced in which one site creation is added to the pair creation an alternative method is the generation of subcritical series expansions which works even for the case of the pure pair creation process in contrast to the supercritical case the subcritical series yields estimates that are compatible with numerical simulations

a stochastic approximation algorithm with varying bounds many optimization problems that are intractable with conventional approaches will yield to stochastic approximation algorithms this is because these algorithms can be used to optimize functions that cannot be evaluated analytically but have to be estimated for instance through simulation or measured thus stochastic approximation algorithms can be used for optimization in simulation unfortunately the classical stochastic approximation algorithm sometimes diverges because of unboundedness problems we study the convergence of a variant of stochastic approximation defined over a growing sequence of compact sets we show that this variant converges under more general conditions on the objective function than the classical algorithm while maintaining the same asymptotic convergence rate we also present empirical evidence that shows that this algorithm sometimes converges much faster than the classical algorithm

an ell _ 1 laplace robust kalman smoother robustness is a major problem in kalman filtering and smoothing that can be solved using heavy tailed distributions e g l1 laplace this paper describes an algorithm for finding the maximum a posteriori map estimate of the kalman smoother for a nonlinear model with gaussian process noise and l1 laplace observation noise the algorithm uses the convex composite extension of the gauss newton method this yields convex programming subproblems to which an interior point path following method is applied the number of arithmetic operations required by the algorithm grows linearly with the number of time points because the algorithm preserves the underlying block tridiagonal structure of the kalman smoother problem excellent fits are obtained with and without outliers even though the outliers are simulated from distributions that are not l1 laplace it is also tested on actual data with a nonlinear measurement model for an underwater tracking experiment the l1 laplace smoother is able to construct a smoothed fit without data removal from data with very large outliers

on line inference for hidden markov models via particle filters we consider the on line bayesian analysis of data by using a hidden markov model where inference is tractable conditional on the history of the state of the hidden component a new particle filter algorithm is introduced and shown to produce promising results when analysing data of this type the algorithm is similar to the mixture kalman filter but uses a different resampling algorithm we prove that this resampling algorithm is computationally efficient and optimal among unbiased resampling algorithms in terms of minimizing a squared error loss function in a practical example that of estimating break points from well log data our new particle filter outperforms two other particle filters one of which is the mixture kalman filter by between one and two orders of magnitude

field theoretic treatment of an effective action for a model of catalyzed autoamplification reaction diffusion models can exhibit continuous phase transitions in behaviors and their dynamics at criticality often exhibit scalings with key parameters that can be characterized by exponents while models with only a single field that transitions between absorbing and nonabsorbing states are well characterized and typically fall in the directed percolation universality class the effects of coupling multiple fields remain poorly understood we recently introduced a model which has three fields one of which relaxes exponentially one of which displays critical behavior and one of which has purely diffusive dynamics but exerts an influence on the critical field tchernookov j chem phys 130 134906 2009 simulations suggested that this model is in a universality class distinct from other reaction diffusion systems studied previously although the three fields give rise to interesting physics they complicate analysis of the model with renormalization group methods here we show how to systematically simplify the action for this model such that analytical expressions for the exponents of this universality class can be obtained by standard means we expect the approach taken here to be of general applicability in reaction diffusion systems with coupled order parameters that display qualitatively different behaviors close to criticality

contact process in disordered and periodic binary two dimensional lattices the critical behavior of the contact process cp in disordered and periodic binary two dimensional 2d lattices is investigated numerically by means of monte carlo simulations as well as via an analytical approximation and standard mean field theory phase separation lines calculated numerically are found to agree well with analytical predictions around the homogeneous point for the disordered case values of static scaling exponents obtained via quasistationary simulations are found to change with disorder strength in particular the finite size scaling exponent of the density of infected sites approaches a value consistent with the existence of an infinite randomness fixed point as conjectured before for the 2d disordered cp at the same time both dynamical and static scaling exponents are found to coincide with the values established for the homogeneous case thus confirming that the contact process in a heterogeneous environment belongs to the directed percolation universality class

extinction survival and dynamical phase transition of branching annihilating random walk we analyze statistical properties of random walkers which disappear when they meet and make offsprings by a controllable rate numerical results for one two and three dimensions and for the sierpinski gasket are assessed in view of the mean field theory predictions universality classes are found to depend on the number of offsprings in space dimension less than 3

an inequality constrained nonlinear kalman bucy smoother by interior point likelihood maximization kalman bucy smoothers are often used to estimate the state variables as a function of time in a system with stochastic dynamics and measurement noise this is accomplished using an algorithm for which the number of numerical operations grows linearly with the number of time points all of the randomness in the model is assumed to be gaussian including other available information for example a bound on one of the state variables is non trivial because it does not fit into the standard kalman bucy smoother algorithm in this paper we present an interior point method that maximizes the likelihood with respect to the sequence of state vectors satisfying inequality constraints the method obtains the same decomposition that is normally obtained for the unconstrained kalman bucy smoother hence the resulting number of operations grows linearly with the number of time points we present two algorithms the first is for the affine case and the second is for the nonlinear case neither algorithm requires the optimization to start at a feasible sequence of state vector values both the unconstrained affine and unconstrained nonlinear kalman bucy smoother are special cases of the class of problems that can be handled by these algorithms

surface growth on diluted lattices by a restricted solid on solid model an influence of diluted sites on surface growth has been investigated using the restricted solid on solid model it was found that with respect to equilibrium growth the surface width and the saturated width exhibited universal power law behaviors i e w ensuremath sim t ensuremath beta and w text sat ensuremath sim l ensuremath zeta regarding all cases with respect to the concentration of diluted sites x 1 ensuremath p with p being the occupation probability on each lattice site for xl x c 1 ensuremath p c p c being the percolation threshold the growth appeared to be similar to that of a regular lattice both in two and three dimensions for x x c the growth yielded nontrivial exponents which were different from those on a regular lattice in nonequilibrium growth a considerable amount of diluted sites x ensuremath le x c appeared to yield nonuniversal growth unlike the case of a regular lattice the cause of nonuniversal growth dynamics has been investigated considering the growth on a backbone cluster and on lattices constructed with periodically and randomly diluted subcells

the marginal likelihood for parameters in a discrete gauss markov process we use laplace s method to approximate the marginal likelihood for parameters in a gauss markov process this approximation requires the determinant of a matrix whose dimensions are equal to the number of state variables times the number of time points we reduce this to sequential evaluation of determinants and inverses of smaller matrices we show this is a numerically stable method

critical behavior and griffiths effects in the disordered contact process we study the nonequilibrium phase transition in the one dimensional contact process with quenched spatial disorder by means of large scale monte carlo simulations for times up to 10 9 and system sizes up to 10 7 sites in agreement with recent predictions of an infinite randomness fixed point our simulations demonstrate activated exponential dynamical scaling at the critical point the critical behavior turns out to be universal even for weak disorder however the approach to this asymptotic behavior is extremely slow with crossover times of the order of 10 4 or larger in the griffiths region between the clean and the dirty critical points we find power law dynamical behavior with continuously varying exponents we discuss the generality of our findings and relate them to a broader theory of rare region effects at phase transitions with quenched disorder

nested multivariate pade approximants abstract a new class of multivariate pade approximants is introduced when dealing with two variables x and y the approach consists in applying the pade approximation with respect to y to the coefficients of the pade approximation with respect to x this technique has a natural extension to n variables and is well adapted to programming in any usual language the advantage of the method is that the algorithm uses only univariate pade approximation as a consequence only small systems need to be solved and the computation of the approximant is much faster if m is the number of derivatives with respect to x and y the algorithm requires o m 4 operations instead of o m 6 when matrix methods are used for solving the coefficients problem or o m 3 operations instead of o m 4 if staircase methods are used another important feature is that the reliable methods which have been developed for choosing the degrees of the numerator and the denominator in the univariate case can be straightforwardly used in the multivariate case

linear glauber model we study the time dependent and the stationary properties of the linear glauber model in a d dimensional hypercubic lattice this model is equivalent to the voter model with noise by using the green function method we get exact results for the two point correlations from which the critical behavior is obtained for vanishing noise the model becomes critical with exponents beta 0 gamma 1 and nu 1 2 for d or 2 with logarithmic corrections at the upper critical dimension d c 2 and beta 0 gamma 1 2 and nu 1 2 for d 1 we show that the model can be mapped into a particular reaction diffusion model

optimal smoothing of non linear dynamic systems via monte carlo markov chains we consider the smoothing problem of estimating a sequence of state vectors given a nonlinear state space model with additive white gaussian noise and measurements of the system output the system output may also be nonlinearly related to the system state often obtaining the minimum variance state estimates conditioned on output data is not analytically intractable to tackle this difficulty a markov chain monte carlo technique is presented the proposal density for this method efficiently draws samples from the laplace approximation of the posterior distribution of the state sequence given the measurement sequence this proposal density is combined with the metropolis hastings algorithm to generate realizations of the state sequence that converges to the proper posterior distribution the minimum variance estimate and confidence intervals are approximated using these realizations simulations of a fed batch bioreactor model are used to demonstrate that the proposed method can obtain significantly better estimates than the iterated kalman bucy smoother

robust recursive estimation in the presence of heavy tailed observation noise a first order approximation is derived for the conditional prior distribution of the state of a discrete time stochastic linear dynamic system in the presence of contaminated normal observation noise this distribution is then used to derive a first order approximation of the conditional mean minimum variance estimator if the observation noise distribution can be represented as a member of the e contaminated normal neighborhood then the conditional prior is also to first order an analogous perturbation from a normal distribution whose first two moments are given by the kalman filter moreover the perturbation is itself of a special form combining distributions whose parameters are given by banks of parallel kalman filters and optimal smoothers

the iterated kalman smoother as a gauss newton method the kalman smoother is known to be the maximum likelihood estimator when the measurement and transition functions are affine i e a linear function plus a constant a new proof of this result is presented that shows that the kalman smoother decomposes a large least squares problem into a sequence of much smaller problems the iterated kalman smoother is then presented and shown to be a gauss newton method for maximizing the likelihood function in the nonaffine case this method takes advantage of the decomposition obtained with the kalman smoother

stochastic lattice models with several absorbing states we study two models with n equivalent absorbing states that generalize the domany kinzel cellular automaton and the contact process numerical investigations show that for n 2 both models belong to the same universality class as branching annihilating walks with an even number of offspring unlike previously known models these models have no explicit parity conservation law

a scaled stochastic approximation algorithm consider a stochastic system of such complexity that its performance can only be evaluated by using simulation or direct experimentation to optimize the expected performance of such systems as a function of several continuous input parameters decision variables we present a scaled stochastic approximation algorithm for finding the zero root of the gradient of the response function in each iteration of the scaled algorithm two independent gradient estimates are sampled at the current estimate of the optimal input parameter vector to compute a scale free estimate of the next search direction we establish sufficient conditions to ensure strong consistency and asymptotic normality of the resulting estimator of the optimal input parameter vector strong consistency is also established for a variant of the scaled algorithm with kesten s acceleration an experimental performance comparison of the scaled algorithm and the classical robbins monro algorithm in two simple queueing systems reveals some of the practical advantages of the scaled algorithm

descent methods for composite nondifferentiable optimization problems we present a framework for the development of globally defined descent algorithms for the minimization of non differentiable objective functionsf h o f withh convex within our structure the global convergence properties of the cauchy modified newton gauss newton and variable metric methods are easily established along with that of several new approaches examples illustrating the calculational techniques are provided

robust estimation with unknown noise statistics the equivalence between the kalman filter and a particular least squares regression problem is established and the regression problem is solved robustly using a statistical approach named m estimation m robust estimators are derived for adaptive estimation of the unknown a priori state and observation noise statistics simultaneously with the system states the feasibility of the approach is demonstrated with simulation

joint maximum a posteriori state path and parameter estimation in stochastic differential equations abstract in this article we introduce the joint maximum a posteriori state path and parameter estimator jme for continuous time systems described by stochastic differential equations sdes this estimator can be applied to nonlinear systems with discrete time sampled measurements with a wide range of measurement distributions we also show that the minimum energy state path and parameter estimator mee obtains the joint maximum a posteriori noise path initial conditions and parameters these estimators are demonstrated in simulated experiments in which they are compared to the prediction error method pem using the unscented kalman filter and smoother the experiments show that the mee is biased for the damping parameters of the drift function furthermore for robust estimation in the presence of outliers the jme attains lower state estimation errors than the pem

study of universality crossover in the contact process we consider a generalization of the contact process stochastic model including an additional autocatalitic process the phase diagram of this model in the proper 2 parameter space displays a line of transitions between an active and an absorbing phase which starts at the critical point of the contact process and ends at the transition point of the voter model thus a crossover between the directed percolation and the compact percolation universality classes is observed at this latter point we study this crossover by a variety of techniques using supercritical series expansions analysed with partial differential approximants we obtain precise estimates of the crossover behaviour of the model in particular we find an estimate for the crossover exponent 2 00 0 02 we also show arguments that support the conjecture 2

critical dynamics of the contact process with quenched disorder we study critical spreading in monte carlo simulations of the two dimensional contact process cp with quenched disorder in the form of random dilution in the pure model spreading from a single particle at the critical point ensuremath lambda c follows power laws with the critical exponents of directed percolation with disorder critical spreading is logarithmic not power law below ensuremath lambda c there is a griffiths phase in which the time dependence is governed by nonuniversal power laws the effects of disorder are also apparent above ensuremath lambda c in the active phase where the relaxation of the survival probability is algebraic rather than exponential as in the pure model our results support the conjecture by bramson durrett and schonmann ann prob 19 960 1991 that in two or more dimensions the disordered cp has only a single phase transition

convergence of markovian stochastic approximation with discontinuous dynamics this paper is devoted to the convergence analysis of stochastic approximation algorithms of the form theta n 1 theta n gamma n 1 h theta n x n 1 where left theta n n in mathbb n right is an mathbb r d valued sequence left gamma n n in mathbb n right is a deterministic stepsize sequence and left x n n in mathbb n right is a controlled markov chain we study the convergence under weak assumptions on smoothness in theta of the function theta mapsto h theta x it is usually assumed that this function is continuous for any x in this work we relax this condition our results are illustrated by considering stochastic approximation algorithms for adaptive quantile estimation and a penalized version of the vector quantization

supercritical series expansion for the contact process in heterogeneous and disordered environments the supercritical series expansion of the survival probability for the one dimensional contact process in heterogeneous and disordered lattices is used for the evaluation of the loci of critical points and critical exponents beta the heterogeneity and disorder are modeled by considering binary regular and irregular lattices of nodes characterized by different recovery rates and identical transmission rates two analytical approaches based on nested pade approximants and partial differential approximants were used in the case of expansions with respect to two variables two recovery rates for the evaluation of the critical values and critical exponents the critical exponents in heterogeneous systems are very close to those for the homogeneous contact process thus confirming that the contact process in periodic heterogeneous environment belongs to the directed percolation universality class the disordered systems in contrast seem to have continuously varying critical exponents

robust estimation in non linear state space models with state dependent noise in this paper we present a robust estimation algorithm for non linear state space models driven by state dependent noise the algorithm is robust to outliers in the data we derive the algorithm step by step from first principles from theory to implementation the implementation is straightforward and consists mainly of two components 1 a slightly modified version of the rauch tung striebel recursions and 2 a backtracking line search strategy since it preserves the underlying chain structure of the problem its computational complexity grows linearly with the number of data the algorithm is iterative and is guaranteed to converge under mild assumptions to a local optimum from any starting point we validate our approach via experiments on synthetic data from a multi variate stochastic volatility model

robustification of kalman filter models abstract kalman filter models based on the assumption of multivariate gaussian distributions are known to be nonrobust this means that when a large discrepancy arises between the prior distribution and the observed data the posterior distribution becomes an unrealistic compromise between the two in this article we discuss a rationale for how to robustify the kalman filter specifically we develop a model wherein the posterior distribution will revert to the prior when extreme outlying observations are encountered and we point out that this can be achieved by assuming a multivariate distribution with student t marginals to achieve fully robust results of the kind desired it becomes necessary to forsake an exact distribution theory approach and adopt an approximation method involving poly t distributions a recursive mechanism for implementing the multivariate t based kalman filter is described its properties are discussed and the procedure is illustrated by an example

mixture kalman filters in treating dynamic systems sequential monte carlo methods use discrete samples to represent a complicated probability distribution and use rejection sampling importance sampling and weighted resampling to complete the on line filtering task we propose a special sequential monte carlo method the mixture kalman filter which uses a random mixture of the gaussian distributions to approximate a target distribution it is designed for on line estimation and prediction of conditional and partial conditional dynamic linear models which are themselves a class of widely used non linear systems and also serve to approximate many others compared with a few available filtering methods including monte carlo methods the gain in efficiency that is provided by the mixture kalman filter can be very substantial another contribution of the paper is the formulation of many non linear systems into conditional or partial conditional linear form to which the mixture kalman filter can be applied examples in target tracking and digital communications are given to demonstrate the procedures proposed

second order necessary and sufficient conditions for convex composite ndo in convex composite ndo one studies the problem of minimizing functions of the formf h f whereh m is a finite valued convex function andf n m is continuously differentiable this problem model has a wide range of application in mathematical programming since many important problem classes can be cast within its framework e g convex inclusions minimax problems and penalty methods for constrained optimization in the present work we extend the second order theory developed by a d ioffe in 11 12 13 for the case in whichh is sublinear to arbitrary finite valued convex functionsh moreover a discussion of the second order regularity conditions is given that illuminates their essentially geometric nature

monte carlo estimation for nonlinear non gaussian state space models we develop a proposal or importance density for state space models with a nonlinear non gaussian observation vector y p y and an unobserved linear gaussian signal vector p the proposal density is obtained from the laplace approximation of the smoothing density p y we present efficient algorithms to calculate the mode of p y and to sample from the proposal density the samples can be used for importance sampling and markov chain monte carlo methods the new results allow the application of these methods to state space models where the observation density p y is not log concave additional results are presented that lead to computationally efficient implementations we illustrate the methods for the stochastic volatility model with leverage

sparse kalman filtering approaches to realized covariance estimation from high frequency financial data estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns market microstructure noise and jumps one technique for addressing both asynchronous returns and market microstructure is the kalman expectation maximization kem algorithm however the kem approach assumes log normal prices and does not address jumps in the return process which can corrupt estimation of the covariance matrix in this paper we extend the kem algorithm to price models that include jumps we propose a sparse kalman filtering approach to this problem in particular we develop a kalman expectation conditional maximization algorithm to determine the unknown covariance as well as detecting the jumps in order to promote a sparse estimate of the jumps we consider both laplace and the spike and slab jump priors numerical results using simulated data show that each of these approaches provide for improved covariance estimation relative to the kem method in a variety of settings where jumps occur