likelihood analysis of non gaussian measurement time series summary in this paper we provide methods for estimating non gaussian time series models these techniques rely on markov chain monte carlo to carry out simulation smoothing and bayesian posterior analysis of parameters and on importance sampling to estimate the likelihood function for classical inference the time series structure of the models is used to ensure that our simulation algorithms are efficient

particle filters for partially observed diffusions in this paper we introduce novel particle filters for a class of partially observed continuous time dynamic models where the signal is given by a multivariate diffusion process we consider a variety of observation schemes including diffusion observed with error observation of a subset of the components of the multivariate diffusion and arrival times of a poisson process whose intensity is a known function of the diffusion cox process unlike currently available methods our particle filters do not require approximations of the transition and or the observation density using time discretisations instead they build on recent methodology for the exact simulation of the diffusion process and the unbiased estimation of the transition density as described in beskos et al 2006 in particular we introduce the generalised poisson estimator which generalises the poisson estimator of beskos et al 2006 thus our filters avoid the systematic biases caused by time discretisations and they have significant computational advantages over alternative continuous time filters these advantages are supported theoretically by a central limit theorem

on line inference for multiple changepoint problems we propose an on line algorithm for exact filtering of multiple changepoint problems this algorithm enables simulation from the true joint posterior distribution of the number and position of the changepoints for a class of changepoint models the computational cost of this exact algorithm is quadratic in the number of observations we further show how resampling ideas from particle filters can be used to reduce the computational cost to linear in the number of observations at the expense of introducing small errors and propose two new optimum resampling algorithms for this problem one a version of rejection control allows the particle filter to automatically choose the number of particles required at each time step the new resampling algorithms substantially out perform standard resampling algorithms on examples we consider and we demonstrate how the resulting particle filter is practicable for segmentation of human gc content

monadic thue systems abstract certain infinite thue systems over a finite alphabet are studied in particular systems s e such that for each a e the set u u a s is a context freelanguage the syntactic structure of sets of ancestors and sets of descendants is considered as well as that of unions of congruence classes taken over infinite context free languages or regular sets the common descendant problem is shown to be tractable while the common ancestor problem is shown to be undecidable even for finite systems the word problem for confluent systems of this type is shown to be tractable the question of whether an infinite system of this type is confluent is shown to be undecidable as is the question of whether the congruence generated by such a system has a confluent presentation

codes modulo finite monadic string rewriting systems a set c is called a code modulo a string rewriting system t if for all v1 v2 vk w1 w2 wm c v1v2 vk tw1w2 wm implies that it is decidable whether a regular set is a code modulo t when t is a finite string rewriting system that is monadic and confluent or that is special and confluent

von karman howarth equation for three dimensional two fluid plasmas we derive the von karman howarth equation for a full three dimensional incompressible two fluid plasma in the long time limit and for very large reynolds numbers we obtain the equivalent of the hydrodynamic four fifths law this exact law predicts the scaling of the third order two point correlation functions and puts a strong constraint on the plasma turbulent dynamics finally we derive a simple expression for the 4 5 law in terms of third order structure functions which is appropriate for comparison with in situ measurements in the solar wind at different spatial ranges

smc 2 an efficient algorithm for sequential analysis of state space models we consider the generic problem of performing sequential bayesian inference in a state space model with observation process y state process x and fixed parameter theta an idealized approach would be to apply the iterated batch importance sampling ibis algorithm of chopin 2002 this is a sequential monte carlo algorithm in the theta dimension that samples values of theta reweights iteratively these values using the likelihood increments p y t y 1 t 1 theta and rejuvenates the theta particles through a resampling step and a mcmc update step in state space models these likelihood increments are intractable in most cases but they may be unbiasedly estimated by a particle filter in the x dimension for any fixed theta this motivates the smc 2 algorithm proposed in this article a sequential monte carlo algorithm defined in the theta dimension which propagates and resamples many particle filters in the x dimension the filters in the x dimension are an example of the random weight particle filter as in fearnhead et al 2010 on the other hand the particle markov chain monte carlo pmcmc framework developed in andrieu et al 2010 allows us to design appropriate mcmc rejuvenation steps thus the theta particles target the correct posterior distribution at each iteration t despite the intractability of the likelihood increments we explore the applicability of our algorithm in both sequential and non sequential applications and consider various degrees of freedom as for example increasing dynamically the number of x particles we contrast our approach to various competing methods both conceptually and empirically through a detailed simulation study included here and in a supplement and based on particularly challenging examples

some undecidability results concerning the property of preserving regularity a finite string rewriting system r preserves regularity if and only if it preserves regularity where is the alphabet containing exactly those letters that have occurrences in the rules of r this proves a conjecture of gyenizse and vagvolgyi 1997 in addition some undecidability results are presented that generalize results of gilleron and tison 1995 from term rewriting systems to string rewriting systems it follows that the property of being regularity preserving is undecidable for term rewriting systems thus answering another question of gyenizse and vagvolgyi 1997 finally it is shown that it is undecidable in general whether a finite lengthreducing and confluent string rewriting system yields a regular set of normal forms for each regular language

particle markov chain monte carlo methods summary markov chain monte carlo and sequential monte carlo methods have emerged as the two main tools to sample from high dimensional probability distributions although asymptotic convergence of markov chain monte carlo algorithms is ensured under weak assumptions the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and or if highly correlated variables are updated independently we show here how it is possible to build efficient high dimensional proposal distributions by using sequential monte carlo methods this allows us not only to improve over standard markov chain monte carlo schemes but also to make bayesian inference feasible for a large class of statistical models where this was not previously so we demonstrate these algorithms on a non linear state space model and a levy driven stochastic volatility model

monte carlo smoothing for nonlinear time series we develop methods for performing smoothing computations in general state space models the methods rely on a particle representation of the filtering distributions and their evolution through time using sequential importance sampling and resampling ideas in particular novel techniques are presented for generation of sample realizations of historical state sequences this is carried out in a forward filtering backward smoothing procedure that can be viewed as the nonlinear non gaussian counterpart of standard kalman filter based simulation smoothers in the linear gaussian case convergence in the mean squared error sense of the smoothed trajectories is proved showing the validity of our proposed method the methods are tested in a substantial application for the processing of speech signals represented by a time varying autoregression and parameterized in terms of time varying partial correlation coefficients comparing the results of our algorithm with those from a simple smoother based on the filte

on nonnegative unbiased estimators we study the existence of algorithms generating almost surely non negative unbiased estimators of some quantities of practical interest our work is motivated by recent exact approximation methods to compute integrals with respect to any target distribution up to any arbitrary precision these techniques can be implemented as long as non negative unbiased estimators of the target density evaluations are available in particular we show that given a non constant function f from r to r and real valued unbiased estimators xn of r there is no algorithm yielding almost surely non negative unbiased estimators of f even if xn is itself a s non negative then there is no algorithm yielding a s non negative unbiased estimators of f if f is not increasing however if the support of xn is a known interval of r then such algorithms exist as long as the function f satisfies some polynomial bound away from zero the results are discussed in the light of several typical situations corresponding to different functions f

random weight particle filtering of continuous time processes it is possible to implement importance sampling and particle filter algorithms where the importance sampling weight is random such random weight algorithms have been shown to be efficient for inference for a class of diffusion models as they enable inference without any time discretization approximation of the underlying diffusion model one difficulty of implementing such random weight algorithms is the requirement to have weights that are positive with probability 1 we show how wald s identity for martingales can be used to ensure positive weights we apply this idea to analysis of diffusion models from high frequency data for a class of diffusion models we show how to implement a particle filter which uses all the information in the data but whose computational cost is independent of the frequency of the data we use the wald identity to implement a random weight particle filter for these models which avoids time discretization error

a closure property of regular languages note given a semigroup homomorphism x p y where p y is the powerset of y we show that the inverse image k 1 of a regular set k y is regular too this theorem has several consequences generalizing previously known results

von karman howarth equation for magnetohydrodynamics and its consequences on third order longitudinal structure and correlation functions a derivation in variable dimension of the scaling laws for mixed third order longitudinal structure and correlation functions for incompressible magnetized flows is given for arbitrary correlation between the velocity and magnetic field with full isotropy homogeneity and incompressibility assumed when close to equipartition between kinetic and magnetic energy the scaling relations involve only structure functions in a manner similar to the frac 4 5 law of kolmogorov

sequential imputations and bayesian missing data problems abstract for missing data problems tanner and wong have described a data augmentation procedure that approximates the actual posterior distribution of the parameter vector by a mixture of complete data posteriors their method of constructing the complete data sets is closely related to the gibbs sampler both required iterations and similar to the em algorithm convergence can be slow we introduce in this article an alternative procedure that involves imputing the missing data sequentially and computing appropriate importance sampling weights in many applications this new procedure works very well without the need for iterations sensitivity analysis influence analysis and updating with new data can be performed cheaply bayesian prediction and model selection can also be incorporated examples taken from a wide range of applications are used for illustration

on line inference for hidden markov models via particle filters we consider the on line bayesian analysis of data by using a hidden markov model where inference is tractable conditional on the history of the state of the hidden component a new particle filter algorithm is introduced and shown to produce promising results when analysing data of this type the algorithm is similar to the mixture kalman filter but uses a different resampling algorithm we prove that this resampling algorithm is computationally efficient and optimal among unbiased resampling algorithms in terms of minimizing a squared error loss function in a practical example that of estimating break points from well log data our new particle filter outperforms two other particle filters one of which is the mixture kalman filter by between one and two orders of magnitude

rates of convergence of the hastings and metropolis algorithms we apply recent results in markov chain theory to hastings and metropolis algorithms with either independent or symmetric candidate distributions and provide necessary and sufficient conditions for the algorithms to converge at a geometric rate to a prescribed distribution pi in the independence case in mathbb r k these indicate that geometric convergence essentially occurs if and only if the candidate density is bounded below by a multiple of pi in the symmetric case in mathbb r only we show geometric convergence essentially occurs if and only if pi has geometric tails we also evaluate recently developed computable bounds on the rates of convergence in this context examples show that these theoretical bounds can be inherently extremely conservative although when the chain is stochastically monotone the bounds may well be effective

extended self similarity in turbulent flows we report on the existence of a hitherto undetected form of self similarity which we call extended self similarity ess ess holds at high as well as at low reynolds number and it is characterized by the same scaling exponents of the velocity differences of fully developed turbulence

application of extended self similarity in turbulence from navier stokes turbulence numerical simulations we show that for the extended self similarity ess method it is essential to take the third order structure function taken with the modulus and called d 3 r rather than the standard third order structure function d3 r itself if this is done we find ess towards scales larger than order 10 eta where eta is the kolmogorov scale if d3 r is used there is no ess we also analyze ess within the batchelor parametrization of the second and third order longitudinal structure function and focus on the scaling of the transversal structure function the re asymptotic inertial range scaling develops only beyond a taylor reynolds number re lambda 500

marginal likelihood from the gibbs output abstract in the context of bayes estimation via gibbs sampling with or without data augmentation a simple approach is developed for computing the marginal density of the sample data marginal likelihood given parameter draws from the posterior distribution consequently bayes factors for model comparisons can be routinely computed as a by product of the simulation hitherto this calculation has proved extremely challenging our approach exploits the fact that the marginal density can be expressed as the prior times the likelihood function over the posterior density this simple identity holds for any parameter value an estimate of the posterior density is shown to be available if all complete conditional densities used in the gibbs sampler have closed form expressions to improve accuracy the posterior density is estimated at a high density point and the numerical standard error of resulting estimate is derived the ideas are applied to probit regression and finite mixture models

filtering via simulation auxiliary particle filters this article analyses the recently suggested particle approach to filtering time series we suggest that the algorithm is not robust to outliers for two reasons the design of the simulators and the use of the discrete support to represent the sequentially updating prior distribution here we tackle the first of these problems

non gaussian ornstein uhlenbeck based models and some of their uses in financial economics non gaussian processes of ornstein uhlenbeck ou type offer the possibility of capturing important distributional deviations from gaussianity and for flexible modelling of dependence structures this paper develops this potential drawing on and extending powerful results from probability theory for applications in statistical analysis their power is illustrated by a sustained application of ou processes within the context of finance and econometrics we construct continuous time stochastic volatility models for financial assets where the volatility processes are superpositions of positive ou processes and we study these models in relation to financial data and theory

nonparametric estimation of diffusions a differential equations approach we consider estimation of scalar functions that determine the dynamics of diffusion processes it has been recently shown that nonparametric maximum likelihood estimation is ill posed in this context we adopt a probabilistic approach to regularize the problem by the adoption of a prior distribution for the unknown functional a gaussian prior measure is chosen in the function space by specifying its precision operator as an appropriate differential operator we establish that a bayesian gaussian conjugate analysis for the drift of one dimensional nonlinear diffusions is feasible using high frequency data by expressing the loglikelihood as a quadratic function of the drift with sufficient statistics given by the local time process and the end points of the observed path computationally efficient posterior inference is carried out using a finite element method we embed this technology in partially observed situations and adopt a data augmentation approach whereby we iteratively generate missing data paths and draws from the unknown functional our methodology is applied to estimate the drift of models used in molecular dynamics and financial econometrics using high and low frequency observations we discuss extensions to other partially observed schemes and connections to other types of nonparametric inference

variance reduction for particle filters of systems with time scale separation we present a particle filter construction for a system that exhibits time scale separation the separation of time scales allows two simplifications that we exploit 1 the use of the averaging principle for the dimensional reduction of the dynamics for each particle during the prediction step and 2 the factorization of the transition probability for the rao blackwellization of the update step the resulting particle filter is faster and has smaller variance than the particle filter based on the original system the method is tested on a multiscale stochastic differential equation and on a multiscale pure jump diffusion motivated by chemical reactions

unbiased monte carlo evaluation of certain functional integrals abstract a monte carlo algorithm for the evaluation of certain functional integrals is introduced this algorithm is not based on the approximation of the functional integral by a finite dimensional integral unbiased estimators for the functional integrals are constructed and corresponding variance reduction techniques are considered numerical results of simulation studies are presented

the simulation smoother for time series models summary recently suggested procedures for simulating from the posterior density of states given a gaussian state space time series are refined and extended we introduce and study the simulation smoother which draws from the multivariate posterior distribution of the disturbances of the model so avoiding the degeneracies inherent in state samplers the technique is important in gibbs sampling with non gaussian time series models and for performing bayesian analysis of gaussian time series