fast and accurate binary response mixed model analysis via expectation propagation expectation propagation is a general prescription for approximation of integrals in statistical inference problems its literature is mainly concerned with bayesian inference scenarios however ex

on stable oscillations and equilibriums induced by small noise we consider the motion of a light particle when the force field is perturbed by a small nose if a certain relation between the mass of the particle and the noise intensity holds the motion of the particle will be close to periodic oscillations or to a stable equilibrium which do not exist without the noise we study various classes of random perturbations in particular we consider the question of computer simulation of these effects and calculate the correction term which appears when the gaussian perturbations are replaced by the simple random walk these are the stochastic resonance type effects and their mathematical description is based on the large deviation theory

an ordered line integral method for computing the quasi potential in the case of variable anisotropic diffusion abstract nongradient stochastic differential equations sdes with position dependent and anisotropic diffusion are often used in biological modeling the quasi potential is a crucial function in the large deviation theory that allows one to estimate transition rates between attractors of the corresponding ordinary differential equation and find the maximum likelihood transition paths unfortunately the quasi potential can rarely be found analytically it is defined as the solution to a certain action minimization problem in this work the recently introduced ordered line integral method olim is extended for computing the quasi potential for 2d sdes with anisotropic and position dependent diffusion scaled by a small parameter on a regular rectangular mesh the presented solver employs the dynamical programming principle at each step a local action minimization problem is solved using straight line path segments and the midpoint quadrature rule the solver is tested on two examples where analytic formulas for the quasi potential are available the dependence of the computational error on the mesh size the update factor k a key parameter of olims as well as the degree and the orientation of anisotropy is established the effect of anisotropy on the quasi potential and the maximum likelihood paths is demonstrated on the maier stein model the proposed solver is applied to find the quasi potential and the maximum likelihood transition paths in a model of the genetic switch in lambda phage between the lysogenic state where the phage reproduces inside the infected cell without killing it and the lytic state where the phage destroys the infected cell

two distinct mechanisms of coherence in randomly perturbed dynamical systems we carefully examine two mechanisms coherence resonance and self induced stochastic resonance by which small random perturbations of excitable systems with large time scale separation may lead to the emergence of new coherent behaviors in the form of limit cycles we analyze what controls the degree of coherence in these two mechanisms and classify their very different properties in particular we show that coherence resonance arises only at the onset of bifurcation and is rather insensitive against variations in the noise amplitude and the time scale separation ratio in contrast self induced stochastic resonance may arise away from bifurcations and the properties of the limit cycle it induces are controlled by both the noise amplitude and the time scale separation ratio

efficiency analysis of multihypothesis motion compensated prediction for video coding overlapped block motion compensation or b frames are examples of multihypothesis motion compensation where several motion compensated signals are superimposed to reduce the bit rate of a video codec this paper extends the wide sense stationary theory of motion compensated prediction mcp for hybrid video codecs to multihypothesis motion compensation the power spectrum of the prediction error is related to the displacement error probability density functions pdfs of an arbitrary number of hypotheses in a closed form expression we then study the influence of motion compensation accuracy on the efficiency of multihypothesis motion compensation as well as the influence of the residual noise level and the gain from optimal combination of n hypotheses for the noise free limiting case doubling the number of equally good hypotheses can yield a gain of up to 1 2 bits sample while doubling the accuracy of motion compensation can additionally reduce the bit rate by up to 1 bit sample independent of n for realistic noise levels it is shown that the introduction of b frames or overlapped block motion compensation can provide larger gains than doubling motion compensation accuracy spatial filtering of the motion compensated candidate signals becomes less important if more hypotheses are combined the critical accuracy beyond which the gain due to more accurate motion compensation is small moves to larger displacement error variances with increasing noise and increasing number of hypotheses n hence sub pel accurate motion compensation becomes less important with multihypothesis mcp the theoretical insights are confirmed by experimental results for overlapped block motion compensation b frames and multiframe motion compensated prediction with up to eight hypotheses from ten previous frames

singularities of fluctuational paths for an overdamped two well system driven by white noise abstract noise induced escape from a metastable state in an overdamped double well system is investigated with the wkb approximation employed the pattern of fluctuational paths is given by using the action plot which gives rise to mpeps the pattern is divided into various regions where fluctuational paths inside show qualitatively distinguished characteristics the global quasi potential is calculated and its value at the saddle on the boundary agrees with the one obtained by action plot we show a contour and 3d plot of the quasi potential to illustrate the relative stabilities of metastable states the bifurcations of singularities as the system parameter varies are analyzed by the second order term of the wkb expansion in the vicinity of x axis its asymptotic behavior determines the positions of cusps and furthermore its evolution turns out to delineate the twisting structure of the lagrangian manifold

crossing the quasi threshold manifold of a noise driven excitable system we consider the noise induced escapes in an excitable system possessing a quasi threshold manifold along which there exists a certain point of minimal quasi potential in the weak noise limit the optimal escaping path turns out to approach this particular point asymptotically making it analogous to an ordinary saddle numerical simulations are performed and an elaboration on the effect of small but finite noise is given which shows that the ridges where the prehistory probability distribution peaks are located mainly within the region where the quasi potential increases gently the cases allowing anisotropic noise are discussed and we found that varying the noise term in the slow variable would dramatically raise the whole level of quasi potentials leading to significant changes in both patterns of optimal paths and exit locations

ordered upwind methods for static hamilton jacobi equations abstract we introduce a family of fast ordered upwind methods for approximating solutions to a wide class of static hamilton jacobi equations with dirichlet boundary conditions standard techniques often rely on iteration to converge to the solution of a discretized version of the partial differential equation our fast methods avoid iteration through a careful use of information about the characteristic directions of the underlying partial differential equation these techniques are of complexity o m log m where m is the total number of points in the domain we consider anisotropic test problems in optimal control seismology and paths on surfaces

a variational maximization maximization algorithm for generalized linear mixed models with crossed random effects we present a variational maximization maximization algorithm for approximate maximum likelihood estimation of generalized linear mixed models with crossed random effects e g item response models with random items random raters or random occasion specific effects the method is based on a factorized variational approximation of the latent variable distribution given observed variables which creates a lower bound of the log marginal likelihood the lower bound is maximized with respect to the factorized distributions as well as model parameters with the proposed algorithm a high dimensional intractable integration is translated into a two dimensional integration problem we incorporate an adaptive gauss hermite quadrature method in conjunction with the variational method in order to increase computational efficiency numerical studies show that under the small sample size conditions that are considered the proposed algorithm outperforms the laplace approximation

the geometric minimum action method a least action principle on the space of curves freidlin wentzell theory of large deviations for the description of the effect of small random perturbations on dynamical systems is exploited as a numerical tool specifically a numerical algorithm is proposed to compute the quasi potential in the theory which is the key object to quantify the dynamics on long time scales when the effect of the noise becomes ubiquitous the equilibrium distribution of the system the pathways of transition between metastable states and their rate etc can all be expressed in terms of the quasi potential we propose an algorithm to compute these quantities called the geometric minimum action method gmam which is a blend of the original minimum action method mam and the string method it is based on a reformulation of the large deviations action functional on the space of curves that allows one to easily perform the double minimization of the original action required to compute the quasi potential the theoretical background of the gmam in the context of large deviations theory is discussed in detail as well as the algorithmic aspects of the method the gmam is then illustrated on several examples a finite dimensional system displaying bistability and modeled by a nongradient stochastic ordinary differential equation an infinite dimensional analogue of this system modeled by a stochastic partial differential equation and an example of a bistable genetic switch modeled by a markov jump process

fast marching methods fast marching methods are numerical schemes for computing solutions to the nonlinear eikonal equation and related static hamilton jacobi equations based on entropy satisfying upwind schemes and fast sorting techniques they yield consistent accurate and highly efficient algorithms they are optimal in the sense that the computational complexity of the algorithms is o n log n where n is the total number of points in the domain the schemes are of use in a variety of applications including problems in shape offsetting computing distances from complex curves and surfaces shape from shading photolithographic development computing first arrivals in seismic travel times construction of shortest geodesics on surfaces optimal path planning around obstacles and visibility and reflection calculations in this paper we review the development of these techniques including the theoretical and numerical underpinnings provide details of the computational schemes including higher order versions and demonstrate the techniques in a collection of different areas

exploring a noisy van der pol type oscillator with a stochastic approach based on conventional ito or stratonovich interpretation zero mean multiplicative noise can induce shifts of attractors or even changes of topology to a deterministic dynamics such phenomena usually introduce additional complications in analysis of these systems we employ in this paper a new stochastic interpretation leading to a straightforward consequence the steady state distribution is boltzmann gibbs type with a potential function severing as a lyapunov function for the deterministic dynamics it implies that an attractor corresponds to the local extremum of the distribution function and the probability is equally distributed right on an attractor we consider a prototype of nonequilibrium processes noisy limit cycle dynamics exact results are obtained for a class of limit cycles including a van der pol type oscillator these results provide a new angle for understanding processes without detailed balance and can be verified by experiments

a sample reuse method for accurate parametric empirical bayes confidence intervals abstract parametric empirical bayes methods of point estimation for a vector of unknown parameters date to the landmark paper of james and stein 1961 the usual approach is to use the mean of the estimated posterior distribution of each parameter where the estimation of the prior parameters hyperparameters is accomplished through the marginal distribution of the data while point estimates computed this way usually perform well interval estimates based on the estimated posterior called naive eb intervals are not they fail to account for the variability in the estimation of the hyperparameters generally resulting in sub nominal coverage probability in the eb sense defined in morris 1983a in this paper we extend the work of carlin and gelfand 1989 who proposed a conditional bias correction method for developing eb intervals which corrects the deficiencies in the naive intervals we show how bias correction can be implemented in general via a type iii parametric bootstrap procedure a sample reuse method first employed by laird and louis 1987

ordered upwind methods for static hamilton jacobi equations theory and algorithms we develop a family of fast methods for approximating the solutions to a wide class of static hamilton jacobi pdes these fast methods include both semi lagrangian and fully eulerian versions numerical solutions to these problems are typically obtained by solving large systems of coupled nonlinear discretized equations our techniques which we refer to as ordered upwind methods oums use partial information about the characteristic directions to decouple these nonlinear systems greatly reducing the computational labor our techniques are considered in the context of control theoretic and front propagation problems we begin by discussing existing oums focusing on those designed for isotropic problems we then introduce a new class of oums which decouple systems for general anisotropic problems we prove convergence of one such scheme to the viscosity solution of the corresponding hamilton jacobi pde next we introduce a set of finite differences methods based on an analysis of the role played by anisotropy in the context of front propagation and optimal trajectory problems the performance of the methods is analyzed and computational experiments are performed using test problems from computational geometry and seismology

approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations summary structured additive regression models are perhaps the most commonly used class of models in statistical applications it includes among others generalized linear models generalized additive models smoothing spline models state space models semiparametric regression spatial and spatiotemporal models log gaussian cox processes and geostatistical and geoadditive models we consider approximate bayesian inference in a popular subset of structured additive regression models latent gaussian models where the latent field is gaussian controlled by a few hyperparameters and with non gaussian response variables the posterior marginals are not available in closed form owing to the non gaussian response variables for such models markov chain monte carlo methods can be implemented but they are not without problems in terms of both convergence and computational time in some practical applications the extent of these problems is such that markov chain monte carlo sampling is simply not an appropriate tool for routine analysis we show that by using an integrated nested laplace approximation and its simplified version we can directly compute very accurate approximations to the posterior marginals the main benefit of these approximations is computational where markov chain monte carlo algorithms need hours or days to run our approximations provide more precise estimates in seconds or minutes another advantage with our approach is its generality which makes it possible to perform bayesian analysis in an automatic streamlined way and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged

when lyapunov exponents fail to exist we describe a simple continuous time flow such that lyapunov exponents fail to exist at nearly every point in the phase space mathbb r 2 despite the fact that the flow admits a unique natural measure this example illustrates that the existence of lyapunov exponents is a subtle question for systems that are not conservative

a scaling theory of bifurcations in the symmetric weak noise escape problem we consider two dimensional overdamped double well systems perturbed by white noise in the weak noise limit the most probable fluctuational path leading from either point attractor to the separatrix the most probable escape path or mpep must terminate on the saddle between the two wells however as the parameters of a symmetric double well system are varied a unique mpep may bifurcate into two equally likely mpeps at the bifurcation point in parameter space the activation kinetics of the system become non arrhenius we quantify the non arrhenius behavior of a system at the bifurcation point by using the maslov wkb method to construct an approximation to the quasistationary probability distribution of the system that is valid in a boundary layer near the separatrix the approximation is a formal asymptotic solution of the smoluchowski equation our construction relies on a new scaling theory which yields critical exponents describing weak noise behavior at the bifurcation point near the saddle

finding the quasipotential for nongradient sdes abstract the quasipotential with respect to a given attractor of a system evolving according to a stochastic differential equation sde with a small noise is a function that allows us to find the expected exit time from its basin of attraction and to estimate the equilibrium probability density in its neighborhood this work is devoted to the study of the quasipotential of nongradient sdes with isotropic diffusion the goals are i to understand the theoretical behavior of the quasipotential and ii to develop tools for its numerical study this work is focused on 2d equations with asymptotically stable equilibria and limit cycles in the theoretical part we establish a number of properties of the quasipotential for 2d linear equations we derive an exact formula for the quasipotential for 2d nonlinear equations we establish how the quasipotential grows near the stable equilibria and the stable limit cycles in the numerical part of this work we adjust the oum for computing the quasipotential on a regular mesh we show that the quasipotential satisfies a hamilton jacobi bellman equation with an anisotropic and unbounded speed function the unbounded speed function may lead to an additional consistency error in specific areas of the phase space however we conduct an error analysis and show that even if this error appears it decays quadratically as we refine the mesh we apply the proposed numerical algorithm to a number of examples linear systems the maier stein model a system with stochastic resonance and a system with two stable limit cycles whenever it is appropriate we compute the minimum action paths by shooting characteristics the minimum action paths found in this way are necessarily the global minimizers of the freidlin wentzell action

expectation propagation in the large data limit summary expectation propagation ep is a widely successful algorithm for variational inference ep is an iterative algorithm used to approximate complicated distributions typically to find a gaussian approximation of posterior distributions in many applications of this type ep performs extremely well surprisingly despite its widespread use there are very few theoretical guarantees on gaussian ep and it is quite poorly understood to analyse ep we first introduce a variant of ep averaged ep which operates on a smaller parameter space we then consider averaged ep and ep in the limit of infinite data where the overall contribution of each likelihood term is small and where posteriors are almost gaussian in this limit we prove that the iterations of both averaged ep and ep are simple they behave like iterations of newton s algorithm for finding the mode of a function we use this limit behaviour to prove that ep is asymptotically exact and to obtain other insights into the dynamic behaviour of ep e g that it may diverge under poor initialization exactly like newton s method ep is a simple algorithm to state but a difficult one to study our results should facilitate further research into the theoretical properties of this important method

hyperbolic periodic orbits in nongradient systems and small noise induced metastable transitions abstract small noise can induce rare transitions between metastable states which can be characterized by maximum likelihood paths mlps nongradient systems contrast gradient systems in that mlp does not have to cross the separatrix at a saddle point but instead possibly at a point on a hyperbolic periodic orbit a numerical approach for identifying such unstable periodic orbits is proposed based on string method in a special class of nongradient systems orthogonal type there are provably local mlps that cross such saddle point or hyperbolic periodic orbit and the separatrix crossing location determines the associated local maximum of transition rate in general cases however the separatrix crossing may not determine a unique local maximum of the rate as we numerically observed a counter example in a sheared 2d space allen cahn spde it is a reasonable conjecture that there are always local mlps associated with each attractor on the separatrix such as saddle point or hyperbolic periodic orbit our numerical experiments did not disprove so

estimability and likelihood inference for generalized linear mixed models using data cloning maximum likelihood estimation for generalized linear mixed models glmm an important class of statistical models with substantial applications in epidemiology medical statistics and many other fields poses significant computational difficulties in this article we use data cloning a simple computational method that exploits advances in bayesian computation in particular the markov chain monte carlo method to obtain maximum likelihood estimators of the parameters in these models this method also leads to a simple estimator of the asymptotic variance of the maximum likelihood estimators determining estimability of the parameters in a mixed model is in general a very difficult problem data cloning provides a simple graphical test to not only check if the full set of parameters is estimable but also and perhaps more importantly if a specified function of the parameters is estimable one of the goals of mixed models is to predict random effects we suggest a frequentist method to obtain predict