a direct approach to the exit problem this paper considers the problem of exit for a dynamical system driven by small white noise from the domain of attraction of a stable state a direct singular perturbation analysis of the forward equation is presented based on kramers approach in which the solution to the stationary fokker planck equation is constructed for a process with absorption at the boundary and a source at the attractor in this formulation the boundary and matching conditions fully determine the uniform expansion of the solution without resorting to external selection criteria for the expansion coefficients such as variational principles or the lagrange identity as in our previous theory the exit density and the mean first passage time to the boundary are calculated from the solution of the stationary fokker planck equation as the probability current density and as the inverse of the total flux on the boundary respectively as an application a uniform expansion is constructed for the escape rate in kramers problem o

random perturbations of heteroclinic attractors estimates are derived for the mean recurrence time of orbits in the neighborhood of an attracting homoclinic orbit or heteroclinic cycle in an ordinary differential equation subject to small additive random noise the theory presented is illustrated with numerical simulations of several systems including ones invariant under symmetry groups for which such heteroclinic attractors are structurally stable the physical implications of the work presented are briefly discussed

uniqueness and stability of optimal policies of finite state markov decision processes in this paper we consider infinite horizon discrete time optimal control of markov decision processes mdps with finite state spaces and compact action sets we restrict attention to unicost mdps which form a class that contains all the weakly communicating mdps the unicost mdps are characterized as those mdps for which there exists a solution to the single optimality equation we address the problem of uniqueness and stability of minimizing markov actions our main result asserts that when we endow the set of unicost mdps with a certain natural metric under which it is complete then the class of mdps with essentially unique and stable minimizing markov actions contains the intersection of countably many open dense sets hence is itself dense thus the property of having essentially unique and stable minimizing markov actions is generic for unicost mdps

a period doubling bifurcation with slow parametric variation and additive noise slow sinusoidal modulation of a control parameter can maintain a low period orbit into parameter regions where the low period orbit is locally unstable and a higherperiod orbit would normally occur whether or not a bifurcation to higher period becomes evident during the modulation depends on the competing effects of stabilization by the modulation and destabilization by inherent very low level system noise a transition often rapid from a locally unstable period 1 orbit to period 2 for example can be triggered by noise the competing effects are examined here for a period doubling bifurcation of a general unimodal map a nested set of three matched asymptotic expansions a triple deck is used to describe the combined period 1 and period 2 response the resulting solution gives estimates of whether and where an apparent period doubling bifurcation occurs typical period 1 stability boundaries are obtained that include the effect of the amplitude and frequency of the variation the noise level in the system and the allowable maximum threshold level of period 2 response

surmounting oscillating barriers path integral approach for weak noise we consider the thermally activated escape of an overdamped brownian particle over a potential barrier in the presence of periodic driving a time dependent path integral formalism is developed which allows us to derive asymptotically exact weak noise expressions for both the instantaneous and the time averaged escape rate our results comprise a conceptually different systematic treatment of the rate prefactor multiplying the exponentially leading arrhenius factor moreover an estimate for the deviations at finite noise strengths is provided and a supersymmetry type property of the time averaged escape rate is verified for piecewise parabolic potentials the rate expression can be evaluated in closed analytical form while in more general cases as exemplified by a cubic potential an action integral remains to be minimized numerically our comparison with very accurate numerical results demonstrates an excellent agreement with the theoretical predictions over a wide range of driving strengths and driving frequencies

sensitivity analysis of the stochastically and periodically forced brusselator abstract the problem of sensitivity of nonlinear system limit cycle with respect to small stochastic and periodic disturbances is considered sensitivity analysis on the basis of quasipotential function is performed the quasipotential is used widely in statistical physics for instance by graham for analysis of nonequilibrium thermodynamics problem we consider an application of quasipotential technique to sensitivity problem for the plane orbit case an approximation of quasipotential is expressed by some scalar function this function sensitivity function is introduced as a base tool of a quantitative description for a system response on the external disturbances new cycle characteristics sensitivity factor parameter of stiffness are considered the analysis of the forced brusselator based on sensitivity function is shown from this analysis the critical value of brusselator parameter is found the dynamics of forced brusselator for this critical value is investigated for small stochastic disturbances the burst of response amplitude is shown for small periodic disturbances the period doubling regime of the transition to chaos scenario is demonstrated

the effect of noise on β cell burst period bursting electrical behavior is commonly observed in a variety of nerve and endocrine cells including that in electrically coupled cells located in intact pancreatic islets however individual cells usually display either spiking or very fast bursting behavior and the difference between isolated and coupled cells has been suggested to be due to stochastic fluctuations of the plasma membrane ion channels which are supposed to have a stronger effect on single cells than on cells situated in clusters the channel sharing hypothesis this effect of noise has previously been studied using numerical simulations we show here how the application of two recent methods allows an analytic treatment of the stochastic effects on the location of the saddle node and homoclinic bifurcations which determine the burst period thus the stochastic system can be analyzed similarly to the deterministic system but with a quantitative description of the effect of noise this approach supports previous investigations

beyond the fokker planck equation pathwise control of noisy bistable systems we introduce a new method allowing one to describe slowly time dependent langevin equations through the behaviour of individual paths this approach yields considerably more information than the computation of the probability density in particular scaling laws can be obtained easily the main idea is to show that for sufficiently small noise intensity and slow time dependence the vast majority of paths remain in small space time sets typically in the neighbourhood of potential wells the size of these sets often has a power law dependence on the small parameters with universal exponents the overall probability of exceptional paths is exponentially small with an exponent also showing power law behaviour the results cover time spans up to the maximal kramers time of the system we apply our method to three phenomena characteristic for bistable systems stochastic resonance dynamical hysteresis and bifurcation delay where it yields precise bounds on transition probabilities and the distribution of hysteresis areas and first exit times we also discuss the effect of coloured noise

control of noisy heteroclinic cycles abstract we develop a control strategy for strongly nonlinear dynamical systems possessing heteroclinic cycles connecting saddle points and subject to small random perturbations in the absence of control attracting cycles lead to solutions which exhibit intermittent behavior consisting of quiescent periods interrupted by rapid bursts we show that suitable feedback control can retain solutions near the saddle points and hence reduce the typical bursting rate the model studied and the form of control are motivated by our interest in reducing turbulence production in the boundary layer

solution of the boundary value problem for optimal escape in continuous stochastic systems and maps topologies of invariant manifolds and optimal trajectories are investigated in stochastic continuous systems and maps a topological method is introduced that simplifies the solution of boundary value problems the activation energy is calculated as a function of a set of parameters characterizing the initial conditions of the escape path the method is applied explicitly to compute the optimal escape path and the activation energy for a variety of dynamical systems and maps

domain formation in transitions with noise and a time dependent bifurcation parameter the characteristic size for spatial structure that emerges when the bifurcation parameter in model partial differential equations is slowly increased through its critical value depends logarithmically on the size of added noise numerics and analysis are presented for the real ginzburg landau and swift hohenberg equa tions

destabilization by noise of transverse perturbations to heteroclinic cycles a simple model and an example from dynamo theory we show that transverse perturbations from structurally stable heteroclinic cycles can be destabilized by surprisingly small amounts of noise even when each individual fixed point of the cycle is stable to transverse modes a condition that favours this process is that the linearization of the dynamics in the transverse direction be characterized by a non normal matrix the phenomenon is illustrated by a simple two dimensional switching model and by a simulation of a convectively driven dynamo

analogy between a 10d model for nonlinear wave wave interaction in a plasma and the 3d lorenz dynamics this paper investigates nonlinear wave wave interactions in a system that describes a modified decay instability and consists of three langmuir and one ion sound waves as a means to establish that the underlying dynamics exists in a 3d space and that it is of the lorenz type both continuous and discrete time multivariable global models were obtained from data these data were obtained from a 10d dynamical system that describes the modified decay instability obtained from zakharov s equations which characterise langmuir turbulence this 10d model is equivariant under a continuous rotation symmetry and a discrete order 2 rotation symmetry when the continuous rotation symmetry is modded out that is when the dynamics are represented with the continuous rotation symmetry removed under a local diffeomorphism it is shown that a 3d system may describe the underlying dynamics for certain parameter values the models obtained using global modelling techniques from three time series from the 10d dynamics with the continuous rotation symmetry modded out generate attractors which are topologically equivalent these models can be simulated easily and due to their simplicity are amenable for analysis of the original dynamics after symmetries have been modded out moreover it is shown that all of these attractors are topologically equivalent to an attractor generated by the well known lorenz system 2003 elsevier science b v all rights reserved

escape through an unstable limit cycle resonant activation we consider a brownian particle acted on by a linear conservative force a nonlinear frictional force and multiplicative colored and additive white noises the frictional force can be negative when the external energy supply is large enough we numerically calculate the mean first passage time mfpt for the particle to escape from an unstable limit cycle and find resonant activation i e the mfpt first decreases followed by a rise after passing through a minimum with increasing noise correlation time tau for a fixed noise variance for fixed noise strength of the multiplicative noise the mfpt increases linearly with tau this is in sharp contrast to the case of fluctuations of nonlinear potentials in which the mfpt first increases nonlinearly before reaching a limiting value our model could be useful for understanding some biological processes

convergence to equilibrium of the stochastic heisenberg model the stochastic heisenberg model is a probabilistic model of the time evolution of a classical heisenberg ferromagnet it is proved that the stochastic process converges to equilibrium at sufficiently high temperatures and that the equilibrium state is a gibbs state of the hamiltonian possessing the global markov property the principal technique employed is the expansion of observables on the state space into laplace series

colored noise and a characteristic level crossing problem in this paper we discuss the activation of a dynamical system of one spatial variable forced by an external gaussian stationary colored noise the model is an analog of the smoluchowski equation when external fluctuations are present we assume that the underlying deterministic dynamics are derived from a potential forming a well we enlarge the state space by including the noise variable and then use half range expansion techniques singular perturbations and matched asymptotics to solve the boundary value problem for the steady state probability density function in the asymptotic limit of small correlation time and over the entire regime of noise strength we find a uniformly valid expression for the activation rate to the top of the potential barrier a characteristic point for the unforced dynamical system we show the important effects of the boundary behavior of the process on its activation rate if the intensity of the noise is small compared to the autocorrelation of the noise then the effective activation energy is modified by the noise autocorrelation we consider an example of the activation process in a double well potential and compare our uniformly valid results with results reported in the literature

an algorithm to identify and compute average optimal policies in multichain markov decision processes this paper concerns discrete time finite state multichain mdps with compact action sets the optimality criterion is long run average cost simple examples illustrate that optimal stationary markov policies do not always exist we establish the existence of e optimal policies that are stationary markovian and develop an algorithm that computes these approximate optimal policies we establish a necessary and sufficient condition for the existence of an optimal policy that is stationary markovian and in case that such an optimal policy exists the algorithm computes it

an ordered line integral method for computing the quasi potential in the case of variable anisotropic diffusion abstract nongradient stochastic differential equations sdes with position dependent and anisotropic diffusion are often used in biological modeling the quasi potential is a crucial function in the large deviation theory that allows one to estimate transition rates between attractors of the corresponding ordinary differential equation and find the maximum likelihood transition paths unfortunately the quasi potential can rarely be found analytically it is defined as the solution to a certain action minimization problem in this work the recently introduced ordered line integral method olim is extended for computing the quasi potential for 2d sdes with anisotropic and position dependent diffusion scaled by a small parameter on a regular rectangular mesh the presented solver employs the dynamical programming principle at each step a local action minimization problem is solved using straight line path segments and the midpoint quadrature rule the solver is tested on two examples where analytic formulas for the quasi potential are available the dependence of the computational error on the mesh size the update factor k a key parameter of olims as well as the degree and the orientation of anisotropy is established the effect of anisotropy on the quasi potential and the maximum likelihood paths is demonstrated on the maier stein model the proposed solver is applied to find the quasi potential and the maximum likelihood transition paths in a model of the genetic switch in lambda phage between the lysogenic state where the phage reproduces inside the infected cell without killing it and the lytic state where the phage destroys the infected cell

testing transition state theory on kac zwanzig model a variant of the kac zwanzig model is used to test the prediction of transition state theory tst and variational transition state theory vtst the model describes the evolution of a distinguished particle moving in a double well external potential and coupled to n free particles through linear springs while the kac zwanzig model is deterministic under appropriate choice of the model parameters the evolution of the distinguished particle can be approximated by a two state markov chain whose transition rate constants can be computed exactly in suitable limit here these transition rate constants are compared with the predictions of tst and vtst it is shown that the application of tst with a naive albeit natural choice of dividing surface leads to the wrong prediction of the transition rate constants this is due to crossings of the dividing surface that do not correspond to actual transition events however optimizing over the dividing surface within vtst allows one to eliminate completely these spurious crossings and therefore derive the correct transition rate constants for the model the reasons why vtst is successful in this model are discussed which allows one to speculate on the reliability of vtst in more complicated systems

the exit problem at weak noise the two variable quasipotential and the kramers problem a new approach to the weak noise analysis of exit problems removes an intrinsic contradiction of an existing method it applies to both the mean time and the location of the exits novel results mainly concern the exits from an entire domain of attraction

discrete time controlled markov processes with average cost criterion a survey this work is a survey of the average cost control problem for discrete time markov processes the authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades the exposition ranges from finite to borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies the authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography the authors have also identified several important questions that are still open to investigation

some flesh on the skeleton the bifurcation structure of bimodal maps abstract one of the main tools in the numerical study of two parameter families of one dimensional maps is the drawing of curves in parameter space corresponding to the existence of superstable periodic orbits we use kneading theory to describe the structure of these sets of curves for the case of maps with at most two turning points then we explain how the bifurcation structure hangs on this skeleton

multidimensional activated rate processes with slowly relaxing mode abstract the multidimensional kramers problem is considered in the case of highly anisotropic friction as a development of our previous results involving the study of several new regimes of activated rate process we construct a theory which incorporates all these regimes and treats the process in every case in a uniform manner to do this we take advantage of the friction anisotropy and reduce the initial multidimensional fokker planck equation to a set of two effective one dimensional equations these equations describe diffusive motion along the slow coordinate in two effective potential wells and inter well transitions due to the fast coordinate motion to calculate the rate constant on the basis of the set we use an original method which is a generalization of the kramers method as a result we obtain a new formula for the rate constant which depending on the friction anisotropy level is smoothly changed from the well known kramers langer formula to new ones obtained in our previous papers

disease extinction in the presence of random vaccination we investigate disease extinction in an epidemic model described by a birth death process we show that in the absence of vaccination the effective entropic barrier for extinction displays scaling with the distance to the bifurcation point with an unusual critical exponent even a comparatively weak poisson distributed random vaccination leads to an exponential increase in the extinction rate with the exponent that strongly depends on the vaccination parameters

a computational analysis for mean exit time under non gaussian levy noises abstract complex dynamical systems are often subject to non gaussian random fluctuations the exit phenomenon i e escaping from a bounded domain in state space is an impact of randomness on the evolution of these dynamical systems the existing work is about asymptotic estimate on mean exit time when the noise intensity is sufficiently small in the present paper however the authors analyze mean exit time for arbitrary noise intensity via numerical investigation the mean exit time for a dynamical system driven by a non gaussian discontinuous with jumps stable levy motion is described by a differential equation with nonlocal interactions a numerical approach for solving this nonlocal problem is proposed a computational analysis is conducted to investigate the relative importance of jump measure diffusion coefficient and non gaussianity in affecting mean exit time

topological features of large fluctuations to the interior of a limit cycle received 29 october 1996 we investigate the pattern of optimal paths along which a dynamical system driven by weak noise moves with overwhelming probability when it fluctuates far away from a stable state our emphasis is on systems that perform self sustained periodic vibrations and have an unstable focus inside a stable limit cycle we show that in the vicinity of the unstable focus the flow field of optimal paths generically displays a pattern of singularities in particular it contains a switching line that separates areas to which the system arrives along optimal paths of topologically different types the switching line spirals into the focus and has a self similar structure depending on the behavior of the system near the focus it may be smooth or have finite length branches our results are based on an analysis of the topology of the lagrangian manifold for an auxiliary purely dynamical problem that determines the optimal paths we illustrate our theory by studying both theoretically and numerically a van der pol oscillator driven by weak white noise s1063 651x 97 02703 7 many nonlinear dissipative systems display stable periodic oscillations i e their state spaces contain stable limit cycles 1 examples include lasers 2 radiofrequency generators 1 a chemical 3 a and biological 3 b systems in many cases the vibrations are nonsinusoidal as in a multimode laser with strong mode coupling 4 passive optically bistable elements 5 and various sorts of engineering structure 6 fluctuations in periodically oscillating systems play an extremely important role they eventually destroy periodicity of the self sustained oscillations so that the oscillation phase becomes random and a stationary probability distribution in the state space of the system is formed 7 8 this distribution reaches a maximum at the cycle does not vary much along the cycle and falls off rapidly in directions transverse to the cycle the shape of the maximum depends on the character of small transverse fluctuations away from the cycle if the fluctuations are due to the system being perturbed by weak noise of intensity d the transverse falloff will usually be approximately gaussian 7 9 with standard deviation proportional to d 1 2 as d 0

diffusion across characteristic boundaries with critical points we consider the problems of the effect of small white noise perturbations on a deterministic dynamical system in the plane with i an asymptotically stable equilibrium point or limit cycle and ii an equilibrium point surrounded by closed trajectories the mean exit time and the distribution of exit points for each problem is determined by solving singularly perturbed elliptic boundary value problems in domains with closed characteristic boundaries with critical points uniformly valid asymptotic solutions are constructed for each of the problems for the asymptotically stable equilibrium point the method of matched asymptotic expansions with the integral condition of matkowsky and schuss is employed a method of averaging combined with boundary layer analysis is used for the problem of an equilibrium point surrounded by closed trajectories the influence on the solutions of the critical points on the boundary is exhibited and explained an application to the physical pendulum is given finally our r

the slow passage through a hopf bifurcation delay memory effects and resonance this paper explores analytically and numerically in the context of the fitzhugh nagumo model of nerve membrane excitability an interesting phenomenon that has been described as a delay or memory effect it can occur when a parameter passes slowly through a hopf bifurcation point and the system s response changes from a slowly varying steady state to slowly varying oscillations on quantitative observation it is found that the transition is realized when the parameter is considerably beyond the value predicted from a straightforward bifurcation analysis which neglects the dynamic aspect of the parameter variation this delay and its dependence on the speed of the parameter variation are described the model involves several parameters and particular singular limits are investigated one in particular is the slow passage through a low frequency hopf bifurcation where the system s response changes from a slowly varying steady state to slowly varying relaxation oscillations we find in this case the onset o

noise induced escape in an excitable system we consider the stochastic dynamics of escape in an excitable system the fitzhugh nagumo fhn neuronal model for different classes of excitability we discuss first the threshold structure of the fhn model as an example of a system without a saddle state we then develop a nonlinear nonlocal stability approach based on the theory of large fluctuations including a finite noise correction to describe noise induced escape in the excitable regime we show that the threshold structure is revealed via patterns of most probable optimal fluctuational paths the approach allows us to estimate the escape rate and the exit location distribution we compare the responses of a monostable resonator and monostable integrator to stochastic input signals and to a mixture of periodic and stochastic stimuli unlike the commonly used local analysis of the stable state our nonlocal approach based on optimal paths yields results that are in good agreement with direct numerical simulations of the langevin equation

non periodic driving of coupled oscillators a spherical swing abstract nonlinearly coupled damped oscillators at 1 1 frequency ratio one oscillator being driven coherently for efficient excitation are exemplified by a spherical swing with some phase mismatch between drive and response for certain damping range excitation is found to succeed if it lags behind but to produce a chaotic attractor if it leads the response although a period doubling sequence for damping increasing leads to the attractor this is actually born as a hard as regards amplitude bifurcation at a zero growth rate parametric line as damping decreases an unstable fixed point crosses an invariant plane to enter as saddle focus a phase space domain of physical solutions a second hard bifurcation occurs at the zero mismatch line the saddle focus leaving that domain times on the attractor diverge when approaching either line leading to exactly one dimensional and noninvertible limit maps which are analytically determined

a stochastic partial differential equation with values in a manifold abstract we investigate a certain stochastic partial differential equation which is defined on the unit interval with periodic boundary condition and takes values in a manifold such equation has particularly two different applications namely it determines the evolution law of an interacting constrained system of continuum distributed over the unit circle while it defines a diffusive motion of loops on a manifold we establish the existence and uniqueness results and then show the smoothness property of the solutions some examples are given in the final section

uncertainty transformation via hopf bifurcation in fast slow systems propagation of uncertainty in dynamical systems is a significant challenge here we focus on random multiscale ordinary differential equation models in particular we study hopf bifurcation in the fast subsystem for random initial conditions we show that a random initial condition distribution can be transformed during the passage near a delayed dynamic hopf bifurcation i to certain classes of symmetric copies ii to an almost deterministic output iii to a mixture distribution with differing moments and iv to a very restricted class of general distributions we prove under which conditions the cases i iv occur in certain classes vector fields

mean exit time and escape probability for dynamical systems driven by levy noises the mean first exit time and escape probability are utilized to quantify dynamical behaviors of stochastic differential equations with non gaussian alpha stable type levy motions an efficient and accurate numerical scheme is developed and validated for computing the mean exit time and escape probability from the governing differential integral equation an asymptotic solution for the mean exit time is given when the pure jump measure in the levy motion is small from both the analytical and numerical results it is observed that the mean exit time depends strongly on the domain size and the value of alpha in the alpha stable levy jump measure the mean exit time and escape probability could become discontinuous at the boundary of the domain when the value of alpha is in 0 1

the policy iteration algorithm for average reward markov decision processes with general state space the average cost optimal control problem is addressed for markov decision processes with unbounded cost it is found that the policy iteration algorithm generates a sequence of policies which are c regular where c is the cost function under consideration this result only requires the existence of an initial c regular policy and an irreducibility condition on the state space furthermore under these conditions the sequence of relative value functions generated by the algorithm is bounded from below and nearly decreasing from which it follows that the algorithm is always convergent under further conditions it is shown that the algorithm does compute a solution to the optimality equations and hence an optimal average cost policy these results provide elementary criteria for the existence of optimal policies for markov decision processes with unbounded cost and recover known results for the standard linear quadratic gaussian problem in particular in the control of multiclass queueing networks it is found that there is a close connection between optimization of the network and optimal control of a far simpler fluid network model

slow passage through a hopf bifurcation from oscillatory to steady state solutions this paper investigates the slow passage through a supercritical hopf bifurcation from a branch of slowly varying periodic solutions to a branch of slowly varying steady states this analysis is motivated by a recent numerical study of bursting oscillations in an enzymatic system it was found that the transition from oscillations to steady states is delayed even if the rate of change of the control parameter is extremely small the delay due to the slow passage is characterized by determining the amplitude of the oscillations at the bifurcation point defining varepsilon as the rate of change of the bifurcation parameter it is shown that the amplitude is an o varepsilon 1 4 quantity as varepsilon to 0 in addition a particular class of equations leading to relaxation oscillations is considered it is assumed that frequency omega of the oscillations at the hopf bifurcation can be controlled using a second parameter distinct from the bifurcation parameter it is then shown that the ampl

pathwise description of dynamic pitchfork bifurcations with additive noise the slow drift with speed of a parameter through a pitchfork bifurcation point known as the dynamic pitchfork bifurcation is characterized by a significant delay of the transition from the unstable to the stable state we describe the effect of an additive noise of intensity by giving precise estimates on the behaviour of the individual paths we show that until time after the bifurcation the paths are concentrated in a region of size around the bifurcating equilibrium with high probability they leave a neighbourhood of this equilibrium during a time interval after which they are likely to stay close to the corresponding deterministic solution we derive exponentially small upper bounds for the probability of the sets of exceptional paths with explicit values for the exponents

noise induced escape through a fractal basin boundary we study noise induced escape within a discrete dynamical system that has two co existing chaotic attractors in phase space separated by a locally disconnected fractal basin boundary it is shown that escape occurs via a unique accessible point on the fractal boundary the structure of escape paths is determined by the original saddles forming the homoclinic structure of the system and by their hierarchical interrelations

asymptotic series for singularly perturbed kolmogorov fokker planck equations we derive limit theorems for the transition densities of diffusion processes and develop asymptotic expansions for solutions of a class of singularly perturbed kolmogorov fokker planck equations the model under consideration can be viewed as a markov process having two time scales one of them is a rapidly changing scale and the other is a slowly varying one the study is motivated by a wide range of applications involving singularly perturbed markov processes in manufacturing systems reliability analysis queueing networks statistical physics population biology financial economics and many other related fields in this work the asymptotic expansion is constructed explicitly it is shown that the initial layer terms in the expansion decay at an exponential rate error bounds on the remainder terms also are obtained the validity of the expansion is rigorously justified

a note on the convergence of policy iteration in markov decision processes with compact action spaces the undiscounted unichain finite state markov decision process with compact action space is studied we provide a counterexample for a result in hordijk and puterman 1987 and give an alternate proof of the convergence of policy iteration under the condition that there exists a state that is recurrent under every stationary policy the analysis essentially uses a two term matrix representation for the relative value vectors generated by policy iteration procedure

three dimensional forced damped dynamical systems with rich dynamics bifurcations chaos and unbounded solutions abstract we consider certain autonomous three dimensional dynamical systems that can arise in mechanical and fluid dynamical contexts extending a previous study in craik and okamoto 2002 to include linear forcing and damping we find that the four leaf structure discovered in that paper and unbounded orbits persist but may now be accompanied by three distinct period doubling cascades to chaos and by orbits that approach stable equilibrium points this rich structure is investigated both analytically and numerically distinguishing three main cases determined by the damping and forcing parameter values

stochastic calculus application to dynamic bifurcations and threshold crossings for the dynamic pitchfork bifurcation in the presence of white noise the statistics of the last time at zero are calculated as a function of the noise level and the rate of change of the parameter the threshold crossing problem used for example to model the firing of a single cortical neuron is considered concentrating on quantities that may be experimentally measurable but have so far received little attention expressions for the statistics of pre threshold excursions occupation density and last crossing time of zero are compared with results from numerical generation of paths

switching path distribution in multidimensional systems we explore the distribution of paths followed in fluctuation induced switching between coexisting stable states we introduce a quantitative characteristic of the path distribution in phase space that does not require a priori knowledge of system dynamics the theory of the distribution is developed and its direct measurement is performed in a micromechanical oscillator driven into parametric resonance the experimental and theoretical results on the shape and position of the distribution are in excellent agreement with no adjustable parameters in addition the experiment provides the first demonstration of the lack of time reversal symmetry in switching of systems far from thermal equilibrium the results open the possibility of efficient control of the switching probability based on the measured narrow path distribution

probability densities for noisy delay bifurcations the delay of a transition in a nonlinear system due to a slowly varying control parameter can be significantly reduced by very small noise a new asymptotic approximation for the time dependent probability density function gives a complete description of the process into the transition region and is easily interpreted in terms of the noisy dynamics it is also used to calculate mean transition times the method is applied to two nonlinear systems with noise a one dimensional canonical model for a steady bifurcation and the noisy fitzhugh nagumo model

chaos and the effect of noise in a model of three wave mode coupling abstract the equations governing the nonlinear interaction of three resonant waves are studied in the limit that one wave is slightly unstable and the other two modes more heavily damped it is shown that the dynamics can be described in terms of a bimodal one dimensional map which allows bifurcation sequences etc to be simply constructed the effect of small additive noise can be studied in the same framework it is shown that even at very low levels the effect of noise can be extremely important in determining the period and amplitude of the oscillations

monotonicity of the free energy in the stochastic heisenberg model the specific free energy of the state at timet of the stochastic heisenberg model is shown to be non increasing witht and to strictly decrease whenever this state is not a gibbs state of the hamiltonian the initial state is assumed to be translation invariant and suitably smooth for such states a convergence theorem is obtained

noise and slow fast dynamics in a three wave resonance problem recent research on the dynamics of certain fluid dynamical instabilities shows that when there is a slow invariant manifold subject to fast timescale instability the dynamics are extremely sensitive to noise the behaviour of such systems can be described in terms of a one dimensional map and previous work has shown how the effect of noise can be modelled by a simple adjustment to the map here we undertake an in depth investigation of a particular set of equations using the methods of stochastic integration we confirm the prediction of the earlier studies that the noise becomes important when mu log epsilon o 1 where mu is the small timescale ratio and epsilon is the noise level in addition we present detailed information about the statistics of the solution when the noise is a dominant effect the analytical results show excellent agreement with numerical simulations

rates and mean first passage times the relation between mean first passage times t and transition rates f in noisy dynamical systems with metastahle states is investigated it is shown that the inverse mean first passage time to the separatrix of the noiseless system may deviate from twice the rate not only because in general the deterministic separatrix is not the locus in the state space from which a noisy trajectory goes to either side with equal probability a further cause of a deviation from the often assumed relation ft 1 2 between rates and mean first passage times is given if the noisy dynamics is discontinuous i e shows jumps with finite probability then the value of the splitting probability at the separatrix does not fix the value of ft since the system need not visit the separatrix during a transition from one to the other side most important for discontinuous processes the deviation from the ft rule survives even in the weak noise limit a mathematical relation for the product of the rate and the mean first passage time is proposed for markovian processes and numerically confirmed for a particular one dimensional noisy map

escape problem for irreversible systems the problem of noise induced escape from a metastable state arises in physics chemistry biology systems engineering and other areas the problem is well understood when the underlying dynamics of the system obey detailed balance when this assumption fails many of the results of classical transition rate theory no longer apply and no general method exists for computing the weak noise asymptotic behavior of fundamental quantities such as the mean escape time in this paper we present a general technique for analyzing the weak noise limit of a wide range of stochastically perturbed continuous time nonlinear dynamical systems we simplify the original problem which involves solving a partial differential equation into one in which only ordinary differential equations need be solved this allows us to resolve some old issues for the case when detailed balance holds when it does not hold we show how the formula for the asymptotic behavior of the mean escape time depends on the dynamics of the system along the most probable escape path we also present results on short time behavior and discuss the possibility of ital focusing along the escape path

a three dimensional autonomous system with unbounded bending solutions abstract we consider the system x ayz bz cy y dzx ex fz z gxy hy kx for real functions x t y t and z t where the overdot denotes differentiation with respect to a time like independent variable t and the coefficients a to k are real constants such equations arise in mechanical and fluid dynamical contexts depending on parameter values solutions may exhibit blowup in finite time or they may be bounded oscillatory or unbounded as time t the local shape of the latter unbounded solutions is typically helical sometimes with and sometimes without a 90 bend in the axis of the helix complete solutions are obtained in cases where certain coefficients are zero other cases are investigated numerically and asymptotically the numerical solutions reveal an interesting four leaf structure connected to the helical trajectories this structure largely determines whether these trajectories bend through 90 or not a fluid dynamical application is discussed in appendix a

local and global behavior near homoclinic orbits we study the local behavior of systems near homoclinic orbits to stationary points of saddle focus type we explicitly describe how a periodic orbit approaches homoclinicity and with the help of numerical examples discuss how these results relate to global patterns of bifurcations

the 2 1 steady hopf mode interaction in the two layer be nard problem abstract the interaction of hopf and steady modes with wavenumber ratio 2 1 is investigated for a critical situation two different immiscible liquids lie in layers between horizontal walls and are heated from below a situation with a pair of complex conjugate eigenvalues at wavenumber and a real eigenvalue at wavenumber 2 is at criticality weakly nonlinear amplitude equations are derived for the interaction of these oscillatory and steady modes the two modes generate a two parameter bifurcation the coefficients involved in the equations are determined numerically based on the physical parameters of the system at criticality three obvious equilibrium solutions of the amplitude equations are the steady solution the traveling waves and the mixed standing waves the eigenvalues governing the stability of these solutions are found explicitly numerical results and bifurcation diagrams are given for the critical situation the steady solution and the traveling wave solution are unstable there is a region of stability for the standing wave solution a new equilibrium solution the asymmetric mixed mode is found to be stable in a parameter range bifurcations from the standing wave solution and the asymmetric mixed mode are described

the asymptotic behavior of the principal eigenvalue in a singular perturbation problem with invariant boundaries we consider diffusion random perturbations of a dynamical systemst in a domaing rm which in particular may be invariant under the action ofst continuing the study of k1 k4 we find the asymptotic behavior of the principal eigenvalue of the corresponding generator when the diffusion term tends to zero

large deviations from the thermodynamic limit in globally coupled maps abstract systems of a large number n of globally coupled maps have become popular as a relatively simple prototype of high dimensional dynamics showing many interesting and typical phenomena like synchronisation cluster formation and multistability and having potential applications in systems like josephson junction arrays or in biophysical models while much progress has been made in the explanation of the macroscopic behaviour of such systems in the limit n there is still need for a sound theory about the asymptotic behaviour of finite n systems as n approaches infinity this article introduces a method by which it is possible to obtain asymptotic estimates for finite size effects that show up as long term deviations from the thermodynamic limit behaviour this method works for systems of globally coupled noisy maps and is based upon the concept of quasipotentials originally developed by freidlin wentzell and others for describing the influence of small random perturbations on the long term behaviour of dynamical systems the problems of explicitly computing quasipotentials in the present context and potential approximation schemes are discussed all the concepts described in this article are illustrated with a simple example

derivation of amplitude equations for nonlinear oscillators subject to arbitrary forcing by using a generalization of the multiple scales technique we develop a method to derive amplitude equations for zero dimensional forced systems the method allows to consider either additive or multiplicative forcing terms and can be straightforwardly applied to the case that the forcing is white noise we give examples of the use of this method to the case of the van der pol duffing oscillator the writing of the amplitude equations in terms of a lyapunov potential allow us to obtain an analytical expression for the probability distribution function which reproduces reasonably well the numerical simulation results

escape through an unstable limit cycle driven by multiplicative colored non gaussian and additive white gaussian noises in a previous paper bag and hu phys rev e 73 061107 2006 we studied the mean lifetime mlt for the escape of a brownian particle through an unstable limit cycle driven by multiplicative colored gaussian and additive gaussian white noises and found resonant activation ra behavior in the present paper we switch from gaussian to non gaussian multiplicative colored noise we find that in the ra phenomenon the minimum appears at a smaller noise correlation time tau for non gaussian noises compared to gaussian noises in the plot of mlt vs tau for a fixed noise variance the same plot for a given noise strength increases linearly and the increasing rate is smaller for non gaussian noises than for the gaussian noises the plot of logarithm of inverse of mlt vs inverse of the strength of additive noise is arrhenius like for gaussian colored noise and it becomes similar to the quantum kramers rate if the multiplicative noise is non gaussian

predictability of noise controlled dynamics abstract noise controlled dynamics runs counter to usual intuition the larger the noise the more regular the solutions we present numerical and analytical results for a set of three stochastic partial differential equations in one space dimension motivated by the intermittent destabilization of tall thin convection cells by horizontal shear time series are predictable in the sense that they follow limit cycles with a small variation in amplitude from cycle to cycle closer inspection reveals that the amplitude is determined by very small amounts of noise

hyperbolic periodic orbits in nongradient systems and small noise induced metastable transitions abstract small noise can induce rare transitions between metastable states which can be characterized by maximum likelihood paths mlps nongradient systems contrast gradient systems in that mlp does not have to cross the separatrix at a saddle point but instead possibly at a point on a hyperbolic periodic orbit a numerical approach for identifying such unstable periodic orbits is proposed based on string method in a special class of nongradient systems orthogonal type there are provably local mlps that cross such saddle point or hyperbolic periodic orbit and the separatrix crossing location determines the associated local maximum of transition rate in general cases however the separatrix crossing may not determine a unique local maximum of the rate as we numerically observed a counter example in a sheared 2d space allen cahn spde it is a reasonable conjecture that there are always local mlps associated with each attractor on the separatrix such as saddle point or hyperbolic periodic orbit our numerical experiments did not disprove so

decision directed versus non data aided plls a comparative review the performance of discrete time decision directed dd and non data aided nda first order phase looked loops pll for a qpsk modulated signal is evaluated under the disturbance of thermal noise and 1 f2 phase noise a well known continuous time model is presented that closely approximates the loop s behavior using the continuous time model fokker planck techniques are applied to compute the mean time to lose lock mtll for both plls we optimize the loop filter using the mtll as the optimization criterion on the basis of the optimal mtll we determine the regions where one pll outperforms the other monte carlo simulation results are presented to validate the analytic analysis for the setting under study the nda pll outperforms the dd pll for snr below 4 5db regardless of the phase noise level applicability of the results for a second order pll is discussed

on transition densities of singularly perturbed diffusions with fast and slow components we derive asymptotic properties of transition densities for singularly perturbed diffusion processes with fast and slow components our study focuses on the kolmogorov fokker planck equations the model can be viewed as a diffusion process having two time scales and is motivated by a wide variety of applications involving singularly perturbed markov processes in manufacturing systems homogenization reliability analysis queueing networks statistical physics population biology financial economics and many other related fields by virtue of the methods of matched singular perturbation asymptotic expansion is constructed for the transition density the expansion includes both regular part and boundary layer corrections detailed justification of the asymptotic expansion is given and error bounds are also provided

gradient particle solutions of fokker planck equations for noisy delay bifurcations a gradient particle method is used to compute probability densities for processes with delay bifurcations that are sensitive to very small noise the method is particularly useful for computing the probability density in the transition region where asymptotic approximations may not be valid for a one dimensional steady bifurcation problem and a two dimensional fitzhugh nagumo model we solve the fokker planck equation fpe and compare the results with direct simulations and asymptotic approximations

the invariant density of a chaotic dynamical system with small noise abstract we consider a chaotic dynamical system perturbed by noise and calculate an approximate invariant density when the noise level is small because of the special structure of the dynamical system the effective support of the invariant density is much smaller than the noiseless attractor this behavior is captured by the asymptotic form of the invariant density which is given explicitly

a scaling theory of bifurcations in the symmetric weak noise escape problem we consider two dimensional overdamped double well systems perturbed by white noise in the weak noise limit the most probable fluctuational path leading from either point attractor to the separatrix the most probable escape path or mpep must terminate on the saddle between the two wells however as the parameters of a symmetric double well system are varied a unique mpep may bifurcate into two equally likely mpeps at the bifurcation point in parameter space the activation kinetics of the system become non arrhenius we quantify the non arrhenius behavior of a system at the bifurcation point by using the maslov wkb method to construct an approximation to the quasistationary probability distribution of the system that is valid in a boundary layer near the separatrix the approximation is a formal asymptotic solution of the smoluchowski equation our construction relies on a new scaling theory which yields critical exponents describing weak noise behavior at the bifurcation point near the saddle

towards a theory of transition paths we construct a statistical theory of reactive trajectories between two pre specified sets a and b i e the portionsof the path of a markov process during which the path makes a transition from a to b this problem is relevant e g in the context of metastability in which case the two sets a and b are metastable sets though the formalism we propose is independent of any such assumptions on a and b we show that various probability distributions on the reactive trajectories can be expressed in terms of the equilibrium distribution of the process and the so called committor functions which give the probability that the process reaches first b before reaching a either backward or forward in time using these objects we obtain i the distribution of reactive trajectories which gives the proportion of time reactive trajectories spend in sets outside of a and b ii the hitting point distribution of the reactive trajectories on a surface which measures where the reactive trajectories hit the surface when they cross it iii the last hitting point distribution of the reactive trajectories on the surface iv the probability current of reactive trajectories the integral of which on a surface gives the net average flux of reactive trajectories across this surface v the average frequency of reactive trajectories which gives the average number of transitions between a and b per unit of time and vi the traffic distribution of reactive trajectories which gives some information about the regions the reactive trajectories visit regardless of the time they spend in these regions

noise stabilized random attractor a two dimensional flow model is introduced with deterministic behavior consisting of bursts that become successively larger with longer interburst time intervals between them the system is symmetric in one variable and there are bursts on either side of separated by the presence of an invariant manifold at in the presence of arbitrarily small additive noise in the direction the successive bursts have bounded amplitudes and interburst intervals this system with noise is proposed as a model for edge localized modes in tokamaks with noise the bursts can switch from positive to negative and vice versa the probability distribution of burst heights and interburst periods is studied as is the dependence of the statistics on the noise variance the modification of this behavior as the symmetry in is broken is studied showing qualitatively similar behavior if the symmetry breaking is small enough experimental observations of a nonlinear circuit governed by the same equations are presented showing good agreement

large deviations results for the exit problem with characteristic boundary abstract we consider the exit problem for an asymptotically small random perturbation of a stable dynamical system in a region d we show the standard large deviations results for the exit distribution and mean exit time as obtained by wentzell and freidlin under the assumption of nontangential drift b n

limiting exit location distributions in the stochastic exit problem consider a two dimensional continuous time dynamical system with an attracting fixed point s if the deterministic dynamics are perturbed by white noise random perturbations of strength epsilon the system state will eventually leave the domain of attraction omega of s we analyze the case when as epsilon to0 the exit location on the boundary partial omega is increasingly concentrated near a saddle point h of the deterministic dynamics we show using formal methods that the asymptotic form of the exit location distribution on partial omega is generically non gaussian and asymmetric and classify the possible limiting distributions a key role is played by a parameter mu equal to the ratio lambda s h lambda u h of the stable and unstable eigenvalues of the linearized deterministic flow at h if mu 1 then the exit location distribution is generically asymptotic as epsilon to0 to a weibull distribution with shape parameter 2 mu on the cal o epsilon mu 2

a generic hard transition to chaos abstract a hard in amplitude transition to chaos in a class of dissipative flows of broad applicability is presented for positive values of a parameter no matter how small a fully developed chaotic attractor exists within some domain of additional parameters whereas no chaotic behavior exists for 0 as is made positive an unstable fixed point reaches an invariant plane to enter a phase half space of physical solutions the ghosts of a line of fixed points and a rich heteroclinic structure existing at 0 make the limits t 0 non commuting and allow an exact description of the chaotic flow the formal structure of flows that exhibit the transition is determined a subclass of such flows coupled oscillators in near resonance at any 2 q frequency ratio with representing linear excitation of the first oscillator is fully analysed

aperiodic stochastic resonance stochastic resonance sr is a phenomenon wherein the response of a nonlinear system to a weak periodic input signal is optimized by the presence of a particular level of noise recently we presented a method and theory for characterizing sr type behavior in excitable systems with aperiodic i e broadband input signals phys rev e 52 r3321 1995 we coined the term aperiodic stochastic resonance asr to describe this general type of behavior in that earlier study we demonstrated asr in the fitzhugh nagumo neuronal model here we demonstrate asr in three additional systems a bistable well system an integrate and fire neuronal model and the hodgkin huxley hh neuronal model we present computational and theoretical results for each system in the context of the hh model we develop a general theory for asr in excitable membranes this work clearly shows that sr type behavior is not limited to systems with periodic inputs thus in general noise can serve to enhance the response of a nonlinear system to a weak input signal regardless of whether the signal is periodic or aperiodic textcopyright 1996 the american physical society

singularly perturbed linear two point boundary value problems finding asymptotic solutions to two point boundary value problems for linear singularly perturbed second order ordinary differential equations is now quite well understood troublesome exceptions involving turning points known as problems of boundary layer resonance have been studied by many experts since the initial work of ackerberg and o malley stud appl math 49 1970 pp 277 295 this significant work is surveyed here closely related literature involves the canard phenomena new generally more geometric approaches including gevrey asymptotics and blowup promise improved understanding of such sensitive asymptotics and their generalizations

master equation simulations of a model of a thermochemical system master equation approach is used to study the influence of fluctuations on the dynamics of a model thermochemical system for appropriate values of parameters the deterministic description of the system gives the subcritical or supercritical hopf bifurcations for small systems containing 100 000 particles close to the supercritical hopf bifurcation the stochastic trajectories obtained from numerical simulations do not allow to distinguish between damped oscillations around a stable focus and sustained oscillations around a small stable limit cycle this uncertainty disappears if the number of particles in the system is increased up to 1 000 000 close to subcritical hopf bifurcation the stochastic trajectory of the system jumps from the basin of attraction of a stable focus to the basin of attraction of a stable limit cycle in this case the time dependencies of temperature and concentration of reactant in the system are apparently similar to intermittent chaotic oscillations the mean first passage time for the transitions from the stable focus to the stable limit cycle show the characteristic exponential dependence on the number of particles this passage time depends very strongly on the bifurcation parameter reaction heat which determines the distance between the stable focus and an unstable limit cycle

on the convergence of policy iteration in finite state undiscounted markov decision processes the unichain case we study the convergence of policy iteration for the undiscounted finite state discrete time markov decision problem with compact action space and unichain transition structure using a newton method type representation for policy iteration we establish the existence of a solution to the optimality equation we show that to find an average optimal policy it is sufficient to solve the optimality equation on the recurrent set of the maximizing policy under the additional assumption of a unique maximizing policy at each stage of the policy iteration procedure we show that the iterates are convergent and the resulting policy is blackwell optimal

the exit distribution on the stochastic separatrix in kramers exit problem kramers exit problem is concerned with noise activated escape from a potential well in the case when the noise strength varepsilon temperature measured in units of potential barrier height is small this becomes a singular perturbation problem unexpectedly the noise induced escape from the domain of attraction of the stable equilibrium at the bottom of the well does not occur at points on the separatrix where the energy is minimal rather the distribution of exit points on the separatrix in the phase plane is shown to be spread away from the saddle point where the energy is minimal it is shown that actually most of the probability of the exit points on the separatrix is located at a distance o sqrt varepsilon from the saddle point and the probability vanishes altogether at the saddle point in this problem large deviations theory fails to predict the distribution of the exit point for finite noise we construct an asymptotic approximation to the solution of the fokker planck eq