# Mean first-passage times of non-Markovian random walkers in confinement.

T Guérin, N Levernier, O Bénichou, R Voituriez

<figure><img src="image_1.png" /></figure>

## To cite this version:

T Guérin, N Levernier, O Bénichou, R Voituriez. Mean first-passage times of non-Markovian random walkers in confinement. Nature, 2016, 534 (7607), pp.356-9. 10.1038/nature18272. hal-01344629

HAL Id: hal-01344629

https://hal.science/hal-01344629

Submitted on 12 Jul 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Mean first-passage times of non-Markovian random walkers in confinement

T. Guérin<fnref n="1" />, N. Lernier², O. Bénichou² & R. Voituriez²,³

The first-passage time, defined as the time a random walker takes to reach a target point in a confining domain, is a key quantity in the theory of stochastic processes<fnref n="1" />. Its importance comes from its crucial role in quantifying the efficiency of processes as varied as diffusion-limited reactions²,³, target search processes⁴ or the spread of diseases⁵. Most methods of determining the properties of first-passage time in confined domains have been limited to Markovian (memoryless) processes³,⁶,⁷. However, as soon as the random walker interacts with its environment, memory effects cannot be neglected: that is, the future motion of the random walker does not depend only on its current position, but also on its past trajectory. Examples of non-Markovian dynamics include single-file diffusion in narrow channels⁸, or the motion of a tracer particle either attached to a polymeric chain⁹ or diffusing in simple¹⁰ or complex fluids such as nematics¹¹, dense soft colloids¹² or viscoelastic solutions¹³,¹⁴. Here we introduce an analytical approach to calculate, in the limit of a large confining volume, the mean first-passage time of a Gaussian non-Markovian random walker to a target. The non-Markovian features of the dynamics are encompassed by determining the statistical properties of the fictitious trajectory that the random walker would follow after the first-passage event takes place, which are shown to govern the first-passage time kinetics. This analysis is applicable to a broad range of stochastic processes, which may be correlated at long times. Our theoretical predictions are confirmed by numerical simulations for several examples of non-Markovian processes, including the case of fractional Brownian motion in one and higher dimensions. These results reveal, on the basis of Gaussian processes, the importance of memory effects in first-passage statistics of non-Markovian random walkers in confinement.

It has long been recognized that the kinetics of reactions is influenced by the properties of the transport process that brings reactants into contact<fnref n="1" />,². Transport can even be the rate-limiting step, and in this diffusion-controlled regime, the reaction kinetics is quantified by the properties of the first encounter between molecules². First-passage time (FPT) properties have been studied intensively in the past few decades<fnref n="1" />,³,¹⁵ and are now well understood when the stochastic motion of the reactants satisfies the Markov property, that is, is memoryless (uninfluenced by previous states, only by the current state). Under this assumption, exact asymptotic formulas characterizing the FPT of a tracer to a target located inside⁶,⁷,¹⁶ or at the boundary¹⁵ of a large confining volume have been obtained. These studies reveal that the geometrical parameters, as well as the complex properties of the stochastic transport process (such as subdiffusion), can have a strong impact on the reaction kinetics³,⁶,⁷.

However, as a general rule, the dynamics of a given reactant results from its interactions with its environment and cannot be described as a Markov process. Indeed, although the evolution of the set of all microscopic degrees of freedom of the system is Markovian, the dynamics restricted to the reactant only is not. This is typically the case

for a tagged monomer, whose non-Markovian motion results from the structural dynamics of the whole chain to which it is attached⁹,¹⁷,¹⁸, as observed for example, in proteins¹⁹. Other experimentally observed examples of non-Markovian dynamics include the diffusion of tracers in crowded narrow channels⁸ or in complex fluids such as nematics¹¹ or viscoelastic solutions¹³,¹⁴. Even in simple fluids, hydrodynamic memory effects and thus non-Markovian dynamics have been recently observed¹⁰. So far, most theoretical results on the first-passage properties of non-Markovian processes have been limited to specific examples¹⁷,¹⁸,²⁰⁻²² or to unconfined systems, where non-trivial persistence exponents characterizing its long time decay have been calculated²³⁻²⁵. However, in many situations, geometric confinement has a key role in first-passage kinetics³,⁶,⁷. Here, we develop a theoretical framework with which to determine the mean FPT of non-Markovian random walkers in confinement.

More precisely, we consider a non-Markovian Gaussian stochastic process $x(t)$, defined in unconfined space, which represents the position of a random walker at time $t$, starting from $x_0$ at $t=0$. As the process is non-Markovian, the FPT statistics in fact depend also on $x(t)$ for $t<0$. For the sake of simplicity, we assume that at $t=0$ the process of constant average $x_0$ is in the stationary state (see Supplementary Information for more general initial conditions), with increments $x(t+\tau) - x(t)$ independent of $t$. The process $x(t)$ is then entirely characterized by its mean square displacement (MSD): $\psi(\tau) = \langle [x(t+\tau) - x(t)]^2 \rangle$. Such a quantity is routinely measured in single particle tracking experiments and in fact includes all the memory effects in the case of Gaussian processes. At long times, the MSD is assumed to diverge and thus, typically, the particle does not remain close to its initial position. Last, the process is continuous and non-smooth²⁵ ($\langle \dot{x}(t)^2 \rangle = +\infty$), meaning that the trajectory is irregular and of fractal type, similar to standard Brownian motion. Note that the class of random walks that we consider here covers a broad spectrum of non-Markovian processes used in physics, and in particular the examples mentioned above.

The random walker is now confined in a domain of volume $V$ with reflecting walls, and we focus on its mean FPT to reach a target of position $x=0$ (see Fig. 1). Note that this setting also gives access to the reaction kinetics of a reactant in the presence of a concentration $c=1/V$ of targets in infinite space. Although the theory can be developed in any space dimension (see Supplementary Information for an explicit treatment of the two-dimensional and three-dimensional cases), it is presented here for clarity in one dimension (see Fig. 1b). Our starting point is the following generalization of the renewal equation<fnref n="1" />

$$p(0,t) = \int_{0}^{t} d\tau F(\tau) p(0, t | \text{FPT} = \tau) \quad (1)$$

which results from a partition over the first-passage event. In this equation, $p(0, t)$ stands for the probability density of being at position $x=0$ at time $t$, $F$ is the FPT density and $p(0, t|\text{FPT} = \tau)$ is the probability that

<footnote n="1">¹Laboratoire Ondes et Matière d'Aquitaine, University of Bordeaux, Unité Mixte de Recherche 5798, CNRS, F-33400 Talence, France. ²Laboratoire de Physique Théorique de la Matière Condensée, CNRS/Université Pierre et Marie Curie, 4 Place Jussieu, 75005 Paris, France. ³Laboratoire Jean Perrin, CNRS/Université Pierre et Marie Curie, 4 Place Jussieu, 75005 Paris, France.</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1 | Mean FPT of a random walker in confinement.** a, What is the mean time $\langle T \rangle$ needed for a random walker starting at $x_0$ (blue dot) to reach a target (red dot) in a confining volume $V$? Here we answer this question for random walkers with memory. b, In one dimension, the problem is to quantify the FPT of a random trajectory (in blue) in the presence of a reflecting boundary. We show here that $\langle T \rangle$ is controlled by the average trajectory $\mu(\tau)$ (in red) followed by the walker in the future of its first passage to the target.</figcaption></figure>

$x=0$ at time $t$ given that the first-passage event occurred at time $\tau$. Owing to the confinement, for large times $p(0, t)$ reaches the stationary value $1/V$. Next, subtracting $1/V$ on both sides of equation (1) and integrating over $t$ from 0 to infinity yields an exact expression for the mean FPT:

$$ \frac{\langle T \rangle}{V} = \int_{0}^{\infty} dt [q_{\pi}(t) - p(0, t)] \qquad (2) $$

where $q_{\pi}(t)dx$ is the probability of observing the random walker in the interval $[0, dx]$ at time $t$ after the first passage to the target. The exact formula (2) is a generalization of the expression obtained for Markovian processes$^{6,26}$ and holds for any non-smooth non-Markovian process

with stationary increments (even non-Gaussian). Even if $q_{\pi}(t)$ is a priori a non-trivial quantity because it is conditioned by first-passage events, this equation is of great practical use in determining the mean FPT, as shown below.

To proceed further, we first consider the large volume limit $V \to \infty$ (where it is assumed that all boundary points are sent to infinity) and, second, we assume that the stochastic process in the future of the FPT, defined by $y(t) \equiv x(t + \text{FTP})$, is Gaussian with mean $\mu(t)$ and the same covariance as the initial process $x(t)$ (see Fig. 1b). Simulations and the perturbation theory below show the broad validity of this approach. Equation (2) then leads to:

$$ \langle T \rangle = V \int_{0}^{\infty} dt \frac{e^{-\mu(t)^2/2\psi(t)} - e^{-x_0^2/2\psi(t)}}{[2\pi\psi(t)]^{1/2}} \qquad (3) $$

Relying on a generalization of equation (1) to link the $n$ times probability distribution functions of $x(t_1), x(t_2), \dots$ and the FPT density, we obtain an equation for the probability of the future trajectories $y(t)$ leading to (see Supplementary Information for details):

$$ \int_{0}^{\infty} \frac{dt}{\sqrt{\psi(t)}} \{[\mu(t+\tau)-\mu(t)K(t, \tau)]e^{-\mu(t)^2/2\psi(t)} - x_0[1-K(t, \tau)]e^{-x_0^2/2\psi(t)}\} = 0 \qquad (4) $$

where $\mu(0) = 0$ and $K(t, \tau) = [\psi(t+\tau) + \psi(t) - \psi(\tau)]/[2\psi(t)]$. Equation (4), which allows for a self-consistent determination of the mean future trajectory $\mu(t)$, together with equation (3), provide the mean FPT and constitute our main result.

At this stage, several remarks can be made. (1) The mean FPT depends linearly on the confining volume $V$, which extends the result obtained for Markovian processes$^6$. (2) Our approach reveals the key role of the mean trajectory $\mu(t)$ followed by the walker in the future of the first-passage event. In other words, even if the real motion is

<figure><img src="image_4.png" /><figcaption>**Figure 2 | Mean FPT of one-dimensional non-Markovian random walks.** Mean FPT as a function of the initial position $x_0$ (a–d) and average reactive trajectory $\mu(t)$ in the future of the FPT as a function of time $t$ (e–h) for various one-dimensional Gaussian stochastic processes. Solid lines are predictions of the non-Markovian theory from equations (3) and (4); dashed lines are the Markovian approximation (in which $\mu(t) = 0$); and symbols represent numerical simulations using the circulant matrix algorithm (see Supplementary Information). In a and e, the correlator $\Psi(t)$ is indicated with $D=1$, $D_0=30$, $\lambda=1$ (arbitrary units). Time is in units of $1/\lambda$ and lengths are in units of $(D/\lambda)^{1/2}$. In e symbols represent different</figcaption></figure>

<figure><img src="image_5.png" /></figure>

volumes (hexagons, $V=40$; squares, $V=60$; and diamonds, $V=120$); the superposition confirms that $\mu(t)$ does not depend on $V$. In b–d and f–h, fractional Brownian motion (FBM) is shown for $K=1$ (arbitrary units). Time is in units of $V^{1/H}/K^{1/2H}$. Note that the theory is derived for the limit of large volume, or equivalently $x_0 \ll V$. When significant, error bars give the s.e.m. of the numerical simulations. Number $n$ of simulated trajectories: in a and e $n=173,285$ (for $V=40$), $n=180,641$ (for $V=60$), and $n=96,623$ (for $V=120$); in b and f $n=19,224$; in c and g $n=22,422$; and in d and h $n=40,685$.
<figure><img src="image_6.png" /><figcaption>**Figure 3** | Mean FPT of two- and three-dimensional non-Markovian random walks. Mean FPT to a target of radius a = 1 (arbitrary units) as a function of the initial position r₀ (a, c, e) and average reactive trajectory μ(t) in the future of the FPT as a function of time t (b, d, f) for different two-dimensional (2D) (a–d) and three-dimensional (3D) (e, f) Gaussian stochastic processes. Solid lines are predictions of the non-Markovian theory from equations (3) and (4); dashed lines are the Markovian approximation, in which μ(t) remains equal to the radius a = 1 of the target; and symbols represent numerical simulations using the circulant matrix algorithm. In a and b, the correlator Ψ(t) of each coordinate in two</figcaption></figure>

stopped at the first encounter with the target, the mean FPT is con-
trolled by the statistical properties of the fictitious path that the walker
would follow if allowed to continue after the first encounter event. (3)
Assuming that ψ(t) ∝ t 2H at large times, with 0 < H < 1, it can be shown
from the asymptotic analysis of equation (4) that:

$$
\mu(t) \approx x_0 - A \, t^{2H-1} \quad (t \to \infty) \tag{5}
$$

where A is a coefficient depending on the entire MSD function ψ(t) (at all timescales) and on x₀ (it generally has the same sign as x₀). Thus, for processes that are subdiffusive at long times (so that the MSD grows slower than linearly with time, *H* < 1/2), μ(t) comes back to the initial position x₀ of the walker, which is consequently not forgotten. On the contrary, asymptotically superdiffusive walkers (*H* > 1/2) keep going away from the target in the future of the FPT with a non-trivial exponent. These behaviours reflect the anticorrelation and correlation of successive steps of subdiffusive and superdiffusive walks, respectively. Note that even for asymptotically diffusive processes (*H* = 1/2), μ(t) tends to a non-vanishing constant, in contrast to a pure (Markovian) Brownian motion. (4) The importance of non-Markovian effects can be appreciated by comparing the mean FPT to the result obtained by setting μ(t) = 0, which amounts to neglecting the memory of the trajectory before the first passage. As shown by equation (5), μ(t) is actually not small, so that memory effects are important. They are especially marked for *H* < 1/3, where setting μ(t) = 0 in equation (3) leads to an infinite mean FPT, as opposed to our finite non-Markovian prediction.

We now confirm the validity of these analytical results by com-
paring them to numerical simulations of representative examples of
non-Markovian processes defined by the MSD ψ(t). First, the choice

dimensions is indicated for D = 1, D₀ = 30, V = 100, λ = 1 (arbitrary units).
Time is in units of 1/λ and lengths are in units of a. In c and d, fractional
Brownian motion in two dimensions is shown, with K = 1, V = 60²
(arbitrary units). Time is in units of a¹/²H/K¹/²H and lengths in units of a.
In e and f, the correlator Ψ(t) of each coordinate in three dimensions is
indicated for D = 1, D₀ = 10, λ = 1 (arbitrary units). Time is in units of 1/λ
and lengths in units of a. The confining volume is a sphere of radius R = 70
or a cube of volume V = 116³. When significant, error bars give the s.e.m.
of the numerical simulations. Number n of simulated trajectories: in a and b
n = 35,334, in c and d n = 37,314; and in e and f n = 16,900.

$\psi(t) = D_0(1 - e^{-\lambda t}) + Dt = \psi_D(t)$ corresponds to the generic case where the position $x(t)$ is coupled to other degrees of freedom at the single timescale $1/\lambda$ (Fig. 2a, e). It is typically relevant to tracers moving in nematics¹¹ or solutions of non-adsorbing polymers²⁷.

Second, the choice ψ(t) = Kt²H where 0 < H < 1 and K is a positive transport coefficient (Fig. 2b–d, f–h), corresponds to the fractional Brownian motion used in fields as varied as hydrology²⁸, finance²⁹ and biophysics¹³,³⁰; it is a particularly good description of anomalous diffusion in various physical situations such as telomere motion³⁰ or tracer diffusion in viscoelastic fluids¹³. This process is strongly non-Markovian, as shown by its long-range correlation functions. For fractional Brownian motion, the solution of equation (4) is of the form:

$$
\mu(t) = x_0 \tilde{\mu}_H(t K^{1/2H} / x_0^{1/H})
$$

so that the mean FPT reads:

$$
\langle T \rangle = V \beta_H x_0^{1/H-1} K^{-1/2H} \quad (6)
$$

with β<sub>H</sub> a numerical coefficient given in Supplementary Information.
This equation gives the explicit dependence of the mean FPT on x<sub>0</sub> and
generalizes the results obtained for Markovian processes<sup>6</sup>.

Third, the theory can be extended to higher dimensions with the supplementary assumption that the random walk is isotropic. Two-dimensional and three-dimensional versions of both of the choices of ψ(t) considered above have been analysed explicitly (Fig. 3).

In fact, as shown in the Supplementary Information, the theory
is exact at order ε² when one considers a MSD function of the type
ψ(t) = Dt + εψ₁ + ε²ψ₂ + ... where the small parameter ε measures the
deviation from a Markovian process (see Supplementary Information). Figures 2 and 3 reveal very good quantitative agreement between the analytical predictions and the numerical simulations far beyond this perturbative regime. Both the volume and the source-target distance dependence of the mean FPT are unambiguously captured by the theoretical analysis, at all the length scales involved in the problem. Note that, even if the theoretical prediction relies on large-volume asymptotics, numerical simulations show that it is accurate even for small confining systems (with various shapes of confining volumes, such as spherical or cubic). The very different nature of these examples (one, two or three dimensions, diffusive, superdiffusive or subdiffusive at long times...) demonstrates the wide range of applicability of our approach. Remarkably, the amplitude of memory effects is important in the examples shown in Figs 2 and 3, where the multiplicative factor between Markovian and non-Markovian estimates of the mean FPT can be up to 15 (Fig. 2c). As discussed above, this factor is even infinite for the fractional Brownian motion as soon as $H < 1/3$. Interestingly, even for the process defined by $\psi(t) = \psi_D(t)$ above, which is diffusive both at short and long times, for which one could thus expect memory effects to be negligible, this factor is not small (typically 5; see Fig. 2a). The accuracy of our analytical predictions for the mean FPT traces back to the quantitative prediction for the trajectories in the future of the FPT $\mu(t)$, as shown in Figs 2 and 3. The strong dependence of $\mu(t)$ on the starting point $x_0$, predicted by our approach and confirmed numerically, is a direct manifestation of the non-Markovian feature of the random walks. Together, our results demonstrate and quantify the importance of memory effects in the first-passage properties of non-Markovian random walks in confined geometry.

1. Redner, S. *A Guide to First-Passage Processes* (Cambridge Univ. Press, 2001).

2. ben Avraham, D. & Havlin, S. *Diffusion and Reactions in Fractals and Disordered Systems* (Cambridge Univ. Press, 2000).

3. Bénichou, O. & Voituriez, R. From first-passage times of random walks in confinement to geometry-controlled kinetics. *Phys. Rep.* **539**, 225–284 (2014).

4. Shlesinger, M. F. Mathematical physics: search research. *Nature* **443**, 281–282 (2006).

5. Lloyd, A. L. & May, R. M. Epidemiology—how viruses spread among computers and people. *Science* **292**, 1316–1317 (2001).

6. Condamin, S., Bénichou, O., Tejedor, V., Voituriez, R. & Klafter, J. First-passage times in complex scale-invariant media. *Nature* **450**, 77–80 (2007).

7. Bénichou, O., Chevalier, C., Klafter, J., Meyer, B. & Voituriez, R. Geometry-controlled kinetics. *Nat. Chem.* **2**, 472–477 (2010).

8. Wei, Q.-H., Bechinger, C. & Leiderer, P. Single-file diffusion of colloids in one-dimensional channels. *Science* **287**, 625–627 (2000).

9. Panja, D. Anomalous polymer dynamics is non-Markovian: memory effects and the generalized Langevin equation formulation. *J. Stat. Mech.* **2010**, P06011 (2010).

10. Franosch, T. et al. Resonances arising from hydrodynamic memory in Brownian motion. *Nature* **478**, 85–88 (2011).

11. Turiv, T. et al. Effect of collective molecular reorientations on Brownian motion of colloids in nematic liquid crystal. *Science* **342**, 1351–1354 (2013).

12. Démery, V., Bénichou, O. & Jacquin, H. Generalized Langevin equations for a driven tracer in dense soft colloids: construction and applications. *New J. Phys.* **16**, 053032 (2014).

13. Ernst, D., Hellmann, M., Köhler, J. & Weiss, M. Fractional Brownian motion in crowded fluids. *Soft Matter* **8**, 4886–4889 (2012).

14. Mason, T. G. & Weitz, D. Optical measurements of frequency-dependent linear viscoelastic moduli of complex fluids. *Phys. Rev. Lett.* **74**, 1250 (1995).

15. Holcman, D. & Schuss, Z. The narrow escape problem. *SIAM Rev.* **56**, 213–257 (2014).

16. Bénichou, O., Meyer, B., Tejedor, V. & Voituriez, R. Zero constant formula for first-passage observables in bounded domains. *Phys. Rev. Lett.* **101**, 130601–130604 (2008).

17. Guérin, T., Bénichou, O. & Voituriez, R. Non-markovian polymer reaction kinetics. *Nat. Chem.* **4**, 568–573 (2012).

18. Bénichou, O., Guérin, T. & Voituriez, R. Mean first-passage times in confined media: from Markovian to non-Markovian processes. *J. Phys. A* **48**, 163001 (2015).

19. Kou, S. & Xie, X. S. Generalized Langevin equation with fractional Gaussian noise: subdiffusion within a single protein molecule. *Phys. Rev. Lett.* **93**, 180603 (2004).

20. Wilemski, G. & Fixman, M. Diffusion-controlled intrachain reactions of polymers. *1. Theory*. *J. Chem. Phys.* **60**, 866–877 (1974).

21. Hanggi, P. & Talkner, P. First-passage time problems for non-Markovian processes. *Phys. Rev. A* **32**, 1934–1937 (1985).

22. Masoliver, J., Lindenberg, K. & West, B. J. First-passage times for non-Markovian processes: correlated impacts on bound processes. *Phys. Rev. A* **34**, 2351–2363 (1986).

23. Krug, J. et al. Persistence exponents for fluctuating interfaces. *Phys. Rev. E* **56**, 2702 (1997).

24. Molchan, G. Maximum of a fractional Brownian motion: probabilities of small values. *Commun. Math. Phys.* **205**, 97–111 (1999).

25. Bray, A. J., Majumdar, S. N. & Schehr, G. Persistence and first-passage properties in nonequilibrium systems. *Adv. Phys.* **62**, 225–361 (2013).

26. Noh, J. D. & Rieger, H. Random walks on complex networks. *Phys. Rev. Lett.* **92**, 118701 (2004).

27. Ochab-Marcinek, A. & Holyst, R. Scale-dependent diffusion of spheres in solutions of flexible and rigid polymers: mean square displacement and autocorrelation function for FCS and DLS measurements. *Soft Matter* **7**, 7366–7374 (2011).

28. Mandelbrot, B. B. & Wallis, J. R. Noah, Joseph, and operational hydrology. *Wat. Resour. Res.* **4**, 909–918 (1968).

29. Cutland, N. J., Kopp, P. E. & Willinger, W. Stock price returns and the Joseph effect: a fractional version of the Black–Scholes model. In *Seminar on Stochastic Analysis, Random Fields and Applications* 327–351 (Springer, 1995).

30. Burnecki, K. et al. Universal algorithm for identification of fractional Brownian motion. A case of telomere subdiffusion. *Biophys. J.* **103**, 1839–1847 (2012).

**Acknowledgements** This work was supported by ERC grant FPTOpt-277998.

**Author Contributions** All authors contributed equally to this work.