ANNALES

DE LA FACULTÉ
DES SCIENCES

DE

TOULOUSE

Mathématiques

FLORENT MALRIEU

Some simple but challenging Markov processes

Tome XXIV, n° 4 (2015), p. 857-883.

<http://afst.cedram.org/item?id=AFST_2015_6_24_4_857_0>

© Université Paul Sabatier, Toulouse, 2015, tous droits réservés.

L'accès aux articles de la revue « Annales de la faculté des sciences de Toulouse Mathématiques » (http://afst.cedram.org/), implique l'accord avec les conditions générales d'utilisation (http://afst.cedram.org/legal/). Toute reproduction en tout ou partie de cet article sous quelque forme que ce soit pour tout usage autre que l'utilisation à fin strictement personnelle du copiste est constitutive d'une infraction pénale. Toute copie ou impression de ce fichier doit contenir la présente mention de copyright.

cedram

Article mis en ligne dans le cadre du

Centre de diffusion des revues académiques de mathématiques

http://www.cedram.org/
# Some simple but challenging Markov processes

FLORENT MALRIEU(1)

**RÉSUMÉ.** — Ces notes rassemblent l'étude du comportement en temps long de plusieurs processus de Markov déterministes par morceaux. Ces processus ont le double intérêt d'être motivés par la modélisation (biologie, réseaux de communication, chimie, ...) et d'impliquer de nombreux outils mathématiques : couplage, approches spectrales, équations aux dérivées partielles non locales ou encore inégalités fonctionnelles. Ces exemples permettent enfin de formuler des questions ouvertes.

**ABSTRACT.** — In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry, ...) and they are mathematically rich. Their mathematical study relies on coupling method, spectral decomposition, PDE technics, functional inequalities. We also relate these simple examples to recent and open problems.

## 1. Introduction

A Piecewise deterministic Markov processes (PDMP¹) is a stochastic process involving deterministic motion punctuated by random jumps. This large class of non diffusive stochastic models was introduced in the literature by Davis [20, 21] (see also [34]). As it will be stressed below, these processes arise naturally in many application areas: biology, communication networks, reliability of complex systems for example. From a mathematical point of view, they are simple to define but their study may require a broad spectrum of tools as stochastic coupling, functional inequalities, spectral analysis, dynamical systems, partial differential equations.

<footnote>(1) Laboratoire de Mathématiques et Physique Théorique (UMR CNRS 6083), Fédération Denis Poisson (FR CNRS 2964), Université François-Rabelais, Parc de Grand-mont, 37200 Tours, France.

florent.malrieu(AT)univ-tours.fr</footnote>

<footnote>(1) This may also mean "Persi Diaconis: Mathemagician and Popularizer".</footnote>
The aim of the present paper is to present simple examples of PDMP appearing in different applied frameworks and to investigate their long time behavior. Rather than using generic technics (as Meyn-Tweedie-Foster-Lyapunov...strategy) we will focus on as explicit as possible estimates. Several open and motivating questions (stability criteria, regularity of the invariant measure(s), explicit rate of convergence...) are also listed along the paper.

Roughly speaking the dynamics of a PDMP on a set $E$ depends on three local characteristics, namely, a flow $\varphi$, a jump rate $\lambda$ and a transition kernel $Q$. Starting from $x$, the motion of the process follows the flow $t \mapsto \varphi_t(x)$ until the first jump time $T_1$ which occurs in a Poisson-like fashion with rate $\lambda(x)$. More precisely, the distribution of the first jump time is given by

$$ \mathbb{P}_x(T_1 > t) = \exp \left( - \int_0^t \lambda(\varphi_s(x)) ds \right). $$

Then, the location of the process at the jump time $T_1$ is selected by the transition measure $Q(\varphi_{T_1}(x), \cdot)$ and the motion restarts from this new point as before. This motion is summed up by the infinitesimal generator:

$$ Lf(x) = F(x) \cdot \nabla f(x) + \lambda(x) \int_E (f(y) - f(x)) Q(x, dy), \quad (1.1) $$

where $F$ is the vector field associated to the flow $\varphi$. In several examples, the process may jump when it hits the boundary of $E$. The boundary of the space $\partial E$ can be seen as a region where the jump rate is infinite (see for example [18] for the study of billiards in a general domain with random reflections).

In the sequel, we denote by $\mathcal{P}(\mathbb{R}^d)$ the set of probability measures on $(\mathbb{R}^d, \mathcal{B}(\mathbb{R}^d))$ and, for any $p \ge 1$, by $\mathcal{P}_p(\mathbb{R}^d)$ the set of probability measures on $(\mathbb{R}^d, \mathcal{B}(\mathbb{R}^d))$ with a finite $p^{th}$-moment: $\mu \in \mathcal{P}_p(\mathbb{R}^d)$ if

$$ \int_{\mathbb{R}^d} |x|^p \mu(dx) < +\infty. $$

The total variation distance on $\mathcal{P}(\mathbb{R}^d)$ is given by

$$ \begin{aligned} \|\nu - \tilde{\nu}\|_{\text{TV}} &= \inf \left\{ \mathbb{P}(X \neq \tilde{X}) : X \sim \nu, \tilde{X} \sim \tilde{\nu} \right\} \\ &= \sup \left\{ \iint f \, d\nu - \iint f \, d\tilde{\nu} : f \text{ bounded by } 1/2 \right\}. \end{aligned} $$

If $\nu$ and $\tilde{\nu}$ are absolutely continuous with respect to $\mu$ with density functions $g$ and $\tilde{g}$, then

$$ \|\nu - \tilde{\nu}\|_{\text{TV}} = \frac{1}{2} \int_{\mathbb{R}^d} |g - \tilde{g}| \, d\mu. $$
For $p \ge 1$, the Wasserstein distance of order $p$, defined on $P_p(\mathbb{R}^d)$, is given by

$$W_p(\nu, \tilde{\nu}) = \inf \left\{ \left[ \mathbb{E} \left( |X - \tilde{X}|^p \right) \right]^{1/p} : X \sim \nu, \tilde{X} \sim \tilde{\nu} \right\}.$$

Similarly to the total variation distance, the Wasserstein distance of order 1 has a nice dual formulation:

$$W_1(\nu, \tilde{\nu}) = \sup \left\{ \iint f d\nu - \iint f d\tilde{\nu} : f \text{ is 1-Lipschitz} \right\}.$$

A generic dual expression can be formulated for $W_p$ (see [62]).

## 2. Storage models, with a bandit...

Let us consider the PDMP driven by the following infinitesimal generator:

$$Lf(x) = -\beta x f'(x) + \alpha \int_{0}^{\infty} (f(x+y) - f(x)) e^{-y} dy.$$

Such processes appear in the modeling of storage problems or pharmacokinetics that describe the evolution of the concentration of a chemical product in the human body. The present example is studied in [59, 6]. More realistic models are studied in [11, 14]. Similar processes can also be used as stochastic gene expression models (see [42, 65]).

In words, the current stock $X_t$ decreases exponentially at rate $\beta$, and increases at random exponential times by a random (exponentially distributed) amount. Let us introduce a Poisson process $(N_t)_{t \ge 0}$ with intensity $\alpha$ and jump times $(T_i)_{i \ge 0}$ (with $T_0 = 0$) and a sequence $(E_i)_{i \ge 1}$ of independent random variables with an exponential law of parameter 1 independent of $(N_t)_{t \ge 0}$. The process $(X_t)_{t \ge 0}$ starting from $x \ge 0$ can be constructed as follows: for any $i \ge 0$,

$$X_t = \begin{cases} e^{-\beta(t-T_i)} X_{T_i} & \text{if } T_i \le t < T_{i+1}, \\ e^{-\beta(T_{i+1}-T_i)} X_{T_i} + E_{i+1} & \text{if } t = T_{i+1}. \end{cases}$$

This model is sufficiently naïve to express the Laplace transform of $X$.

**LEMMA 2.1 (Laplace transform).** — For any $t \ge 0$ and $s < 1$, the Laplace transform of $X_t$ is given by

$$L(t, s) := \mathbb{E}(e^{sX_t}) = L(0, se^{-\beta t}) \left( \frac{1 - se^{-\beta t}}{1 - s} \right)^{\alpha/\beta},$$
where $L(0, \cdot)$ stands for the Laplace transform of $X_0$. In particular, the invariant distribution of $X$ is the Gamma distribution with density

$$x \mapsto \frac{x^{\alpha/\beta - 1} e^{-x}}{\Gamma(\alpha/\beta)} \mathbb{I}_{[0, +\infty)}(x).$$

*Proof*. — Applying the infinitesimal generator to $x \mapsto e^{sx}$, one deduces that the function $L$ is solution of the following partial differential equation:

$$\partial_t L(t, s) = -\beta s \partial_s L(t, s) + \frac{\alpha s}{1-s} L(t, s).$$

More generally, if the random income is no longer exponentially distributed but has a Laplace transform $L_i$ then $L$ is solution of

$$\partial_t L(t, s) = -\beta s \partial_s L(t, s) + \alpha(L_i(s) - 1)L(t, s).$$

As a consequence, if $G$ is given by $G(t, s) = \log L(t, s) + (\alpha/\beta) \log(1-s)$ then

$$\partial_t G(t, s) = -\beta s \partial_s G(t, s).$$

The solution of this partial differential equation is given by $G(t, s) = G(0, se^{-\beta t})$. $\square$

The next step is to investigate the convergence to equilibrium.

**THEOREM 2.2 (Convergence to equilibrium).** — Let us denote by $\nu P_t$ the law of $X_t$ if $X_0$ is distributed according to $\nu$. For any $x, y \ge 0$ and $t \ge 0$ and $p \ge 1$,

$$W_p(\delta_x P_t, \delta_y P_t) \le |x - y|e^{-\beta t},$$

and (when $\alpha \ne \beta$)

$$\|\delta_x P_t - \delta_y P_t\|_{\text{TV}} \le e^{-\alpha t} + |x-y|\alpha \frac{e^{-\beta t} - e^{-\alpha t}}{\alpha - \beta}. \quad (2.1)$$

Moreover, if $\mu$ is the invariant measure of the process $X$, we have for any probability measure $\nu$ with a finite first moment and $t \ge 0$,

$$\|\nu P_t - \mu\|_{\text{TV}} \le \|\nu - \mu\|_{\text{TV}} e^{-\alpha t} + W_1(\nu, \mu) \alpha \frac{e^{-\beta t} - e^{-\alpha t}}{\alpha - \beta}.$$

*Remark 2.3 (Limit case).* — In the case $\alpha = \beta$, the upper bound (2.1) becomes

$$\|\delta_x P_t - \delta_y P_t\|_{\text{TV}} \le (1 + |x - y|\alpha t)e^{-\alpha t}.$$
*Remark 2.4 (Optimality).* — Applying $L$ to the test function $f(x) = x^n$ allows us to compute recursively the moments of $X_t$. In particular,

$$ \mathbb{E}_x(X_t) = \frac{\alpha}{\beta} + \left(x - \frac{\alpha}{\beta}\right) e^{-\beta t}. $$

This relation ensures that the rate of convergence for the Wasserstein distance is sharp. Moreover, the coupling for the total variation distance requires at least one jump. As a consequence, the exponential rate of convergence is greater than $\alpha$. Thus, Equation (2.1) provides the optimal rate of convergence $\alpha \wedge \beta$.

*Proof of Theorem 2.2.* — Firstly, consider two processes $X$ and $Y$ starting respectively at $x$ and $y$ and driven by the same randomness (i.e. Poisson process and jumps). Then the distance between $X_t$ and $Y_t$ is deterministic:

$$ X_t - Y_t = (x - y)e^{-\beta t}. $$

Obviously, for any $p \ge 1$ and $t \ge 0$,

$$ W_p(\delta_x P_t, \delta_y P_t) \le |x - y|e^{-\beta t}. $$

Let us now construct explicitly a coupling at time $t$ to get the upper bound (2.1) for the total variation distance. The jump times of $(X_t)_{t \ge 0}$ and $(Y_t)_{t \ge 0}$ are the ones of a Poisson process $(N_t)_{t \ge 0}$ with intensity $\alpha$ and jump times $(T_i)_{i \ge 0}$. Let us now construct the jump heights $(E_i^X)_{1 \le i \le N_t}$ and $(E_i^Y)_{1 \le i \le N_t}$ of $X$ and $Y$ until time $t$. If $N_t = 0$, no jump occurs. If $N_t \ge 1$, we choose $E_i^X = E_i^Y$ for $1 \le i \le N_t - 1$ and $E_{N_t}^X$ and $E_{N_t}^Y$ in order to maximise the probability

$$ \mathbb{P}(X_{T_{N_t}} + E_{N_t}^X = Y_{T_{N_t}} + E_{N_t}^Y | X_{T_{N_t}}, Y_{T_{N_t}}). $$

This maximal probability of coupling is equal to

$$ \exp(-|X_{T_{N_t}} - Y_{T_{N_t}}|) = \exp(-|x - y|e^{-\beta T_{N_t}}) \ge 1 - |x - y|e^{-\beta T_{N_t}}. $$

As a consequence, we get that

$$ \begin{aligned} \|\delta_x P_t - \delta_y P_t\|_{\text{TV}} &\le 1 - \mathbb{E}[(1 - |x - y|e^{-\beta T_{N_t}}) \mathbb{1}_{\{N_t \ge 1\}}] \\ &\le e^{-\alpha t} + |x - y|\mathbb{E}(e^{-\beta T_{N_t}} \mathbb{1}_{\{N_t \ge 1\}}). \end{aligned} $$

The law of $T_n$ conditionally on the event $\{N_t = n\}$ has the density

$$ u \mapsto n \frac{u^{n-1}}{t^n} \mathbb{1}_{[0,t]}(u). $$
This ensures that

$$ \mathbb{E}(e^{-\beta T_{N_t}} \mathbb{I}_{\{N_t \ge 1\}}) = \int_0^1 e^{-\beta t v} \mathbb{E}(N_t v^{N_t-1}) dv. $$

Since the law of $N_t$ is the Poisson distribution with parameter $\alpha t$, one has

$$ \mathbb{E}(N_t v^{N_t-1}) = \alpha t e^{\alpha t (v-1)}. $$

This ensures that

$$ \mathbb{E}(e^{-\beta N_t} \mathbb{I}_{\{N_t \ge 1\}}) = \alpha \frac{e^{-\beta t} - e^{-\alpha t}}{\alpha - \beta}, $$

which completes the proof. Finally, to get the last estimate, we proceed as follows: if $N_t$ is equal to 0, a coupling in total variation of the initial measures is done, otherwise, we use the coupling above. $\square$

*Remark 2.5 (Another example).* — Surprisingly, a process of the same type appears in [37] in the study of the so-called bandit algorithm. The authors have to investigate the long time behavior of the process driven by

$$ Lf(y) = (1 - p - py)f'(y) + qy \frac{f(y + g) - f(y)}{g}, $$

where $0 < q < p < 1$ and $g > 0$. This can be done following the lines of the proof of Theorem 2.2.

### 3. The TCP model with constant jump rate

This section is devoted to the process on $[0, +\infty)$ driven by the following infinitesimal generator

$$ Lf(x) = f'(x) + \lambda(f(x/2) - f(x)) \quad (x \ge 0). $$

In other words, the process grows linearly between jump times that are the one of a homogeneous Poisson process with parameter $\lambda$ and it is divided by 2 at these instants of time. See Section 3.4 for concrete motivations and generalizations.

#### 3.1. Spectral decomposition

Without loss of generality, we choose $\lambda = 1$ in this section. The generator $L$ of the naïve TCP process preserves the degree of polynomials. As a
consequence, for any $n \in \mathbb{N}$, the eigenvalue $\lambda_n = -(1 - 2^{-n})$ is associated
to a polynomials $P_n$ with degree $n$. As an example,

$$
P_0(x) = 1, \quad P_1(x) = x - 2 \quad \text{and} \quad P_2(x) = x^2 - 8x + 32/3.
$$

Moreover, one can explicitly compute the moments of the invariant measure
$\mu$ (see [39]): for any $n \in \mathbb{N}$

$$
\int x^n \mu(dx) = \frac{n!}{\prod_{k=1}^{n} (1 - 2^{-k})}.
$$

Roughly speaking, this relation comes from the fact that the functions $m_n$:
$t \in [0, \infty) \mapsto \mathbb{E}(X_t^n)$ for $n \ge 0$ are solution of

$$
m'_{n}(t) = nm_{n-1}(t) + (2^{-n} - 1)m_{n}(t).
$$

It is also shown in [24] that the Laplace transform of $\mu$ is finite on a neigh-
borhood of the origin. As a consequence, the polynomials are dense in $L^2(\mu)$.
Unfortunately, the eigenvectors of $L$ are not orthogonal in $L^2(\mu)$. For ex-
ample,

$$
\int P_1 P_2 d\mu = -\frac{64}{27}.
$$

This lack of symmetry (due to the fact that the invariant measure $\mu$ is
not reversible) prevents us to easily deduce an exponential convergence to
equilibrium in $L_{\mu}^2$.

When the invariant measure is reversible, the spectral decomposition
(and particularly its spectral gap) of $L$ provides fine estimates for the con-
vergence to equilibrium. See for example [41] and the connection with cou-
pling strategies and strong stationary times introduced in [1].

OPEN QUESTION 1 (Spectral proof of ergodicity). — *Despite the lack of reversibility, is it possible to use the spectral properties of L to get some estimates on the long time behavior of X?*

Remark 3.1. — *This spectral approach has been fruitfully used in [28, 45]*
*to study (nonreversible) hypocoercive models.*

**3.2. Convergence in Wasserstein distances**

The convergence in Wasserstein distance is obvious.

LEMMA 3.2 (Convergence in Wasserstein distance [57, 16]). — For any $p \ge 1$,

$$
W_p(\delta_x P_t, \delta_y P_t) \le |x - y| e^{-\lambda_p t} \quad \text{with} \quad \lambda_p = \frac{\lambda(1 - 2^{-p})}{p}. \quad (3.1)
$$
*Remark 3.3 (Alternative approach).* — The case $p = 1$ is obtained in [57] by PDEs estimates using the following alternative formulation of the Wasserstein distance on $\mathbb{R}$. If the cumulative distribution functions of the two probability measures $\nu$ and $\tilde{\nu}$ are $F$ and $\tilde{F}$ then

$$W_1(\nu, \tilde{\nu}) = \int_{\mathbb{R}} |F(x) - \tilde{F}(x)| dx.$$

The general case $p \ge 1$ is obvious from the probabilistic point of view: choosing the same Poisson process $(N_t)_{t \ge 0}$ to drive the two processes provides that the two coordinates jump simultaneously and

$$|X_t - Y_t| = |x - y|2^{-N_t}.$$

As a consequence, since the law of $N_t$ is the Poisson distribution with parameter $\lambda t$, one has

$$\mathbb{E}_{x,y}(|X_t - Y_t|^p) = |x - y|^p \mathbb{E}(2^{-pN_t}) = |x - y|^p e^{-p\lambda_p t}.$$

This coupling turns out to be sharp. Indeed, one can compute explicitly the moments of $X_t$ (see [39, 52]): for every $n \ge 0$, every $x \ge 0$, and every $t \ge 0$,

$$\mathbb{E}_x(X_t^n) = \frac{n!}{\prod_{k=1}^n \theta_k} + n! \sum_{m=1}^n \left( \sum_{k=0}^m \frac{x^k}{k!} \prod_{\substack{j=k \\ j \ne m}}^n \frac{1}{\theta_j - \theta_m} \right) e^{-\theta_m t}, \quad (3.2)$$

where $\theta_n = \lambda(1 - 2^{-n}) = n\lambda_n$ for any $n \ge 1$. Obviously, assuming for example that $x > y$,

$$W_n(\delta_x P_t, \delta_y P_t)^n \ge \mathbb{E}_x((X_t)^n) - \mathbb{E}_y((Y_t)^n) \\ \sim_{t \to \infty} n! \left( \sum_{k=0}^n \frac{x^k - y^k}{k!} \prod_{j=k}^{n-1} \frac{1}{\theta_j - \theta_n} \right) e^{-\theta_n t}.$$

As a consequence, the rate of convergence in Equation (3.1) is optimal for any $n \ge 1$.

### 3.3. Convergence in total variation distance

The estimate for the Wasserstein rate of convergence does not provide on its own any information about the total variation distance between $\delta_x P_t$ and $\delta_y P_t$. It turns out that this rate of convergence is the one of the $W_1$ distance. This is established in [57, Thm 1.1]. Let us provide here an improvement of this result by a probabilistic argument.
**THEOREM 3.4** (Convergence in total variation distance). — For any $x, y \ge 0$ and $t \ge 0$,

$$ \|\delta_x P_t - \delta_y P_t\|_{\text{TV}} \le \lambda e^{-\lambda t/2} |x - y| + e^{-\lambda t}. \quad (3.3) $$

As a consequence, for any measure $\nu$ with a finite first moment and $t \ge 0$,

$$ \|\nu P_t - \mu\|_{\text{TV}} \le \lambda e^{-\lambda t/2} W_1(\nu, \mu) + e^{-\lambda t} \|\nu - \mu\|_{\text{TV}}. \quad (3.4) $$

**Remark 3.5** (Propagation of the atom). — Note that the upper bound obtained in Equation (3.3) does not go to zero as $y \to x$. This is due to the fact that $\delta_x P_t$ has an atom at $y + t$ with mass $e^{-\lambda t}$.

*Proof of Theorem 3.4.* — The coupling is a slight modification of the Wasserstein one. The paths of $(X_s)_{0 \le s \le t}$ and $(Y_s)_{0 \le s \le t}$ starting respectively from $x$ and $y$ are determined by their jump times $(T_n^X)_{n \ge 0}$ and $(T_n^Y)_{n \ge 0}$ up to time $t$. These sequences have the same distribution than the jump times of a Poisson process with intensity $\lambda$.

Let $(N_t)_{t \ge 0}$ be a Poisson process with intensity $\lambda$ and $(T_n)_{n \ge 0}$ its jump times with the convention $T_0 = 0$. Let us now construct the jump times of $X$ and $Y$. Both processes make exactly $N_t$ jumps before time $t$. If $N_t = 0$, then

$$ X_s = x + s \quad \text{and} \quad Y_s = y + s \quad \text{for } 0 \le s \le t. $$

Assume now that $N_t \ge 1$. The $N_t - 1$ first jump times of $X$ and $Y$ are the ones of $(N_t)_{t \ge 0}$:

$$ T_k^X = T_k^Y = T_k \quad 0 \le k \le N_t - 1. $$

In other words, the Wasserstein coupling acts until the penultimate jump time $T_{N_t-1}$. At that time, we have

$$ X_{T_{N_t-1}} - Y_{T_{N_t-1}} = \frac{x-y}{2^{N_t-1}}. $$

Then we have to define the last jump time for each process. If they are such that

$$ T_{N_t}^X = T_{N_t}^Y + X_{T_{N_t-1}} - Y_{T_{N_t-1}}, $$

then the paths of $X$ and $Y$ are equal on the interval $(T_{N_t}^X, t)$ and can be chosen to be equal for any time larger than $t$.

Recall that conditionally on the event $\{N_t = 1\}$, the law of $T_1$ is the uniform distribution on $(0, t)$. More generally, if $n \ge 2$, conditionally on the set $\{N_t = n\}$, the law of the penultimate jump time $T_{n-1}$ has a density $s \mapsto n(n-1)t^{-n}(t-s)s^{n-2}\mathbb{I}_{(0,t)}(s)$ and conditionally on the event $\{N_t = n, T_{n-1} = s\}$, the law of $T_n$ is uniform on the interval $(s, t)$.
Conditionally on $N_t = n \ge 1$ and $T_{n-1}, T_n^X$ and $T_n^Y$ are uniformly distributed on $(T_{n-1}, t)$ and can be chosen such that

$$ \mathbb{P}\left(T_n^X = T_n^Y + \frac{x-y}{2^{n-1}} \middle| N_t^X = N_t^Y = n, T_{n-1}^X = T_{n-1}^Y = T_{n-1}\right) \\ = \left(1 - \frac{|x-y|}{2^{n-1}(t-T_{n-1})}\right) \vee 0 \ge 1 - \frac{|x-y|}{2^{n-1}(t-T_{n-1})}. $$

This coupling provides that

$$ \|\delta_x P_t - \delta_y P_t\|_{\text{TV}} \le 1 - \mathbb{E}\left[\left(1 - \frac{|x-y|}{2^{N_t-1}(t-T_{N_t-1})}\right) \mathbb{1}_{\{N_t \ge 1\}}\right] \\ \le e^{-\lambda t} + |x-y| \mathbb{E}\left(\frac{2^{-N_t+1}}{(t-T_{N_t-1})} \mathbb{1}_{\{N_t \ge 1\}}\right). $$

For any $n \ge 2$,

$$ \mathbb{E}\left(\frac{1}{t - T_{N_t-1}} \middle| N_t = n\right) = \frac{n(n-1)}{t^n} \int_0^t u^{n-2} du = \frac{n}{t}. $$

This equality also holds for $n = 1$. Thus we get that

$$ \mathbb{E}\left(\frac{2^{-N_t+1}}{(t-T_{N_t-1})} \mathbb{1}_{\{N_t \ge 1\}}\right) = \frac{1}{t} \mathbb{E}(N_t 2^{-N_t+1}) = \lambda e^{-\lambda t/2}, $$

since $N_t$ is distributed according to the Poisson law with parameter $\lambda t$. This provides the estimate (3.3). The general case (3.4) is a straightforward consequence: if $N_t$ is equal to 0, a coupling in total variation of the initial measures is done, otherwise, we use the coupling above. $\square$

### 3.4. Some generalizations

This process on $\mathbb{R}_+$ belongs to the subclass of the AIMD (Additive Increase Multiplicative Decrease) processes. Its infinitesimal generator is given by

$$ Lf(x) = f'(x) + \lambda(x) \int_0^1 (f(ux) - f(x)) \nu(du), \quad (3.5) $$

where $\nu$ is a probability measure on $[0, 1]$ and $\lambda$ is a non negative function. It can be viewed as the limit behavior of the congestion of a single channel (see [24, 31] for a rigorous derivation of this limit). In [44], the authors give a generalization of the scaling procedure to interpret various PDMPs as the limit of discrete time Markov chains and in [40] more general increase and decrease profiles are considered as models for TCP. In the real world
(Internet), the AIMD mechanism allows a good compromise between the minimization of network congestion time and the maximization of mean throughput. See also [12] for a simplified TCP windows size model. See [40, 43, 52, 53, 54, 51, 33] for other works dedicated to this process. Generalization to interacting multi-class transmissions are considered in [29, 30].

Such processes are also used to model the evolution of the size of bacteria or polymers which mixes growth and fragmentation: they growth in a deterministic way with a growth speed $x \mapsto \tau(x)$, and split at rate $x \mapsto \lambda(x)$ into two (for simplicity) parts $y$ and $x-y$ according a kernel $\beta(x, y)dy$. The infinitesimal generator associated to this dynamics writes

$$Lf(x) = \tau(x)f'(x) + \lambda(x) \int_{0}^{x} (f(y) - f(x))\beta(x, y) dy.$$

If the initial distribution of the size has a density $u(\cdot, 0)$ then this density is solution of the following integro-differential PDE:

$$\partial_t u(x,t) = -\partial_x(\tau(x)u(x,t)) - \lambda(x)u(x,t) + \int_x^\infty \lambda(y)\beta(y,x)u(y,t) dy.$$

If one is interesting in the density of particles with size $x$ at time $t$ in the growing population (a splitting creates two particles), one has to consider the PDE

$$\partial_t u(x,t) = -\partial_x(\tau(x)u(x,t)) - \lambda(x)u(x,t) + 2 \int_x^\infty \lambda(y)\beta(y,x)u(y,t) dy.$$

This growth-fragmentation equations have been extensively studied from a PDE point of view (see for example [56, 23, 15, 46]). A probabilistic approach is used in [10] to study the pure fragmentation process.

## 4. Switched flows and motivating examples

Let $E$ be the set $\{1, 2, \dots, n\}$, $(\lambda(\cdot, i, j))_{i,j \in E}$ be nonnegative continuous functions on $\mathbb{R}^d$, and, for any $i \in E$, $F^i(\cdot) : \mathbb{R}^d \mapsto \mathbb{R}^d$ be a smooth vector field such that the ordinary differential equation

$$\begin{cases} \dot{x}_t = F^i(x_t) & \text{for } t > 0, \\ x_0 = x \end{cases}$$

has a unique and global solution $t \mapsto \varphi_t^i(x)$ on $[0, +\infty)$ for any initial condition $x \in \mathbb{R}^d$. Let us consider the Markov process

$$(Z_t)_{t \ge 0} = ((X_t, I_t))_{t \ge 0} \text{ on } \mathbb{R}^d \times E$$
defined by its infinitesimal generator $L$ as follows:

$$Lf(x, i) = F^i(x) \cdot \nabla_x f(x, i) + \sum_{j \in E} \lambda(x, i, j)(f(x, j) - f(x, i))$$

for any smooth function $f: \mathbb{R}^d \times E \to \mathbb{R}$.

These PDMP are also known as hybrid systems. They have been intensively studied during the past decades (see for example the review [64]). In particular, they naturally appear as the approximation of Markov chains mixing slow and fast dynamics (see [19]). They could also be seen as a continuous time version of iterated random functions (see the excellent review [22]).

In this section, we present few examples from several applied areas and describe their long time behavior.

## 4.1. A surprising blow up for switched ODEs

The main probabilistic results of this section are established in [38]. Consider the Markov process $(X, I)$ on $\mathbb{R}^2 \times \{0, 1\}$ driven by the following infinitesimal generator:

$$Lf(x, i) = (A_i x) \cdot \nabla_x f(x, i) + r(f(x, 1 - i) - f(x, i)) \quad (4.1)$$

where $r > 0$ and $A_0$ and $A_1$ are the two following matrices

$$A_0 = \begin{pmatrix} -\alpha & 1 \\ 0 & -\alpha \end{pmatrix} \quad \text{and} \quad A_1 = \begin{pmatrix} -\alpha & 0 \\ -1 & -\alpha \end{pmatrix} \quad (4.2)$$

for some positive $\alpha$. In other words, $(I_t)_{t \ge 0}$ is a Markov process on $\{0, 1\}$ with constant jump rate $r$ (from 0 to 1 and from 1 to 0) and $(X_t)_{t \ge 0}$ is the solution of $\dot{X}_t = A_{I_t} X_t$.

The two matrices $A_0$ and $A_1$ are Hurwitz matrices (all eigenvalues have strictly negative real parts). Moreover, it is also the case for the matrix $A_p = pA_1 + (1-p)A_0$ with $p \in [0, 1]$ since the eigenvalues of $A_p$ are $-\alpha \pm i\sqrt{p(1-p)}$. Then, for any $p \in [0, 1]$, there exists $K_p \ge 1$ and $\rho > 0$ such that

$$\|x_t\| \le K_p \|x_0\| e^{-\rho t},$$

for any solution $(x_t)_{t \ge 0}$ of $\dot{x}_t = A_p x_t$.
### 4.1.1. Asymptotic behavior of the continuous component

The first step is to use polar coordinates to study the large time behavior of $R_t = \|X_t\|$ and $U_t$ the point on the unit circle $S^1$ given by $X_t/R_t$. One gets that

$$ \dot{R}_t = R_t \langle A_{I_t} U_t, U_t \rangle $$

$$ \dot{U}_t = A_{I_t} U_t - \langle A_{I_t} U_t, U_t \rangle U_t. $$

As a consequence, $(U_t, I_t)$ is a Markov process on $S^1 \times \{0, 1\}$. One can show that it admits a unique invariant measure $\mu$. Therefore, if $\mathbb{P}(R_0 = 0) = 0$,

$$ \frac{1}{t} \log R_t = \frac{1}{t} \log R_0 + \frac{1}{t} \int_0^t \langle A_{I_s} U_s, U_s \rangle \, ds \xrightarrow{a.s.} \int \langle A_i u, u \rangle \mu(du, i). $$

The stability of the Markov process depends on the sign of

$$ L(\alpha, r) := \int \langle A_i u, u \rangle \mu(du, i). $$

An "explicit" formula for $L(\alpha, r)$ can be formulated in terms of the classical trigonometric functions

$$ \cot(x) = \frac{\cos(x)}{\sin(x)}, \quad \sec(x) = \frac{1}{\cos(x)} \quad \text{and} \quad \csc(x) = \frac{1}{\sin(x)}. $$

**THEOREM 4.1** (Lyapunov exponent [38]). — For any $r > 0$ and $\alpha > 0$,

$$ L(\alpha, r) = G(r) - \alpha \quad \text{where} \quad G(r) = \int_{0}^{2\pi} (p_{0}(\theta; r) - p_{1}(\theta; r)) \cos(\theta) \sin(\theta) \, d\theta > 0 $$

and $p_0$ and $p_1$ are defined as follows: for $\theta \in (-\pi/2, 0)$

$$ H(\theta; r) = \exp(-2r \cot(2\theta)) \int_{\theta}^{0} \exp(2r \cot(2y)) \sec^2(y) \, dy, $$

$$ C(r) = \left[ 4 \int_{-\frac{\pi}{2}}^{0} \sec^2(x) + (\csc^2(x) - \sec^2(x))rH(x; r) \, dx \right]^{-1}, $$

$$ p_0(\theta; r) = C(r) \csc^2(\theta) r H(\theta; r), $$

$$ p_1(\theta; r) = C(r) \sec^2(\theta) [1 - rH(\theta; r)], $$

and for any $\theta \in \mathbb{R}$,

$$ p_i(\theta; r) = p_{1-i}(\theta + \pi/2; r) = p_i(\theta + \pi; r). $$
Sketch of proof of Theorem 4.1. — Let us denote by $(\Theta_t)_{t \ge 0}$ the lift of $(U_t)_{t \ge 0}$.
The process $(\Theta, I)$ is also Markovian. Moreover, its infinitesimal generator
is given by

$$
\mathcal{L}f(\theta, i) = -[i \cos^2(\theta) + (1-i) \sin^2(\theta)] \partial_{\theta} f(\theta, i) + r[f(\theta, 1-i) - f(\theta, i)].
$$

Notice that the dynamics of $(\Theta, I)$ does not depend on the parameter $\alpha$.
This process has a unique invariant measure $\mu$ (depending on the jump
rate $r$). With the one-to-one correspondence between a point on $S^1$ and a
point in $[0, 2\pi)$, let us write the invariant probability measure $\mu$ as

$$
\mu(d\theta, i) = p_i(\theta; r) \mathbb{I}_{[0,2\pi)}(\theta) d\theta,
$$

The functions $p_0$ and $p_1$ are solution of

$$
\begin{cases}
\partial_{\theta}(\sin^2(\theta)p_0(\theta)) + r(p_1(\theta) - p_0(\theta)) = 0, \\
\partial_{\theta}(\cos^2(\theta)p_1(\theta)) + r(p_0(\theta) - p_1(\theta)) = 0.
\end{cases}
$$

These relations provide the desired expressions.

<figure><img src="image_1.png" /></figure>

The previous technical result provides immediately the following result
on the (in)stability of the process.

COROLLARY 4.2 ((In)Stability [38]). — There exist α > 0, a > 0 and
b > 0 such that L (α, r) is negative if r < a or r > b and L (α, r) is positive
for some r ∈ (a, b).

From numerical experiments, see Figure 1, one can formulate the follow-
ing conjecture on the function G.

<figure><img src="image_2.png" /><figcaption>Figure 1. — Shape of the function G defined in Theorem 4.1.</figcaption></figure>
**CONJECTURE 4.3 (Shape of G).** — There exists $r_c \sim 4.6$ such that $G'(r) > 0$ for $r < r_c$ and $G'(r) < 0$ for $r > r_c$ and $G(r_c) \sim 0.2$. Moreover,

$$ \lim_{r \to 0} G(r) = 0 \quad \text{and} \quad \lim_{r \to \infty} G(r) = 0. $$

**OPEN QUESTION 2 (Shape of the instability domain).** — Is it possible to prove Conjecture 4.3? This would imply that the set

$$ U_{\alpha} = \{r > 0 : \|X_t\| \xrightarrow{\frac{p.s.}{t \to \infty}} +\infty\} = \{r > 0 : L(r, \alpha) > 0\} = \{r > 0 : G(r) > \alpha\} $$
is empty for $\alpha > G(r_c)$ and is a non empty interval if $\alpha < G(r_c)$.

*Remark 4.4 (On the irreducibility of (U, I)).* — Notice that one can modify the matrices $A_0$ and $A_1$ in such a way that (U, I) has two ergodic invariant measures (see [9]).

**OPEN QUESTION 3 (Oscillations of the Lyapunov exponent).** — Is it possible to choose the two $2 \times 2$ matrices $A_0$ and $A_1$ in such a way that the set of jump rates $r$ associated to unstable processes is the union of several intervals?

### 4.1.2. A deterministic counterpart

Consider the following ODE

$$ \dot{x}_t = (1 - u_t)A_0x_t + u_tA_1x_t, \quad (4.3) $$

where $u$ is a given measurable function from $[0, \infty)$ to $\{0, 1\}$. The system is said to be *unstable* if there exists a starting point $x_0$ and a measurable function $u : [0, \infty) \to \{0, 1\}$ such that the solution of (4.3) goes to infinity.

In [13, 4, 5], the authors provide necessary and sufficient conditions for the solution of (4.3) to be unbounded for two matrices $A_0$ and $A_1$ in $M_2(\mathbb{R})$. In the particular case (4.2), this result reads as follows.

**THEOREM 4.5 (Criterion for stability [5]).** — If $A_0$ and $A_1$ are given by (4.2), the system (4.3) is unbounded if and only if

$$ R(\alpha^2) := \frac{1 + 2\alpha^2 + \sqrt{1 + 4\alpha^2}}{2\alpha^2} e^{-2\sqrt{1+4\alpha^2}} > 1. \quad (4.4) $$

More precisely, the result in [5] ensures that

* if $2\alpha > 1$ (case $S1$ in [5]) then there exists a common quadratic Lyapunov function for $A_0$ and $A_1$ (and $\|X_t\|$ goes to 0 exponentially fast as $t \to \infty$ for any function $u$),
• if $2\alpha \le 1$ (case S4 in [5]) then, the system is

- globally uniformly asymptotically stable (and $\|X_t\|$ goes to 0 exponentially fast as $t \to \infty$ for any function $u$) if $R(\alpha^2) < 1$,

- uniformly stable (but for some functions $u$, $\|X_t\|$ does not converge to 0) if $R(\alpha^2) = 1$,

- unbounded if $R(\alpha^2) > 1$,

where $R(\alpha^2)$ is given by (4.4).

*Proof of Theorem 4.5.* — The general case is considered in [5]. The main idea is to construct the so-called worst trajectory choosing at each instant of time the vector field that drives the particle away from the origin. The solutions $x_t = (y_t, z_t)$ of $\dot{x}_t = A_0x_t$ and $\dot{z}_t = A_1x_t$ starting from $x_0 = (y_0, z_0)$ are respectively given by

$$ \begin{cases} y_t = (z_0 t + y_0)e^{-\alpha t} \\ z_t = z_0 e^{-\alpha t} \end{cases} \quad \text{and} \quad \begin{cases} y_t = y_0 e^{-\alpha t} \\ z_t = (-y_0 t + z_0)e^{-\alpha t}. \end{cases} $$

Let us define, for $x = (y, z)$,

$$ Q(x) = \det(A_0 x, A_1 x) = \alpha y^2 - yz - \alpha z^2. $$

Then the set of the points where $A_0x$ and $A_1x$ are collinear is given by

$$ \{x \in \mathbb{R}^2 : Q(x) = 0\} = \{x = (y, z) : y = \gamma^+ z \text{ or } y = \gamma^- z\} $$

where

$$ \gamma^+ = \frac{1 + \sqrt{1 + 4\alpha^2}}{2\alpha} > 0 \quad \text{and} \quad \gamma^- = \frac{1 - \sqrt{1 + 4\alpha^2}}{2\alpha} < 0. $$

Let us start with $x_0 = (0,1)$ and $I_0 = 0$. Choose $t_1 = \gamma^+$ in such a way that:

$$ x_{t_1} = (\gamma^+ e^{-\alpha \gamma^+}, e^{-\alpha \gamma^+}). $$

Now, set $t_2 = t_1 + \gamma^+ - \gamma^-$ and $I_t = 1$ for $t \in [t_1, t_2)$ in such a way that $y_{t_2} = \gamma^- z_{t_2}$ i.e. $y_{t_2} = -(\gamma^+)^{-1} z_{t_2}$. Then, one has

$$ x_{t_2} = (\gamma^+ e^{-\alpha(2\gamma^+-\gamma^-)}, -(\gamma^+)^2 e^{-\alpha(2\gamma^+-\gamma^-)}). $$

Finally, choose $t_3 = t_2 - \gamma^-$ and $I_t = 0$ for $t \in [t_2, t_3)$ in such a way that $y_{t_3} = 0$. Then, one has

$$ x_{t_3} = (0, -(\gamma^+)^2 e^{-2\alpha(\gamma^+-\gamma^-)}). $$

The process is unbounded if and only if $\|x_{t_3}\| > 1$. This is equivalent to (4.4). □
Some simple but challenging Markov processes

<figure><img src="image_3.png" /><figcaption>Figure 2. — The *worth trajectory* with $\alpha = 0.32$ (on the left), $\alpha = 0.3314$ (in the middle) and $\alpha = 0.34$ (on the right). The system evolves clock-wisely from (0, 1).</figcaption></figure>
## 4.2. Invariant measure(s) of switched flows

In order to avoid the possible explosions studied in Section 4.1, one can impose that the state space of the continuous variable is a compact set.

In [7], it is shown thanks to an example that the number of the invariant measures may depend on the jump rate for fixed vector fields (as for the problem of (un)-stability described in the previous section). Moreover Hörmander-like conditions on the vector fields are formulated in [2, 7] to ensure that the first marginal of the invariant measure(s) may be absolutely continuous with respect to the Lebesgue measure on $\mathbb{R}^d$. However the density may blow up as it is shown in the example below.

**Example 4.6** (Possible blow up of the density near a critical point). — Consider the process on $\mathbb{R} \times \{0, 1\}$ associated to the infinitesimal generator

$$Lf(x, i) = -\alpha_i(x - i)\partial_x f(x, i) + \lambda_i(f(x, 1 - i) - f(x)).$$

This process is studied in [36, 58]. The support of its invariant measure $\mu$ is the set $[0, 1] \times \{0, 1\}$ and $\mu$ is given by

$$\int f d\mu = \frac{\lambda_1}{\lambda_0 + \lambda_1} \int_0^1 f(x, 0) p_0(x) dx + \frac{\lambda_0}{\lambda_0 + \lambda_1} \int_0^1 f(x, 1) p_1(x) dx,$$

where $p_0$ and $p_1$ are Beta distributions:

$$p_0(x) = \frac{x^{\lambda_0/\alpha_0 - 1}(1-x)^{\lambda_1/\alpha_1}}{B(\lambda_0/\alpha_0, \lambda_1/\alpha_1 + 1)} \quad \text{and} \quad p_1(x) = \frac{x^{\lambda_0/\alpha_0}(1-x)^{\lambda_1/\alpha_1 - 1}}{B(\lambda_0/\alpha_0 + 1, \lambda_1/\alpha_1)}.$$

The density of the invariant measure possibly explodes near 0 or 1.

The paper [3] is a detailed analysis of invariant measures for switched flows in dimension one. In particular, the authors prove smoothness of the invariant densities away from critical points and describe the asymptotics of the invariant densities at critical points.

The situation is more intricate for higher dimensions.

**Example 4.7** (Possible blow up of the density in the interior of the support). — Consider the process on $\mathbb{R}^2 \times \{0, 1\}$ associated to the constant jump rates $\lambda_0$ and $\lambda_1$ for the discrete component and the vector fields

$$F^0(x) = Ax \text{ and } F^1(x) = A(x - a) \text{ where } A = \begin{pmatrix} -1 & -1 \\ 1 & -1 \end{pmatrix} \text{ and } a = \begin{pmatrix} 1 \\ 0 \end{pmatrix}. \quad (4.5)$$
<figure><img src="image_4.png" /><figcaption>Figure 3. — Path of the process associated to $F^0$ and $F^1$ given by (4.5) starting from the origin. Red (resp. blue) pieces of path correspond to $I = 1$ (resp. $I = 0$).</figcaption></figure>

The origin and $a$ are the respective unique critical points of $F^0$ and $F^1$. Thanks to the precise estimates in [3], one can prove the following fact. If $\lambda_0$ is small enough then, as for one-dimensional example, the density of the invariant measure blows up at the origin. This also implies that the density is infinite on the set $\{\varphi_t^1(0) : t \ge 0\}$.

**OPEN QUESTION 4.** — *What can be said on the smoothness of the density of the invariant measure of such processes?*

### 4.3. A convergence result

This section sums up the study of the long time behavior of certain switched flows presented in [8]. See also [61] for another approach. To focus on the main lines of this paper, the hypotheses below are far from the optimal ones.

**HYPOTHESIS 4.8** (Regularity of the jump rates). — There exist $a > 0$ and $\kappa > 0$ such that, for any $x, \tilde{x} \in \mathbb{R}^d$ and $i, j \in E$,

$$a(x, i, j) \ge \underline{a} \quad \text{and} \quad \sum_{j \in E} |a(x, i, j) - a(\tilde{x}, i, j)| \le \kappa \|x - \tilde{x}\|.$$

The lower bound condition insures that the second — discrete — coordinate of $Z$ changes often enough (so that the second coordinates of two independent copies of $Z$ coincide sufficiently often).
HYPOTHESIS 4.9 (Strong dissipativity of the vector fields). — There ex-
ists $\alpha > 0$ such that,

$$
\langle x - \tilde{x}, F^i(x) - F^i(\tilde{x}) \rangle \le -\alpha \|x - \tilde{x}\|^2, \quad x, \tilde{x} \in \mathbb{R}^d, i \in E. \quad (4.6)
$$

Hypothesis 4.9 ensures that, for any $i \in E$,

$$
\| \varphi_t^i(x) - \varphi_t^i(\tilde{x}) \| \le e^{-\alpha t} \|x - \tilde{x}\|, \quad x, \tilde{x} \in \mathbb{R}^d.
$$

As a consequence, the vector fields $F^i$ has exactly one critical point $\sigma(i) \in \mathbb{R}^d$. Moreover it is exponentially stable since, for any $x \in \mathbb{R}^d$,

$$
\| \varphi_t^i(x) - \sigma(i) \| \le e^{-\alpha t} \|x - \sigma(i)\|.
$$

In particular, $X$ cannot escape from a sufficiently large ball $\bar{B}(0, M)$. Define the following distance $W_1$ on the probability measures on $B(0, M) \times E$: for $\eta, \tilde{\eta} \in \mathcal{P}(B(0, M) \times E)$,

$$
W_1(\eta, \tilde{\eta}) = \inf \left\{ \mathbb{E}|X - \tilde{X}| + \mathbb{P}(I \neq \tilde{I}) : (X, I) \sim \eta \text{ and } (\tilde{X}, \tilde{I}) \sim \tilde{\eta} \right\}.
$$

THEOREM 4.10 (Long time behavior [8]). — Assume that Hypotheses 4.8 and 4.9 hold.

Then, the process has a unique invariant measure and its support is
included in $\bar{B}(0, M) \times E$. Moreover, let $\nu_0$ and $\tilde{\nu}_0$ be two probability measures
on $\bar{B}(0, M) \times E$. Denote by $\nu_t$ the law of $Z_t$ when $Z_0$ is distributed as $\nu_0$
Then there exist positive constants $c$ and $\gamma$ such that

$W_1(\eta_t, \tilde{\eta}_t) \le ce^{-\gamma t}.$

The constants $c$ and $\gamma$ can be explicitly expressed in term of the pa-
rameters of the model (see [8]). The proof relies on the construction of an
explicit coupling. See also [17, 48].

OPEN QUESTION 5. — One can apply Theorem 4.10 to the processes defined in Examples 4.6 and 4.7. The associated time reversal processes are associated to unstable vector fields and unbounded jump rates. What can be said about their convergence to equilibrium?

Section 4.4 presents an application of this theorem to a biological model.
In Section 4.5, we describe a naïve model for the movement of bacteria that
can also be seen as an ergodic telegraph process.
## 4.4. Neuron activity

The paper [55] establishes limit theorems for a class of stochastic hybrid systems (continuous deterministic dynamic coupled with jump Markov processes) in the fluid limit (small jumps at high frequency), thus extending known results for jump Markov processes. The main results are a functional law of large numbers with exponential convergence speed, a diffusion approximation, and a functional central limit theorem. These results are then applied to neuron models with stochastic ion channels, as the number of channels goes to infinity, estimating the convergence to the deterministic model. In terms of neural coding, the central limit theorems allows to estimate numerically the impact of channel noise both on frequency and spike timing coding.

The Morris-Lecar model introduced in [49] describes the evolution in time of the electric potential $V(t)$ in a neuron. The neuron exchanges different ions with its environment via ion channels which may be open or closed. In the original – deterministic – model, the proportion of open channels of different types are coded by two functions $m(t)$ and $n(t)$, and the three quantities $m$, $n$ and $V$ evolve through the flow of an ordinary differential equation.

Various stochastic versions of this model exist. Here we focus on a model described in [63], to which we refer for additional information. This model is motivated by the fact that $m$ and $n$, being proportions of open channels, are better coded as discrete variables. More precisely, we fix a large integer $K$ (the total number of channels) and define a PDMP $(V, u_1, u_2)$ with values in $\mathbb{R} \times \{0, 1/K, 2/K, \dots, 1\}^2$ as follows.

Firstly, the potential $V$ evolves according to

$$ \frac{dV(t)}{dt} = \frac{1}{C} \left( I - \sum_{i=1}^{3} g_i u_i(t) (V - V_i) \right) \quad (4.7) $$

where $C$ and $I$ are positive constants (the capacitance and input current), the $g_i$ and $V_i$ are positive constants (representing conductances and equilibrium potentials for different types of ions), $u_3(t)$ is equal to 1 and $u_1(t)$, $u_2(t)$ are the (discrete) proportions of open channels for two types of ions.

These two discrete variables follow birth-death processes on $\{0, 1/K, \dots, 1\}$ with birth rates $\alpha_1$, $\alpha_2$ and death rates $\beta_1$, $\beta_2$ that depend
on the potential V :

$$
\begin{align}
\alpha_i(V) &= c_i \cosh \left( \frac{V - V_i'}{2V_i''} \right) \left( 1 + \tanh \left( \frac{V - V_i'}{V_i''} \right) \right) \nonumber \\
\beta_i(V) &= c_i \cosh \left( \frac{V - V_i'}{2V_i''} \right) \left( 1 - \tanh \left( \frac{V - V_i'}{V_i''} \right) \right) \tag{4.8}
\end{align}
$$

where the $c_i$ and $V_i'$, $V_i''$ are constants.

Let us check that Theorem 4.10 can be applied in this example. Formally
the process is a PDMP with $d = 1$ and the finite set $E = \{0, 1/K, \dots, 1\}^2$.
The discrete process $(u_1, u_2)$ plays the role of the index $i \in E$, and the fields
$F^{(u_1, u_2)}$ are defined (on $\mathbb{R}$) by (4.7) by setting $u_1(t) = u_1$, $u_2(t) = u_2$.

The constant term $u_3g_3$ in (4.7) ensures that the uniform dissipation
property (4.6) is satisfied: for all $(u_1, u_2)$,

$$
\begin{align*}
\langle V - \tilde{V}, F^{(u_1, u_2)}(V) - F^{(u_1, u_2)}(\tilde{V}) \rangle &= -\frac{1}{C} \sum_{i=1}^{3} u_i g_i (V - \tilde{V})^2 \\
&\leq -\frac{1}{C} u_3 g_3 (V - \tilde{V})^2.
\end{align*}
$$

The Lipschitz character and the bound from below on the rates are not
immediate. Indeed the jump rates (4.8) are not bounded from below if V is
allowed to take values in R.

However, a direct analysis of (4.7) shows that $V$ is essentially bounded : all the fields $F^{(u_1,u_2)}$ point inward at the boundary of the (fixed) line segment $S = [0, \max(V_1, V_2, V_3 + (I+1)/g_3 u_3)]$, so if $V(t)$ starts in this region it cannot get out. The necessary bounds all follow by compactness, since $\alpha_i(V)$ and $\beta_i(V)$ are $C^1$ in $S$ and strictly positive.

4.5. Chemotaxis

Let us briefly describe how bacteria move (see [50, 26, 25] for details). They alternate two basic behavioral modes: a more or less linear motion, called a run, and a highly erratic motion, called tumbling, the purpose of which is to reorient the cell. During a run the bacteria move at approximately constant speed in the most recently chosen direction. Run times are typically much longer than the time spent tumbling. In practice, the tumbling time is neglected. An appropriate stochastic process for describing the motion of cells is called the *velocity jump process* which is deeply studied in [50]. The velocity belongs to a compact set (the unit sphere for example)
and changes by random jumps at random instants of time. Then, the position is deduced by integration of the velocity. The jump rates may depend on the position when the medium is not homogeneous: when bacteria move in a favorable direction i.e. either in the direction of foodstuffs or away from harmful substances the run times are increased further. Sometimes, a diffusive approximation is available [50, 60].

In the one-dimensional simple model studied in [27], the particle evolves in $\mathbb{R}$ and its velocity belongs to $\{-1, +1\}$. Its infinitesimal generator is given by:

$$Af(x, v) = v\partial_x f(x, v) + (a + (b - a)\mathbb{1}_{\{xv>0\}})(f(x, -v) - f(x, v)), \quad (4.9)$$

with $0 < a < b$. The dynamics of the process is simple: when $X$ goes aways from 0, (resp. goes to 0), $V$ flips to $-V$ with rate $b$ (resp. $a$). Since $b > a$, it is quite intuitive that this Markov process is ergodic. One could think about it as an analogue of the diffusion process solution of

$$dZ_t = dB_t - \operatorname{sign}(Z_t) dt.$$

More precisely, under a suitable scaling, one can show that $X$ goes to $Z$. Finally, this process is an ergodic version of the so-called telegraph process. See for example [35, 32].

Of course, this process does not satisfy the hypotheses of Theorem 4.10 since the vector fields have no stable point. It is shown in [27] that the invariant measure $\mu$ of $(X, V)$ driven by (4.9) is the product measure on $\mathbb{R}_+ \times \{-1, +1\}$ given by

$$\mu(dx, dv) = (b-a)e^{-(b-a)x} dx \otimes \frac{1}{2}(\delta_{-1} + \delta_{+1})(dv).$$

One can also construct an explicit coupling to get explicit bounds for the convergence to the invariant measure in total variation norm [27]. See also [47] for another approach, linked with functional inequalities.

**OPEN QUESTION 6** (More realistic models). — Is it possible to establish quantitative estimates for the convergence to equilibrium for more realistic dynamics (especially in $\mathbb{R}^3$) as considered in [50, 26, 25]?

**Acknowledgements.** — FM deeply thanks Persi Diaconis for his energy, curiosity and enthusiasm and Laurent Miclo for the perfect organisation of the stimulating workshop "Talking Across Fields" in Toulouse during March 2014. This paper has been improved thanks to the constructive com-
ments of two referees. FM acknowledges financial support from the French ANR project ANR-12-JS01-0006 - PIECE.

Bibliography

[1] ALDOUS (D.) and DIACONIS (P.). — Strong uniform times and finite random walks, Adv. in Appl. Math. 8, no. 1, p. 69-97 (1987).

[2] BAKHTIN (Y.) and HURTH (T.). — Invariant densities for dynamical systems with random switching, Nonlinearity 25 no. 10, p. 2937-2952 (2012).

[3] BAKHTIN (Y.), HURTH (T.), and MATTINGLY (J. C.). — Regularity of invariant densities for 1D-systems with random switching, arXiv:1406.5425, (2014).

[4] BALDE (M.) and BOSCAIN (U.). —, Stability of planar switched systems: the non-diagonalizable case, Commun. Pure Appl. Anal. 7, no. 1, p. 1-21 (2008).

[5] BALDE (M.), BOSCAIN (U.), and MASON (P.). — A note on stability conditions for planar switched systems, Internat. J. Control 82, no. 10, p. 1882-1888 (2009).

[6] BARDET (J.-B.), CHRISTEN (A.), GUILLIN (A.), MALRIEU (A.), and ZITT (P.-A.). Total variation estimates for the TCP process, Electron. J. Probab. 18, no. 10, p. 1-21 (2013).

[7] BENAÎM (M.), LE BORGNE (S.), MALRIEU (F.), and ZITT (P.-A.). — Qualitative properties of certain piecewise deterministic Markov processes, Ann. Inst. Henri Poincar Probab. Stat. 51, no. 3, p. 1040-1075 (2015).

[8] BENAÎM (M.), LE BORGNE (S.), MALRIEU (F.), and ZITT (P.-A.). — Quantitative ergodicity for some switched dynamical systems, Electron. Commun. Probab. 17, no. 56, p. 14 (2012).

[9] BENAÎM (M.), LE BORGNE (S.), MALRIEU (F.), and ZITT (P.-A.). — On the stability of planar randomly switched systems, Ann. Appl. Probab. 24, no. 1, p. 292-311 (2014).

[10] BERESTYCKI (J.), BERTOIN (J.), HAAS (B.), and MIERMONT (G.). — Quelques aspects fractals des fragmentations aléatoires, Quelques interactions entre analyse, probabilités et fractals, Panor. Synthèses, vol. 32, Soc. Math. France, Paris, p. 191-243 (2010).

[11] BERTAIL (P.), S. CLÉMENÇON (S.), and TRESSOU (J.). — A storage model with random release rate for modeling exposure to food contaminants, Math. Biosci. Eng. 5, no. 1, p. 35-60 (2008).

[12] BORKOVEC (M.), DASGUPTA (A.), RESNICK (S.), and SAMORODNITSKY (G.). — A single channel on/off model with TCP-like control, Stoch. Models 18, no. 3, p. 333-367 (2002).

[13] BOSCAIN (U.), — Stability of planar switched systems: the linear single input case, SIAM J. Control Optim. 41 no. 1, p. 89-112 (2002).

[14] BOUGUET (F.). — Quantitative exponential rates of convergence for exposure to food contaminants, To appear in ESAIM PS, arXiv:1310.3948, (2013).

[15] CALVEZ (V.), DOUMIC JAUFFRET (M.), and GABRIEL (P.). — Self-similarity in a general aggregation-fragmentation problem. Application to fitness analysis, J. Math. Pures Appl. (9) 98, no. 1, p. 1-27 (2012).

[16] CHAFAÏ (D.), MALRIEU (F.), and PAROUX (K.). — On the long time behavior of the TCP window size process, Stochastic Process. Appl. 120, no. 8, p. 1518-1534 (2010).
[17] CLOEZ (B.) and HAIRER (M.). — Exponential ergodicity for Markov processes with random switching, Bernoulli 21, no. 1, p. 505-536 (2015).

[18] COMETS (F.), POPOV (S.), SCHÜTZ (M.), and VACHKOVSKAIA (M.). — Billiards in a general domain with random reflections, Arch. Ration. Mech. Anal. 191, no. 3, p. 497-537 (2009).

[19] CRUDU (A.), DEBUSSCHE (A.), MULLER (A.), and RADULESCU (O.). Convergence of stochastic gene networks to hybrid piecewise deterministic processes, Ann. Appl. Probab. 22, no. 5, p. 1822-1859 (2012).

[20] DAVIS (M. H. A.). — Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models, J. Roy. Statist. Soc. Ser. B 46, no. 3, p. 353-388, With discussion (1984).

[21] DAVIS (M. H. A.). — Markov models and optimization, Monographs on Statistics and Applied Probability, vol. 49, Chapman & Hall, London, (1993).

[22] DIACONIS (P.) and FREEDMAN (P.). — Iterated random functions, SIAM Rev. 41, no. 1, p. 45-76 (1999).

[23] DOUMIC JAUFFRET (M.) and GABRIEL (P.). — Eigenelements of a general aggregation-fragmentation model, Math. Models Methods Appl. Sci. 20, no. 5, p. 757-783 (2010).

[24] DUMAS (V.), GUILLEMIN (F.), and ROBERT (Ph.). — A Markovian analysis of additive-increase multiplicative-decrease algorithms, Adv. in Appl. Probab. 34, no. 1, p. 85-111 (2002).

[25] ERBAN (R.) and OTHMER (H. G.). — From individual to collective behavior in bacterial chemotaxis, SIAM J. Appl. Math. 65, no. 2, p. 361-391 (electronic) (2004/05).

[26] ERBAN (R.) and OTHMER (H. G.). — From signal transduction to spatial pattern formation in E. coli: a paradigm for multiscale modeling in biology, Multiscale Model. Simul. 3, no. 2, p. 362-394 (electronic) (2005).

[27] FONTBONA (J.), GUÉRIN (H.), and MALRIEU (F.). — Quantitative estimates for the long-time behavior of an ergodic variant of the telegraph process, Adv. in Appl. Probab. 44, no. 4, p. 977-994 (2012).

[28] GADAT (S.) and MICLO (L.). — Spectral decompositions and $L^2$-operator norms of toy hypocoercive semi-groups, Kinet. Relat. Models 6, no. 2, p. 317-372 (2013).

[29] GRAHAM (C.) and ROBERT (Ph.). — Interacting multi-class transmissions in large stochastic networks, Ann. Appl. Probab. 19, no. 6, p. 2334-236 (2009).

[30] GRAHAM (C.) and ROBERT (Ph.). — Self-adaptive congestion control for multiclass intermittent connections in a communication network, Queueing Syst. 69, no. 3-4, p. 237-257 (2011).

[31] GUILLEMIN (F.), ROBERT (Ph.), and ZWART (B.). — AIMD algorithms and exponential functionals, Ann. Appl. Probab. 14, no. 1, p. 90-117 (2004).

[32] HERRMANN (S.) and VALLOIS (P.). — From persistent random walk to the telegraph noise, Stoch. Dyn. 10, no. 2, p. 161-196 (2010).

[33] HESPANHA (J. P.). — A model for stochastic hybrid systems with application to communication networks, Nonlinear Anal. 62, no. 8, p. 1353-1383 (2005).

[34] JACOBSEN (M.). — Point process theory and applications, Probability and its Applications, Birkhuser Boston Inc., Boston, MA, Marked point and piecewise deterministic processes (2006).

[35] KAC (M.). — A stochastic model related to the telegraphers equation, Rocky Mountain J. Math. 4, p. 497-509 (1974).

[36] KARMAKAR (R.) and BOSE (I.). — Graded and binary responses in stochastic gene expression, Physical Biology 197 no. 1, p. 197-214 (2004).
[37] LAMBERTON (D.) and PAGÈS (G.). — A penalized bandit algorithm, Electron. J. Probab. 13, no. 13, p. 341-373 (2008).

[38] LAWLEY (S. D.), MATTINGLY (J. C.), and REED (M. C.). — Sensitivity to switching rates in stochastically switched ODEs, Commun. Math. Sci. 12, no. 7, p. 1343-1352 (2014).

[39] VAN LEEUWAARDEN (J. S. H.) and LÖPKER (A. H.). — Transient moments of the TCP window size process, J. Appl. Probab. 45, no. 1, p. 163-175 (2008).

[40] VAN LEEUWAARDEN (J. S. H.), LÖPKER (A. H.), and OTT (T. J.). — TCP and iso-stationary transformations, Queueing Syst. 63, no. 1-4, p. 459-475 (2009).

[41] LEVIN (D. A.), PERES (Y.), and WILMER (E. L.). — Markov chains and mixing times, American Mathematical Society, Providence, RI, With a chapter by James G. Propp and David B. Wilson (2009).

[42] MACKEY (M. C.), TYRAN-KAMINSKA (M.), and YVINEC (R.). — Dynamic behavior of stochastic gene expression models in the presence of bursting, SIAM J. Appl. Math. 73, no. 5, p. 1830-1852 (2013).

[43] MAULIK (K.) and ZWART (B.). — Tail asymptotics for exponential functionals of Lévy processes, Stochastic Process. Appl. 116, no. 2, p. 156-177 (2006).

[44] MAULIK (K.) and ZWART (B.). — An extension of the square root law of TCP, Ann. Oper. Res. 170, p. 217-232 (2009).

[45] MICLO (L.) and MONMARCHÉ (P.). —, tude spectrale minutieuse de processus moins indécis que les autres, Séminaire de Probabilités XLV, Lecture Notes in Math., vol. 2078, Springer, Cham, p. 459-481 (2013).

[46] MISCHLER (S.) and SCHER (J.). — Spectral analysis of semigroups and growth-fragmentation equations, arXiv:1310.7773, (2013).

[47] MONMARCHÉ (P.). — Hypocoercive relaxation to equilibrium for some kinetic models, Kinet. Relat. Models 7, no. 2, p. 341-360 (2014).

[48] MONMARCHÉ (P.). — On $H^1$ and entropic convergence for contractive PDMP, arXiv:1404.4220 (2014).

[49] MORRIS (C.) and LECAR (H.). — Voltage oscillations in the barnacle giant muscle fiber, Biophys. J. 35, no. 1, p. 193-213 (1981).

[50] OTHMER (H. G.), DUNBAR (S. R.), and ALT (W.). — Models of dispersal in biological systems, J. Math. Biol. 26, no. 3, p. 263-298 (1988).

[51] OTT (T. J.). — Rate of convergence for the square root formula in the internet transmission control protocol, Adv. in Appl. Probab. 38, no. 4, p. 1132-1154 (2006).

[52] OTT (T. J.) and KEMPERMAN (J. H. B.). — Transient behavior of processes in the TCP paradigm, Probab. Engrg. Inform. Sci. 22, no. 3, p. 431-471 (2008).

[53] OTT (T. J.), KEMPERMAN (J. H. B.), and MATHIS (M.). — The stationary behavior of ideal TCP congestion avoidance, unpublished manuscript available at http://www.teunisott.com/ (1996).

[54] OTT (T. J.) and SWANSON (J.). — Asymptotic behavior of a generalized TCP congestion avoidance algorithm, J. Appl. Probab. 44, no. 3, p. 618-635 (2007).

[55] PAKDAMAN (K.), THIEULLEN (M.), and WAINRIB (G.). — Fluid limit theorems for stochastic hybrid systems with application to neuron models, Adv. in Appl. Probab. 42, no. 3, p. 761-794 (2010).

[56] PERTHAME (B.). — Transport equations in biology, Frontiers in Mathematics, Birkhäuser Verlag, Basel (2007).

[57] PERTHAME (B.) and RYZHIK (L.). — Exponential decay for the fragmentation or cell-division equation, J. Differential Equations 210, no. 1, p. 155-177 (2005).
[58] RADULESCU (O.), MULLER (A.), and CRUDU (A.). — Théorèmes limites pour des processus de Markov à sauts. Synthèse des résultats et applications en biologie moléculaire, Technique et Science Informatiques 26, no. 3-4, p. 443-469 (2007).

[59] ROBERTS (G. O.) and TWEEDIE (R. L.). — Rates of convergence of stochastically monotone and continuous time Markov models, J. Appl. Probab. 37, no. 2, p. 359-373 (2000).

[60] ROUSSET (M.) and SAMAЕY (G.). — Individual-based models for bacterial chemotaxis in the diffusion asymptotics, Math. Models Methods Appl. Sci. 23, no. 11, p. 2005-2037 (2013).

[61] SHAO (J.). — Ergodicity of regime-switching diffusions in Wasserstein distances, Stochastic Process. Appl. 125, no. 2, p. 739-758 (2015).

[62] VILLANI (C.). — Topics in optimal transportation, Graduate Studies in Mathematics, vol. 58, American Mathematical Society, Providence, RI (2003).

[63] WAINRIB (G.), THIEULLEN (M.), and PAKDAMAN (K.). — Intrinsic variability of latency to first-spike, Biol. Cybernet. 103, no. 1, p. 43-56 (2010).

[64] YIN (G. G.) and ZHU (C.). — Hybrid switching diffusions, Stochastic Modelling and Applied Probability, vol. 63, Springer, New York, Properties and applications (2010).

[65] YVINEC (R.), ZHUGE (C.), LEI (J.), and MACKEY (M. C.). — Adiabatic reduction of a model of stochastic gene expression with jump Markov process, J. Math. Biol. 68, no. 5, p. 1051-1070 (2014).