<figure><img src="image_1.png" /></figure>

Rubrique

Chemotherapy models

Fayssal Charif(1), Mohamed Helal(1) and Abdelkader Lakmeche(1,2)

¹Biomathematics Laboratory
Univ Sidi-Bel-Abbès
Sidi-Bel-Abbès
Algeria
mat.faycal@hotmail.fr, mhelal_abbes@yahoo.fr and lakmeche@yahoo.fr

²INRIA Lyon La Doua
Villeurbanne
France
lakmeche@yahoo.fr

**RÉSUMÉ.** Nous considérons un modèle de chimiothérapie pour une population de cellules avec résistance. Nous considérons le cas de deux médicaments le premier avec effet impulsif et le deuxième avec effet continu. Nous étudions la stabilité des solutions p'ériodiques triviales et l'apparition des solutions périodiques non-triviales en utilisant la bifurcation de Lyapunov-Schmidt.

**ABSTRACT.** A chemotherapeutic treatment model for cell population with resistant tumor is considered. We consider the case of two drugs one with pulsed effect and the other one with continuous effect. We investigate stability of the trivial periodic solutions and the onset of nontrivial periodic solutions by the mean of Lyapunov-Schmidt bifurcation.

**MOTS-CLÉS** : Modèle de chimiothérapie, Equations différentielles impulsifs, Solutions périodiques, Stabilité exponentielle, Bifurcation de Lyapunov-Schmidt.

**KEYWORDS** : Chemotherapy model, Impulsive differential equations, Periodic solutions, Exponential stability, Lyapunov-Schmidt bifurcation.
# 1. Introduction

Impulsive differential systems are found in the description of phenomena issued from applied sciences as physics, chemistry, biology and medicine. Recently interesting works in biomathematics have been published, we can cite those considering impulsive chemotherapeutic treatment of tumor diseases ([1], [3]-[5], [8], [9]), impulsive pest control strategies ([6]), impulsive harvesting ([7], [12]), and impulsive vaccination ([10], [11]). In this paper, we consider a mathematical model for cell population under chemotherapy treatment by two drugs, one with instantaneous effect and the other with continuous effect. We study the stability of the tumor eradication and the consequences of the loss of its stability. In the last case we could have a bifurcation of positive nontrivial solutions which correspond to the persistence of the tumor.

More specifically, we consider a cell population constituted by three subpopulations: normal cells, sensitive tumor cells and resistant tumor cells, with mutation and resistance to the drug with instantaneous effect. The model is described by the following impulsive differential equations:

$$ \frac{dx_1}{dt} = r_1 x_1 \left( 1 - \frac{x_1}{k_1} - \lambda_1 (x_2 + x_3) \right) - \theta_1 x_1, \quad [1] $$

$$ \frac{dx_2}{dt} = r_2 x_2 \left( 1 - \frac{x_2 + x_3}{k_2} - \lambda_2 (x_1 + x_3) \right) - m x_2 - \theta_2 x_2, \quad [2] $$

$$ \frac{dx_3}{dt} = r_3 x_3 \left( 1 - \frac{x_2 + x_3}{k_3} - \lambda_3 (x_1 + x_2) \right) + m x_2 - \theta_3 x_3, \quad [3] $$

$$ x_1(t_i^+) = T_1 x_1(t_i), \quad [4] $$

$$ x_2(t_i^+) = (T_2 - R) x_2(t_i), \quad [5] $$

$$ x_3(t_i^+) = T_3 x_3(t_i) + R x_2(t_i), \quad [6] $$

where $t_{i+1} - t_i = \tau > 0, \forall i \in \mathbb{N}, \theta_1 < r_1, \theta_2 < r_2 + m$ and $\theta_3 < r_3$.

The variables and parameters are:

$\tau$: period between two successive drug with instantaneous effect,

$x_j$: normal (resp. sensitive tumor, resistant tumor) cells biomass for $j = 1$ (resp. 2,3),

$r_j$: growth rates of the normal (resp. sensitive tumor, resistant tumor) cells for $j = 1$ (resp. 2,3),

$k_j$: carrying capacities of the normal (resp. sensitive tumor, resistant tumor) cells for $j = 1$ (resp. 2,3),

$\lambda_j$: competitive parameters of the normal (resp. sensitive tumor, resistant tumor) cells for $j = 1$ (resp. 2,3),

$T_j$: survival fractions of the normal (resp. sensitive tumor, resistant tumor) cells, their values are completely determined by the quantity of injected drug with instantaneous effect.

$\theta_j$: elimination rate of normal (resp. sensitive tumor, resistant tumor) cells by the drug with continuous effect.

$R$: Fraction of cells mutating due to the dose of the drug with instantaneous effect, which is less than $T_2$.

$m$: acquired resistance parameter usually it is very small.

Note that if $\theta_i = 0$, then we obtain the models studied in [1] and [5].
## 2. Analysis of the model

In the following, we proceed to analyze our model. To this purpose, we shall use a fixed point approach.

Let $\Phi(t, X_0)$ be the solution of the system (1)-(6) for the initial condition $X_0$.
We define the mappings $F_1, F_2, F_3: \mathbf{R}^3 \to \mathbf{R}$ by

$$F_1(x_1, x_2, x_3) = r_1 x_1 \left( 1 - \frac{x_1}{k_1} - \lambda_1(x_2 + x_3) \right) - \theta_1 x_1,$$

$$F_2(x_1, x_2, x_3) = r_2 x_2 \left( 1 - \frac{x_2+x_3}{k_2} - \lambda_2(x_1 + x_3) \right) - m x_2 - \theta_2 x_2,$$

$$F_3(x_1, x_2, x_3) = r_3 x_3 \left( 1 - \frac{x_2+x_3}{k_3} - \lambda_3(x_1 + x_2) \right) + m x_2 - \theta_3 x_3,$$

and $\Theta_1, \Theta_2, \Theta_3: \mathbf{R}^3 \to \mathbf{R}$ by

$$\Theta_1(x_1, x_2, x_3) = T_1 x_1,$$

$$\Theta_2(x_1, x_2, x_3) = (T_2 - R)x_2,$$

$$\Theta_3(x_1, x_2, x_3) = T_3 x_3 + Rx_2.$$

Let $\Theta := (\Theta_1, \Theta_2, \Theta_3)$ and $\Psi: \mathbf{R}_+ \times \mathbf{R}^3 \to \mathbf{R}^3$ be the operator defined by

$$\Psi(\tau, X_0) = \Theta(\Phi(\tau, X_0)), \quad [7]$$

and denote by $D_X \Psi$ the derivative of $\Psi$ with respect to $X$. Then $\xi = \Phi(., X_0)$ is a $\tau$-periodic solution of (1)-(6) if and only if

$$\Psi(\tau, X_0) = X_0, \quad [8]$$

i.e. $X_0$ is a fixed point of $\Psi(\tau, .)$, and it is exponentially stable if and only if the spectral radius $\rho(D_X \Psi(\tau, .))$ is strictly less than 1 ([2]). A fixed point $X_0$ of $\Psi(\tau, .)$ is the initial state of (1)-(6) which gives a $\tau$-periodic solution $\xi$ verifying $\xi(0) = X_0$.

We reduce the problem of finding a periodic solution of (1)-(6) to a fixed point problem. Here, $\xi$ is a periodic solution of period $\tau$ for (1)-(6) if and only if $X_0$ is a fixed point for (8). Consequently, to establish the existence of nontrivial periodic solutions of (1)-(6), one needs to prove the existence of nontrivial fixed points of $\Psi(\tau, .)$.

### REMARQUE. —

The problem (1),(4), obtained by taking $x_2 = 0$ and $x_3 = 0$, has a $\tau_0$-periodic solution $x(t, x_0) = x_s(t)$, where

$$x_s(t) = k_1 \frac{r_1 - \theta_1}{r_1} \frac{(T_1 - \exp(-(r_1 - \theta_1)\tau_0)) \exp((r_1 - \theta_1)t)}{\exp((r_1 - \theta_1)t)(T_1 - \exp(-(r_1 - \theta_1)\tau_0)) + (1 - T_1)}, \quad 0 < t \le \tau_0, \quad [9]$$

$$\text{with } x_0 = k_1 \frac{r_1 - \theta_1}{r_1} \frac{(T_1 - \exp(-(r_1 - \theta_1)\tau_0))}{1 - \exp(-(r_1 - \theta_1)\tau_0)}.$$

We denote by $\zeta = (x_s, 0, 0)$ which is a solution of (1)-(6), we call it trivial solution.

### 2.1. Stability of $\zeta$

In the case without tumor we have $x_2 = x_3 = 0$, then (1)-(6) is reduced to (1),(4) which has a unique non-trivial positive periodic solution $x_s$ given by (9). It is defined and stable for $T_1 > \exp(-(r_1 - \theta_1)\tau_0)$. That is

$$\tau_0 > \frac{1}{r_1 - \theta_1} \ln \left( \frac{1}{T_1} \right). \quad [10]$$
To determine the stability of the trivial solution $\zeta = (x_s, 0, 0)$ in the three dimensional space, we must calculate $D_X \Psi(\tau_0, X_0)$ where $X_0 = (x_0, 0, 0)$. We have

$$D_X \Psi(\tau_0, X_0) = D_X \Theta(\Phi(\tau_0, X_0)) \frac{\partial \Phi}{\partial X}(\tau_0, X_0) = \begin{pmatrix} T_1 \frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_1} & T_1 \frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_2} & T_1 \frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_3} \\ 0 & (T_2 - R) \frac{\partial \Phi_2(\tau_0, X_0)}{\partial x_2} & 0 \\ 0 & R \frac{\partial \Phi_2(\tau_0, X_0)}{\partial x_2} + T_3 \frac{\partial \Phi_3(\tau_0, X_0)}{\partial x_2} & T_3 \frac{\partial \Phi_3(\tau_0, X_0)}{\partial x_3} \end{pmatrix}$$

The solution $\zeta$ is exponentially stable if and only if the spectral radius is less than one, that is

$$|T_j \frac{\partial \Phi_j}{\partial x_j}(\tau_0, X_0)| < 1, \text{ for } j = 1, 2, 3.$$

Using the variational equation associated to the system (1)-(6)

$$\frac{d}{dt}(D_X \Phi(t, X_0)) = D_X F(\Phi(t, X_0))(D_X \Phi(t, X_0)), \quad [11]$$

with the initial condition $D_X \Phi(0, X_0) = Id_{\mathbb{R}^3}$ we obtain

$$\begin{align*}
\frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_1} &= T_1^{-2} e^{-(r_1 - \theta_1)\tau_0}, \\
\frac{\partial \Phi_2(\tau_0, X_0)}{\partial x_2} &= T_1 \frac{-r_3\lambda_2 K_1}{r_1} e^{(r_2 - \theta_2 - m - (r_1 - \theta_1)\frac{r_2\lambda_2 K_1}{r_1})\tau_0}, \\
\frac{\partial \Phi_3(\tau_0, X_0)}{\partial x_3} &= T_1 \frac{-r_3\lambda_3 K_1}{r_1} e^{(r_3 - \theta_3 - (r_1 - \theta_1)\frac{r_3\lambda_3 K_1}{r_1})\tau_0}, \\
\frac{\partial \Phi_3(\tau_0, X_0)}{\partial x_2} &= \frac{me^{(r_3 - \theta_3 - (r_1 - \theta_1)\frac{r_3\lambda_3 K_1}{r_1})\tau_0} (1 - e^{-(r_1 - \theta_1)\tau_0}) \frac{r_2\lambda_2 K_1}{r_1}}{T_1 \frac{r_3\lambda_3 K_1}{r_1} (1 - e^{-(r_1 - \theta_1)\tau_0}) \frac{r_3\lambda_3 K_1}{r_1}} \int_0^{\tau_0} e^{(r_2 - \theta_2 - (r_3 - \theta_3) - m)u} I^{\left(\frac{r_3\lambda_3 - r_2\lambda_2}{r_1}\right) \frac{K_1}{r_1}}(u)du, \\
\frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_3} &= -\frac{(r_1 - \theta_1)\lambda_1 K_1 (T_1 - e^{-(r_1 - \theta_1)\tau_0}) e^{-(r_1 - \theta_1)\tau_0}}{T_1^2 (1 - e^{-(r_1 - \theta_1)\tau_0})^2 - \frac{r_3\lambda_3 K_1}{r_1}} \int_0^{\tau_0} e^{(r_3 - \theta_3)u} I^{\left(1 - \frac{r_3\lambda_3 K_1}{r_1}\right) \frac{K_1}{r_1}}(u)du, \\
\frac{\partial \Phi_1(\tau_0, X_0)}{\partial x_2} &= -\frac{(r_1 - \theta_1)\lambda_1 K_1 (T_1 - e^{-(r_1 - \theta_1)\tau_0}) e^{-(r_1 - \theta_1)\tau_0}}{T_1^2 (1 - e^{-(r_1 - \theta_1)\tau_0})^2 - \frac{r_2\lambda_2 K_1}{r_1}} \left\{ \int_0^{\tau_0} e^{(r_2 - \theta_2 - m)u} I^{\left(1 - \frac{r_2\lambda_2 K_1}{r_1}\right) \frac{K_1}{r_1}}(u)du \right. \\
&\qquad \left. + m \int_0^{\tau_0} e^{(r_3 - \theta_3)u} I^{\left(1 - \frac{r_3\lambda_3 K_1}{r_1}\right) \frac{K_1}{r_1}}(u) \left( \int_0^u e^{(r_2 - \theta_2 - (r_3 - 3) - m)p} I^{\left(\frac{r_3\lambda_3 - r_2\lambda_2}{r_1}\right) \frac{K_1}{r_1}}(p)dp \right)du \right\}
\end{align*}$$

for $0 < t \le \tau_0$ where $I(t) = (T_1 - e^{-(r_1 - \theta_1)\tau_0})e^{(r_1 - \theta_1)t} + (1 - T_1)$ (Fore more details, see [5]).

In view of the fact that $\lambda_2 K_1 < 1$ and $\lambda_3 K_1 < 1$ (see [8]), we have

$$T_2 < T_1 \frac{r_2\lambda_2 K_1}{r_1} + R \quad [12]$$

and

$$T_3 < T_1 \frac{r_3\lambda_3 K_1}{r_1}. \quad [13]$$

$\zeta$ is exponentially stable as an equilibrium for the full system (1)-(6) if and only if

$$\frac{\ln(\frac{1}{T_1})}{r_1 - \theta_1} < \tau_0 < \min \left( \frac{\ln\left(T_1 \frac{r_2\lambda_2 K_1}{r_1} (T_2 - R)^{-1}\right)}{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2\lambda_2 K_1}{r_1}}, \frac{\ln\left(T_1 \frac{r_3\lambda_3 K_1}{r_1} T_3^{-1}\right)}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3\lambda_3 K_1}{r_1}} \right). \quad [14]$$

### REMARQUE. —

The trivial periodic solution $(x_s, 0, 0)$ represents the healthy equilibrium. So, our aim is to obtain its stability, this corresponds to the eradication of the tumor. We have the following result.
**Theorem 2.1** If (12)-(14) are satisfied, then $\zeta$ is exponentially stable.

If conditions (12), (13) are satisfied and

$$ T_2 > T_1 \frac{\frac{r_2 \lambda_2 K_1}{r_1} - \frac{r_3 \lambda_3 K_1}{r_1} \frac{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}}}{T_3 \frac{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}}} + R, \quad [15] $$

we have min $$ \left( \frac{\ln\left( T_1^{\frac{r_2 \lambda_2 K_1}{r_1}} (T_2 - R)^{-1} \right)}{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}, \frac{\ln\left( T_1^{\frac{r_2 \lambda_3 K_1}{r_1}} T_3^{-1} \right)}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}} \right) = \frac{\ln\left( T_1^{\frac{r_2 \lambda_2 K_1}{r_1}} (T_2 - R)^{-1} \right)}{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}} $$

That is, the tumor eradication solution $\zeta$ is stable for

$$ \frac{\ln\left(\frac{1}{T_1}\right)}{r_1 - \theta_1} < \tau_0 < \frac{\ln\left( T_1^{\frac{r_2 \lambda_2 K_1}{r_1}} (T_2 - R)^{-1} \right)}{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}. \quad [16] $$

If conditions (12), (13) are satisfied and

$$ T_2 < T_1 \frac{\frac{r_2 \lambda_2 K_1}{r_1} - \frac{r_3 \lambda_3 K_1}{r_1} \frac{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}}}{T_3 \frac{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}}} + R, \quad [17] $$

we have min $$ \left( \frac{\ln\left( T_1^{\frac{r_2 \lambda_2 K_1}{r_1}} (T_2 - R)^{-1} \right)}{r_2 - \theta_2 - m - (r_1 - \theta_1) \frac{r_2 \lambda_2 K_1}{r_1}}, \frac{\ln\left( T_1^{\frac{r_3 \lambda_3 K_1}{r_1}} T_3^{-1} \right)}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}} \right) = \frac{\ln\left( T_1^{\frac{r_3 \lambda_3 K_1}{r_1}} T_3^{-1} \right)}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}} $$

That is, we have stability of $\zeta$ for

$$ \frac{\ln\left(\frac{1}{T_1}\right)}{r_1 - \theta_1} < \tau_0 < \frac{\ln\left( T_1^{\frac{r_3 \lambda_3 K_1}{r_1}} T_3^{-1} \right)}{r_3 - \theta_3 - (r_1 - \theta_1) \frac{r_3 \lambda_3 K_1}{r_1}}. \quad [18] $$

### REMARQUE.

From Theorem 2.1, we deduce that we have stability of the trivial equilibrium if the treatment amplitudes and period between two successive administrations of the treatment satisfy (12)-(14), in this case we have eradication of the tumor.

## 2.2. Bifurcation analysis

We use Lyapunov Schmidt bifurcation to find nontrivial periodic solution of the model (1)-(6). A necessary condition for the bifurcation of nontrivial periodic solutions near $\zeta$ is

$$ \det(D_X M(0, (0, 0, 0))) = 0. $$

To find a nontrivial periodic solution of period $\tau$ with initial data $X_0 = (x_0, 0, 0)$, we need to solve the fixed point problem

$$ X = \Psi(\tau, X). \quad [19] $$

Let $\bar{\tau}$ and $\bar{X}$ such that $\tau = \tau_0 + \bar{\tau}$ and $X = X_0 + \bar{X}$. The equation (19) is equivalent to

$$ M(\bar{\tau}, \bar{X}) = 0, \quad [20] $$
where

$$M(\bar{\tau}, \bar{X}) = (M_1(\bar{\tau}, \bar{X}), M_2(\bar{\tau}, \bar{X}), M_3(\bar{\tau}, \bar{X})) = X_0 + \bar{X} - \Psi(\tau_0 + \bar{\tau}, X_0 + \bar{X}).$$

If $(\bar{\tau}, \bar{X})$ is a zero of $M$, then $(X_0 + \bar{X})$ is a fixed point of $\Psi(\tau_0 + \bar{\tau}, \cdot)$. Since $\zeta$ is a trivial $\tau_0$-periodic solution (1)-(6), then it is associated to the trivial fixed point $X_0$ of $\Psi(\tau_0, \cdot)$. Let

$$D_X M(0, (0, 0, 0)) = \begin{pmatrix} a'_0 & b'_0 & c'_0 \\ 0 & e'_0 & 0 \\ 0 & h'_0 & i'_0 \end{pmatrix}. \qquad [21]$$

It follows that (see [5] for details):

$$\begin{align*} a'_0 &= T_1^{-1}(T_1 - e^{-(r_1-\theta_1)\tau_0}), \\ b'_0 &= \frac{(r_1-\theta_1)\lambda_1 K_1(T_1 - e^{-(r_1-\theta_1)\tau_0})e^{-(r_1-\theta_1)\tau_0}}{T_1(1-e^{-(r_1-\theta_1)\tau_0})^{2-\frac{r_2\lambda_2K_1}{r_1}}} \left\{ \int_0^{\tau_0} e^{(r_2-\theta_2-m)u} I^{1-\frac{r_2\lambda_2K_1}{r_1}}(u)du \right. \\ &\qquad \left. + m \int_0^{\tau_0} e^{(r_3-\theta_3)u} I^{1-\frac{r_3\lambda_3K_1}{r_1}}(u)(\int_0^u e^{(r_2-\theta_2-(r_3-3)m)p} I^{(r_3\lambda_3-r_2\lambda_2)\frac{K_1}{r_1}}(p)dp)du \right\}, \\ c'_0 &= \frac{(r_1-\theta_1)\lambda_1 K_1(T_1 - e^{-(r_1-\theta_1)\tau_0})e^{-(r_1-\theta_1)\tau_0}}{T_1(1-e^{-(r_1-\theta_1)\tau_0})^{2-\frac{r_3\lambda_3K_1}{r_1}}} \int_0^{\tau_0} e^{(r_3-\theta_3)u} I^{1-\frac{r_3\lambda_3K_1}{r_1}}(u)du, \end{align*}$$

$$\begin{align*} e'_0 &= 1 - (T_2 - R)T_1 \frac{\frac{r_2\lambda_2K_1}{r_1}}{r_1} e^{(r_2-\theta_2-m-(r_1-\theta_1)\frac{r_2\lambda_2K_1}{r_1})\tau_0}, \\ h'_0 &= -RT_1 \frac{\frac{r_2\lambda_2K_1}{r_1}}{r_1} e^{(r_2-\theta_2-m-(r_1-\theta_1)\frac{r_2\lambda_2K_1}{r_1})\tau_0} \\ &\quad -T_3 \frac{me^{(r_3-\theta_3-(r_1-\theta_1)\frac{r_3\lambda_3K_1}{r_1})\tau_0}(1-e^{-(r_1-\theta_1)\tau_0})\frac{r_2\lambda_2K_1}{r_1}}{T_1\frac{r_3\lambda_3K_1}{r_1}(1-e^{-(r_1-\theta_1)\tau_0})\frac{r_3\lambda_3K_1}{r_1}} \int_0^{\tau_0} e^{(r_2-\theta_2-(r_3-\theta_3)-m)u} I^{(r_3\lambda_3-r_2\lambda_2)\frac{K_1}{r_1}}(u)du \end{align*}$$

and

$$i'_0 = 1 - T_3 T_1 \frac{\frac{r_3\lambda_3K_1}{r_1}}{r_1} e^{(r_3-\theta_3-(r_1-\theta_1)\frac{r_3\lambda_3K_1}{r_1})\tau_0}.$$

From the stability of the solution $x_s$ in the one dimensional space we have $a'_0 > 0$, it follows that $e'_0 i'_0 = 0$ is necessary for the bifurcation. Equality

$$\tau_0 = \frac{\ln\left(T_1^{\frac{r_2\lambda_2K_1}{r_1}} (T_2 - R)^{-1}\right)}{r_2 - \theta_2 - m - (r_1 - \theta_1)\frac{r_2\lambda_2K_1}{r_1}}, \qquad [22]$$

corresponds to $e'_0 = 0$ and equality

$$\tau_0 = \frac{\ln\left(T_1^{\frac{r_3\lambda_3K_1}{r_1}} T_3^{-1}\right)}{r_3 - \theta_3 - (r_1 - \theta_1)\frac{r_3\lambda_3K_1}{r_1}}, \qquad [23]$$

corresponds to $i'_0 = 0$. We have three cases :

1) Case 1 : $e'_0 = 0$ and $i'_0 \neq 0$. Suppose that (10)-(13), (15) and (22) are satisfied. With the above notations, we deduce that $M(0, (0, 0, 0)) = 0$, $\dim(\ker[D_X M(0, (0, 0, 0))]) = 1$ with $\ker[D_X M(0, (0, 0, 0))] = \text{span}\{(\frac{c'_0 h'_0}{a'_0 i'_0} - \frac{b'_0}{a'_0}, 1, -\frac{h'_0}{i'_0})\}$. Then equation (20) is equivalent to

$$\left\{ \begin{array}{l} M_1(\bar{\tau}, \alpha Y_0 + Z) = 0, \\ M_2(\bar{\tau}, \alpha Y_0 + Z) = 0, \\ M_3(\bar{\tau}, \alpha Y_0 + Z) = 0, \end{array} \right. \qquad [24]$$
where $Y_0 = \left( \frac{c'_0 h'_0}{a'_0 i'_0} - \frac{b'_0}{a'_0}, 1, -\frac{h'_0}{i'_0} \right)$, $Z = (z_1, 0, z_3)$, $\bar{X} = \alpha Y_0 + Z$ and $(\alpha, z_1, z_3) \in \mathbb{R}^3$.

From the first and last equations of (24), we see that

$$ \det \left( \frac{\partial M_1(0,0,0,0)}{\partial z_1} \quad \frac{\partial M_1(0,0,0,0)}{\partial z_3} \quad \frac{\partial M_3(0,0,0,0)}{\partial z_1} \quad \frac{\partial M_3(0,0,0,0)}{\partial z_3} \right) = \det \left( \begin{array}{cc} a'_0 & c'_0 \\ 0 & i'_0 \end{array} \right) = a'_0 i'_0 \neq 0. $$

From the implicit function theorem, we can solve $M_1(\bar{\tau}, \alpha Y_0 + Z) = 0$ and $M_3(\bar{\tau}, \alpha Y_0 + Z) = 0$ near $(0, (0, 0, 0))$ with respect to $Z$ as a function of $\bar{\tau}$ and $\alpha$ and find a unique continuous function $Z^*$, such that $Z^*(\bar{\tau}, \alpha) = (z_1^*(\bar{\tau}, \alpha), 0, z_3^*(\bar{\tau}, \alpha))$, $Z^*(0, 0) = (0, 0, 0)$,

$$ M_1 \left( \bar{\tau}, \left( \left( \frac{c'_0 h'_0}{a'_0 i'_0} - \frac{b'_0}{a'_0} \right) \alpha + z_1^*(\bar{\tau}, \alpha), \alpha, -\frac{h'_0}{i'_0} \alpha + z_3^*(\bar{\tau}, \alpha) \right) \right) = 0 $$

and

$$ M_3 \left( \bar{\tau}, \left( \left( \frac{\dot{c}_0 \dot{h}_0}{\dot{a}_0 \dot{i}_0} - \frac{\dot{b}_0}{\dot{a}_0} \right) \alpha + z_1^*(\bar{\tau}, \alpha), \alpha, -\frac{\dot{h}_0}{\dot{i}_0} \alpha + z_3^*(\bar{\tau}, \alpha) \right) \right) = 0, $$

for every $(\bar{\tau}, \alpha)$ small enough.

Moreover, we have $\frac{\partial Z^*}{\partial \alpha}(0, 0) = (0, 0, 0)$ and $\frac{\partial Z^*}{\partial \bar{\tau}}(0, 0) = \frac{(r_1 - \theta_1)^2 k_1 (1 - T_1) e^{-(r_1 - \theta_1)\tau_0}}{r_1 (1 - e^{-(r_1 - \theta_1)\tau_0})^2}$, $0, 0)$.

Then $M(\bar{\tau}, \bar{X}) = 0$ if and only if

$$ f_2(\bar{\tau}, \alpha) = M_2 \left( \bar{\tau}, \left( \left( \frac{\dot{c}_0 \dot{h}_0}{\dot{a}_0 \dot{i}_0} - \frac{\dot{b}_0}{\dot{a}_0} \right) \alpha + z_1^*(\bar{\tau}, \alpha), \alpha, -\frac{\dot{h}_0}{\dot{i}_0} \alpha + z_3^*(\bar{\tau}, \alpha) \right) \right) = 0. \quad [25] $$

We now proceed to solve equation (25).

We have $f_2(0, 0) = 0$.

From the Taylor development of $f_2$ around $(\bar{\tau}, \alpha) = (0, 0)$, we find that $\frac{\partial f_2(0, 0)}{\partial \bar{\tau}} = \frac{\partial f_2(0, 0)}{\partial \alpha} = 0$.

Let $A_2 = \frac{\partial^2 f_2(0, 0)}{\partial \bar{\tau}^2}$, $B_2 = \frac{\partial^2 f_2(0, 0)}{\partial \bar{\tau} \partial \alpha}$ and $C_2 = \frac{\partial^2 f_2(0, 0)}{\partial \alpha^2}$. It's shown that $A_2 = 0$. Further, for $\lambda_2 = 0$ we have $B_2 = -(T_2 - R)(r_2 - \theta_2 - m)e^{(r_2 - \theta_2 - m)\tau_0} < 0$ and

$$ C_2 = (T_2 - R) \frac{r_2 e^{(r_2 - \theta_2 - m)\tau_0}}{k_2(r_2 - \theta_2 - m)} (e^{(r_2 - \theta_2 - m)\tau_0} - 1) \\ + (T_2 - R) \frac{2r_2 m e^{(r_2 - \theta_2 - m)\tau_0}}{k_2} \int_0^{\tau_0} e^{(r_3 - \theta_3)u} I_{\frac{-r_3 \lambda_3 K_1}{r_1}}(u) \left( \int_0^u \frac{e^{(r_2 - \theta_2 - (r_3 - \theta_3) - m)s}}{I_{\frac{-r_3 \lambda_3 K_1}{r_1}}(s)} ds \right) du \\ + 2(T_2 - R) \left( \frac{-h'_0}{i'_0} \right) \left\{ \frac{r_2 e^{(r_2 - \theta_2 - m)\tau_0}}{k_2(1 - e^{-(r_1 - \theta_1)\tau_0})^{\frac{-r_3 \lambda_3 K_1}{r_1}}} \int_0^{\tau_0} e^{(r_3 - \theta_3)u} I_{\frac{-\frac{(r_3 - \theta_3)\lambda_3 K_1}{r_1}}{r_1}}(u) du \right\}, $$

(for more details, see [5]). From conditions cited above, we have $i'_0 > 0$ and $h'_0 < 0$, then $C_2 > 0$, therefore $B_2 C_2 < 0$. Hence

$$ f_2(\bar{\tau}, \alpha) = B_2 \bar{\tau} \alpha + C_2 \frac{\alpha^2}{2} + o\left(|\alpha|^2 + |\bar{\tau}|^2\right). $$
By taking $\bar{\tau} = \sigma\alpha$, we have $f_2(\sigma\alpha, \alpha) = \frac{\alpha^2}{2}g_2(\sigma, \alpha)$ where $g_2(\sigma, \alpha) = 2B_2\sigma + C_2 + o_\alpha(1+\sigma^2)$. Moreover $\frac{\partial g_2}{\partial \sigma}(\sigma, 0) = 2B_2 < 0$ and $g_2(\sigma, 0) = 2B_2\sigma + C_2$. So, for $\sigma_0 = -\frac{C_2}{2B_2}$ we have $g_2(\sigma_0, 0) = 0$ and $\frac{\partial g_2}{\partial \sigma}(\sigma_0, 0) \neq 0$.

Using the implicit function theorem we find a function $\sigma(\alpha)$ such that for $\alpha$ small enough $g_2(\sigma(\alpha), \alpha) = 0$ and $\sigma(0) = \sigma_0 = -\frac{C_2}{2B_2}$. Then, for $\alpha$ near 0 and $\bar{\tau}(\alpha) = \sigma(\alpha)\alpha$ we have $f_2(\bar{\tau}(\alpha), \alpha) = 0$.

**Theorem 2.2** If conditions (10)-(13), (15) and (22) hold, then we have a bifurcation of one nontrivial $\tau(\alpha)$-periodic solution of (1)-(6) with initial condition $\left(x_0 + \left(\frac{c'_0h'_0}{a'_0i'_0} - \frac{b'_0}{a'_0}\right)\alpha + z_1^*(\bar{\tau}(\alpha), \alpha), \alpha, -\frac{h'_0}{i'_0}\alpha + z_3^*(\bar{\tau}(\alpha), \alpha)\right)$ and period $\tau(\alpha) = \tau_0 + \bar{\tau}(\alpha)$ for $\alpha(>0)$ and $\lambda_2$ small enough, where $\bar{\tau}(\alpha) = -\frac{C_2}{2B_2}\alpha + \circ(\alpha)$ and $z_1^*(\bar{\tau}(\alpha), \alpha) = -\frac{C_2}{2B_2}\frac{(r_1-\theta_1)^2k_1(1-T_1)e^{-(r_1-\theta_1)\tau_0}}{r_1(1-e^{-(r_1-\theta_1)\tau_0})^2}\alpha + \circ(\alpha)$.

2) Case 2: $e'_0 \neq 0$ and $i'_0 = 0$. Suppose that (10)-(13), (17) and (23) are satisfied. We have $M(0, (0, 0, 0)) = 0$, $\dim(\ker[D_X M(0, (0, 0, 0))]) = 1$ with $\ker[D_X M(0, (0, 0, 0))] = \operatorname{span}\left\{\left(\frac{-c'_0}{a'_0}, 0, 1\right)\right\}$. Let $Y_0 = \left(\frac{-c'_0}{a'_0}, 0, 1\right)$, $Z = (z_1, z_2, 0)$, $\bar{X} = \alpha Y_0 + Z$ and $(\alpha, z_1, z_2) \in \mathbb{R}^3$.

From the first and second equations of (24), we have

$$ \det \left( \frac{\frac{\partial M_1(0,(0,0,0))}{\partial z_1}}{\frac{\partial M_2(0,(0,0,0))}{\partial z_1}} \quad \frac{\frac{\partial M_1(0,(0,0,0))}{\partial z_2}}{\frac{\partial M_2(0,(0,0,0))}{\partial z_2}} \right) = \det \left( \begin{array}{cc} a' & b' \\ 0 & e' \end{array} \right) = a'_0 e'_0 \neq 0. $$

From the implicit function theorem, we can solve $M_1(\bar{\tau}, \alpha Y_0 + Z) = 0$ and $M_2(\bar{\tau}, \alpha Y_0 + Z) = 0$ near $(0, (0, 0, 0))$ with respect to $Z$ as a function of $\bar{\tau}$ and $\alpha$ and find a unique continuous function $Z^*$, such that $Z^*(\bar{\tau}, \alpha) = (z_1^*(\bar{\tau}, \alpha), z_2^*(\bar{\tau}, \alpha), 0)$, $Z^*(0, 0) = (0, 0, 0)$,

$$ M_1 \left( \bar{\tau}, \left( -\frac{c'_0}{a'_0} \alpha + z_1^*(\bar{\tau}, \alpha), z_2^*(\bar{\tau}, \alpha), \alpha \right) \right) = 0 $$

and

$$ M_2 \left( \bar{\tau}, \left( -\frac{c'_0}{a'_0} \alpha + z_1^*(\bar{\tau}, \alpha), z_2^*(\bar{\tau}, \alpha), \alpha \right) \right) = 0, $$

for every $(\bar{\tau}, \alpha)$ small enough.

Moreover, we have $\frac{\partial Z^*}{\partial \alpha}(0, 0) = (0, 0, 0)$ and $\frac{\partial Z^*}{\partial \bar{\tau}}(0, 0) = \frac{(r_1 - \theta_1)^2 k_1 (1 - T_1) e^{-(r_1 - \theta_1) \tau_0}}{r_1 (1 - e^{-(r_1 - \theta_1) \tau_0})^2}$, $0, 0)$.

Then $M(\bar{\tau}, \bar{X}) = 0$ if and only if

$$ f_3(\bar{\tau}, \alpha) = M_3 \left( \bar{\tau}, \left( -\frac{c'_0}{a'_0} \alpha + z_1^*(\bar{\tau}, \alpha), z_2^*(\bar{\tau}, \alpha), \alpha \right) \right) = 0. \quad [26] $$

We now proceed to solve equation (26).

We have $f_3(0, 0) = 0$.

From the Taylor development of $f_3$ around $(\bar{\tau}, \alpha) = (0, 0)$, we find that $\frac{\partial f_3(0,0)}{\partial \bar{\tau}} = \frac{\partial f_3(0,0)}{\partial \alpha} = 0$.

Let $A_3 = \frac{\partial^2 f_3(0,0)}{\partial \bar{\tau}^2}$, $B_3 = \frac{\partial^2 f_2(0,0)}{\partial \bar{\tau} \partial \alpha}$ and $C_3 = \frac{\partial^2 f_3(0,0)}{\partial \alpha^2}$. It's shown that $A_3 = 0$.
Further, for $\lambda_3 = 0$ we have $B_3 = -(r_3 - \theta_3)T_3e^{(r_3-\theta_3)\tau_0} < 0$ and $C_3 = 2r_3\tau_0K_3^{-1}T_3e^{(r_3-\theta_3)\tau_0} > 0$. Hence

$$f_3(\bar{\tau}, \alpha) = B_3\bar{\tau}\alpha + C_3\frac{\alpha^2}{2} + o(|\alpha|^2 + |\bar{\tau}|^2).$$

Using the same arguments as in the case 1, we have the following results.

**Theorem 2.3** If conditions (10)-(13), (17) and (23) hold, then we have a bifurcation of one nontrivial $\tau(\alpha)$-periodic solution of (1)-(6) with initial condition $(x_0 + (\frac{-\dot{c}_0}{\dot{a}_0})\alpha + z_1^*(\bar{\tau}(\alpha), \alpha), 0, \alpha)$ and period $\tau(\alpha) = \tau_0 + \bar{\tau}(\alpha)$ for $\alpha(>0)$ and $\lambda_3$ small enough, where $z_1^*(\bar{\tau}(\alpha), \alpha) = -\frac{C_3}{2B_3}\frac{(r_1-\theta_1)^2k_1(1-T_1)e^{-(r_1-\theta_1)\tau_0}}{r_1(1-e^{-(r_1-\theta_1)\tau_0})^2}\alpha + \circ(\alpha)$ and $\bar{\tau}(\alpha) = -\frac{C_3}{2B_3}\alpha + \circ(\alpha)$.

3) Case 3: $e'_0 = 0 = i'_0$.

(i) If $h'_0 \neq 0$, then $A_2 = B_2 = C_2 = 0$, which is an undetermined case, to study it we need to determine the higher derivatives of $f_2$.

(ii) If $h'_0 = 0$, then $\dim \ker(E) = 2$, in this case the approach above can not be applied.

REMARQUE. —

From Theorems 2.2 and 2.3, we deduce that the loss of stability for some values of the treatment amplitudes and the period between two successive administration of the treatment we have the onset of the tumor.

## 3. Conclusion

In this work we have considered a model of chemotherapy treatment by two drugs for population with normal cells, sensitive tumor cells and resistant tumor cells, one with instantaneous effect and the other with continuous effect. We have studied the stability of the trivial solution corresponding to the eradication of the tumor, and we find necessary conditions to have eradication of the tumor. Otherwise, we lose stability and bifurcation of nontrivial periodic solutions will appear, it corresponds to the persistence of the tumor. We have treated two cases, for the third one we need an other approach. It will be interesting to consider the resistance with respect to the drug with continuous effect.

## 4. Bibliographie

[1] A. BOUDERMINE, M. HELAL AND A. LAKMECHE, « Bifurcation of nontrivial periodic solutions for pulsed chemotherapy model », *Mathematical Sciences And Applications E-Notes*, vol. 2, n° 2, 2014, pp. 22-44.

[2] G. IOOSS, « Bifurcation of maps and applications, Study of mathematics », *North Holland*, 1979.

[3] A. LAKMECHE AND O. ARINO, « Bifurcation of non trivial periodic solutions of impulsive differential equations arising in chemotherapeutic treatment », *Dynamics Cont. Discr. Impl. Syst.*, vol. 7, 2000, 265-287.

[4] A. LAKMECHE AND O. ARINO, « Nonlinear mathematical model of pulsed-therapy of heterogeneous tumors », *Nonlinear Anal. Real World Appl.*, vol. 2, 2001, 455-465.
[5] A. LAKMECHE, M. HELAL AND A. LAKMECHE, « Pulsed chemotherapy model », *Electronic journal of mathematical analysis and applications*, vol. 2, n° 1, 2014, 127-148.

[6] X. N. LIU AND L.S. CHEN, « Complex dynamics of Holling type II Lotka Volterra predator prey system with impulsive perturbations on the predator », *Chaos Solitons Fract.* vol. 16, 2003, 311-320.

[7] K. NEGI AND S. GAKKHAR, « Dynamics in a Beddington DeAngelis prey predator system with impulsive harvesting », *Ecol. Model*. vol. 206, 2007, 421-430.

[8] J. C. PANETTA, « A mathematical model of periodically pulsed chemotherapy : tumor recurrence and metastasis in a competition environment », *Bull. Math. Biol.*, vol. 58, 1996, 425-447.

[9] J. C. PANETTA, « A mathematical model of drug resistance : Heterogeneous tumors », *Math. Biosci.*, vol. 147, 1998, 41-61

[10] B. SHULGIN, L. STONE AND Z. AGUR, « Theoretical examination of the pulse vaccination policy in the SIR epidemic model », *Math. Comput. Model.*, vol. 31, 2000, 207-215.

[11] G. Z. ZENG, L. S. CHEN AND L. H. SUN, « Complexity of an SIR epidemic dynamics model with impulsive vaccination control », *Chaos Solitons Fract.*, vol. 26, 2005, 495-505.

[12] X. ZHANG, Z. SHUAI AND K. WANG, « Optimal impulsive harvesting policy for single population », *Nonlinear Anal.*, vol. 4, 2003, 639-651.