# Instability Induced by Cross-Diffusion in a Predator-Prey Model with Sex Structure

Authors: Shengmao Fu; Lina Zhang
Journal: Journal of Applied Mathematics
 (2012)

---

## Abstract

In this paper, we consider a cross-diffusion predator-prey model with sex structure. We prove that cross-diffusion can destabilize a uniform positive equilibrium which is stable for the ODE system and for the weakly coupled reaction-diffusion system. As a result, we find that stationary patterns arise solely from the effect of cross-diffusion.


---

## Body

## 1. Introduction

Sex ratio means the comparison between the number of male and female in the species. The sex ratio is generally regarded as 1 : 1. But for wildlife, the sex ratio of species varies with the category, environment condition, community behavior, orientation and heredity, and so forth. The animal’s sex ratio in the different life history stages may vary with different animals. Take a bird as an example: the number of the older males is larger than that of the older females with the increase in age, which is contrary to the case of the mammal where the number of the older females is larger than that of the older males with the increase in age [1, 2]. Sex ratio is the basis of analyzing the dynamic state of different species, the variation of which has a huge influence on the dynamic state of the species [1–7]. Abrahams and Dill [5] have provided evidence that male and female guppies forage differently in the presence of predators and that sexual differences in the energetic equivalence of the risk of predation exist. Eubanks and Miller [6] have found that female Gladicosa pulchra (Lycosidae) wolf spiders climb trees significantly more often than males in the presence of forest floor predators. It is found that the sex of voles affects the risk of predation by mammals and female voles are more easily predated than male voles in [7].Incorporating the sex of prey in a classical Lotka-Volterra model, Liu et al. [1] considered the following sex-structure model:(1.1)u′(t)=b1v-u(D1+ku+kv+c1w),v′(t)=v(b2-D2-ku-kv-c1w),w′(t)=w(-D3+c2u+c2v-c3w),
where u, v, and w are the population densities of the male prey, the female prey, and the predator species respectively. The parameters D1, D2, and D3 are their mortality rates, b1 and b2 are the birth rates of the male prey and the female prey, c1 and c2 are the predation rate and the conversion rate of the predators, and k and c3 are the intraspecific competition rates of the prey and predators. All the parameters in model (1.1) are positive.Ifβ=b2-D2>0, then two obvious nonnegative equilibria of model (1.1) are u0=(0,0,0) and u1=(u1,v1,0), where(1.2)u1=b1βk(b1+D1+β),v1=β(D1+β)k(b1+D1+β).
Moreover, model (1.1) has a positive equilibrium if and only ifH1R:=c2β-kD3>0.
In this case the positive equilibrium is uniquely given by ũ=(ũ,ṽ,w̃), where(1.3)ũ=b1(c3β+c1D3)(b1+D1+β)(c3k+c1c2),ṽ=(D1+β)(c3β+c1D3)(b1+D1+β)(c3k+c1c2),w̃=c2β-kD3c3k+c1c2.
It turns out that R plays an important role in determining the stability of u1 and ũ [1]. To be precise, u1 is locally asymptotically stable if R<0, while ũ is locally asymptotically stable if R>0. This shows that a uniform coexistence state exists and is stable when the intrinsic growth rate β of the female prey is larger than the critical value kD3/c2.Taking account of the inhomogeneous distribution of the prey and the predator in different spatial locations within a fixed bounded domainΩ at any given time, and the natural tendency of each species to diffuse to areas of smaller population concentration, Liu and Zhou in [8] investigated the following weakly coupled reaction-diffusion system:(1.4)ut-d1Δu=b1v-u(D1+ku+kv+c1w),x∈Ω,t>0,vt-d2Δv=v(β-ku-kv-c1w),x∈Ω,t>0,wt-d3Δw=w(-D3+c2u+c2v-c3w),x∈Ω,t>0,∂ηu(x,t)=∂ηv(x,t)=∂ηw(x,t)=0,x∈∂Ω,t>0,u(x,0)=u0(x),v(x,0)=v0(x),w(x,0)=w0(x),x∈Ω,
where η is the outward unit normal vector of the boundary ∂Ω which is smooth, ∂η=∂/∂η. The homogeneous Neumann boundary condition indicates that the predator-prey system is self-contained with zero population flux across the boundary. The constants d1, d2, and d3, called diffusion coefficients, are positive, and the initial values u0(x), v0(x), and w0(x) are nonnegative smooth functions which are not identically zero. Liu and Zhou in [8] found that the nonnegative constant steady states have the same stability properties as the ODE model (1.1). Therefore, Turing instability cannot occur for this reaction-diffusion system.However, in model (1.4), only diffusion of each individual species is taken into account. In some cases, the reality is that the female prey is easily predated because of physiological factor, while the male prey can congregate and form a huge group to protect itself from the attack of the predators [7, 9]; therefore, the predators tend to keep away from their male prey. Similarly as in [10–12], we model this by the cross-diffusion term Δ(d3w+d4uw) for the predators, where d4>0, called the cross-diffusion coefficient. Thus, the cross-diffusion system that we will study is the following:(1.5)ut-d1Δu=b1v-u(D1+ku+kv+c1w):=G1(u,v,w),x∈Ω,t>0,vt-d2Δv=v(β-ku-kv-c1w):=G2(u,v,w),x∈Ω,t>0,wt-Δ(d3w+d4uw)=w(-D3+c2u+c2v-c3w):=G3(u,v,w),x∈Ω,t>0,∂ηu(x,t)=∂ηv(x,t)=∂ηw(x,t)=0,x∈∂Ω,t>0,u(x,0)=u0(x),v(x,0)=v0(x),w(x,0)=w0(x),x∈Ω.
To our knowledge, only a few works investigated the effect of cross-diffusion on population structure and dynamics in the above model. Recently, H. Xu and S. Xu in [13] investigated the global existence of solutions for the corresponding full SKT model of (1.5) when the space dimension is less than ten.An interesting feature of (1.5) is that the interaction between the predators and the male prey gives rise to a cross-diffusion term. The resulting mathematical model is a strongly coupled system of three equations which is mathematically much more complex than those considered earlier. In this paper, we will show that cross-diffusion can destabilize the uniform equilibrium ũ which is stable for models (1.1) and (1.4). Moreover, we will demonstrate that the nonlinear dispersive force can give rise to a spatial segregation of these species.Our paper is organized as follows. In Section2, we analyze the local stability of ũ for (1.5) and calculate the fixed point index, which is important for our later discussions on the existence of nonconstant positive steady states. In Section 3, we prove global asymptotic stability of ũ with d4=0, that is, when no cross-diffusion occurs in the model. This implies that cross-diffusion has a destabilizing effect. In Section 4, we establish a priori upper and lower bounds for all possible positive steady states of (1.5). In Section 5, we study the global existence of nonconstant positive steady states of (1.5) for suitable values of the parameters. This is done by using the Leray-Schauder degree theory and the results obtained in Sections 2, 3, and 4. In Section 6, we discuss the nonexistence of nonconstant positive steady states of (1.5). In the last section, we give a brief discussion about our model.


## 2. Local Stability Analysis and Fixed Point Index ofũ

Letu=(u,v,w)T, Φ(u)=(d1u,d2v,d3w+d4uw)T, and G(u)=(G1(u),G2(u),G3(u))T. Then the stationary problem of (1.5) can be written as(2.1)-ΔΦ(u)=G(u)inΩ;∂ηu=0on⁡∂Ω.
In this section, we study the linearization of (2.1) at ũ and calculate the fixed point index.Similar to [14, 15], let 0=μ1<μ2<μ3<μ4⋯ be the eigenvalues of the operator -Δ on Ω with the homogeneous Neumann boundary condition, and let E(μi) be the eigenspace corresponding to μi in H1(Ω). Let {ϕij:j=1,2,…,dimE(μi)} be the orthonormal basis of E(μi), X=[H1(Ω)]3, and Xij={cϕij:c∈ℝ3}. Then(2.2)X=⨁i=1+∞Xi,Xi=⨁j=1dim⁡E(μi)Xij.LetY=[C1(Ω¯)]3, Y+={u∈Y:u,v,w>0 on Ω¯}, and B(C)={u∈Y:C-1<u,v,w<C on Ω¯} for C>0. Since detΦu(u)=d1d2(d3+d4u)>0 for all nonnegative u, Φu-1(u) exists and det{Φu-1(u)} is positive. Hence, u is a positive solution to (2.1) if and only if(2.3)F(u):=u-(I-Δ)-1{Φu-1(u)[G(u)+∇uΦuu(u)∇u]+u}=0inY+,where(I-Δ)-1 is the inverse of I-Δ under homogeneous Neumann boundary conditions.Further, we note thatDuF(ũ)=I-(I-Δ)-1{Φu-1(ũ)Gu(ũ)+I} and λ is an eigenvalue of DuF(ũ) if and only if, for some i≥1, it is an eigenvalue of the matrix(2.4)Bi:=I-11+μi[Φu-1(ũ)Gu(ũ)+I]=11+μi[μiI-Φu-1(ũ)Gu(ũ)].Writing(2.5)H(μ)=H(ũ;μ):=det⁡{μI-Φu-1(ũ)Gu(ũ)},
we see that if H(μi)≠0, then for each integer 1≤j≤dimE(μi), the number of negative eigenvalues of DuF(ũ) on Xij is odd if and only if H(μi)<0. As a consequence, we have the following proposition.Proposition 2.1 (see [16]).
Suppose that, for alli≥1,  H(μi)≠0. Then
(2.6)index(F(⋅),ũ)=(-1)γ,
where
(2.7)γ=∑i≥1,H(μi)<0dim⁡E(μi).To facilitate our computation ofindex(F(·),ũ), we need to determine the sign of H(μi). In particular, as the aim of this paper is to study the existence of stationary patterns of (2.1) with respect to the cross-diffusion coefficient d4, we will concentrate on the dependence of H(μi) on d4. At this point, we note that H(μ)=det{Φu-1(ũ)}det{μΦu(ũ)-Gu(ũ)}. Since det{Φu-1(ũ)} is positive, we will need only to consider det{μΦu(ũ)-Gu(ũ)}. By(2.8)Gu(ũ)=(-kũ-D1-βb1-kũ-c1ũ-kṽ-kṽ-c1ṽc2w̃c2w̃-c3w̃),Φu(ũ)=(d1000d20d4w̃0d3+d4ũ),
we have(2.9)det⁡{μΦu(ũ)-Gu(ũ)}=C3(d4)μ3+C2(d4)μ2+C1(d4)μ-det⁡Gu(ũ):=C(d4;μ),
where(2.10)C3(d4)=d1d2(d3+d4ũ),C2(d4)=d1d2c3w̃+d1k(d3+d4ũ)ṽ+d2(d3+d4ũ)(kũ+D1+β)-d2d4c1ũw̃,C1(d4)=d1kc3ṽw̃+d2c3(kũ+D1+β)w̃+(d3+d4ũ)(kũ+D1+β)kṽ+d4c1ṽw̃(kũ-b1)-c1ũ(-d2c2w̃+kd4ṽw̃)+c1c2d1ṽw̃-(d3+d4ũ)kṽ(kũ-b1),det⁡Gu(ũ)=ṽw̃[-(kc3+c1c2)(kũ+D1+β)+(c1c2+kc3)(kũ-b1)].Notice thatkũ-b1<0; thus detGu(ũ)<0. We consider the dependence of 𝒞 on d4. Let μ̃1(d4), μ̃2(d4), and μ̃3(d4) be the three roots of 𝒞(d4;μ)=0 with Re{μ̃1(d4)}≤Re{μ̃2(d4)}≤Re{μ̃3(d4)}. It follows that μ̃1(d4)μ̃2(d4)μ̃3(d4)<0 from detGu(ũ)<0. Thus, among μ̃1(d4), μ̃2(d4), μ̃3(d4) at least one is real and negative, and the product of the other two is positive.Consider the following limits:(2.11)lim⁡d4→∞C3(d4)d4=d1d2ũ:=a3,lim⁡d4→∞C2(d4)d4=d1kũṽ+d2ũ(kũ+b1ṽũ)-d2c1ũw̃=d1kũṽ+d2(kũ2+b1ṽ-c1ũw̃):=a2,lim⁡d4→∞C1(d4)d4=ũ(kũ+b1ṽũ)kṽ+c1ṽw̃(kũ-b1)-c1ũkṽw̃-ũkṽ(kũ-b1)=b1ṽ(β-2c1w̃):=a1.Therefore,a1<0 ifH2β<2c1w̃.
In the following, we restrict our attention to β<2c1w̃. In this range, a1<0 and C1(d4)<0 for all sufficiently large d4. Notice that(2.12)lim⁡d4→∞C(d4;μ)d4=a3μ3+a2μ2+a1μ=μ(a3μ2+a2μ+a1)anda1<0<a3. A continuity argument shows that, when d4 is large, μ̃1(d4) is real and negative. Furthermore, as μ̃2(d4)μ̃3(d4)>0, μ̃2(d4) and μ̃3(d4) are real and positive, and(2.13)lim⁡d4→∞μ̃1(d4)=-a2-a22-4a1a32a3<0,lim⁡d4→∞μ̃2(d4)=0,lim⁡d4→∞μ̃3(d4)=-a2+a22-4a1a32a3:=μ̃>0.
Thus we have the following proposition.Proposition 2.2.
Assume that  (H1) and  (H2) hold. Then there exists a positive number  d4* such that, when  d4≥d4*, the three roots  μ̃1(d4), μ̃2(d4), μ̃3(d4) of  𝒞(d4;μ)=0 are all real and satisfy  (2.13). Moreover, for all  d4≥d4*,
(2.14)-∞<μ̃1(d4)<0<μ̃2(d4)<μ̃3(d4),C(d4;μ)<0,μ∈(-∞,μ̃1(d4))∪(μ̃2(d4),μ̃3(d4)),C(d4;μ)>0,μ∈(μ̃1(d4),μ̃2(d4))∪(μ̃3(d4),+∞).Remark 2.3.
Proposition2.2 gives a criterion for the instability of ũ when μ̃>μ2 and the cross-diffusion coefficient d4 is large enough. We further check conditions (H1) and (H2). Let the parameters d1, d2, d3, b1, D1, k, c1, c2, and c3 be fixed. Condition (H1) is equivalent to β>kD3/c2, and condition (H2) is equivalent to β>2kc1D3/γ1 for some γ1:=c1c2-kc3>0. Notice that 2c1/γ1>1/c2, so there exists an unbounded region U1={(D3,β)∈ℝ+2:β>2kc1D3/γ1}, such that for any (D3,β)∈U1, ũ is an unstable equilibrium with respect to (1.5) when μ̃>μ2 and the cross-diffusion coefficient d4 is sufficiently large.


## 3. Global Asymptotic Stability ofũ for (1.4)

The aim of this section is to prove Theorem3.2 which shows that model (1.4) has no nonconstant positive steady state no matter what the diffusion coefficients d1, d2, and d3 are; in other words, diffusion alone (without cross-diffusion) cannot drive instability and cannot generate patterns for this predator-prey model. For this, we will make use of the following result.Lemma 3.1 (see [17]).
Leta and b be positive constants. Assume that φ,ψ∈C1([a,+∞)), ψ(t)≥0, and φ is bounded from below. If φ′(t)≤-bψ(t) and ψ′(t) is bounded in [a,+∞), then limt→∞ψ(t)=0.Theorem 3.2.
Let the parametersd1, d2, d3, b1, D1, D3, k, c1, c2, c3, and β be fixed positive constants that satisfy (H1) and
H3b12<4kũ(D1+b1).
Let (u,v,w) be a positive solution of (1.4). Then
(3.1)‖u(⋅,t)-ũ‖L2(Ω)⟶0,‖v(⋅,t)-ṽ‖L2(Ω)⟶0,‖w(⋅,t)-w̃‖L2(Ω)⟶0ast⟶∞.Proof.
Notice from [8] that ũ is uniformly and locally asymptotically stable in the sense of [18]. We only need to prove the global stability of ũ. Define
(3.2)V1(u,v,w)=12∫Ω(u-ũ)2dx+λ∫Ω[(v-ṽ-ṽln⁡vṽ)dx+ρ∫Ω(w-w̃-w̃ln⁡ww̃)]dx,where λ=ũ, ρ=c1ũ/c2. Obviously, V1(u,v,w) is nonnegative and V1(u,v,w)=0 if and only if (u,v,w)=(ũ,ṽ,w̃). The time derivative of V1(u,v,w) for the system (1.4) satisfies
(3.3)dV1(u,v,w)dt=∫Ω((u-ũ)ut+λv-ṽvvt+ρw-w̃wwt)dx:=-I1(t)-I2(t),where
(3.4)I1(t)=∫Ω[d1|∇u|2+λd2ṽv2|∇v|2+ρd3w̃w2|∇w|2]dx,I2(t)=∫Ω[1(D1+ku+kũ+kv+c1w)(u-ũ)2+λk(v-ṽ)2+ρc3(w-w̃)2+(2kũ-b1)(u-ũ)(v-ṽ)(D1+ku+kũ+kv+c1w)2]dx,≥∫Ω[(D1+kũ)(u-ũ)2+λk(v-ṽ)2+ρc3(w-w̃)2+(2kũ-b1)(u-ũ)(v-ṽ)]dx.If the matrix
(3.5)(D1+kũ12(2kũ-b1)012(2kũ-b1)λk000ρc3)is positive definite, then the quadratic form
(3.6)(D1+kũ)(u-ũ)2+λk(v-ṽ)2+ρc3(w-w̃)2+(2kũ-b1)(u-ũ)(v-ṽ)is positive definite. A direct calculation shows that the matrix is positive definite if (H3) holds. Meanwhile, for every δ such that 0<δ<min{c3ρ,(4kũ(D1+b1)-b12)/(4(D1+2kũ))}, we have
(3.7)I2(t)≥δ∫Ω[(u-ũ)2+(v-ṽ)2+(w-w̃)2]dx.Thus,
(3.8)dV1(u,v,w)dt≤-δ∫Ω[(u-ũ)2+(v-ṽ)2+(w-w̃)2]dx.Similarly to [19, Theorem 2.1], we can prove that the solution (u,v,w) is bounded, and so are the derivatives of ∫Ω[(u-ũ)2+(v-ṽ)2+(w-w̃)2]dx by the equations in (1.4). Using Lemma 3.1, we have
(3.9)‖u(⋅,t)-ũ‖L2(Ω)⟶0,‖v(⋅,t)-ṽ‖L2(Ω)⟶0,‖w(⋅,t)-w̃‖L2(Ω)⟶0ast⟶∞.By the fact that V1(u,v,w) is decreasing for t≥0, it is obvious that (ũ,ṽ,w̃) is globally asymptotically stable, and the proof of Theorem 3.2 is completed.Remark 3.3.
Notice that condition (H3) is equivalent to
(3.10)γ2β>b1(kc3+c1c2)4k-c1D3,γ2=3kb1c3+4kc3D1-b1c1c24k(b1+D1).If γ2<0, it is easy to verify that -c1/γ2>k/c2. Hence, there exists an unbounded region
(3.11)U2={(D3,β)∈R+2:β>kD3c2,γ2β>b1(kc3+c1c2)4k-c1D3},such that for any (D3,β)∈U2, ũ is the unique positive steady state with respect to (1.4).Remark 3.4.
From Remarks2.3 and 3.3, there exists an unbounded region
(3.12)U3=U1∩U2={(D3,β)∈R+2:γ1>0,β>2kc1γ1D3,γ2β>b1(kc3+c1c2)4k-c1D3},such that for any (D3,β)∈U3, cross-diffusion can destabilize the uniform equilibrium ũ of (1.5) when μ̃>μ2 and d4 is sufficiently large.


## 4. A Priori Estimates

In the following, the generic constantsC, C*, and so forth, will depend on the domain Ω and the dimension N. However, as Ω and the dimension N are fixed, we will not mention the dependence explicitly. Also, for convenience, we will write Λ instead of the collective constants (b1,D1,D3,c1,c2,c3,k,β). The main purpose of this section is to give a priori positive upper and lower bounds for the positive solutions to (2.1) when R>0. For this, we will cite the following two results.Lemma 4.1 (Harnack’s inequality [20]).
Letw∈C2(Ω)∩C1(Ω¯) be a positive solution to Δw(x)+c(x)w(x)=0,  where c∈C(Ω¯),  satisfying the homogeneous Neumann boundary condition. Then there exists a positive constant C* which depends only on ∥c∥∞ such that maxΩ¯w≤C*minΩ¯w.Lemma 4.2 (maximum principle [21]).
Letg∈C(Ω×ℝ1) and  bj∈C(Ω¯), j=1,2,…,N. (i)
Ifw∈C2(Ω)∩C1(Ω¯) satisfies
(4.1)Δw(x)+∑j=1Nbj(x)wxj+g(x,w(x))≥0inΩ,∂ηw≤0on⁡∂Ω,
and w(x0)=maxΩ¯w(x), then g(x0,w(x0))≥0.(ii)
Ifw∈C2(Ω)∩C1(Ω¯) satisfies
(4.2)Δw(x)+∑j=1Nbj(x)wxj+g(x,w(x))≤0inΩ,∂ηw≥0on⁡∂Ω,
and w(x0)=minΩ¯w(x), then g(x0,w(x0))≤0.Theorem 4.3 (upper bound).
Letd and  d* be two fixed positive constants. Assume that  di≥d,i=1,2,3, and  0≤d4≤d*. Then every possible positive solution  (u,v,w) of  (2.1) satisfies
(4.3)max⁡Ω¯u≤b1k,max⁡Ω¯v≤βk,max⁡Ω¯w≤(1+d*b1dk)c2(b1+β)c3k.Proof.
A direct application of the maximum principle to (2.1) gives v≤β/k on Ω¯. Let u(x0)=maxΩ¯u. Using the maximum principle again, we have b1v(x0)≥u(x0)[D1+ku(x0)+kv(x0)+c1w(x0)]. Thus, ku(x0)v(x0)≤b1v(x0) and u(x0)≤b1/k.
Defineφ=d3w+d4uw; then, φ satisfies(4.4)-Δφ=w(-D3+c2u+c2v-c3w)inΩ,∂ηφ=0on⁡∂Ω.Let φ(x1)=maxΩ¯φ. By Lemma 4.2, we have
(4.5)-D3+c2u(x1)+c2v(x1)-c3w(x1)≥0.It follows that
(4.6)w(x1)≤c2c3[u(x1)+v(x1)]≤c2c3(b1k+βk)=c2(b1+β)c3k.Hence,
(4.7)φ(x1)=[d3+d4u(x1)]w(x1)≤(d3+d4b1k)c2(b1+β)c3k,max⁡Ω¯w=max⁡Ω¯φ(d3+d4u)≤(1+d4b1d3k)c2(b1+β)c3k≤(1+d*b1dk)c2(b1+β)c3kfor any d3≥d and 0≤d4≤d*.Turning now to the lower bound, we first need some preliminary results.Lemma 4.4.
Letdij be positive constants,  i=1,2,3,4, j=1,2,…, and let  (uj,vj,wj) be the corresponding positive solution of  (2.1) with  di=dij. If  (uj,vj,wj)→(u*,v*,w*) uniformly on  Ω¯ as  j→∞ and  (u*,v*,w*) is a constant vector, then  (u*,v*,w*) must satisfy
(4.8)b1v*-u*(D1+ku*+kv*+c1w*)=0,β-ku*-kv*-c1w*=0,-D3+c2u*+c2v*-c3w*=0.Moreover, if u*, v*, and w* are positive constants, then (u*,v*,w*)=(ũ,ṽ,w̃).Proof.
It is easy to see that for allj,
(4.9)∫ΩG1(uj,vj,wj)dx=0.If G1(u*,v*,w*)>0, then G1(uj,vj,wj)>0 when j is large since (uj,vj,wj)→(u*,v*,w*). This is impossible. Similarly, G1(u*,v*,w*)<0 is impossible. Therefore, G1(u*,v*,w*)=0. The same argument shows that β-ku*-kv*-c1w*=0 and -D3+c2u*+c2v*-c3w*=0. Consequently, (u*,v*,w*)=(ũ,ṽ,w̃).Lemma 4.5.
The system(4.10)ut-d1Δu=b1v-u(D1+ku+kv),x∈Ω,t>0,vt-d2Δv=v(β-ku-kv),x∈Ω,t>0,∂ηu=∂ηv=0,x∈∂Ω,t>0,u(x,0)≥(≢)0,v(x,0)≥(≢)0,x∈Ω,
has a unique positive constant steady state (u1,v1) which is globally asymptotically stable, where u1 and v1 are given by (1.2).Proof.
Let(4.11)V2(u,v)=12∫Ω(u-u1)2dx+ρ∫Ω(v-v1-v1ln⁡vv1)dx,where ρ=(b1-ku1)/k=b1(b1+D1)/[k(b1+D1+β)]>0, and let (u,v) be a positive solution of (4.10). Then a direct computation gives
(4.12)dV2dt=-∫Ω[d1|∇u|2+ρd2v1v2|∇v|2+(D1+ku+ku1+kv)(u-u1)2+ρk(v-v1)2]dx≤0,and dV2/dt=0 holds if and only if (u,v)=(u1,v1). By Lemma 3.1, we can conclude that (u1,v1) is globally asymptotically stable.Theorem 4.6 (lower bound).
Letd and  d* be two fixed positive constants. Assume that  di≥d, i=1,2,3, and  0≤d4≤d*. Then there exists a positive constant  C=C(Λ,d,d*), such that every possible positive solution  (u,v,w) of  (2.1) satisfies
(4.13)min⁡Ω¯⁡u,min⁡Ω¯v,min⁡Ω¯w≥C.Proof.
If the conclusion does not hold, then there exists a sequence{d1j,d2j,d3j,d4j}j=1∞ with d1j,d2j,d3j≥d and 0≤d4j≤d* such that the corresponding positive solution (uj,vj,wj) of (2.1) satisfies
(4.14)min⁡{min⁡Ω¯uj,min⁡Ω¯vj,min⁡Ω¯wj}⟶0,asj⟶∞.Moreover, we assume that dij→di∈[d,∞] for i=1,2,3, and d4j→d4∈[0,d*]. By Theorem 4.3 and the standard regularity theory for the elliptic equations, we may also assume that (uj,vj,wj)→(u,v,w) in [C2(Ω¯)]3 for some nonnegative functions u,v,w. It is easy to see that (u,v,w) also satisfies estimate (4.3), and min{minΩ¯u,minΩ¯v,minΩ¯w}=0. Moreover, we observe that, if d1,d2,d3<∞, then (u,v,w) satisfies (2.1).
Next we derive a contradiction for all possible cases.
Firstly, we consider the cased1,d2,d3<∞.
(1)
In view of (2.1), minΩ¯v=0 implies v=0 on Ω¯ from the Harnack inequality. In this case, by the strong maximum principle and the Hopf boundary lemma, it follows that u=w=0 on Ω¯. This is a contradiction to Lemma 4.4. Thus, minΩ¯v>0.(2)
IfminΩ¯u=0, we denote u(x0)=minΩ¯u=0. By the maximum principle we have b1v(x0)≤u(x0)[D1+ku(x0)+kv(x0)+c1w(x0)]=0, and so minΩ¯v=0. This is a contradiction to minΩ¯v>0. Thus minΩ¯u>0.(3)
IfminΩ¯w=0, let φ=d3w+d4uw. Then minΩ¯φ=0 and φ satisfies(4.15)-Δφ=φ(-D3+c2u+c2v-c3w)(d3+d4u)-1inΩ,∂ηφ=0on⁡∂Ω.
The Harnack inequality shows that minΩ¯φ=0 implies φ=0 on Ω¯. Hence, w=0 on Ω¯. From Lemma 4.5, we have (u,v)=(u1,v1). Define ŵj=wj/∥wj∥∞; then, (uj,vj,ŵj,wj) satisfies
(4.16)-d1jΔuj=b1vj-uj(D1+kuj+kvj+c1wj)inΩ,-d2jΔvj=vj(β-kuj-kvj-c1wj)inΩ,-Δ(d3jŵj+d4jujŵj)=ŵj(-D3+c2uj+c2vj-c3wj)inΩ,∂ηuj=∂ηvj=∂ηŵj=0on⁡∂Ω.Similarly to the above, we can prove that there exists a subsequence of {ŵj}, denoted by itself, and a nonnegative function ŵ, such that ŵj→ŵ in C2(Ω¯) and ∥ŵ∥∞=1. Moreover, ŵ satisfies
(4.17)-(d3+d4u1)Δŵ=ŵ(-D3+c2u1+c2v1)inΩ,∂ηŵ=0on⁡∂Ω.Since ∥ŵ∥∞=1, by the strong maximum principle and the Hopf boundary lemma, we find that ŵ>0 on Ω¯. Applying the maximum principle again, we have -D3+c2u1+c2v1=0. Thus, u1+v1=D3/c2. Noting that u1+v1=β/k in (1.2), it follows that c2β-kD3=0, which is a contradiction to the condition R=c2β-kD3>0.
Next, we consider the remaining cases.
Integrating by parts, we obtain that(4.18)b1∫Ωvjdx=∫Ωuj(D1+kuj+kvj+c1wj)dx,∫Ωvj(β-kuj-kvj-c1wj)dx=0,∫Ωwj(-D3+c2uj+c2vj-c3wj)dx=0,for j=1,2,…. Moreover, (u,v,w) satisfies
(4.19)b1∫Ωvdx=∫Ωu(D1+ku+kv+c1w)dx,∫Ωv(β-ku-kv-c1w)dx=0,∫Ωw(-D3+c2u+c2v-c3w)dx=0.
Ifd1=∞, then u satisfies(4.20)-Δu=0inΩ,∂ηu=0on⁡∂Ω.Hence, u is a constant. If u=0, from (4.19), we have in turn that v=w=0. This contradicts Lemma 4.4. So, u is a positive constant and either minΩ¯v=0 or minΩ¯w=0.
Ifd2<∞. In the case of minΩ¯v=0, similarly to the arguments of (1), we have v=0 on Ω¯. This contradicts the first equation of (4.19). Thus minΩ¯v>0 and minΩ¯w=0. Note that w satisfies(4.21)-(d3+d4u)Δw=w(-D3+c2u-c3w)inΩ,∂ηw=0on⁡∂Ω.
Ifd3<∞, the Harnack inequality implies that w=0 on Ω¯. If d3=∞, then w ia a constant. Since minΩ¯w=0, so w=0 on Ω¯. Therefore, by Lemma 4.5, (u,v,w)=(u1,v1,0). Similarly to the arguments of (3), we arrive at c2β-kD3=0, which is a  contradiction.
Similarly, we can derive contradictions for all the other cases.


## 5. Existence of Stationary Patterns for the Model (1.5)

In this section we discuss the existence of nonconstant positive solutions to (2.1). These solutions are obtained for large cross-diffusion coefficient d4, with the other parameters d1, d2, d3, b1, D1, D3, k, c1, c2, c3, and β suitably fixed. Our main result is as follows.Theorem 5.1.
Let the parametersd1, d2, d3, b1, D1, D3, k, c1, c2, c3, and β be fixed such that (H1), (H2), and (H3) hold. Let μ̃ be given by the limit (2.13). If μ̃∈(μn,μn+1) for some n≥2 and the sum ∑i=2ndimE(μi) is odd, then there exists a positive constant d4* such that (2.1) has at least one nonconstant positive solution for d4>d4*.Proof.
By Proposition2.2 and our assumption on μ̃, there exists a positive constant d4* such that (2.14) holds if d4>d4*, and
(5.1)μ̃1(d4)<0=μ1<μ̃2(d4)<μ2,μ̃3(d4)∈(μn,μn+1).
We will prove that for any d4>d4*, (2.1) has at least one nonconstant positive solution. The proof, which is by contradiction, is based on the homotopy invariance of the topological degree.
Suppose on the contrary that the assertion is not true for somed4=d̃4>d4*. In the following we fix d4=d̃4.
Forθ∈[0,1], define Φ(θ;u)=(d1u,d2v,d3w+θd4uw)T and consider the problem(5.2)-ΔΦ(θ;u)=G(u)inΩ,∂ηu=0,on⁡∂Ω.
Then u is a positive nonconstant solution of (2.1) if and only if it is such a solution of (5.2) for θ=1. It is obvious that ũ is the unique constant positive solution of (5.2) for any 0≤θ≤1. As we observed in Section 2, for any 0≤θ≤1, u is a positive solution of (5.2) if and only if
(5.3)F(θ;u):=u-(I-Δ)-1{Φu-1(θ;u)[G(u)+∇uΦuu(θ;u)∇u]+u}=0inY+.
It is obvious that F(1;u)=F(u). Theorem 3.2 shows that F(0;u)=0 has only the positive solution ũ in Y+. By a direct computation,
(5.4)DuF(θ;ũ)=I-(I-Δ)-1{Φu-1(θ;ũ)Gu(ũ)+I}.In particular,
(5.5)DuF(0;ũ)=I-(I-Δ)-1{D-1Gu(ũ)+I},where D=diag(d1,d2,d3) and
(5.6)DuF(1;ũ)=I-(I-Δ)-1{Φu-1Gu(ũ)+I}=DuF(ũ).From (2.5) and (2.9) we see that
(5.7)H(μ)=det⁡{Φu-1(ũ)}C(d4;μ).In view of (2.14) and (5.1), it follows that
(5.8)H(μ1)=H(0)>0,H(μi)<0,2≤i≤n,H(μi)>0,i≥n+1.Therefore, zero is not an eigenvalue of the matrix μiI-Φu-1(ũ)Gu(ũ) for all i≥1, and
(5.9)∑i≥1,H(μi)<0dim⁡E(μi)=∑i=2ndim⁡E(μi)=σn,whichisodd.Thanks to Proposition 2.1, we have
(5.10)index(F(1;⋅),ũ)=(-1)γ=(-1)σn=-1.
Similarly, we can easily show that
(5.11)index(F(0;⋅),ũ)=(-1)0=1.
Now, by Theorems 4.3 and 4.6, there exists a positive constant C such that, for all 0≤θ≤1, the positive solutions of (2.1) satisfy 1/C<u,v,w<C. Therefore, F(θ;u)≠0 on ∂B(C) for all 0≤θ≤1. By the homotopy invariance of the topological degree,
(5.12)deg⁡(F(1;⋅),0,B(C))=deg⁡(F(0;⋅),0,B(C)).
On the other hand, by our supposition, both equations F(1;u)=0 and F(0;u)=0 have only the positive solution ũ in B(C). Hence, by (5.10) and (5.11), we have
(5.13)deg⁡(F(1;⋅),0,B(C))=index(F(1;⋅),ũ)=-1,deg⁡(F(0;⋅),0,B(C))=index(F(0;⋅),ũ)=1.
This contradicts (5.12), and thus we complete the proof of Theorem 5.1.Remark 5.2.
Assume that all the conditions hold in Theorem5.1. Theorem 3.2 shows that ũ is a globally asymptotically stable equilibrium for the system (1.4). However, Theorem 5.1 implies that the cross-diffusion system (1.5) has at least one nonconstant positive steady state. Our results demonstrate that stationary patterns can be found due to the emergence of cross-diffusion.


## 6. Nonexistence of Nonconstant Positive Solution of (2.1)

In this section, we discuss the nonexistence of nonconstant positive solution of (2.1) when the cross-diffusion coefficient d4>0 is small.Theorem 6.1.
If the parametersd1, d3, d4, b1, D1, D3, k, c1, c2, c3, and β satisfy (H1), (H3), and
H4c1d42ũw̃c2<4d1d3,
then the problem (2.1) has no nonconstant positive solution.Proof.
Assume that(u,v,w) is a positive solution of (2.1). Let λ=ũ, ρ=c1ũ/c2. Multiplying the equations of (2.1) by (u-ũ), λ(v-ṽ)/v, and ρ(w-w̃)/w, respectively, and integrating by parts, as in the proof of Theorem 3.2, we obtain 0=-I3-I4, where
(6.1)I3=∫Ω[d1|∇u|2+λd2ṽv2|∇v|2+ρw̃(d3+d4u)w2|∇w|2+ρd4w̃w∇u⋅∇w]dx,I4=∫Ω[(D1+ku+kũ+kv+c1w)(u-ũ)2+λk(v-ṽ)2+ρc3(w-w̃)2+(2kũ-b1)(u-ũ)(v-ṽ)]dx.Applying (H3) and (H4), it is easy to prove that I3≥0 and I4≥0. This implies that (u,v,w)=(ũ,ṽ,w̃) on Ω¯ and the proof is complete.Remark 6.2.
Theorem6.1 shows that the problem (2.1) has no nonconstant positive solutions if one of d1 and d3 is sufficiently large; that is, unlimitedly increasing one of the diffusion rates d1 and d3 will eventually wipe out all nonconstant solutions of (2.1). However, Theorem 6.1 does not tell us the effect of the diffusion rate d2 on the stationary problem (2.1). Using the similar arguments in Section 2, we can find that d2 does not cause instability of ũ. Therefore, we conjecture that the problem (2.1) has no nonconstant positive solutions if d2 is sufficiently large.Remark 6.3.
Theorems5.1 and 6.1 seem to indicate that diffusion tends to suppress pattern formation, while cross-diffusion seems to help create patterns.


## 7. Discussion

In this paper, we have introduced a more realistic mathematical model for a diffusive predator-prey system where the prey has a sex structure comprising male and female members. In this model, we model the tendency of the predators to keep away from the male prey by a cross-diffusion. As a result, our model is a strongly coupled cross-diffusion system, which is mathematically more complex than systems used to model sex-structured predator-prey behavior hitherto [1, 8]. What is noteworthy about this model is that, as the cross-diffusion term arises, it is precisely this cross-diffusion that destabilizes the uniform positive equilibrium and gives rise to stationary patterns for the model. Indeed, stationary patterns do not arise for the ODE (spatially independent) model, nor the PDE model without cross-diffusion.In fact, one can see that this particular cross-diffusion term is also significant from the mathematical point of view. The following system represents the general form of SKT-type cross-diffusion [22] in the predator-prey model(7.1)ut-Δ(d1u+d12uv+d13uw)=G1(u,v,w),x∈Ω,t>0,vt-Δ(d2v+d21uv+d23vw)=G2(u,v,w),x∈Ω,t>0,wt-Δ(d3w+d31uw+d32vw)=G3(u,v,w),x∈Ω,t>0,∂ηu(x,t)=∂ηv(x,t)=∂ηw(x,t)=0,x∈∂Ω,t>0,u(x,0)=u0(x),v(x,0)=v0(x),w(x,0)=w0(x),x∈Ω,
where dij, i,j=1,2,3, i≠j, are cross-diffusion coefficients. Using the method in Section 2 of this paper, we have investigated the stability of ũ for each dij, respectively. We found that either d13 or d23 does not cause instability of ũ, while each of the other dij can induce the instability of ũ. For d32≠0, we can obtain the similar conclusions as Theorem 5.1 by the same mathematical treatment in this paper. Unfortunately, the existence of nonconstant positive steady states has not been obtained when d12≠0 or d21≠0, because we cannot establish a priori lower bounds for all possible positive steady states of (7.1).On the other hand, as pointed out in [23], a Lotka-Volterra-type model can be regarded as a local approximation to a nonlinear system. In the present paper, we only consider the case that the interaction terms on the right-hand side of (1.5) are linear. For the nonlinear case, it will be extremely difficult to analyze positive steady states.In this paper, we do not discuss the stability and the number of the nonconstant positive solutions. We will consider them in the coming papers.



---

*Source: 240432-2012-03-20.xml*