# Global Dynamics of a Predator-Prey System with Cooperative Hunting
## 4 Stability of Equilibria
### Stability of Boundary Equilibria  
In this subsection, we show the local qualitative behaviors of System (3) near the boundary equilibria \(E_{0}=(0,0)\) and \(E_{1}=(0,1)\).  
**Theorem 3**.: \(1\). \(E_{0}=(0,0)\) _is always a saddle._
2. \(E_{1}=(0,1)\) _is a stable node if either_ \(d>c\) _or_ \(d=c\) _and_ \(\beta=1\)_, whereas it is a saddle if_ \(d<c\)_, with its stable manifold along the_ \(y\)_-axis and its unstable manifold (denoted by_ \(\Gamma_{E_{1}}^{u}\)_) entering the interior of_ \(\mathbb{R}_{+}^{2}\)_. If_ \(d=c\) _and_ \(\beta\
eq 1\)_,_ \(E_{1}\) _is a saddle node:_ 1. _When_ \(\beta<1\)_,_ \(E_{1}\) _has a parabolic sector in the first quadrant and it is an attracting saddle node;_  
Figure 2: The number of interior equilibria is determined by the number of intersection points of \(u=F(x)\) and \(u=d\) in the first quadrant. (**a**) The case of \(\beta\leq 1\). (**b**) The case of \(\beta>1\).  
_._
2. _When_ \(\beta>1\)_,_ \(E_{1}\) _has two hyperbolic sectors in the first quadrant._  
Proof.: From the Jacobian matrix (9) at \(E_{0}\), it is easy to see that \(E_{0}\) is always a saddle.  
At \(E_{1}=(0,1)\), the eigenvalues of the Jacobian matrix (9) are \(\kappa_{1}=c-d\) and \(\kappa_{2}=-1\). Therefore, \(E_{1}\) is a stable node if \(d>c\) and a saddle if \(d<c\). Now, we discuss the critical case \(d=c\), where \(E_{1}\) becomes a non-hyperbolic equilibrium. We use the center manifold theorem and the polar blow-up technique [34] to determine the type of \(E_{1}\).  
By performing a transformation of \(x=X,y=Y+1\) and then setting \(X=u\), \(Y=-u+v\), System (3) becomes  
\[\begin{split} u^{\prime}&=c(\beta-1)u^{2}+cuv-c \beta u^{3}+c\beta u^{2}v,\\ v^{\prime}&=-v+(\beta(c-1)-c)u^{2}+(c+1)uv-v^{2}- \beta(c-1)u^{3}+\beta(c-1)u^{2}v.\end{split} \tag{10}\]  
It is easy to check that the flow on the center manifold is given by  
\[u^{\prime}=c(\beta-1)u^{2}+c(\beta c-2\beta-c)u^{3}+O(u^{4}).\]  
Thus, we know from the center manifold theorem [34] that \(E_{1}\) is a saddle node when \(\beta\
eq 1\), whereas it is a stable node when \(\beta=1\). In order to determine the qualitative behavior of the solutions near \(E_{1}\) in \(\mathbb{R}_{+}^{2}\) when \(\beta\
eq 1\), the blow-up technique [34] is applied to System (10) below. By applying the polar coordinate transformation \(u=\rho\cos(\eta)\), \(v=\rho\sin(\eta)\) to (10), we obtain the following system of polar coordinates  
\[\frac{\mathrm{d}\rho}{\mathrm{d}t}=-\rho\sin^{2}(\eta)+\rho^{2}p(\eta)+O(\rho ^{3}),\quad\frac{\mathrm{d}\eta}{\mathrm{d}t}=-\cos(\eta)\sin(\eta)+O(\rho), \tag{11}\]  
where  
\[\begin{split} p(\eta)=&\cos(\eta)\Big{[}c(\beta-1 )\cos^{2}(\eta)+c\cos(\eta)\sin(\eta)\Big{]}\\ &+\sin(\eta)\Big{[}(\beta c-\beta-c)\cos^{2}(\eta)+(c+1)\cos( \eta)\sin(\eta)-\sin^{2}(\eta)\Big{]}.\end{split} \tag{12}\]  
On the unit circle \(\{0\}\times\mathbb{S}\), System (11) clearly has four singular points \((0,\eta_{i})\), \(\eta_{i}=0\), \(\pi\), \(\frac{\pi}{2}\), \(\frac{3\pi}{2}\), corresponding to \(i=1,2,3,4\). When \(0<\rho\ll 1\), we have \(\frac{\mathrm{d}\rho}{\mathrm{d}t}<0\) along the directions \(\eta=\eta_{3}\) and \(\eta_{4}\). If \(\beta>1\), then \(\frac{\mathrm{d}\rho}{\mathrm{d}t}>0\) along the direction \(\eta=\eta_{1}\) and \(\frac{\mathrm{d}\rho}{\mathrm{d}t}<0\) along the direction \(\eta=\eta_{2}\). If \(\beta<1\), then \(\frac{\mathrm{d}\rho}{\mathrm{d}t}<0\) along the direction \(\eta=\eta_{1}\) and \(\frac{\mathrm{d}\rho}{\mathrm{d}t}>0\) along the direction \(\eta=\eta_{2}\). Therefore, when \(\beta>1\), \(E_{1}\) is a saddle node that consists of two hyperbolic sectors and one parabolic sector, and its hyperbolic sectors are in the first quadrant; when \(\beta<1\), \(E_{1}\) has its parabolic sector in the first quadrant, and it is an attracting saddle node.