# Some Dynamic Aspects of a Sextic Galactic Potential in a Rotating Reference Frame
## 4 Stability Analysis
### Linear Stability  
This section completes the study in Section 4.1 by investigating the linear stability of the equilibrium positions given in Theorem 4. The linearization system corresponding to the Hamiltonian system in Equation (3) takes the form  
\[\dot{\mathbf{z}}=\mathcal{J}\mathbf{z}, \tag{24}\]  
where \(\mathbf{z}=(z_{1},z_{2},z_{3},z_{4})\) are the perturbed variables for \(x,y,p_{x}\), and \(p_{y}\), respectively. The matrix \(\mathcal{J}\) is the Jacobian matrix evaluated at the equilibrium position \((x_{0},y_{0})\) given by  
\[\mathcal{J}(x_{0},y_{0})=\left( \begin{array}{cccc}0&\Omega&1&0\\ -\Omega&0&0&1\\ A&B&0&\Omega\\ B&C&-\Omega&0\end{array} \right). \tag{25}\]  
where \(A=-5 ax_{0}^{4}-6 cx_{0}^{2}y_{0}^{2}-dy_{0}^{4}-1\), \(B=-4x_{0}y_{0}(cx_{0}^{2}+dy_{0}^{2})\), and \(C=-5 by_{0}^{4}-cx_{0}^{4}-6 dx_{0}^{2}y_{0}^{2}-1\). The eigenvalues of the Jacobian matrix in Equation (25) are the roots of the characteristic equation  
\[\lambda^{4}+M\lambda^{2}+N=0, \tag{26}\]  
where \(M\) and \(N\) are given by  
\[M=2\Omega^{2}-(A+C),\;\;N=(\Omega^{2}+A)(\Omega^{2}+C)-B^{2}. \tag{27}\]  
The equilibrium position \((x_{0},y_{0})\) is linearly stable if the zeros of the characteristic equation (eigenvalues of the Jacobian matrix in Equation (25)) have pure imaginary roots, and the Jacobian matrix is semi-simple. Thus, the conditions for linear stability are  
\[N\geq 0,M\geq 0,M^{2}-4N\geq 0. \tag{28}\]It is worth mentioning that if at least one of the conditions in Equation (28) is not satisfied at \((x_{0},y_{0})\), then the equilibrium position \((x_{0},y_{0})\) is unstable, and the problem does not require further investigation. Based on the [PERSON] theorem (see, for example, [41]), the instability of the equilibrium position \((x_{0},y_{0})\) in a linear approximation is still valid when the nonlinear terms in the Hamilton system in Equation (3) are considered, while the stability of the equilibrium position \((x_{0},y_{0})\) is not necessarily still valid when the nonlinear terms are taken into account. This means the linear approximation analysis gives the necessary conditions for stability and sufficient conditions for instability. According to Theorem 6, the equilibrium point \(\mathcal{E}_{0}\) is a maximum critical point for the effective potential in Equation (19) if \(\Omega>1\) and the eigenvalues of the Jacobian matrix are \(\pm(\Omega-1)i\) and \(\pm(\Omega+1)i\). Hence, \(\mathcal{E}_{0}\) is a linearly stable maximum. This proves the first statement in the following theorem:  
**Theorem 7**. : _For the equilibrium positions of the Hamiltonian system (3), we have_  
1. _If all the coefficients_ \(a,b,c,d<0\) _and_ \(\Omega>1\)_, then_ \(\mathcal{E}_{0}\) _is a linearly stable maximum._
2. _Assuming_ \(b(\Omega^{2}-1)>0\)_, the positions_ \(\mathcal{E}_{1,2}\) _are linearly stable maximums if_ \(d\in[b\mu_{1},b[\) _and_ \(\Omega\in]\frac{1}{\sqrt{2}},1[\)_. Otherwise, they are Lyapunov unstable._
3. _Assuming_ \(a(\Omega^{2}-1)>0\)_, the positions_ \(\mathcal{E}_{3,4}\) _are linearly stable maximums if_ \(c\in[a\mu_{1},a[\) _and_ \(\Omega\in]\frac{1}{\sqrt{2}},1[\)_. Otherwise, they are Lyapunov unstable._  
_where_ \(\mu_{1}=(5-\Omega^{2}-8\Omega\sqrt{1-\Omega^{2}})/(1-\Omega^{2})\)_._  
Proof. : By Theorem 6, the equilibrium positions \(\mathcal{E}_{1,2}\) are the maximums for the effective potential in Equation (19) if \(\frac{d}{b}>1,0\leq\Omega<1\). To maintain the equilibrium condition, since \(\Omega<1\), \(b\) must be negative, then \(d<b\). Let \(\mu=\frac{d}{b}\). By calculating the expressions for \(M\) and \(N\) at the equilibrium positions \(\mathcal{E}_{1,2}\), we have  
\[M =7\Omega^{2}-3+\mu(\Omega^{2}-1), \tag{29a}\] \[N =4(\Omega^{2}-1)^{2}(\mu-1),\] (29b) \[M^{2}-4N =\left[7\Omega^{2}-3+\mu(\Omega^{2}-1)\right]^{2}-16(\Omega^{2}-1 )^{2}(\mu-1). \tag{29c}\]  
The equilibrium positions \(\mathcal{E}_{1,2}\) are linearly stable if the conditions in Equation (28) are satisfied, taking into account the existence of the condition \(b<0,0\leq\Omega<1\).