## 3 Multi-Player Zero-Sum Differential Game for Decoupled Multi-Agent System
### The Equivalent Algebraic Riccati Equation  
It can be seen from the above results that the Nash equilibrium solution can be obtained by solving the coupled HJ equation (22). In this subsection, it will be shown that the coupled HJ equation (22) can be equivalent to an Algebraic Riccati equation (ARE). Defining the optimal value function \(V_{i}^{*}=\delta_{i}^{T}P_{i}\delta_{i}\), the corresponding optimal \(u_{i}^{*}\) and \(\widetilde{u}_{-i}^{*}\) can be obtained as  
\[ \begin{array}{c}u_{i}^{*}=-2(d_{i}+g_{i})R_{ii}^{-1}B_{i}^{T}P_{i}\delta_{i }\\ \widetilde{u}_{j}^{*}=-\frac{2}{\gamma_{i}^{2}}a_{ij}R_{ii}^{-1}B_{j}^{T}P_{i }\delta_{i}\,\ \ j\in N_{i}\end{array}  \tag{30}\]  
Substituting (30) and \(V_{i}^{*}=\delta_{i}^{T}P\delta_{i}\) into (21) yields  
\[ \begin{array}{c}\delta_{i}^{T}(P_{i}A+A^{T}P_{i})\delta_{i}+\frac{1}{2} \delta_{i}^{T}Q_{i}\delta_{i}&-2(d_{i}+g_{i})^{2}\delta_{i}^{T}P_{i}B_{i}R_{ ii}^{-1}B_{i}^{T}P_{i}\delta_{i}\\ &+\frac{2}{\gamma_{ij}^{2}}\delta_{i}^{T}(\sum\limits_{j\in N_{i}}a_{ij}^{2}P_ {i}B_{j}R_{ij}^{-1}B_{j}^{T}P_{i})\delta_{i}=0\end{array}  \tag{31}\]  
The above equation can be equivalent to  
\[ \begin{array}{c}P_{i}A+A^{T}P_{i}+\frac{1}{2}Q_{i}-2(d_{i}+g_{i})^{2}P_{i} B_{i}R_{ii}^{-1}B_{i}^{T}P_{i}\\ +\frac{2}{\gamma_{ij}^{2}}\sum\limits_{j\in N_{i}}a_{ij}^{2}P_{i}B_{j}R_{ij}^{ -1}B_{j}^{T}P_{i}=0\end{array}  \tag{32}\]  
Defining the integrated matrix as  
\[ \begin{array}{c}R=diag(\frac{R_{ii}}{2(d_{i}+g_{i})^{2}},-\frac{\gamma_{i}^ {2}R_{ij_{1}}}{2a_{ij_{1}}},-\frac{\gamma_{i}^{2}R_{ij_{2}}}{2a_{ij_{2}}}, \cdots,-\frac{\gamma_{i}^{2}R_{ij_{d_{i}}}}{2a_{ij_{d_{i}}}})\\ B=[B_{i}B_{j_{1}}B_{j_{2}}\cdots B_{j_{i_{d_{i}}}}],\ j_{1},\ j_{2},\ \cdots,j_{d_{i}}\in N_{i}\end{array} \]  
Then, (32) can be rewritten as the ARE  
\[P_{i}A+A^{T}P_{i}+\frac{1}{2}Q_{i}-P_{i}BR^{-1}B^{T}P_{i}=0 \tag{33}\]  
**Theorem 2**. : _Assume that all the real control policies \(u_{-i}=\left\{\ u_{j}|\ j\in N_{i}\right\}\) of neighboring agents satisfy the bounded condition as follows,_  
\[\left\|u_{j}\right\|_{2}=u_{j}^{T}u_{j}\leq\xi,\ \ j\in N_{i} \tag{34}\]  
_Selecting the first equation of (30) as the control policies \(u_{i^{\prime}}\)  
**Proof of Theorem 2. ** Selecting the optimal positive value function \(V_{i}^{*}=\delta_{i}^{T}P_{i}\delta_{i}\geq 0\), in terms of the solution of the coupled HJ equation (21), as the Lyapunov function. According to Lemma 2, the derivative of \(V_{i}^{*}\) is  
\[ \begin{array}{rl}\frac{dV_{i}^{*}}{dt}=&-\frac{1}{2}\Bigg{(}\delta_{i}^{T}Q_{i }\delta_{i}+u_{i}^{T}R_{ii}u_{i}-\gamma_{i}^{2}\sum\limits_{j\in N_{i}}u_{j}^{T }R_{ij}u_{j}\Bigg{)}\\ &+\frac{1}{2}(u_{i}-u_{i}^{*})^{T}R_{ii}(u_{i}-u_{i}^{*})-\sum\limits_{j\in N_{ i}}\frac{\gamma_{i}}{2}(u_{j}-\widetilde{u_{j}}^{*})^{T}R_{ij}(u_{j}-\widetilde{u_{j} }^{*})\end{array}  \tag{36}\]  
Selecting \(u_{i}=u_{i}^{*}\) as the first equation of (30) yields  
\[ \begin{array}{rl}\frac{dV_{i}^{*}}{dt}=&-\frac{1}{2}\delta_{i}^{T}Q_{i} \delta_{i}-2(d_{i}+g_{i})^{2}\delta_{i}^{T}P_{i}B_{i}R_{ii}^{-1}B_{i}^{T}P_{i }\delta_{i}\\ &+\frac{\gamma_{i}^{2}}{2}\sum\limits_{j\in N_{i}}u_{j}^{T}R_{ij}u_{j}-\sum \limits_{j\in N_{i}}\frac{\gamma_{i}}{2}(u_{j}-\widetilde{u_{j}}^{*})^{T}R_{ij }(u_{j}-\widetilde{u_{j}}^{*})\\ &\leq&-\delta_{i}^{T}\big{(}\frac{1}{2}Q_{i}+2(d_{i}+g_{i})^{2}P_{i}B_{i}R_{ii }^{-1}B_{i}^{T}P_{i}\big{)}\delta_{i}+\frac{\gamma_{i}^{2}\sum\limits_{j\in N _{i}}\frac{\gamma_{i}}{2}(K_{ij})}{2}\end{array}  \tag{37}\]  
Let \(\frac{dV_{i}^{*}}{dt}\leq 0\), we can realize that  
\[ \begin{array}{rl}\delta_{i}^{T}\delta_{i}=\left\|\delta_{i}\right\|_{2} \geq\frac{\gamma_{i}^{2}\xi\sum\limits_{j\in N_{i}}\overline{\lambda}(R_{ij}) }{2\underline{\lambda}(\frac{1}{2}Q_{i}+2(d_{i}+g_{i})^{2}P_{i}B_{i}R_{ii}^{- 1}B_{i}^{T}P_{i})}\end{array}  \tag{38}\]  
Defining \(\overline{Q}_{i}=\frac{1}{2}Q_{i}+2(d_{i}+g_{i})^{2}P_{i}B_{i}R_{ii}^{-1}B_{ i}^{T}P_{i}\) yields the bounded invariant set (35). \(\square\)  
**Remark 2.