## 3 Multi-Player Zero-Sum Differential Game for Decoupled Multi-Agent System
### Multi-Player Zero-Sum Differential Game
 \tag{18}\]  
If the \(V_{i}^{*}\) is the Nash equilibrium solution of the multi-player zero-sum game, that is  
\[V_{i}^{*}(\delta_{i})=\underset{u_{i}}{\text{minmax}}J_{i}(\delta_{i}\,,u_{i} \,\,,\,\widehat{u}_{-i}) \tag{19}\]  
we can obtain  
\[\underset{\widehat{u}_{-i}}{\text{minmax}}H_{i}(\delta_{i},\
abla V_{i}^{*}, u_{i}\,\widehat{u}_{-i})=0 \tag{20}\]  
Substituting the optimal strategy determined by (18) into (20), the coupled Hamilton-Jacobi (HJ) equations yield  
\[ \begin{array}{c}0=\frac{1}{2}\delta_{i}^{T}Q_{i}\delta_{i}+\frac{1}{2}(d_{i }+g_{i})^{2}\
abla V_{i}^{T}B_{i}R_{ii}^{-1}B_{i}^{T}\
abla V_{i}-\frac{1}{2 \gamma_{i}^{2}}\sum\limits_{j\in N_{i}}a_{ij}^{2}\
abla V_{i}^{T}B_{j}R_{ij}^ {-1}B_{j}^{T}\
abla V_{i}\\ \hskip 14.226378 pt+\
abla V_{i}^{T}(A\delta_{i}-(d_{i}+g_{i})^{2}B_{i}R_{ii}^{- 1}B_{i}^{T}\
abla V_{i}+\frac{1}{\gamma_{i}^{2}}\sum\limits_{j\in N_{i}}a_{ij}^ {2}B_{j}R_{ij}^{-1}B_{j}^{T}\
abla V_{i})\,\,,\,\,V_{i}(0)=0\,,\,\,\,i\in N\end{array}  \tag{21}\]  
For a given solution \(V_{i}^{*}\), in order to define \(u_{i}^{*}=u_{i}(V_{i}^{*})\) and \(\widehat{u}_{j}^{*}=\widehat{u}_{j}(V_{i}^{*})\) in the same way as (18), (21) can be written as  
\[H_{i}(\delta_{i},\
abla V_{i}^{*},u_{i}^{*},\widehat{u}_{-i}^{*})=0\text{, }V_{i}^{*}(0)=0 \tag{22}\]  
**Lemma 2**. : _For any policies \(u_{i}\), \(\widehat{u}_{-i}\), the following equation holds_  
\[H_{i}(\delta_{i},\
abla V_{i}^{*},u_{i},\widehat{u}_{-i})=\frac{1}{2}(u_{i}-u _{i}^{*})^{T}R_{ii}(u_{i}-u_{i}^{*})-\sum\limits_{j\in N_{i}}\frac{\gamma_{i}} {2}(\widehat{u}_{j}-\widehat{u}_{j}^{*})^{T}R_{ij}(\widehat{u}_{j}-\widehat{u }_{j}^{*}) \tag{23}\]  
**Proof of Lemma 2**. Substituting \(u_{i}\), \(\widehat{u}_{-i}\) for \(u_{i}^{*}\), \(\widehat{u}_{-i}^{*}\) in (22)  
\[ \begin{array}{c}H_{i}(\delta_{i},\
abla V_{i}^{*},u_{i},\widehat{u}_{-i})= \
abla V_{i}^{*T}(A\delta_{i}+(d_{i}+g_{i})B_{i}u_{i}-\sum\limits_{j\in N_{i} }a_{ij}B_{j}\widehat{u}_{j}\\ \hskip 14.226378 pt+(d_{i}+g_{i})B_{i}u_{i}^{*}-(d_{i}+g_{i})B_{i}u_{i}^{*}-\sum \limits_{j\in N_{i}}a_{ij}B_{j}\widehat{u}_{j}^{*}+\sum\limits_{j\in N_{i}}a_{ ij}B_{j}\widehat{u}_{j}^{*})\\ \hskip 14.226378 pt+\frac{1}{2}\Bigg{(}\delta_{i}^{T}Q_{i}\delta_{i}+u_{i}^{T}R_ {ii}u_{i}-\gamma_{i}^{2}\sum\limits_{j\in N_{i}}\widehat{u}_{j}^{T}R_{ij} \widehat{u}_{j}\Bigg{)}\\ \hskip 14.226378 pt+\frac{1}{2}\Bigg{(}u_{i}^{*T}R_{ii}u_{i}^{*}-u_{i}^{*T}R_ {ii}u_{i}^{*}-\gamma_{i}^{2}\sum\limits_{j\in N_{i}}\widehat{u}_{j}^{*T}R_{ij} \widehat{u}_{j}^{*}+\gamma_{i}^{2}\sum\limits_{j\in N_{i}}\widehat{u}_{j}^{*T}R_ {ij}\widehat{u}_{j}^{*}\Bigg{)}\end{array}  \tag{24}\]Substituting \(H_{i}(\delta_{i},\
abla V_{i}^{*},u_{i}^{*},\widetilde{u}_{-i}^{*})=0\) into (24), we can obtain  
\[ \begin{split} H_{i}(\delta_{i},\
abla V_{i}^{*},u_{i},\widetilde{u }_{-i})&=\
abla V_{i}^{*T}((d_{i}+g_{i})B_{i}(u_{i}-u_{i}^{*})- \sum\limits_{j\in N_{i}}a_{ij}B_{j}(\widetilde{u}_{j}-\widetilde{u}_{j}^{*}) )\\ &\quad+\frac{1}{2}\left(u_{i}^{T}R_{ii}u_{i}-u_{i}^{*T}R_{ii}u_{i}^ {*}\right)-\frac{1}{2}\left(\gamma_{i}^{2}\sum\limits_{j\in N_{i}}\widetilde{u }_{j}^{T}R_{ij}\widetilde{u}_{j}-\gamma_{i}^{2}\sum\limits_{j\in N_{i}} \widetilde{u}_{j}^{*T}R_{ij}\widetilde{u}_{j}^{*}\right)\end{split}  \tag{25}\]  
Completing the squares in (25) gives (23) upon the relationship between \(u_{i}^{*}\), \(\widetilde{u}_{-i}^{*}\) and \(\
abla V_{i}^{*}\). \(\square\)  
**Remark 1**. : _The polics \(\widetilde{u}_{-i}\) in Section 3 are not the real policies of neighboring nodes. These are only defined as the virtual coupling input from neighboring nodes, which are regarded as the external disturbances and have the same channels as the control inputs of neighboring nodes. In this way, the bounded \(L_{2}\)-gain attenuation for the real coupling inputs from neighboring nodes can be realized. In addition, the complex relationships among agents are decoupled virtually during the control protocol design process, and the solving process of zero-sum game is effectively simplified.