## 3 Synthesis of the Algorithm for the Solution of the Zero-Sum Differential Game
### Gradient Calculation  
In this subsection, the relations for calculating the gradients that appear in (22) i.e., (23) is derived. In order to perform chain rule for derivatives backward in time, first the system (15) needs to be rewritten in discrete-time state-space form. For this purpose,we use the multistep [PERSON] method. Compared to the most popular four stage Runge-Kutta method, which requires solving the approximation problem four times at each step, the [PERSON] method requires only one calculation. The explicit Adams approximation of system (15) can be formulated as follows. Let the time interval \(\left[t_{0},\ t_{f}\right]\) consists of \(t_{i}=t_{0}+i\tau\) for \(i=0,\,1,\,2,\,\ldots,\,N-1\), such that \(\tau=(t_{f}-t_{0})/N\) is the time step length, and let \(\hat{\mathbf{x}}\) be the extended \(4n\)-dimensional state vector (\(n\) is dimension of state vector \(\mathbf{x}\) and \(4\) is order of [PERSON] method). Then, the system can be written in the following discrete-time state-space form:  
\[\hat{\mathbf{x}}(i+1)=\hat{\mathbf{f}}(\hat{\mathbf{x}}(i))+\mathbf{a}\otimes \mathbf{B}(\mathbf{x}(i))\boldsymbol{\pi}+\mathbf{a}\otimes\mathbf{D}(\mathbf{ x}(i))\boldsymbol{\rho},\ \ \hat{\mathbf{x}}(0)=\hat{\mathbf{x}}_{0}, \tag{27}\]  
where the vector \(\mathbf{a}\) contains a non-zero coefficients of [PERSON] method (see ([39], page 358)), and where, for simplicity, we introduced  
\[\mathbf{B}(\mathbf{x})\equiv\mathbf{G}_{1}(\mathbf{x})\boldsymbol{\Theta}( \mathbf{x}),\quad\mathbf{D}(\mathbf{x})\equiv\mathbf{G}_{2}(\mathbf{x}) \boldsymbol{\
abla}(\mathbf{x}). \tag{28}\]  
General considerations regarding [PERSON] method are given in [39], while several novel schemes for multistep [PERSON] method are proposed in [40]. The discrete-time form of the objective function (24) is  
\[J=\tau\sum_{i=0}^{N-1}\left(\mathbf{h}^{\mathrm{T}}(\mathbf{x}(i))\mathbf{h}( \mathbf{x}(i))+\boldsymbol{\pi}^{\mathrm{T}}\mathbf{P}(\mathbf{x}(i)) \boldsymbol{\pi}-\mu\boldsymbol{\rho}^{\mathrm{T}}\mathbf{R}(\mathbf{x}(i)) \boldsymbol{\rho}\right), \tag{29}\]  
where for simplicity we introduced  
\[\mathbf{P}(\mathbf{x})\equiv\boldsymbol{\Theta}^{\mathrm{T}}(\mathbf{x}) \boldsymbol{\Theta}(\mathbf{x}),\quad\mathbf{R}(\mathbf{x})\equiv\boldsymbol{ \
abla}^{\mathrm{T}}(\mathbf{x})\boldsymbol{\
abla}(\mathbf{x}). \tag{30}\]  
The following is a derivation of recursive matrix relations for calculating \(\
abla_{\boldsymbol{\pi}}J\) using the basic chain rule arithmetic and matrix differentials calculus as well as certain properties of vectorization of matrices and Kronecker algebra. The relations for calculating \(\
abla_{\boldsymbol{\rho}}J\) are obtained in an analogous way. The gradient of the objective function (29) with respect to the vector \(\boldsymbol{\pi}\) is  
\[\
abla_{\boldsymbol{\pi}}J=\tau\sum_{i=0}^{N-1}\frac{\partial F(i)}{\partial \boldsymbol{\pi}}, \tag{31}\]  
where  
\[F(i)=\mathbf{h}^{\mathrm{T}}(\mathbf{x}(i))\mathbf{h}(\mathbf{x}(i))+ \boldsymbol{\pi}^{\mathrm{T}}\mathbf{P}(\mathbf{x}(i))\boldsymbol{\pi}-\mu \boldsymbol{\rho}^{\mathrm{T}}\mathbf{R}(\mathbf{x}(i))\boldsymbol{\rho}. \tag{32}\]  
Then, an operation of partial derivative with respect to vector \(\boldsymbol{\pi}\) is performed over the expression (32), which gives:  
\[ \begin{split}&\
abla_{\boldsymbol{\pi}}F(i)=2\mathbf{h}^{ \mathrm{T}}(\mathbf{x}(i))\cdot\
abla_{\mathbf{x}}\mathbf{h}(\mathbf{x}(i)) \cdot\
abla_{\boldsymbol{\pi}}\hat{\mathbf{x}}(i)-\mu[\boldsymbol{\rho} \otimes\boldsymbol{\rho}]^{\mathrm{T}}\cdot\
abla_{\mathbf{x}}\mathrm{vec}( \mathbf{R}(\mathbf{x}(i)))\cdot\
abla_{\boldsymbol{\pi}}\hat{\mathbf{x}}(i)\\ &+[\mathrm{vec}(\mathbf{P}(\mathbf{x}(i)))]^{\mathrm{T}}\cdot( \boldsymbol{\pi}\oplus\boldsymbol{\pi})+(\boldsymbol{\pi}\otimes\boldsymbol{ \pi})^{\mathrm{T}}\cdot\
abla_{\mathbf{x}}\mathrm{vec}(\mathbf{P}(\mathbf{x}( i)))\cdot\
abla_{\boldsymbol{\pi}}\hat{\mathbf{x}}(i).\end{split}