## 3 Identification Algorithms
### The AM-MISG Algorithm  
The AM-SG algorithm only uses the current dataset to update the parameter estimates. Therefore, it has a slow convergence rate and low estimation accuracy. To improve the identification performance of the AM-SG algorithm, an innovation length \(p\) is introduced to derive an auxiliary model-based multi-innovation stochastic gradient (AM-MISG) algorithm. Considering the most recent \(p\) sets of input-output data, define the stacked output vector \(\mathbf{Y}_{i}(p,k)\), the stacked noise vector \(\mathbf{V}_{i}(p,k)\) and the stacked information matrix \(\mathbf{\Psi}_{i}(p,k)\) as:  
\[\mathbf{Y}_{i}(p,k) =[y_{i}(k),\ y_{i}(k-1),\ \cdots,\ y_{i}(k-p+1)]^{\mathrm{ T}}\in\mathbb{R}^{p},\] \[\mathbf{V}_{i}(p,k) =[v_{i}(k),\ v_{i}(k-1),\ \cdots,\ v_{i}(k-p+1)]^{\mathrm{ T}}\in\mathbb{R}^{p},\] \[\mathbf{\Psi}_{i}(p,k) =[\mathbf{\varphi}_{i}(k),\ \mathbf{\varphi}_{i}(k-1),\ \cdots,\ \mathbf{\varphi}_{i}(k-p+1)]\in\mathbb{R}^{n_{0}\times p}.\]  
Equation (5) can be expanded into the following matrix form:  
\[\mathbf{Y}_{i}(p,k)=\mathbf{\Psi}_{i}^{\mathrm{ T}}(p,k)\mathbf{\theta}_{i}+\mathbf{V}_{i}(p,k). \tag{9}\]  
However, the information vectors \(\mathbf{\varphi}_{i}(k-l),\ l=0,\ 1,\cdots,p-1\) in \(\mathbf{\Psi}_{i}(p,k)\) include unknown noise-free outputs. Let \(\hat{\mathbf{\varphi}}_{i}(k-l)\) be their estimates; the estimate of \(\mathbf{\Psi}_{i}(p,k)\) can be defined as:  
\[\hat{\mathbf{\Psi}}_{i}(p,k)=[\hat{\mathbf{\varphi}}_{i}(k),\ \hat{\mathbf{\varphi}}_{i}(k-1),\ \cdots,\ \hat{\mathbf{\varphi}}_{i}(k-p+1)]\in \mathbb{R}^{n_{0}\times p}.\]  
Define the following criterion function:  
\[J_{2}(\mathbf{\theta}_{i})=\left\|\mathbf{V}_{i}(p,k)\right\|^{2}=\left\|\mathbf{Y}_{i}(p,k)-\mathbf{\Psi}_{i}^{\mathrm{ T}}(p,k)\mathbf{\theta}_{i}\right\|^{2}, \tag{10}\]  
where \(\left\|\mathbf{X}\right\|^{2}=\mathrm{tr}[\mathbf{X}\mathbf{X}^{\mathrm{ T}}]\) represents the norm of the matrix \(\mathbf{X}\). The gradient of \(J_{2}(\mathbf{\theta}_{i})\) with respect to \(\mathbf{\theta}_{i}\) is given by:  
\[\mathrm{grad}[J_{2}(\mathbf{\theta}_{i})]=\frac{\partial J_{2}(\mathbf{\theta}_{i})}{ \partial\mathbf{\theta}_{i}}=-2\hat{\mathbf{\Psi}}_{i}(p,k)[\mathbf{Y}_{i}(p,k)-\hat{\mathbf{ \Psi}}_{i}^{\mathrm{ T}}(p,k)\mathbf{\theta}_{i}].\]  
Applying the stochastic gradient search principle to minimize the criterion function in (10), we have:  
\[\hat{\mathbf{\theta}}_{i}(k) =\hat{\mathbf{\theta}}_{i}(k-1)-\mu_{i}(k)\mathrm{grad}J_{2}[\hat{ \mathbf{\theta}}_{i}(k-1)]\] \[=\hat{\mathbf{\theta}}_{i}(k-1)+2\mu_{i}(k)\mathbf{\Psi}_{i}(p,k)[\mathbf{Y}_ {i}(p,k)-\mathbf{\Psi}_{i}^{\mathrm{ T}}(p,k)\hat{\mathbf{\theta}}_{i}(k-1)], \tag{11}\]  
where \(\mu_{i}(k)>0\) is called the step size or the convergence factor. For the convenience of formula derivation, let \(\mu_{i}(k)=\frac{1}{2r_{i}(k)}\); Equation (11) can be rewritten into:  
\[\hat{\mathbf{\theta}}_{i}(k) =\hat{\mathbf{\theta}}_{i}(k-1)+\frac{1}{r_{i}(k)}\hat{\mathbf{\Psi}}_{i} (p,k)[\mathbf{Y}_{i}(p,k)-\hat{\mathbf{\Psi}}_{i}^{\mathrm{ T}}(p,k)\hat{\mathbf{\theta}}_{i}(k-1)] \tag{12}\] \[\mathbf{V}_{i}(p,k) =[v_{i}(k),\ v_{i}(k-1),\ \cdots,\ v_{i}(k-p+1)]^{\mathrm{ T}}\in\mathbb{R}^{p},\]0To guarantee the convergence of this recursive algorithm, all eigenvalues of the matrix \(\left[\mathbf{I}-\frac{1}{r_{i}(k)}\mathbf{\hat{\Psi}}_{i}(p,k)\mathbf{\hat{\Psi}}_{i}^{ \mathrm{T}}(p,k)\right]\) should be located inside the unit circle. Therefore, a conservative choice of \(\frac{1}{r_{i}(k)}\) is:  
\[\mathbf{V}_{i}(p,k) =[v_{i}(k),\ v_{i}(k-1),\ \cdots,\ v_{i}(k-p+1)]^{\mathrm{ T}}\in\mathbb{R}^{p},\]1  
In this paper, we take a common choice:  
\[\mathbf{V}_{i}(p,k) =[v_{i}(k),\ v_{i}(k-1),\ \cdots,\ v_{i}(k-p+1)]^{\mathrm{ T}}\in\mathbb{R}^{p},\]2  
where \(\lambda_{i}\in(0,\ 1]\) is the forgetting factor.