# An Effective Adaptive Combination Strategy for Distributed Learning Network
## Appendix A Mean Convergence Analysis  
We define the network weight error  
\[\tilde{\mathbf{w}}(t)\triangleq\text{col}\{\tilde{\mathbf{w}}_{1}(t),\cdots,\tilde{ \mathbf{w}}_{N}(t)\}\in\mathbb{C}^{MN}, \tag{11}\]  
where \(\tilde{\mathbf{w}}_{k}(t)\triangleq\mathbf{w}_{\text{o}}-\mathbf{w}_{k}(t),k=1,2,\cdots,N\). We define the global regressor,  
\[\mathbf{u}(t)\triangleq\text{col}\{\mathbf{u}_{1}(t),\cdots,\mathbf{u}_{N}(t)\}, \tag{12}\]  
and the global covariance matrix,  
\[\mathbf{R}\triangleq\text{diag}\{\mathbf{R}_{1},\cdots,\mathbf{R}_{N}\}, \tag{13}\]  
where the covariance matrix \(\mathbf{R}_{k}\triangleq\mathbb{E}\{\mathbf{u}_{k}(t)\mathbf{u}_{k}^{\text{H}}(t)\},k=1, 2,\cdots,N\). We also introduce the following diagonal matrices,  
\[\mathbf{\mathcal{V}}(t) \triangleq\text{diag}\{v_{1}(t),\cdots,v_{N}(t)\}\otimes\mathbf{I}_{M}, \tag{14}\] \[\mathbf{\mathcal{D}} \triangleq\text{diag}\{\mu_{1},\cdots,\mu_{N}\}\otimes\mathbf{I}_{M}, \tag{15}\]  
and the extended combination matrix  
\[\mathbf{\mathcal{A}}(t)\triangleq\mathbf{A}(t)\otimes\mathbf{I}_{M}. \tag{16}\]  
Following [1, 12], we can obtain the iteration of the network weight error  
\[\tilde{\mathbf{w}}(t)=\hat{\mathbf{F}}(t)\tilde{\mathbf{w}}(t-1)-\mathbf{\mathcal{A}}^{\text{ T}}(t)\mathbf{\mathcal{D}}\mathbf{z}(t), \tag{17}\]  
where \(\hat{\mathbf{F}}(t)\triangleq\mathbf{\mathcal{A}}^{\text{T}}(t)\big{(}\mathbf{I}_{MN}- \mathbf{\mathcal{D}}\hat{\mathbf{R}}(t)\big{)}\) with \(\hat{\mathbf{R}}(t)\triangleq\text{diag}\big{\{}\hat{\mathbf{R}}_{1}(t),\cdots,\hat{ \mathbf{R}}_{N}(t)\big{\}}\) denoting the instantaneous estimate of the global covariance matrix \(\mathbf{R}\) and \(\hat{\mathbf{R}}_{k}(t)\triangleq\mathbf{u}_{k}(t)\mathbf{u}_{k}^{\text{H}}(t)\), and \(\mathbf{z}(t)\triangleq\mathbf{\mathcal{V}}^{s}(t)\mathbf{u}(t)\). Obviously, we have \(\mathbb{E}\{\mathbf{z}(t)\}=\mathbf{0}\). According to Assumption 1 and Assumption 2, taking mathematical expectation of (17) yields  
\[\mathbb{E}\{\tilde{\mathbf{w}}(t)\}=\mathbf{F}(t)\mathbb{E}\{\tilde{\mathbf{w}}(t-1)\}, \tag{18}\]where  
\[\mathbf{F}(t)\triangleq\mathbb{E}\big{\{}\mathbf{\hat{F}}(t)\big{\}}=\mathbb{E}\Big{\{} \mathbf{\mathcal{A}}^{\mathsf{T}}(t)\Big{\}}(\mathbf{I}_{\text{MN}}-\mathbf{\mathcal{D}} \mathbf{R}). \tag{10}\]  
(10) can be further expressed as  
\[\mathbb{E}\{\mathbf{\hat{w}}(t)\}=\mathbf{\mathcal{F}}(t)\mathbb{E}\{\mathbf{\hat{w}}(0)\}, \tag{11}\]  
where \(\mathbf{\mathcal{F}}(t)\triangleq\prod_{i=t}^{1}\mathbf{F}(i)\triangleq\mathbf{F}(t)\cdots \mathbf{F}(1)\). To facilitate our analysis, we herein introduce the submultiplicative matrix norm (A submultiplicative matrix norm satisfies \(\|\mathbf{A}\|\leq\|\mathbf{A}\|\|\mathbf{B}\|\)[23]). For any square matrix \(\mathbf{X}\), and any \(\epsilon>0\), there exists a submultiplicative matrix norm \(\|\cdot\|_{\varrho}\) such that \(\varrho(\mathbf{X})\leq\|\mathbf{X}\|_{\varrho}\leq\varrho(\mathbf{X})+\epsilon\), where \(\varrho(\mathbf{X})\) denotes the spectrum radius of \(\mathbf{X}\)[23; 24]. Accordingly, we have \(\|\mathbf{\mathcal{F}}(t)\|_{\varrho}\leq\prod_{i=t}^{1}\|\mathbf{F}(i)\|_{\varrho}\). Notice that the diffusion LMS algorithm converges if and only if \(\lim_{t\to\infty}\|\mathbf{\mathcal{F}}(t)\|_{\varrho}=0\). Hence, a sufficient condition for the diffusion LMS algorithm to converge is \(\|\mathbf{F}(t)\|_{\varrho}\leq\varrho(\mathbf{F}(t))+\epsilon<1\). Therefore, the diffusion LMS algorithm converges if \(\varrho(\mathbf{F}(t))<1\) for all \(t\) with a sufficiently small \(\epsilon\) chosen. Thus, the diffusion LMS algorithm converges if \(\mathbf{F}(t)\) in (10) is stable at each \(t\). \(\mathbb{E}\Big{\{}\mathbf{\mathcal{A}}^{\mathsf{T}}(t)\Big{\}}\) in (10) is left-stochastic since each column of it sums to 1. Thus, according to ([1], Appexdix I), \(\mathbf{F}(t)\) is stable if, and only if \(\mathbf{I}_{\text{MN}}-\mathbf{\mathcal{D}}\mathbf{R}\) is stable, i.e., \(\max\{|1-\lambda(\mathbf{\mathcal{D}}\mathbf{R})|\}<1\).