# Optimal Model-Free Mean-Square Consensus for Multi-Agents with Markov Switching Topology
## 3 Main Results
### Stability Analysis  
**Theorem 2**.: _For \(\forall i\), the \(V_{i}^{*}(e_{i}(k))\) and \(u_{i}^{*}(k)\) satisfy (21) and (14), respectively. Then, the LNTE \(e_{i}(k)\) are mean-square convergent, i.e., \(E[\|e_{i}(k)\|^{2}]\to 0\), when \(k\to\infty\), and the leader and all of the agents achieve consensus._  
Proof.: Considering that \(V_{i}^{*}(e_{i}(k))\) meets (26), it yields that  
\[V_{i}^{*}(e_{i}(k))-\gamma V_{i}^{*}(e_{i}(k+1))=E\big{[}r(e_{i}(k),u_{i}^{*} (k),u_{-i}^{*}(k))\big{]}. \tag{47}\]  
Then,  
\[\gamma^{k}V_{i}^{*}(e_{i}(k))-\gamma^{k+1}V_{i}^{*}(e_{i}(k+1))=\gamma^{k}E \big{[}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\big{]}. \tag{48}\]  
Choose \(\gamma^{k}V_{i}^{*}(e_{i}(k))\) as the Lyapunov function candidate and define the difference as  
\[\triangle(\gamma^{k}V_{i}^{*}(e_{i}(k)))=\gamma^{k+1}V_{i}^{*}(e_{i}(k+1))- \gamma^{k}V_{i}^{*}(e_{i}(k)). \tag{49}\]  
According to the definition of \(r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\) in (15), one can obtain  
\[\triangle(\gamma^{k}V_{i}^{*}(e_{i}(k)))=-\gamma^{k}E\big{[}r(e_{i}(k),u_{i}^ {*}(k),u_{-i}^{*}(k))\big{]}<0. \tag{50}\]  
From (50), it is evident that the LNTE \(e_{i}(k)\) is asymptotically stable at \(u_{i}^{*}(k)\) for \(\forall i\), i.e., \(e_{i}(k)\to 0\) as \(k\to\infty\).  
Choose a large enough \(H\); for all \(k\) from \(0\) to \(H\), sum the differences to obtain  
\[\begin{split}\sum_{k=0}^{H}\triangle(\gamma^{k}V_{i}^{*}(e_{i}(k )))&=-\sum_{k=0}^{H}\gamma^{k}E\big{[}r(e_{i}(k),u_{i}^{*}(k),u_{ -i}^{*}(k))\big{]}\\ &=-\sum_{k=0}^{H}\gamma^{k}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k ))\cdot Pr\{\sigma(k)=\tau\}.\end{split} \tag{51}\]  
Since \(r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\) is positive definite, the cumulation \(\sum_{k=0}^{H}\gamma^{k}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\cdot Pr\{ \sigma(k)=\tau\}\) is the sum of a series of non-negative numbers. If we can prove that this sum is finite, then we can say that the cumulative increment in the Lyapunov function is finite. If \(\lim_{k\to\infty}e_{i}(k)=0\), based on the Barbalat Lemma, \(\lim_{k\to\infty}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))=0\). Hence, for sufficiently large \(H,\sum_{k=0}^{H}\gamma^{k}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\cdot Pr\{ \sigma(k)=\tau\}\) can be made arbitrarily small.  
Since \(\sum_{k=0}^{H}\triangle(\gamma^{k}V_{i}^{*}(e_{i}(k)))\) is non-positive and \(\sum_{k=0}^{H}\gamma^{k}r(e_{i}(k),u_{i}^{*}(k),u_{-i}^{*}(k))\cdot Pr\{ \sigma(k)=\tau\}\) can be arbitrarily small, this means that \(\sum_{k=0}^{\infty}\triangle(\gamma^{k}V_{i}^{*}(e_{i}(k)))\) must be finite. Otherwise, if this sum is infinite, then \(V_{i}^{*}(e_{i}(k))\) would grow without a bound, denying the boundedness of \(V_{i}^{*}(e_{i}(k))\). Then, \(\lim_{k\to\infty}V_{i}^{*}(e_{i}(k))=0\).  
It is easy to obtain the following as  
\[e_{i}^{T}(k)Q_{ii}e_{i}(k)\geqslant\lambda(Q_{ii})e_{i}^{T}(k)e_{i}(k)=\lambda_{ min}(Q_{ii})\|e_{i}(k)\|^{2}, \tag{52}\]  
where \(\lambda_{min}(Q_{ii})\) is the smallest eigenvalue of \(Q_{ii}\). Then,  
\[E[\|e_{i}(k)\|^{2}] \leqslant\frac{E[e_{i}^{T}(k)Q_{ii}e_{i}(k)]}{\lambda_{min}(Q_{ii})} \tag{53}\] \[\leqslant\frac{E[e_{i}^{T}(k)Q_{ii}e_{i}(k)+(u_{i}^{*}(k))^{T}R_{ii }u_{i}^{*}(k)+\sum_{j\in\mathcal{N}_{i}}(u_{j}^{*}(k))^{T}R_{ji}u_{j}^{*}(k)]}{ \lambda_{min}(Q_{ii})}\] \[\leqslant\frac{V_{i}^{*}(k)}{\lambda_{min}(Q_{ii})}.\]  
Therefore, it can be obtained that  
\[\lim_{k\to\infty}E[\|e_{i}(k)\|^{2}]\leqslant\frac{\lim_{k\to\infty}V_{i}^{*} (k)}{\lambda_{min}(Q_{ii})}=0. \tag{54}\]  
From Lemma 1, the agents will eventually reach an agreement with the leader if all of the LNTEs converge to zero.