# Optimal Model-Free Mean-Square Consensus for Multi-Agents with Markov Switching Topology
## 2 Preliminaries
### Problem Formulation  
Suppose that there are \(N\) agents in a discrete-time MAS and that the dynamics of agents are described as  
\[x_{i}(k+1)=Ax_{i}(k)+B_{i}u_{i}(k),~{}~{}i=1,2,\ldots,N, \tag{1}\]  
where \(x_{i}(k)\in\mathbb{R}^{n}\) is the state and \(u_{i}(k)\) is the control input. The system parameters are \(A\in\mathbb{R}^{n\times n}\) and \(B_{i}\in\mathbb{R}^{n\times m}\), and they are not known. The definition of a reference system, often known as the leader, is  
\[x_{0}(k+1)=Ax_{0}(k), \tag{2}\]  
where the target state of consensus is \(x_{0}(k)\in\mathbb{R}^{n}\). For \(\forall i\), the followers and leader are said to reach consensus if \(x_{i}(k)\) satisfies  
\[lim_{k\rightarrow\infty}\|x_{i}(k)-x_{0}(k)\|=0~{}~{},i=1,2,\ldots,N. \tag{3}\]According to (3), the consensus error \(\xi(k)\) is defined as  
\[\xi(k)=x(k)-\bar{x}_{0}(k). \tag{4}\]  
where \(x(k)=[x_{1}^{T}(k)\ x_{2}^{T}(k)\ \cdots\ x_{N}^{T}(k)]^{T}\in\mathbb{R}^{N\times n}\) and \(\bar{x}_{0}(k)=[x_{0}^{T}(k)\ x_{0}^{T}(k)\ \cdots\ x_{0}^{T}(k)]^{T}\in\mathbb{R}^{N \times n}\). **Definition 1**. : _(Mean-Square Consensus) [27] For MASs (1) and (2) under Markovian switching topologies, the control protocols achieve mean-square consensus if the following criteria are met:_  
\[\lim_{k\to\infty}\mathbb{E}\Big{[}\|x_{i}(k)-x_{0}(k)\|^{2}\Big{]}\ =\ 0,\quad i=1, \ldots,N, \tag{5}\]  
For agent \(i\), define the LNTE as  
\[e_{i}(k)=\sum_{j\in\mathcal{N}_{i}}a_{ij}^{\sigma(k)}(x_{i}(k)-x_{j}(k))+b_{i }^{\sigma(k)}(x_{i}(k)-x_{0}(k)). \tag{6}\]  
A summary of the overall tracking error vector is  
\[e(k)=((\mathcal{L}^{\sigma(k)}+\mathcal{B}^{\sigma(k)})\otimes I_{n})(x(k)- \bar{x}_{0}(k)), \tag{7}\]  
where \(e(k)=[e_{1}^{T}(k)\ e_{2}^{T}(k)\ \cdots\ e_{N}^{T}(k)]^{T}\in\mathbb{R}^{N \times n}\); \(\otimes\) denotes the Kronecker product; and \(I_{n}\) is the \(n\)-dimension identity matrix. On the basis of (6), one may derive the dynamics of the LNTE as  
\[e_{i}(k+1)=Ae_{i}(k)+(l_{ii}^{\sigma(k)}+b_{i}^{\sigma(k)})B_{i}u_{i}(k)-\sum_ {j\in\mathcal{N}_{i}}a_{ij}^{\sigma(k)}B_{j}u_{j}(k). \tag{8}\]  
Further, (7) can be rewritten as  
\[e(k)=((\mathcal{L}^{\sigma(k)}+\mathcal{B}^{\sigma(k)})\otimes I_{n})\xi(k). \tag{9}\]  
In terms of (9), the link between \(e(k)\) and \(\xi(k)\) is as follows. **Lemma 1**. : _Suppose that \(\sigma(k)\) is a homogeneous and ergodic Markov chain with state space \(\mathcal{F}=\{1,2,\ldots,f\}\). If \(\lim_{k\to\infty}E[\|e(k)\|^{2}]=0\); then, \(\lim_{k\to\infty}E[\|\xi(k)\|^{2}]=0\)._  
Proof. : Set \(\Lambda^{\sigma(k)}\triangleq(\mathcal{L}^{\sigma(k)}+\mathcal{B}^{\sigma(k )})\otimes I_{n}\); then,  
\[E[e(k)e(k)^{T}] =E[\Lambda^{\sigma(k)}\xi(k)\xi(k)^{T}(\Lambda^{\sigma(k)})^{T}]\] \[x_{0}(k+1)=Ax_{0}(k), \tag{2}\]0 \[x_{0}(k+1)=Ax_{0}(k), \tag{2}\]1 \[x_{0}(k+1)=Ax_{0}(k), \tag{2}\]2  
where \(\tau\) is any value of \(\mathcal{F}\). Since \(\{\sigma(k),\ k\in\mathbb{N}^{+}\}\) is ergodic, there is a stationary distribution, denoted by \(\pi=[\pi(1),\ldots,\pi(\tau),\ldots,\pi(f)]^{T}\), where \(\pi(\tau)>0\) for \(\forall\tau\in\mathcal{F}\). Therefore, it follows from the ergodicity of the Markov chain that \(\lim_{k\to\infty}Pr\{\sigma(k)=\tau\}=\pi(\tau)\). From (10), it has  
\[x_{0}(k+1)=Ax_{0}(k), \tag{2}\]3Furthermore, it follows from (10) that  
\[ \begin{split} E[\|e(k)\|^{2}]&=tr(E[e(k)e(k)^{T}])\\ &=tr(\sum_{\tau\in\mathcal{F}}\Lambda^{\tau}E[\xi(k)\xi(k)^{T}]( \Lambda^{\tau})^{T}\cdot Pr\{\sigma(k)=\tau\})\\ &=tr(E[\xi(k)\xi(k)^{T}]\sum_{\tau\in\mathcal{F}}\Lambda^{\tau}( \Lambda^{\tau})^{T}\cdot Pr\{\sigma(k)=\tau\}).\end{split}  \tag{12}\]  
Then,  
\[\lim_{k\to\infty}E[\|e(k)\|^{2}]=\lim_{k\to\infty}tr(E[\xi(k)\xi(k)^{T}]\sum_{ \tau\in\mathcal{F}}\Lambda^{\tau}(\Lambda^{\tau})^{T}\cdot\pi(\tau)). \tag{13}\]  
The joint connected topology \(\mathcal{G}_{im}=\bigcup_{t\in\mathcal{F}}G_{\tau}\) has a directed spanning tree, so \(\sum_{\tau\in\mathcal{F}}\Lambda^{\tau}(\Lambda^{\tau})^{T}\) is nonsingular [24]. In addition, \(\pi(\tau)>0\).