# Optimal Model-Free Mean-Square Consensus for Multi-Agents with Markov Switching Topology
## 1 Introduction
In [30], an optimal consensus control approach grounded in data was introduced for MASs with switching topologies, featuring an enhanced local neighborhood tracking error (LNTE) system defined through a joint communication graph to estimate agent states. All of the above articles considered the cost of control but analyzed Markov switching indirectly by estimating the state or defining a new system, while the direct analysis of Markov switching is blank. For these reasons, there is substantial research value in the optimal consensus of MASs in switching topology. Table 1 has summarized the main research works surveyed in this section along with the extra details. Consequently, considering the aforementioned analysis, this research introduces a data-based approach to tackle the optimal consensus problem for MASs under switching topologies. This paper makes the following main contributions. 1. In contrast to [29; 30], which estimate or define new tracking error dynamics, this paper introduces a mean-square consensus definition and directly proves the mean-square consistent convergence of the tracking error under Markov switching topology without altering the basic form of the tracking error. The connection between the tracking error and the consensus error is re-evaluated, and it is demonstrated that, when the tracking error achieves mean-square convergence, the consensus error also converges in the mean-square sense. 2. A variable network topology can cause changes in adjacent agents, resulting in the independent variable of the performance function becoming a random variable. We express the state value function in its expected form, which differs from the state value function under fixed topology [14; 15; 16; 17; 18; 19; 20]. Additionally, Ref. [29] also addresses the Markov switching problem, but since we do not use the method of tracking error \begin{table}
\begin{tabular}{c c} \hline \hline
**Sources** & **Contributions** \\ \hline [PERSON], [PERSON] [23] & Two structures of fixed topology and switching topology were proposed. \\ [PERSON], [PERSON] [24] & The condition that the union topology of dynamic network topology has a spanning tree was proposed. \\ [PERSON], [PERSON], [PERSON]. [25] & Addressing the consensus challenge in an undirected and connected network topology. \\ [PERSON], [PERSON]. [26] & The consensus of DT MASs on a joint-connected switching directed graph was studied. \\ [PERSON], [PERSON], [PERSON]. [27] & The mean-square consensus of heterogeneous MASs under Markov switching topology was studied. \\ [PERSON], [PERSON], [PERSON], et al. [28] & A reconstruction mechanism was proposed to deal with the consensus error at the switching instants. \\ [PERSON], [PERSON], [PERSON], et al. [29] & The input delay was added and the tracking error estimation method was proposed. \\ [PERSON], [PERSON], [PERSON], et al. [30] & An enhanced LNTE system was defined to estimate the agent state. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A summary of the main research works on the optimal consensus. estimation and our value function has a different form, its algorithm is not suitable for this scenario. The structure of this manuscript is as outlined below. Section 2 provides an overview of fundamental graph theory and the problem statement. A PI algorithm is detailed in Section 3, including discussions on its convergence and stability properties. Section 4 describes the operation of the PI algorithm using actor-critic NNs. The simulation results are presented in Section 5, and a discussion is given in Section 6. A brief conclusion is provided in Section 7.