SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\lambda_ {2} \leq \frac {2 \tau (e) \ln 2 \frac {1}{5} \delta}{2 \tau (e) \ln 2 e + 1}. \tag {7}
$$
Again by Proposition 2, we have
$$
\left| \left| \pi^ {\tau (e)} (\cdot | G) - \pi^ {*} (\cdot) \right| \right| _ {T V} \leq 2 \left(p ^ {*}\right) ^ {\tau (e)} = e.
$$
Consequently, we arrive at
$$
\tau (e) = \frac {\ln e}{\ln 2 p ^ {*}}
$$
$$
\lambda_ {2} \leq \frac {2 \frac {\ln e}{\ln 2 p ^ {*}} \ln 2 e}{2 \frac {\ln e}{\ln 2 p ^ {*}} \ln 2 e + 1}.
$$
by plugging  $\tau (e)$  into (7).
This completes the proof of the lower bound on the spectral gap  $1 - \lambda_{2}$ .
In the following proposition, we show the sufficient condition for graphs generated by the E-R model being connected.
Proposition 5. Assume  $c$  in setting 1 meets the condition
$$
1 \geq c \geq \frac {1}{2} + \frac {1}{2} \sqrt {1 - (\frac {\epsilon}{M T}) ^ {\frac {2}{M - 1}}},
$$
where  $0 < \epsilon < 1$ . Then, with probability  $1 - \epsilon$ , for any  $t > 0$ ,  $G_{t}$  following the E-R model is connected.
Proof. For  $1 \leq j \leq M$ , we denote the degree of client  $j$  as  $d_j$ .
It is straightforward to have 1)  $\sum_{j=1}^{M} d_{j} = 2 \cdot$  total number of edges, 2)  $E[\text{total number of edges}] = c \cdot \frac{M(M-1)}{2}$  and 3) random variables  $d_{1}, d_{2}, \ldots, d_{M}$  are dependent but follow the same distribution.
Note that  $d_{j}$  follows a binomial distribution with  $E[d_{j}] = c \cdot (M - 1)$  where  $c$  is the probability of an edge. Then by the Chernoff Bound inequality, we have
$$
P (d _ {j} <   \frac {M - 1}{2}) \leq \exp \left\{- (M - 1) \cdot K L (0. 5 | | c) \right\}
$$
where  $KL(0.5||c)$  denotes the KL divergence between Bernoulli(0.5) and Bernoulli(c).
For the term  $KL(0.5||c)$ , we can further show that
$$
K L (0. 5 | | c) = \frac {1}{2} \log \frac {\frac {1}{2}}{c} + \frac {1}{2} \log \frac {\frac {1}{2}}{1 - c} = \frac {1}{2} \log \frac {1}{4 c (1 - c)}
$$
which leads to  $P(d_{j} < \frac{M - 1}{2}) \leq \exp \left\{(M - 1) \cdot \frac{1}{2} \log 4c(1 - c)\right\}$ .
Meanwhile, we have specified the choice of  $c$  as
$$
\frac {1}{2} + \frac {1}{2} \sqrt {1 - \left(\frac {\epsilon}{M T}\right) ^ {\frac {2}{M - 1}}} \} \leq c <   1
$$
which guarantees  $\exp \left\{(M - 1)\cdot \frac{1}{2}\log 4c(1 - c)\right\} \leq \frac{\epsilon}{MT}$  as follow. We observe that
$$
\begin{array}{l} c \geq \frac {1}{2} + \frac {1}{2} \sqrt {1 - (\frac {\epsilon}{M T}) ^ {\frac {2}{M - 1}}} \\ \Longrightarrow 4 c (1 - c) \leq \left(\frac {\epsilon}{M T}\right) ^ {\frac {2}{M - 1}} \\ \Longrightarrow \log 4 c (1 - c) \leq \frac {2 \log \frac {\epsilon}{M T}}{M - 1} \\ \Longrightarrow (M - 1) \cdot \frac {1}{2} \log 4 c (1 - c) \leq \log \frac {\epsilon}{M T} \\ \Longrightarrow \exp \left\{(M - 1) \cdot \frac {1}{2} \log 4 c (1 - c) \right\} \leq \frac {\epsilon}{M T}. \\ \end{array}
$$
This is summarized as for any  $j$
$$
P \left(d _ {j} <   \frac {M - 1}{2}\right) \leq \exp \left\{\left(M - 1\right) \cdot \frac {1}{2} \log 4 c (1 - c) \right\} \leq \frac {\epsilon}{M T}. \tag {8}
$$
Meanwhile, it is known as if  $\delta(G_t) \geq \frac{M - 1}{2}$ , then we have that graph  $G_t$  is connected where  $\delta(G_t) = \min_m d_m$ .
As a result, consider the probability and we obtain that
$$
\begin{array}{l} P (\text {g r a p h} G _ {t} \text {i s c o n n e c t e d}) \\ \geq P \left(\min  _ {j} d _ {j} \geq \frac {M - 1}{2}\right) \\ = P \left(\bigcap_ {j} \left\{d _ {j} \geq \frac {M - 1}{2} \right\}\right) \\ = 1 - P \left(\bigcup_ {j} \left\{d _ {j} <   \frac {M - 1}{2} \right\}\right) \\ \geq 1 - \sum_ {j} P \left(d _ {j} <   \frac {M - 1}{2}\right) \\ = 1 - M P \left(d _ {j} <   \frac {M - 1}{2}\right) \\ \geq 1 - M \frac {\epsilon}{M T} = 1 - \frac {\epsilon}{T} \\ \end{array}
$$
where the second inequality holds by the Bonferroni's inequality and the third inequality uses (8).
Consequently, we obtain
$$
\begin{array}{l} P (\text {g r a p h} G _ {t} \text {i s c o n n e c t e d}) \\ = P \left(\cap_ {t} \{G _ {t} \text {i s c o n n e c t e d} \}\right) \\ \geq 1 - \sum_ {t} P \left(G _ {t} \text {i s n o t c o n n e c t e d}\right) \\ = 1 - \sum_ {t} (1 - P \left(G _ {t} \text {i s c o n n e c t e d}\right)) \\ \geq 1 - \sum_ {t} (1 - (1 - \frac {\epsilon}{T}) = 1 - \epsilon \\ \end{array}
$$
where the first inequality holds again by the Bonferroni's inequality and the second inequality results from the above derivation.
On graphs with the established properties, we next show the results on the transmission gap between two consecutive rounds of communication for any two clients and the number of arm pulls for all clients.
Proposition 6. We have that with probability  $1 - \epsilon$ , for any  $t > L$  and any  $m$ , there exists  $t_0$  such that
$$
t + 1 - \min _ {j} t _ {m, j} \leq t _ {0}, t _ {0} \leq c _ {0} \min _ {l} n _ {l, i} (t + 1)
$$
where  $c_{0} = c_{0}(K,\min_{i\neq i^{*}}\Delta_{i},M,\epsilon ,\delta)$
Proof. The edges in setting 1 follow a Bernoulli distribution with a given parameter  $c$  by definition. Though setting 2 does not explicitly define the edge distribution, the probability of an edge existing in a connected graph, denoted as  $c$ , is deterministic, independent of time since graphs are i.i.d. over time and homogeneous among edges.
Henceforth, it is straightforward that  $c$  satisfies
$$
\frac {M (M - 1)}{2} c = E (N)
$$
and equivalently  $c = \frac{2E(N)}{M(M - 1)}$  where  $N$  denotes the number of edges in a random connected graph.
We observe that  $0 \leq N \leq \frac{M(M - 1)}{2}$ . Furthermore, the existing result in [Trevisan] yields
$$
E [ N ] = M \log M.
$$
Consequently, the probability term  $c$  has an explicit expression  $c = \frac{2E[N]}{M(M - 1)} = \frac{2\log M}{M - 1}$ .
For setting  $s_2, S_2$  we have  $c = \frac{2\log M}{M - 1} \geq \frac{1}{2}$  since  $M < 10$ , while in setting  $s_1, S_1$ , the condition on  $c$  guarantees  $c > \frac{1}{2}$ . Note that  $t + 1 - t_{m,j}$  follows a geometric distribution since each edge follows a Bernoulli distribution, which holds by
$$
\begin{array}{l} P (t + 1 - t _ {m, j} = 1 | t _ {m, j}) \\ = \frac {P (\text {t h e r e i s a n e d g e b e t w e e n m a n d j a t t i m e s t p t + 1 a n d} t _ {m , j})}{P (\text {t h e r e i s a n e d g e b e t w e e n m a n d j a t t i m e s t p t} _ {m , j})} \\ = \frac {(c) ^ {2}}{c} = c \\ \end{array}
$$
$$
\begin{array}{l} P (t + 1 - t _ {m, j} = k \mid t _ {m, j}) \\ = \frac {P (\text {t h e r e i s a n e d g e b e t w e e n m a n d j a t t i m e s t e p t + k a n d} t _ {m , j} , \text {n o e d g e a t t i m e s t e p t} + 1 , \dots , t + k - 1)}{P (\text {t h e r e i s a n e d g e b e t w e e n m a n d j a t t i m e s t e p} t _ {m , j})} \\ = \frac {(1 - c) ^ {k - 1} c ^ {2}}{c} = c (1 - c) ^ {k - 1}. \\ \end{array}
$$
Note that  $P(t + 1 - \min_{j} t_{m,j} \geq t_0)$  which denotes the tail of a geometric distribution depends on the choice of  $c$ . More precisely, the tail probability  $P_0$  is monotone decreasing in  $c$ .
When  $c = \frac{1}{2}$ , we obtain that
$$
P _ {0} = P \left(t + 1 - \min  _ {j} t _ {m, j} > t _ {0}\right) \leq \sum_ {s > t _ {0}} \left(\frac {1}{2}\right) ^ {s} \leq \left(\frac {1}{2}\right) ^ {t _ {0}}. \tag {9}
$$
Choosing  $t_0 = \frac{\ln\frac{M^2T}{\epsilon}}{\ln 2}$  leads to  $P_0 = 1 - \left(\frac{1}{2}\right)^{t_0} = 1 - \frac{\epsilon}{M^2T}$  and
$$
\begin{array}{l} c _ {0} \min  _ {l} n _ {l, i} (t + 1) \geq c _ {0} \min  _ {l} n _ {l, i} (L) \\ \geq c _ {0} \frac {L}{K} \geq c _ {0} \frac {\ln \frac {M ^ {2} T}{\epsilon}}{c _ {0} \ln 2} = \frac {\ln \frac {M ^ {2} T}{\epsilon}}{\ln 2} = t _ {0} \\ \end{array}
$$
where the last inequality holds by the choice of  $L$ . This implies  $\min_{l} n_{l,i}(t + 1) - t_0 \geq (1 - c_0) \min_{l} n_{l,i}(t + 1)$ , i.e.  $t_0 \leq c_0 \min_{l} n_{l,i}(t + 1)$ .
Therefore, with probability  $1 - \frac{\epsilon}{M^2T}$
$$
\begin{array}{l} \min _ {l} n _ {l, i} (t _ {m, l}) \\ \geq \min  _ {l} n _ {l, i} (\min  _ {j} t _ {m, j}) \\ \geq \min  _ {l} n _ {l, i} (t + 1 - t _ {0}) \\ \geq \min  _ {l} n _ {l, i} (t + 1) - t _ {0} \\ \geq \left(1 - c _ {0}\right) \cdot \min  _ {l} n _ {l, i} (t + 1) \\ \end{array}
$$
where the first inequality results from the fact  $t_{m,l} \geq \min_j t_{m,j}$ , the second inequality uses the fact from (9), the third inequality applies the definition of  $n$ , and the last inequality holds by the choice of  $t_0$ .
Consider setting  $s_3, S_3$  where  $M > 10$ . Generally, for a given parameter  $c$ , we obtain
$$
\begin{array}{l} P (t + 1 - \min  _ {j} t _ {m, j} = 1) = c, \\ P (t + 1 - \min  _ {j} t _ {m, j} = 2) = c (1 - c), \\ \end{array}
$$
$$
P (t + 1 - \min  _ {j} t _ {m, j} = n) = c (1 - c) ^ {n - 1}
$$
$$
P _ {0} = P \big (t + 1 - \min _ {j} t _ {m, j} > t _ {0} \big) \leq \sum_ {s > t _ {0}} c (1 - c) ^ {s - 1} \leq c (\frac {1}{c} - \frac {1 - (1 - c) ^ {t _ {0}}}{c}) = (1 - c) ^ {t _ {0}}.
$$
For the probability term  $P_0$ , we further arrive at
$$
P _ {0} \geq 1 - \frac {\epsilon}{M ^ {2} T}
$$
by the choice of  $t_0 \geq \frac{\ln\left(\frac{\epsilon}{M^2T}\right)}{\ln(1 - c)}$ .
Meanwhile, we claim that the choice of  $t_0$  satisfies
$$
t _ {0} \leq c _ {0} \min  _ {l} n _ {l, i} (t + 1)
$$
since  $\frac{\ln(\frac{\epsilon}{M^2T})}{\ln(1 - c)} \leq c_0 \min_l n_{l,i}(t + 1)$  holds by noting  $n_{l,i}(t + 1) \geq n_{l,i}(L) \geq \frac{L}{K}$  and
$$
L \geq \frac {K \ln (\frac {\epsilon}{M ^ {2} T})}{c _ {0} \ln (1 - c)} = \frac {K \ln (\frac {M ^ {2} T}{\epsilon})}{c _ {0} \ln (\frac {1}{1 - c})}.
$$
To summarize, in all the settings, we have that with probability at least  $1 - \frac{\epsilon}{M^2T}$ ,
$$
t + 1 - \min  _ {j} t _ {m, j} \leq t _ {0},
$$
$$
t _ {0} \leq c _ {0} \min  _ {l} n _ {l, i} (t + 1). \tag {10}
$$
Therefore, we obtain that in setting  $s_1, S_1$
$$
P (\forall m, t + 1 - \min  _ {j} t _ {m, j} \leq t _ {0} \leq c _ {0} \min  _ {l} n _ {l, i} (t + 1))
$$
$$
\dot {=} (1 - P _ {0}) ^ {M} \geq 1 - M P _ {0} = 1 - \frac {\epsilon}{M T} \tag {11}
$$
where the inequality is a result of the Bernoulli's inequality.
In setting  $s_2, S_2, s_3, S_3$ ,  $\{t + 1 - \min_j t_{1,j}, \dots, t + 1 - \min_j t_{M,j}\}$  follow the same distribution, but are dependent since they construct a connected graph. However, we have the following result
$$
\begin{array}{l} P (\forall m, t + 1 - \min _ {j} t _ {m, j} \leq t _ {0} \leq c _ {0} \min _ {l} n _ {l, i} (t + 1)) \\ = 1 - P (\cup_ {m} \{t + 1 - \min _ {j} t _ {m, j} \geq t _ {0} \}) \\ \geq 1 - \sum_ {m} P (t + 1 - \min  _ {j} t _ {m, j} \geq t _ {0}) \\ = 1 - M P _ {0} = 1 - \frac {\epsilon}{M T} \tag {12} \\ \end{array}
$$
by the Bonferroni's inequality.
As a consequence, we arrive at that in setting  $s_1, S_1, s_2, S_2, s_3, S_3$ ,
$$
\begin{array}{l} P (\forall t, \forall m, t + 1 - \min  _ {j} t _ {m, j} \leq t _ {0} \leq c _ {0} \min  _ {l} n _ {l, i} (t + 1)) \\ \geq 1 - \sum_ {t} \sum_ {m} P (t + 1 - \min  _ {j} t _ {m, j} \leq t _ {0} \leq c _ {0} \min  _ {l} n _ {l, i} (t + 1)) \\ \geq 1 - M T \left(1 - \left(1 - \frac {\epsilon}{M T}\right)\right) = 1 - \epsilon \\ \end{array}
$$
where the first inequality again uses the Bonferroni's inequality and the second inequality holds by applying (11) and (12).
After establishing the transmissions among clients, we next proceed to show the concentration properties of the network-wide estimators maintained by the clients.
The first is to demonstrate the unbiasedness of these estimators with respect to the global expected rewards.
Proposition 7. Assume the parameter  $\delta$  satisfies that  $0 < \delta < c = f(\epsilon, M, T)$ . For any arm  $i$  and any client  $m$ , at every time step  $t$ , we have
$$
E [ \tilde {\mu} _ {i} ^ {m} (t) | A _ {\epsilon , \delta} ] = \mu_ {i}.
$$
Proof. The result can be shown by induction as follows. We start with the basis step by considering any time step  $t \leq L + 1$ . By the definition of  $\tilde{\mu}_i^m(t) = \tilde{\mu}_i^m(L + 1)$ , we arrive at
$$
\begin{array}{l} E [ \tilde {\mu} _ {i} ^ {m} (t) | A _ {\epsilon , \delta} ] \\ = E \left[ \tilde {\mu} _ {i} ^ {m} (L + 1) \mid A _ {\epsilon , \delta} \right] \\ = E \left[ \sum_ {j = 1} ^ {M} P _ {m, j} ^ {\prime} (L) \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) \mid A _ {\epsilon , \delta} \right] \tag {13} \\ \end{array}
$$
where  $P_{m,j}^{\prime}(L) = \left\{ \begin{array}{ll}\frac{1}{M}, & \text{if } P_L(m,j) > 0\\ 0, & \text{else} \end{array} \right.$ . The definition of  $A_{\epsilon ,\delta}$  and the choice of  $\delta$  guarantee that  $|P_L - cE| < \delta < c$  on event  $A_{\epsilon ,\delta}$ , i.e. we have for any  $t\geq L$ ,  $P_{t} > 0$  and thereby obtaining
$$
P _ {m, j} ^ {\prime} (L) = \frac {1}{M}. \tag {14}
$$
Therefore, we continue with (13) and have
$$
\begin{array}{l} (1 3) = E [ \sum_ {j = 1} ^ {M} \frac {1}{M} \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) | A _ {\epsilon , \delta} ] \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E \left[ \bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) \mid A _ {\epsilon , \delta} \right] \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E \left[ \sum_ {n _ {j, i}} r _ {i} ^ {j} (s) \right. | A _ {\epsilon , \delta} ] \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E [ E [ \sum_ {n _ {j, i}} r _ {i} ^ {j} (s) ] | \sigma (n _ {j, i} (l)) _ {l \leq h _ {m, j} ^ {L}}, A _ {\epsilon , \delta} | A _ {\epsilon , \delta} ] \\ \end{array}
$$
where the last equality uses the law of total expectation.
With the derivations, we further have
$$
\begin{array}{l} (13) = \frac{1}{M}\sum_{j = 1}^{M}E\left[\frac{1}{n_{j,i}(h_{m,j}^{L})} E\left[\sum_{s:n_{j,i}(s) - n_{j,i}(s - 1) = 1}r_{i}^{j}(s)|\sigma (n_{j,i}(l))_{l\leq h_{m,j}^{L}},A_{\epsilon ,\delta}|A_{\epsilon ,\delta}\right.\right. \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E \left[ \frac {1}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \sum_ {s: n _ {j, i} (s) - n _ {j, i} (s - 1) = 1} E \left[ r _ {i} ^ {j} (s) \mid \sigma \left(n _ {j, i} (l)\right) _ {l \leq h _ {m, j} ^ {L}}, A _ {\epsilon , \delta} \mid A _ {\epsilon , \delta} \right] \right. (15) \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E \left[ \frac {1}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \sum_ {s: n _ {j, i} (s) - n _ {j, i} (s - 1) = 1} \mu_ {i} ^ {j} \mid A _ {\epsilon , \delta} \right] (16) \\ = \frac {1}{M} \sum_ {j = 1} ^ {M} E \left[ \mu_ {i} ^ {j} \mid A _ {\epsilon , \delta} \right] = \mu_ {i} (17) \\ \end{array}
$$
where the second equality (15) uses the fact that  $\{s:n_{j,i}(s) - n_{j,i}(s - 1) = 1\}$  is contained in  $\sigma (n_{j,i}(l))_{l\leq L_{m,j}}$  and the third equality (16) results from that  $r_i^j (s)$  is independent of everything else given  $s$  and  $E[r_i^j (s)] = \mu_i^j$
The induction step follows a similar analysis as follows. Suppose that for any  $s \leq t$  we have  $E[\tilde{\mu}_i^m (s)|A_{\epsilon ,\delta}] = \mu_i$ .
For time step  $t + 1$ , we first write it as
$$
\begin{array}{l} E [ \tilde {\mu} _ {i} ^ {m} (t + 1) | A _ {\epsilon , \delta} ] \\ = E \left[ \sum_ {j = 1} ^ {M} P _ {t} ^ {\prime} (m, j) \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(t _ {m, j}\right) + d _ {m, t} \sum_ {j \in N _ {m} (t)} \hat {\tilde {\mu}} _ {i, j} ^ {m} (t) + d _ {m, t} \sum_ {j \notin N _ {m} (t)} \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(t _ {m, j}\right) \mid A _ {\epsilon , \delta} \right] \\ = E \left[ E \left[ \sum_ {j = 1} ^ {M} P _ {t} ^ {\prime} (m, j) \tilde {\mu} _ {i} ^ {j} \left(t _ {m, j}\right) + d _ {m, t} \sum_ {j \in N _ {m} (t)} \bar {\mu} _ {i} ^ {j} (t) + d _ {m, t} \sum_ {j \notin N _ {m} (t)} \bar {\mu} _ {i} ^ {j} \left(t _ {m, j}\right) \mid \sigma \left(n _ {j, i} (t)\right) _ {j, i, t}, A _ {\epsilon , \delta} \mid A _ {\epsilon , \delta} \right] \right. \tag {18} \\ \end{array}
$$
where  $P_{t}^{\prime}(m,j)$  and  $d$  are constants since  $P_{t}(m,j) > 0$  for  $t\geq L$  on event  $A_{\epsilon ,\delta}$  and the last equality is again by the law of total expectation.
This gives us that by the law of total expectation
$$
\begin{array}{l} (1 8) = E \left[ \sum_ {j = 1} ^ {M} P _ {t} ^ {\prime} (m, j) E \left[ \tilde {\mu} _ {i} ^ {j} \left(t _ {m, j}\right) \mid \sigma \left(n _ {j, i} (t)\right) \right. _ {j, i, t}, A _ {\epsilon , \delta} + \right. \\ d _ {m, t} \sum_ {j \in N _ {m} (t)} E [ \bar {\mu} _ {i} ^ {j} (t) | \sigma (n _ {j, i} (t)) _ {j, i, t}, A _ {\epsilon , \delta} + \\ d _ {m, t} \sum_ {j \notin N _ {m} (t)} E \left[ \bar {\mu} _ {i} ^ {j} \left(t _ {m, j}\right) \mid \sigma \left(n _ {j, i} (t)\right) _ {j, i, t}, A _ {\epsilon , \delta} \mid A _ {\epsilon , \delta} \right] \\ = \sum_ {j = 1} ^ {M} P ^ {\prime} (m, j) E [ E [ \tilde {\mu} _ {i} ^ {j} (t _ {m, j}) | \sigma (n _ {j, i} (t)) _ {j, i, t}, A _ {\epsilon , \delta} | A _ {\epsilon , \delta} ] + \\ E \left[ d _ {m, t} \sum_ {j \in N _ {m} (t)} E \left[ \bar {\mu} _ {i} ^ {j} (t) \mid \sigma \left(n _ {j, i} (t)\right) _ {j, i, t}, A _ {\epsilon , \delta} + \right. \right. \\ d _ {m, t} \sum_ {j \notin N _ {m} (t)} E \left[ \bar {\mu} _ {i} ^ {j} \left(t _ {m, j}\right) \mid \sigma \left(n _ {j, i} (t)\right) _ {j, i, t}, A _ {\epsilon , \delta} \mid A _ {\epsilon , \delta} \right] \\ = \sum_ {j = 1} ^ {M} P ^ {\prime} (m, j) E [ \tilde {\mu} _ {i} ^ {j} (t _ {m, j}) | A _ {\epsilon , \delta} ] + E [ d _ {m, t} \sum_ {j} E [ \bar {\mu} _ {i} ^ {j} (t _ {m, j}) | \sigma (n _ {j, i} (t)) _ {j, i, t}, A _ {\epsilon , \delta} | A _ {\epsilon , \delta} ] \\ = \sum_ {j = 1} ^ {M} P ^ {\prime} (m, j) E [ \tilde {\mu} _ {i} ^ {j} (t _ {m, j}) | A _ {\epsilon , \delta} ] + \\ d _ {m, t} \sum_ {j} E [ E [ \frac {1}{n _ {j , i} (t _ {m , j})} \sum_ {s: n _ {j, i} (s) - n _ {j, i} (s - 1) = 1} E \left[ r _ {i} ^ {j} (s) \mid \sigma \left(n _ {j, i} (t)\right) _ {j, i, t}, A _ {\epsilon , \delta} \mid A _ {\epsilon , \delta} \right] \\ = \sum_ {j = 1} ^ {M} P ^ {\prime} (m, j) \mu_ {i} + d _ {m, t} M \mu_ {i} = \left(\sum_ {j = 1} ^ {M} P ^ {\prime} (m, j) + M d _ {m, t}\right) \mu_ {i} = \mu_ {i} \\ \end{array}
$$
where the first equality uses  $P_{t}^{\prime}(m,j)$  and  $d$  are constants on event  $A_{\epsilon,\delta}$ , the second equality is derived by re-organizing the terms, the third equality again uses the law of total expectation and integrates the second term by  $t_{m,j}$ , the fourth equality elaborate the second term and the equality in the last line follows from the induction and (15, 16 17).
This completes the induction step and thus shows the unbiasedness of the network-wide estimators conditional on event  $A_{\epsilon, \delta}$ .
Then we characterize the moment generating functions of the network-wide estimators and conclude that they have similar properties as their local rewards.
Proposition 8. Assume the parameter  $\delta$  satisfies that  $0 < \delta < c = f(\epsilon, M, T)$ . In setting  $s_1, s_2, s_3$  where rewards follow sub-gaussian distributions, for any  $m, i, \lambda$  and  $t > L$  where  $L$  is the length of
the burn-in period, the global estimator  $\tilde{\mu}_i^m (t)$  is sub-Gaussian distributed. Moreover, the conditional moment generating function satisfies that with  $P(A_{\epsilon ,\delta}) = 1 - 7\epsilon$
$$
\begin{array}{l} E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \exp \left\{\frac {\lambda^ {2}}{2} \frac {C \sigma^ {2}}{\min _ {j} n _ {j , i} (t)} \right\} \\ \end{array}
$$
where  $\sigma^2 = \max_{j,i}(\tilde{\sigma}_i^j)^2$  and  $C = \max \left\{\frac{4(M + 2)(1 - \frac{1 - c_0}{2(M + 2)})^2}{3M(1 - c_0)}, (M + 2)(1 + 4Md_m^2,t)\right\}$ .
Proof. We prove the statement on the conditional moment generating functions by induction. Let us start with the basis step.
Note that the definition of  $A_{\epsilon, \delta}$  and the choice of  $\delta$  again guarantee that for  $t \geq L$ ,  $|P_t - cE| < \delta < c$  on event  $A_{\epsilon, \delta}$ . This implies that for any  $t \geq L$ ,  $m$  and  $j$ ,  $P_t(m, j) > 0$ , and if  $t = L$
$$
P _ {t} ^ {\prime} (m, j) = \frac {1}{M} \tag {19}
$$
and if  $t > L$
$$
P _ {t} ^ {\prime} (m, j) = \frac {M - 1}{M ^ {2}}. \tag {20}
$$
Consider the time step  $t \leq L + 1$ . The quantity satisfies that
$$
\begin{array}{l} E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (t) - \mu_ {i}\right) \right\} 1 _ {A _ {e, \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right. \\ = E \left[ \exp \left\{\lambda \left(\tilde {\mu} _ {i} ^ {m} (L + 1) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right. \\ = E \left[ \exp \left\{\lambda \left(\sum_ {j = 1} ^ {M} P _ {m, j} ^ {\prime} (L) \hat {\tilde {\mu}} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \left(\sum_ {j = 1} ^ {M} \frac {1}{M} \hat {\mu} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) - \mu_ {i}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ = E \left[ \exp \left\{\lambda \sum_ {j = 1} ^ {M} \frac {1}{M} \left(\hat {\mu} _ {i, j} ^ {m} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right) \right\} 1 _ {A _ {\epsilon , \delta}} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right] \\ \leq \Pi_ {j = 1} ^ {M} \left(E \left[ \left(\exp \left\{\left(\lambda \frac {1}{M} \left(\bar {\mu} _ {i} ^ {j} \left(h _ {m, j} ^ {L}\right) - \mu_ {i} ^ {j}\right) \right\} 1 _ {A _ {e, \delta}}\right) ^ {M} \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right)\right) \right]\right) ^ {\frac {1}{M}} \tag {21} \\ \end{array}
$$
where the third equality holds by (19), the fourth equality uses the definition  $\mu_{i} = \frac{1}{M}\sum_{i = 1}^{M}\mu_{i}^{j}$ , and the last inequality results from the generalized hoeffding inequality as in Lemma 3 and the fact that  $\hat{\bar{\mu}}_{i,j}^{m}(h_{m,j}^{L}) = \bar{\mu}_{i}^{j}(h_{m,j}^{L})$ .
Note that for any client  $j$ , we have
$$
\begin{array}{l} E [ (\exp \{(\lambda \frac {1}{M} (\bar {\mu} _ {i} ^ {j} (h _ {m, j} ^ {L}) - \mu_ {i} ^ {j}) \} 1 _ {A _ {\epsilon , \delta}}) ^ {M} | \sigma (\{n _ {m, i} (t) \} _ {t, i, m})) ] \\ = E [ \exp \left\{(\lambda (\bar {\mu} _ {i} ^ {j} (h _ {m, j} ^ {L}) - \mu_ {i} ^ {j}) \} 1 _ {A _ {\epsilon , \delta}}) | \sigma (\{n _ {m, i} (t) \} _ {t, i, m}) \right] \\ = E [ \exp \left\{\left(\lambda \frac {\sum_ {s} \left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} 1 _ {A _ {\epsilon , \delta}} \right| \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) ] \\ = E \left[ \exp \left\{\sum_ {s} \left(\lambda \frac {\left(r _ {i} ^ {j} (s) - \mu_ {i} ^ {j}\right)}{n _ {j , i} \left(h _ {m , j} ^ {L}\right)} \right\} 1 _ {A _ {\epsilon , \delta}}\right) \mid \sigma \left(\left\{n _ {m, i} (t) \right\} _ {t, i, m}\right) \right]. \tag {22} \\ \end{array}
$$
It is worth noting that given  $s, r_i^j(s)$  is independent of everything else, which gives us