SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Notice that the summation in the first term is the covariance matrix  $V_{m,t}$  but missing the regularization term  $\lambda I$ ; thus, the first term can be simplified as  $\theta_{m} - \lambda V_{m,t}^{-1}\theta_{m}$ . Applying triangle inequality, we have the following for  $\left\| \hat{\theta}_{m,t} - \theta_{m} \right\|_{V_{m,t}}$ ,
$$
\begin{array}{l} \left\| \hat {\theta} _ {m, t} - \theta_ {m} \right\| _ {V _ {m, t}} \leq \underbrace {\lambda \| \theta_ {m} \| _ {V _ {m , t} ^ {- 1}}} _ {I _ {1}: \text {R e g u l a r i z a t i o n t e r m}} + \underbrace {\left\| \sum_ {i \neq m} \sum_ {k = 1} ^ {t _ {s}} w _ {i , k} x _ {i , k} x _ {i , k} ^ {\top} (\theta_ {i} - \theta_ {m}) \right\| _ {V _ {m , t} ^ {- 1}}} _ {I _ {2}: \text {D i s s i m i l a r i t y t e r m}} \\ + \underbrace {\left\| \sum_ {k = 1} ^ {t - 1} w _ {m , k} x _ {m , k} \eta_ {m , k} + \sum_ {i \neq m} \sum_ {k = 1} ^ {t _ {s}} w _ {i , k} x _ {i , k} \eta_ {i , k} \right\| _ {V _ {m , t} ^ {- 1}}}. _ {I _ {3}: \text {N o i s e t e r m}} \\ \end{array}
$$
For the regularization term  $I_{1}$ , it is upper bounded by,
$$
I _ {1} = \lambda \left\| \theta_ {m} \right\| _ {V _ {m, t} ^ {- 1}} \leq \frac {\lambda}{\lambda_ {\min} \left(V _ {m , t} ^ {1 / 2}\right)} \left\| \theta_ {m} \right\| _ {2} \leq \sqrt {\lambda} \left\| \theta_ {m} \right\| _ {2} \leq \sqrt {\lambda}.
$$
For the dissimilarity term  $I_{2}$ , we have,
$$
\begin{array}{l} I_{2} = \left\| \sum_{i\neq m}\sum_{k = 1}^{t_{s}}w_{i,k}x_{i,k}x_{i,k}^{\top}(\theta_{i} - \theta_{m})\right\|_{V_{m,t}^{-1}} \\ \leq \sum_ {i = 1} ^ {M} \sum_ {k = 1} ^ {t _ {s}} \left\| w _ {i, k} x _ {i, k} x _ {i, k} ^ {\top} (\theta_ {i} - \theta_ {m}) \right\| _ {V _ {m, t} ^ {- 1}} \\ = \sum_ {i = 1} ^ {M} \sum_ {k = 1} ^ {t _ {s}} w _ {i, k} \left| x _ {i, k} ^ {\top} \left(\theta_ {i} - \theta_ {m}\right) \right| \| x _ {i, k} \| _ {V _ {m, t} ^ {- 1}} \\ \leq \sum_ {i = 1} ^ {M} \sum_ {k = 1} ^ {t _ {s}} \| x _ {i, k} \| \| \theta_ {i} - \theta_ {m} \| \alpha \\ \leq \alpha \varepsilon M t _ {s}, \\ \end{array}
$$
where we use triangle inequality in the first inequality, the second inequality holds since  $w_{m,t} \leq \alpha / \| x_{m,t}\|_{V_{m,t}^{-1}}$  by the definition of  $w_{m,t}$ , the last inequality holds due to the bounded parameters assumption and the  $\varepsilon$ -MALCB definition (see Assumption 3.1 and Definition 3.1).
To bound term  $I_{3}$ , we denote vector  $\bar{x}_{m,t} = \sqrt{w_{m,t}} x_{m,t}$ , and the random noise  $\bar{\eta}_{m,t} = \sqrt{w_{m,t}} \eta_{m,t}$ . We have  $w_{m,t} \leq 1$ , therefore,  $\bar{\eta}_{m,t}$  is 1-subGaussian. Furthermore, we rewrite the covariance matrix as  $V_{m,t} = \lambda I + \sum_{s=1}^{t} \bar{x}_{m,s} \bar{x}_{m,s}^{\top} + \sum_{i \neq m} \sum_{k=1}^{t_s} \bar{x}_{i,k} \bar{x}_{i,k}^{\top}$ . We then have the following for the noise term  $I_{3}$ ,
$$
\begin{array}{l} I_{3} = \left\| \sum_{k = 1}^{t - 1}\bar{x}_{m,k}\bar{\eta}_{m,k} + \sum_{i\neq m}\sum_{k = 1}^{t_{s}}\bar{x}_{i,k}\bar{\eta}_{i,k}\right\|_{V_{m,t}^{-1}} \\ \leq \sqrt {2 \log \left(\frac {\operatorname* {d e t} \left(V _ {m , t}\right) ^ {1 / 2} \operatorname* {d e t} (V _ {0}) ^ {- 1 / 2}}{\delta_ {1}}\right)} \\ \leq \sqrt {d \log \left(\frac {1 + M t / (\lambda d)}{\delta_ {1}}\right)}, \\ \end{array}
$$
where we apply Lemma C.4 in the first inequality. Now, putting these three terms together, we have the following Confidence Ellipsoid bound for round  $t < \tau$ ,
$$
\left\| \hat {\theta} _ {m, t} - \theta_ {m} \right\| _ {V _ {m, t}} \leq \left(\sqrt {\lambda} + \alpha \varepsilon M t + \sqrt {d \log \left(\frac {1 + M t / (\lambda d)}{\delta_ {1}}\right)}\right).
$$
We now turn our attention to the round  $t \geq \tau$ . Since agents switch to independent learning, we follow the same argument for confidence ellipsoid as in Theorem 2 of Abbasi-Yadkori et al. [2011], and get
$$
\beta_ {t} = \sqrt {\lambda} + \sqrt {d \log \left(\frac {1 + t / (\lambda d)}{\delta_ {2}}\right)}.
$$
Finally, we complete the proof by taking a union bound over all agents.
Before proving Theorem 4.1, we give an upper bound on the pseudo-regret,  $r_{m,t}$ , in the next proposition. The proof follows standard arguments for linear bandits; most of the arguments can be extracted from Wang et al. [2020], Dubey and Pentland [2020].
Proposition A.1. The pseudo-regret  $r_t$  obtained by any agent  $m$  at round  $t$ , is upper bounded as
$$
r _ {m, t} \leq 2 \beta_ {t} \min  \left(1, \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}}\right).
$$
Proof. Recall that an agent makes decisions optimistically:  $(x_{m,t},\tilde{\theta}_{m,t}) = \operatorname *{arg  max}_{(x,\theta)\in \mathcal{D}_{m,t}\times \mathcal{C}_{m,t}}x^{\top}\theta .$
Let  $x_{m,t}^{\star}$  denote the optimal action at round  $t$  of agent  $m$ , i.e.,  $x_{m,t}^{\star} = \operatorname*{arg  max}_{x\in \mathcal{D}_{m,t}}x^{\top}\theta_{m}$ .
We then have the following for the pseudo-regret
$$
\begin{array}{l} r _ {m, t} = x _ {m, t} ^ {\star} \mathbin {\text {\text {T}}} \theta_ {m} - x _ {m, t} \mathbin {\text {\text {T}}} \theta_ {m} \\ \leq x _ {m, t} ^ {\top} \tilde {\theta} _ {m, t} - x _ {m, t} ^ {\top} \theta_ {m} \quad (\text {S i n c e} (x _ {m, t}, \tilde {\theta} _ {m, t}) \text {i s o p t i m i s t i c}) \\ = x _ {m, t} ^ {\top} \left(\bar {\theta} _ {m, t} - \theta_ {m}\right) \\ = \left\langle V _ {m, t} ^ {- 1 / 2} x _ {m, t}, V _ {m, t} ^ {1 / 2} \left(\tilde {\theta} _ {m, t} - \theta_ {m}\right) \right\rangle \quad \left(V _ {m, t} \succcurlyeq 0\right) \\ \leq \left\| x _ {m, t} \right\| _ {V _ {m, t} ^ {- 1}} \left\| \tilde {\theta} _ {m, t} - \theta_ {m} \right\| _ {V _ {m, t}} \quad (\text {C a u c h y - S c h w a r z ' s i n e q u a l i t y}) \\ \leq \left\| x _ {m, t} \right\| _ {V _ {m, t} ^ {- 1}} \left(\left\| \hat {\theta} _ {m, t} - \tilde {\theta} _ {m, t} \right\| _ {V _ {m, t}} + \left\| \hat {\theta} _ {m, t} - \theta_ {m} \right\| _ {V _ {m, t}}\right) \quad (\text {T r i a n g l e i n e q u a l i t y}) \\ \leq 2 \beta_ {t} \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}} \quad \left(\text {S i n c e} \theta_ {m}, \hat {\tilde {\theta}} _ {m, t} \in \mathcal {C} _ {t}\right) \\ \leq 2 \beta_ {t} \cdot \min  \left(1, \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}}\right), \\ \end{array}
$$
where the last inequality is due to the fact that  $\beta_{t} \geq 1$  and that suboptimality is no more than 2.
We now proceed to prove Theorem 4.1. The proof technique follows the analysis of group regret in Wang et al. [2020] and applies the arguments from Theorem 4.2 of He et al. [2022] to handle the dissimilarity.
Theorem A.1 (Theorem 4.1 restated). For a given  $T$ , and  $\tau \leq T$ , if we set  $D = \tau \log(M\tau) / (dM)$  and  $\beta_t, \forall t \in [\tau]$  according to Lemma 4.1, then, given Assumption 3.1, with probability at least  $1 - M\delta_1$ , the group regret of Algorithm 1 up to round  $\tau$ , is upper bounded as
$$
\begin{array}{l} \mathcal {R} (M, \tau) \leq 4 \sqrt {2} \left(\sqrt {\lambda d M} \tau (\xi_ {\tau}) ^ {1. 5} + \alpha \varepsilon \sqrt {d} (M \tau) ^ {1. 5} \sqrt {\xi_ {\tau}} + d \sqrt {M} \tau \xi_ {\tau} + \frac {\sqrt {\lambda} d \xi_ {\tau}}{\alpha} + \varepsilon d M \tau \xi_ {\tau} \right. \\ \left. + \frac {\left(d \xi_ {\tau}\right) ^ {1 . 5}}{\alpha} + \alpha \varepsilon \sqrt {d} (M \tau \xi_ {\tau}) ^ {1. 5} + d \sqrt {M \tau} \xi_ {\tau} ^ {2}\right), \\ \end{array}
$$
where  $\xi_{t} = \log \left(\frac{1 + Mt / (\lambda d)}{\delta_{1}}\right)$ . Furthermore, if we choose  $\lambda = 1, \alpha = \frac{\sqrt{d}}{\varepsilon M\tau}$ , then the group regret is upper bounded as
$$
\mathcal {R} (M, \tau) \leq 2 0 \sqrt {2} \left(d \sqrt {M \tau} \xi_ {\tau} ^ {2} + \varepsilon d M \tau \xi_ {\tau} ^ {1. 5}\right).
$$
We refer to this group regret as the "collaboration" regret, and denote it as  $\mathcal{R}_{\mathrm{collab}}(M,T)$ .
Proof. Let  $P$  be the total number of synchronization rounds, then, we can index the synchronization matrix  $V_{syn}$  for  $P$  epoch as  $\{V_{syn,p}\}_{p=1}^{P}$ , and define  $V_p = \lambda I + V_{syn,p}$ . Observe that  $\operatorname*{det}(V_0) = \operatorname*{det}(\lambda I) = \lambda^d$  and  $\operatorname*{det}(V_P) \leq (\operatorname{trace}(V_P) / d)^d \leq (\lambda + M\tau / d)^d$ . Therefore,
$$
\log \frac {\det  (V _ {P})}{\det  (V _ {0})} \leq d \log \left(1 + \frac {M \tau}{\lambda d}\right). \tag {1}
$$
By telescoping, we have that  $\log \frac{\operatorname*{det}(V_P)}{\operatorname*{det}(V_0)} = \sum_{p = 1}^{P}\log \frac{\operatorname*{det}(V_p)}{\operatorname*{det}(V_{p - 1})}$ . Therefore, we have at most  $R = \lceil d\log (1 + \frac{M\tau}{\lambda d})\rceil$  epochs such that  $\log \frac{\operatorname*{det}(V_p)}{\operatorname*{det}(V_{p - 1})}\geq 1$ ; otherwise, it violates the condition
in Equation (1). WLOG, we use logarithm base 2 for the determinant ratio. This implies that for all but  $R$  epochs,
$$
1 \leq \frac {\det  (V _ {p})}{\det  (V _ {p - 1})} \leq 2, \tag {2}
$$
We call the epochs satisfying Equation (2) as good epochs. We imagine a single agent playing  $M(\tau - 1)$  actions  $x_{1,1}, x_{2,1}, \ldots, x_{M-1,\tau-1}, x_{M,\tau-1}$  sequentially. Let  $W_{m,t} = \lambda I + \sum_{i=1}^{M} \sum_{s=1}^{t-1} w_{i,s} x_{i,s} x_{i,s}^{\top} + \sum_{j=1}^{m-1} w_{j,t} x_{j,t} x_{j,t}^{\top}$  be the covariance matrix of this imaginary agent when it gets to  $x_{m,t}$ . If  $x_{m,t}$  belongs to a good epoch, say the  $j$ -th epoch, we have the following:
$$
1 \leq \frac {\left\| x _ {m , t} \right\| _ {V _ {m , t} ^ {- 1}}}{\left\| x _ {m , t} \right\| _ {W _ {m , t} ^ {- 1}}} \leq \sqrt {\frac {\det  (W _ {m , t})}{\det  (V _ {m , t})}} \leq \sqrt {\frac {\det  (V _ {j})}{\det  (V _ {j - 1})}} \leq \sqrt {2}, \tag {3}
$$
where the first inequality is due to the fact that  $W_{m,t} \succcurlyeq V_{m,t}$ , and the second inequality follows from Lemma C.2.
Now, applying the Proposition A.1, we bound the pseudo-regret  $r_{m,t}$  of these good epochs as follows:
$$
\begin{array}{l} r _ {m, t} \leq 2 \beta_ {t} \min  \left(1, \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}}\right) \\ \leq 2 \beta_ {t} \min  \left(1, \| x _ {m, t} \| _ {W _ {m, t} ^ {- 1}} \sqrt {\frac {\det  (W _ {m , t})}{\det  (V _ {m , t})}}\right) \\ \leq 2 \sqrt {2} \beta_ {t} \min \left(1, \| x _ {m, t} \| _ {W _ {m, t} ^ {- 1}}\right), \\ \end{array}
$$
where we use Lemma C.2 in the second inequality, and Equation (3) in the third inequality.
Let  $\mathcal{R}_{\mathrm{good}}(M,\tau)$  be the group regret of these good epochs up to round  $\tau$ . Suppose  $P_{\mathrm{good}}$  contains all the good epochs, and  $\mathcal{B}_p$  contains all the pairs  $(m,t)$  belonging to epoch  $p$ . We have
$$
\begin{array}{l} \mathcal {R} _ {\mathrm {g o o d}} (M, \tau) = \sum_ {p \in P _ {\mathrm {g o o d}}} \sum_ {(m, t) \in \mathcal {B} _ {p}} r _ {m, t} \\ \leq \sum_ {p \in P _ {\mathrm {g o o d}}} \sum_ {(m, t) \in \mathcal {B} _ {p}} 2 \sqrt {2} \beta_ {\tau - 1} \min  \left(1, \| x _ {m, t} \| _ {W _ {m, t} ^ {- 1}}\right) \\ = \underbrace {\sum_ {p \in P _ {\text {g o o d}}} \sum_ {(m , t) \in \mathcal {B} _ {p} \wedge w _ {m , t} = 1} 2 \sqrt {2} \beta_ {\tau - 1} \min  \left(1 , \| x _ {m , t} \| _ {W _ {m , t} ^ {- 1}}\right)} _ {I _ {1}: \text {R o u n d s o f g o o d e p o c h w i t h} w _ {m, t} = 1} \\ + \underbrace {\sum_ {p \in P _ {\text {g o o d}}} \sum_ {(m , t) \in \mathcal {B} _ {p} \wedge w _ {m , t} <   1} 2 \sqrt {2} \beta_ {\tau - 1} \min  \left(1 , \| x _ {m , t} \| _ {W _ {m , t} ^ {- 1}}\right)} _ {I _ {2}: \text {R o u n d s o f g o o d e p o c h w i t h} w _ {m, t} <   1}, \\ \end{array}
$$
where in the last equality we split the summation to two cases:  $w_{m,t} = 1$ , and  $w_{m,t} < 1$ .
For term  $I_{1}$ , we consider all the pair  $(m,t)$  in good epochs up to round  $\tau$  such that  $w_{m,t} = 1$ ; we assume that we have a total of such  $K$  pairs, and these pairs can be sequentially listed as  $\{(\bar{m}_1,\bar{t}_1),(\bar{m}_2,\bar{t}_2),\dots ,(\bar{m}_K,\bar{t}_K)\}$ . With this notation, for  $i\leq K$ , we construct the auxiliary covariance matrix  $A_{\bar{m}_i,\bar{t}_i} = \lambda I + \sum_{k = 1}^{i - 1}x_{\bar{m}_k,\bar{t}_k}x_{\bar{m}_k,\bar{t}_k}^\top$ . Thus, we have  $W_{\bar{m}_i,\bar{t}_i}\succcurlyeq A_{\bar{m}_i,\bar{t}_i}$ , which implies  $\| x_{\bar{m}_i,\bar{t}_i}\|_{W_{\bar{m}_i,\bar{t}_i}^{-1}}\leq \| x_{\bar{m}_i,\bar{t}_i}\|_{A_{\bar{m}_i,\bar{t}_i}^{-1}}$ .
We now have the following for term  $I_{1}$ :
$$
\begin{array}{l} I_{1} = \sum_{p\in P_{\text{good}}}\sum_{(m,t)\in \mathcal{B}_{p}\wedge w_{m,t} = 1}2\sqrt{2}\beta_{\tau -1}\min \left(1,\| x_{m,t}\|_{W_{m,t}^{-1}}\right) \\ = \sum_ {k = 1} ^ {K} 2 \sqrt {2} \beta_ {\tau - 1} \min  \left(1, \left\| x _ {\bar {m} _ {k}, \bar {\iota} _ {k}} \right\| _ {W _ {\bar {m} _ {k}, \bar {\iota} _ {k}} ^ {- 1}}\right) \\ \leq \sum_ {k = 1} ^ {K} 2 \sqrt {2} \beta_ {\tau - 1} \min  \left(1, \left\| x _ {\bar {m} _ {k}, \bar {t} _ {k}} \right\| _ {A _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {- 1}}\right) \\ \leq 2 \sqrt {2} \beta_ {\tau - 1} \sqrt {M \tau} \sqrt {\sum_ {i = 1} ^ {K} \min  \left(1 , \left\| x _ {\bar {m} _ {k} , \bar {t} _ {k}} \right\| _ {A _ {\bar {m} _ {k} , \bar {t} _ {k}}} ^ {2}\right)} \\ \leq 4 \beta_ {\tau - 1} \sqrt {2 M \tau \log \left(\frac {\det  (V _ {P})}{\det  (V _ {0})}\right)} \\ \leq 4 \sqrt {2} \beta_ {\tau - 1} \sqrt {d M \tau \log \left(1 + \frac {M \tau}{\lambda d}\right)} \\ \leq 4 \sqrt {2} \beta_ {\tau - 1} \sqrt {d M \tau \xi}, \\ \end{array}
$$
where in the second inequality we use Cauchy Schwarz's inequality and the fact that  $K \leq M\tau$ , and use Lemma C.3 in the second inequality.
For term  $I_{2}$ , since we consider the case such that  $w_{m,t} < 1$ , thus, by the definition of  $w_{m,t}$  we have  $\frac{1}{\alpha} w_{m,t} \cdot \| x_{m,t}\|_{V_{m,t}^{-1}} = 1$ . Therefore, we have,
$$
\begin{array}{l} I _ {2} = 2 \sqrt {2} \sum_ {p \in P _ {\text {g o o d}}} \sum_ {(m, t) \in \mathcal {B} _ {p} \wedge w _ {m, t} <   1} \min  \left(1, \beta_ {\tau} \| x _ {m, t} \| _ {W _ {m, t} ^ {- 1}}\right) \\ = 2 \sqrt {2} \sum_ {p \in P _ {\text {g o o d}}} \sum_ {(m, t) \in \mathcal {B} _ {p} \wedge w _ {m, t} <   1} \min  \left(1, \beta_ {\tau} \frac {w _ {m , t} \| x _ {m , t} \| _ {V _ {m , t} ^ {- 1}}}{\alpha} \| x _ {m, t} \| _ {W _ {m, t} ^ {- 1}}\right) \\ \leq 4 \sum_ {p \in P _ {\text {g o o d}}} \sum_ {(m, t) \in \mathcal {B} _ {p} \wedge w _ {m, t} <   1} \min  \left(1, \beta_ {\tau} \frac {w _ {m , t} \| x _ {m , t} \| _ {W _ {m , t} ^ {- 1}} ^ {2}}{\alpha}\right), \\ \end{array}
$$
where the last inequality follows from Equation (3).
Similarly to what we have done with  $I_{1}$ , we consider all the pair  $(m, t)$  in good epochs up to round  $\tau$  such that  $w_{m,t} < 1$ ; we assume that we have a total of such  $K$  pairs, and these pairs can be sequentially listed as  $\{(\bar{m}_1, \bar{t}_1), (\bar{m}_2, \bar{t}_2), \dots, (\bar{m}_K, \bar{t}_K)\}$ . Furthermore, we define  $\bar{x}_{m,t} = \sqrt{w_{m,t}} x_{m,t}$ , and construct the auxiliary covariance matrix,
$$
\bar {W} _ {\bar {m} _ {i}, \bar {t} _ {i}} = \lambda I + \sum_ {k = 1} ^ {i - 1} w _ {\bar {m} _ {k}, \bar {t} _ {k}} x _ {\bar {m} _ {k}, \bar {t} _ {k}} x _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {\top} = \lambda I + \bar {x} _ {\bar {m} _ {k}, \bar {t} _ {k}} \bar {x} _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {\top}.
$$
Thus, we have  $W_{\bar{m}_i,\bar{\tau}_i}\succcurlyeq \bar{W}_{\bar{m}_i,\bar{\tau}_i}$ , which implies  $\| x_{\bar{m}_i,\bar{\tau}_i}\|_{W_{\bar{m}_i,\bar{\tau}_i}^{-1}}\leq \| x_{\bar{m}_i,\bar{\tau}_i}\|_{\bar{W}_{\bar{m}_i,\bar{\tau}_i}^{-1}}$
$$
\begin{array}{l} I _ {2} \leq 4 \sum_ {k = 1} ^ {K} \min  \left(1, \beta_ {\tau - 1} w _ {\bar {m} _ {k}, \bar {t} _ {k}} / \alpha \left\| x _ {\bar {m} _ {k}, \bar {t} _ {k}} \right\| _ {W _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {- 1}} ^ {2}\right) \\ \leq 4 \sum_ {k = 1} ^ {K} \min  \left(1 + \beta_ {\tau - 1} / \alpha , (1 + \beta_ {\tau - 1} / \alpha) \| \sqrt {w _ {\bar {m} _ {k} , \bar {t} _ {k}}} x _ {\bar {m} _ {k}, \bar {t} _ {k}} \| _ {W _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {- 1}} ^ {2}\right) \\ \leq 4 (1 + \beta_ {\tau - 1} / \alpha) \sum_ {k = 1} ^ {K} \min  \left(1, \| \bar {x} _ {\bar {m} _ {k}, \bar {t} _ {k}} \| _ {\bar {W} _ {\bar {m} _ {k}, \bar {t} _ {k}} ^ {- 1}} ^ {2}\right) \\ \leq 4 \left(1 + \beta_ {\tau - 1} / \alpha\right) d \log \left(1 + \frac {M \tau}{\lambda d}\right) \\ \leq 4 \left(1 + \beta_ {\tau - 1} / \alpha\right) d \xi_ {\tau}, \\ \end{array}
$$
where we use Lemma C.3 in the fourth inequality. Putting the bounds for  $I_{1}$  and  $I_{2}$  together, we have
$$
\begin{array}{l} \mathcal {R} _ {\text {g o o d}} (M, \tau) = I _ {1} + I _ {2} \leq 4 \sqrt {2} \beta_ {\tau - 1} \sqrt {d M \tau \xi_ {\tau}} + 4 \left(1 + \frac {\beta_ {\tau - 1}}{\alpha}\right) d \xi_ {\tau} \\ = 4 \sqrt {2} \left(\beta_ {\tau - 1} \sqrt {d M \tau \xi_ {\tau}} + d \xi_ {\tau} + \frac {\beta_ {\tau - 1}}{\alpha} d \xi_ {\tau}\right). \\ \end{array}
$$
Plugging  $\beta_{\tau -1} = \sqrt{\lambda} +\alpha \varepsilon M\tau +\sqrt{d\log(\frac{1 + M\tau / \lambda}{\delta_1})}$ , the regret of the good epochs is bounded as
$$
\mathcal {R} _ {\text {g o o d}} (M, \tau) \leq 4 \sqrt {2} \left(\sqrt {\lambda d M \tau \xi_ {\tau}} + \alpha \varepsilon \sqrt {d} (M \tau) ^ {1. 5} \sqrt {\xi_ {\tau}} + d \sqrt {M \tau} \xi_ {\tau} + \frac {\sqrt {\lambda} d \xi_ {\tau}}{\alpha} + \varepsilon d M \tau \xi_ {\tau} + \frac {(d \xi_ {\tau}) ^ {1 . 5}}{\alpha}\right).
$$
Now, we focus on the epochs that are not good, which do not satisfy Equation (2). Let  $p$  be one such epoch, and let  $t_0$  be the first round and  $n$  be the length, respectively, of the epoch  $p$ . Recall that at the beginning of each epoch, agents' covariance matrices are synchronized, i.e.,  $V_{m,t_0} = \lambda I + V_{syn}, \forall m \in [M]$ . We can then bound the regret of the (bad) epoch  $p$  as follows.
$$
\begin{array}{l} \mathcal {R} _ {\text {b a d}} (M, p) \leq 2 \beta_ {\tau - 1} \sum_ {m = 1} ^ {M} \sum_ {t = t _ {0}} ^ {t _ {0} + n} \min  \left(1, \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}}\right) \\ \leq 2 \beta_ {\tau - 1} \sum_ {m = 1} ^ {M} \sqrt {n} \sqrt {\sum_ {t = t _ {0}} ^ {t _ {0} + n} \min  \left(1 , \| x _ {m , t} \| _ {V _ {m , t} ^ {- 1}} ^ {2}\right)} \\ \leq 2 \beta_ {\tau - 1} \sum_ {m = 1} ^ {M} \sqrt {n \log \frac {\det  (V _ {m , t _ {0} + n})}{\det  (V _ {m , t _ {0}})}}, \\ \end{array}
$$
where we use Cauchy-Schwarz's inequality and Lemma C.3 in the second inequality.
Letsay the synchronization is triggered on round  $t_0 + n + 1$  ,i.e.,for one of the agents we have that  $(n + 1)\log \frac{\operatorname*{det}(V_{m,t_0 + n + 1})}{\operatorname*{det}(V_{m,t_0})} >D$  . Since  $n\log \frac{\operatorname*{det}(V_{m,t_0 + n})}{\operatorname*{det}(V_{m,t_0})} < D$  for all  $m\in [M]$  ,for this bad epoch (i.e.,from round  $t_0$  to round  $t_0 + n$  ),we can bound the group regret as
$$
\mathcal {R} _ {\text {b a d}} (M, p) \leq 2 \beta_ {\tau - 1} M \sqrt {D}.
$$
As we argued earlier, since  $\operatorname{det} V_p \leq (\lambda + \frac{MT}{d})^d$ , the bad epochs are rare, i.e., there are at most  $R = \lceil d \log(1 + \frac{M\tau}{\lambda d}) \rceil$  (otherwise, if  $\operatorname{det}(V_p) / \operatorname{det}(V_{p-1}) > 2$  for more than  $R$  rounds, then the condition of Equation (1) is violated). Setting  $D = \tau \log(M\tau) / (dM)$  and plugging  $\beta_{\tau-1}$  into the bound above,
$$
\begin{array}{l} \mathcal {R} _ {\text {b a d}} (M, \tau) \leq 2 R \beta_ {\tau - 1} M \sqrt {D} \\ \leq 2 \left(d \xi_ {\tau} \left(\sqrt {\lambda} + \alpha \varepsilon M \tau + \sqrt {d \xi_ {\tau}}\right) M \sqrt {D}\right) \\ \leq 2 \left(\sqrt {\lambda d M \tau} \left(\xi_ {\tau}\right) ^ {1. 5} + \alpha \varepsilon \sqrt {d} \left(M \tau \xi_ {\tau}\right) ^ {1. 5} + d \sqrt {M \tau} \xi_ {\tau} ^ {2}\right). \\ \end{array}
$$
We finish the proof by putting the regret of the good and the bad epochs together,
$$
\begin{array}{l} \mathcal {R} (M, \tau) = \mathcal {R} _ {\text {g o o d}} (M, \tau) + \mathcal {R} _ {\text {b a d}} (M, \tau) \\ \leq 4 \sqrt {2} \left(\sqrt {\lambda d M \tau} \left(\xi_ {\tau}\right) ^ {1. 5} + \alpha \varepsilon \sqrt {d} (M \tau) ^ {1. 5} \sqrt {\xi_ {\tau}} + d \sqrt {M \tau} \xi_ {\tau} + \frac {\sqrt {\lambda} d \xi_ {\tau}}{\alpha} + \varepsilon d M \tau \xi_ {\tau} \right. \\ \left. + \frac {\left(d \xi_ {\tau}\right) ^ {1 . 5}}{\alpha} + \alpha \varepsilon \sqrt {d} (M \tau \xi_ {\tau}) ^ {1. 5} + d \sqrt {M \tau} \xi_ {\tau} ^ {2}\right). \\ \end{array}
$$
Proof. Setting  $\delta_1 = \delta_2 = 1 / (M^2 T)$ , then the group regret caused by failure event, which does not satisfy Lemma 4.1, is at most  $MT \cdot (M\delta_1 + M\delta_2) = \mathcal{O}(1)$ . Thus, we mainly consider the group regret when Lemma 4.1 holds.
From round  $\tau$  onward, agents switch to independent learning mode, this is the group regret from round  $\tau$  to the end, we denote it as  $\mathcal{R}_{\mathrm{ind}}(M,T)$ . Furthermore, for round  $t \geq \tau$ , we have  $V_{m,t} = \lambda I + \sum_{s=\tau}^{t-1} x_{m,s} x_{m,s}^{\top}$  as the gram matrix which only contains agent's data.
Applying Proposition A.1, we have the following
$$
\begin{array}{l} \mathcal {R} _ {\mathrm {i n d}} (M, T) \leq \sum_ {m = 1} ^ {M} \sum_ {t = \tau} ^ {T} 2 \beta_ {t} \min  \left(1, \| x _ {m, t} \| _ {V _ {m, t} ^ {- 1}}\right) \\ \leq 2 \sum_ {m = 1} ^ {M} \beta_ {T} \sqrt {T - \tau} \sqrt {\sum_ {t = \tau} ^ {T} \min  \left(1 , \| x _ {m , t} \| _ {V _ {m , t} ^ {- 1}} ^ {2}\right)} \\ \leq 2 \sum_ {m = 1} ^ {M} \left(\sqrt {\lambda} + \sqrt {d \log \left(\frac {1 + T / (\lambda d)}{\delta_ {2}}\right)}\right) \sqrt {T - \tau} \sqrt {2 d \log \left(1 + \frac {T}{\lambda d}\right)} \\ \leq 4 \sqrt {2} M d \sqrt {T - \tau} \xi_ {T}, \\ \end{array}
$$
where we use Cauchy-Schwarz's inequality in the first inequality, and Lemma 4.1 in the second inequality. Combining with the regret from the beginning up to round  $\tau$  from Theorem 4.1, we have
$$
\begin{array}{l} \mathcal {R} (M, T) = \mathcal {R} _ {\text {c o l l a b}} (M, T) + \mathcal {R} _ {\text {i n d}} (M, T) \\ \leq 2 0 \sqrt {2} \left(d \sqrt {M \tau} + \varepsilon d M \tau + d M \sqrt {T - \tau}\right) \xi_ {T} ^ {2} \tag {4} \\ \end{array}
$$
We notice that Equation (4) has the second term that is linear in term of  $M\tau$ . To avoid linear regret, the choice of  $\tau$  needs to adapt to the dissimilarity level  $\varepsilon$ .
For the last two terms, by Cauchy-Schwarz's inequality, we have  $\varepsilon dM\tau +dM\sqrt{T - \tau}\leq dM\sqrt{2(\varepsilon^2\tau^2 + T - \tau)}$ , this term can be optimized by setting  $\tau = \frac{1}{2\varepsilon^2}$ . We need to restrict  $\tau = \min (\lfloor \frac{1}{2}\varepsilon^{-2}\rfloor ,T)$  since  $\tau$  can not exceed  $T$ . In other words, all agents can collaborate up to round  $\min \left(\lfloor \frac{1}{2}\varepsilon^{-2}\rfloor ,T\right)$ .
Next, we consider the following two cases:
Case 1: For  $\frac{1}{2}\varepsilon^{-2} \leq T$ , we have that  $dM\sqrt{T - \tau} \leq dM\sqrt{T}$ , and  $\varepsilon dM\tau = \frac{dM}{2\varepsilon} \leq dM\sqrt{T}$ . Thus,  $\varepsilon dM\tau + dM\sqrt{T - \tau} \leq 2dM\sqrt{T}$ .
Case 2: For  $\frac{1}{2}\varepsilon^{-2} > T$ , we have  $\tau = T$ . This implies that  $\varepsilon dM\tau +dM\sqrt{T - \tau} = \varepsilon dMT$ . We also have  $\varepsilon dMT < \frac{1}{\sqrt{2}} dM\sqrt{T}$  due to  $\frac{1}{2}\varepsilon^2 >T$
Notice that, in both cases,  $\varepsilon dM\tau +dM\sqrt{T - \tau}$  is evaluated as small as  $\varepsilon dMT$  and always be upper bounded by  $2dM\sqrt{T}$ . Therefore, in Equation (4), the summation of the last two terms of the group regret is upper bounded by  $2\min \{\varepsilon dMT,dM\sqrt{T}\}$ .
By the choice of  $\delta_1, \delta_2, \lambda$ , and assume that  $d \geq 1$ , we can upper bound  $\xi_T \leq 4 \log(MT)$ . Therefore, we have the following for the expected group regret