SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \sum_ {t = T _ {0} + 1} ^ {T} C _ {a _ {t}, t} \leq \beta \sum_ {t = 1} ^ {T} \| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \| _ {M _ {V _ {t}, t - 1} ^ {- 1}} \\ = \beta \sum_ {t \in [ T ]: w _ {i _ {t}, t} = 1} \| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \| _ {M _ {V _ {t}, t - 1} ^ {- 1}} + \beta \sum_ {t \in [ T ]: w _ {i _ {t}, t} <   1} \| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \| _ {M _ {V _ {t}, t - 1} ^ {- 1}}. \tag {44} \\ \end{array}
$$
Then, we prove the following technical lemma.
Lemma 7.
$$
\sum_ {t = T _ {0} + 1} ^ {T} \min  \left\{\mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}, 1 \right\} \leq 2 d \log \left(1 + \frac {T}{\lambda d}\right), \forall j \in [ m ]. \tag {45}
$$
$$
\begin{array}{l} \det  (\boldsymbol {M} _ {V _ {j}, T}) = \det  \left(\boldsymbol {M} _ {V _ {j}, T - 1} + \mathbb {I} \{i _ {T} \in V _ {j} \} \boldsymbol {x} _ {a _ {T}} \boldsymbol {x} _ {a _ {T}} ^ {\top}\right) \\ = \det  (M _ {V _ {j}, T - 1}) \det  \left(\boldsymbol {I} + \mathbb {I} \{i _ {T} \in V _ {j} \} M _ {V _ {j}, T - 1} ^ {- \frac {1}{2}} \boldsymbol {x} _ {a _ {T}} \boldsymbol {x} _ {a _ {T}} ^ {\top} M _ {V _ {j}, T - 1} ^ {- \frac {1}{2}}\right) \\ = \det  (\boldsymbol {M} _ {V _ {j}, T - 1}) \left(1 + \mathbb {I} \left\{i _ {T} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {T}} \| _ {\boldsymbol {M} _ {V _ {j}, T - 1}} ^ {2}\right) \\ = \det  (\boldsymbol {M} _ {V _ {j}, T _ {0}}) \prod_ {t = T _ {0} + 1} ^ {T} \left(1 + \mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}\right) \\ \geq \det (\lambda I) \prod_ {t = T _ {0} + 1} ^ {T} \left(1 + \mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}\right). \tag {46} \\ \end{array}
$$
$$
\begin{array}{l} \sum_ {t = T _ {0} + 1} ^ {T} \min  \left\{\mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}, 1 \right\} \leq 2 \sum_ {t = T _ {0} + 1} ^ {T} \log \left(1 + \mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}\right) \\ = 2 \log \left(\prod_ {t = T _ {0} + 1} ^ {T} \left(1 + \mathbb {I} \{i _ {t} \in V _ {j} \} \| \boldsymbol {x} _ {a _ {t}} \| _ {\boldsymbol {M} _ {V _ {j}, t - 1} ^ {- 1}} ^ {2}\right)\right) \\ \leq 2 \left[ \log \left(\det \left(M _ {V _ {j}, T}\right)\right) - \log \left(\det (\lambda I)\right) \right] \\ \leq 2 \log \left(\frac {\operatorname {t r a c e} \left(\lambda \boldsymbol {I} + \sum_ {t = 1} ^ {T} \mathbb {I} \left\{i _ {t} \in V _ {j} \right\} \boldsymbol {x} _ {a _ {t}} \boldsymbol {x} _ {a _ {t}} ^ {\top}\right)}{\lambda d}\right) ^ {d} \\ \leq 2 d \log \left(1 + \frac {T}{\lambda d}\right). \tag {47} \\ \end{array}
$$
Denote the rounds with  $w_{i_t,t} = 1$  as  $\{\tilde{t}_1,\dots ,\tilde{t}_{l_1}\}$ , and gram matrix  $\tilde{G}_{V_{\tilde{t}_{\tau},\tilde{t}_{\tau} - 1}}\triangleq \lambda I + \sum_{\substack{s\in [\tau ]\\ i_s\in V_{\tilde{t}_{\tau}}}}\boldsymbol{x}_{a_{\tilde{t}_s}}\boldsymbol{x}_{a_{\tilde{t}_s}}^\top$ ; denote the rounds with  $w_{i_t,t} < 1$  as  $\{t_1',\ldots ,t_{l_2}'\}$ , gram matrix  $\pmb{G}_{V_{i_{\tau}^{\prime}},t_{\tau}^{\prime} - 1}^{\prime}\triangleq \lambda I + \sum_{\substack{s\in [\tau ]\\ i_s\in V_{t_{t'}^{\prime}}}}w_{i_{t_s'},t_s'}\boldsymbol{x}_{a_{t_s'}}\boldsymbol{x}_{a_{t_s'}}^\top$ .
$$
\begin{array}{l} \sum_ {t \in [ T ]: w _ {i _ {t}, t} = 1} \| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \| _ {M _ {V _ {t}, t - 1} ^ {- 1}} = \sum_ {j = 1} ^ {m} \sum_ {\tau = 1} ^ {l _ {1}} \mathbb {I} \left\{i _ {\tilde {t} _ {\tau}} \in V _ {j} \right\} \left\| \boldsymbol {x} _ {\boldsymbol {a} _ {\tilde {t} _ {\tau}}} \right\| _ {M _ {V _ {\tilde {t} _ {\tau}, \tilde {t} _ {\tau}} - 1} ^ {- 1}} \leq \sum_ {j = 1} ^ {m} \sum_ {\tau = 1} ^ {l _ {1}} \mathbb {I} \left\{i _ {\tilde {t} _ {\tau}} \in V _ {j} \right\} \left\| \boldsymbol {x} _ {\boldsymbol {a} _ {\tilde {t} _ {\tau}}} \right\| _ {\tilde {\boldsymbol {G}} _ {V _ {\tilde {t} _ {\tau}}, \tilde {t} _ {\tau} - 1}} (48) \\ \leq \sum_ {j = 1} ^ {m} \sqrt {\sum_ {\tau = 1} ^ {l _ {1}} \mathbb {I} \{i _ {\tilde {t} _ {\tau}} \in V _ {j} \} \sum_ {\tau = 1} ^ {l _ {1}} \min  \{1 , \mathbb {I} \{i _ {\tilde {t} _ {\tau}} \in V _ {j} \} \| \boldsymbol {x} _ {\boldsymbol {a} _ {\tilde {t} _ {\tau}}} \| _ {\tilde {\boldsymbol {G}} _ {V _ {\tilde {t} _ {\tau}}, \tilde {t} _ {\tau} - 1} ^ {- 1}} ^ {2} \}} (49) \\ \leq \sum_ {j = 1} ^ {m} \sqrt {T _ {V _ {j , T}} \times 2 d \log \left(1 + \frac {T}{\lambda d}\right)} (50) \\ \leq \sqrt {2 m \sum_ {j = 1} ^ {m} T _ {V _ {j} , T} d \log \left(1 + \frac {T}{\lambda d}\right)} = \sqrt {2 m d T \log \left(1 + \frac {T}{\lambda d}\right)}, (51) \\ \end{array}
$$
where Eq.(48) is because  $\tilde{G}_{V_{\tilde{t}_{\tau}},\tilde{t}_{\tau} - 1}^{-1}\succeq M_{V_{\tilde{t}_{\tau}},\tilde{t}_{\tau} - 1}^{-1}$  in Eq.(49) we use Cauchy-Schwarz inequality, in Eq.(50) we use Lemma 7 and  $\sum_{\tau = 1}^{l_1}\mathbb{I}\{i_{\tilde{t}_\tau}\in V_j\} \leq T_{V_j,T}$ , in Eq.(51) we use Cauchy-Schwarz inequality and  $\sum_{j = 1}^{m}T_{V_{j},T} = T$
For the second part in Eq.(44), Let  $\pmb{x}_{a_{t_{\tau}^{\prime}}}^{\prime}\triangleq \sqrt{w_{i_{t_{\tau}^{\prime}},t_{\tau}^{\prime}}}\pmb{x}_{a_{t_{\tau}^{\prime}}}$ , then
$$
\begin{array}{l} \sum_ {t: w _ {i t}, t <   1} \left\| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \right\| _ {M _ {V _ {t}, t - 1} ^ {- 1}} = \sum_ {t: w _ {i t}, t <   1} \frac {\left\| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \right\| _ {M _ {V _ {t} , t - 1} ^ {- 1}} ^ {2}}{\left\| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \right\| _ {M _ {V _ {t} , t - 1} ^ {- 1}}} = \sum_ {t: w _ {i t}, t <   1} \frac {w _ {i _ {t} , t} \left\| \boldsymbol {x} _ {\boldsymbol {a} _ {t}} \right\| _ {M _ {V _ {t} , t - 1} ^ {- 1}} ^ {2}}{\alpha} (52) \\ = \sum_ {j = 1} ^ {m} \sum_ {\tau = 1} ^ {l _ {2}} \mathbb {I} \{i _ {t _ {\tau} ^ {\prime}} \in V _ {j} \} \frac {w _ {i _ {t _ {\tau} ^ {\prime}} , t _ {\tau} ^ {\prime}}}{\alpha} \left\| \boldsymbol {x} _ {a _ {t _ {\tau} ^ {\prime}}} \right\| _ {\boldsymbol {M} _ {V _ {t _ {\tau} ^ {\prime}} , t _ {\tau} ^ {\prime} - 1} ^ {- 1}} ^ {2} \\ \leq \sum_ {j = 1} ^ {m} \frac {\sum_ {\tau = 1} ^ {l _ {2}} \min  \left\{1 , \mathbb {I} \left\{i _ {t _ {\tau} ^ {\prime}} \in V _ {j} \right\} \left\| \boldsymbol {x} _ {a _ {t _ {\tau} ^ {\prime}}} ^ {\prime} \right\| _ {\boldsymbol {G} _ {V _ {t _ {\tau} ^ {\prime}} , t _ {\tau} ^ {\prime} - 1} ^ {\prime - 1}} ^ {2} \right\}}{\alpha} (53) \\ \leq \sum_ {j = 1} ^ {m} \frac {2 d \log \left(1 + \frac {T}{\lambda d}\right)}{\alpha} = \frac {2 m d \log \left(1 + \frac {T}{\lambda d}\right)}{\alpha} (54) \\ \end{array}
$$
where in Eq.(52) we use the definition of the weights, in Eq.(53) we use  $G_{V_{t_{\tau}^{\prime}}, t_{\tau}^{\prime} - 1}^{\prime - 1} \succeq M_{V_{t_{\tau}^{\prime}}, t_{\tau}^{\prime} - 1}^{-1}$ , and Eq.(54) uses Lemma 7.
Then, with Eq.(54), Eq.(51), Eq.(44), Eq.(40), Eq.(43),  $\delta = \frac{1}{T}$ , and  $\beta = \sqrt{\lambda} + \sqrt{2\log(T) + d\log(1 + \frac{T}{\lambda d})} + \alpha C$ , we can get
$$
\begin{array}{l} R (T) \leq 4 + T _ {0} + \left(2 \sqrt {\lambda} + \sqrt {2 \log (T) + d \log \left(1 + \frac {T}{\lambda d}\right)} + \alpha C\right) \times \left(\sqrt {2 m d T \log \left(1 + \frac {T}{\lambda d}\right)} + \alpha C\right). \\ \left. + \frac {2 m d \log \left(1 + \frac {T}{\lambda d}\right)}{\alpha}\right) \\ = 4 + 1 6 u \log (u T) + 4 u \max  \left\{\frac {2 8 8 d}{\gamma^ {2} \alpha \sqrt {\lambda} \tilde {\lambda} _ {x}} \log (u T), \frac {1 6}{\tilde {\lambda} _ {x} ^ {2}} \log (\frac {8 d T}{\tilde {\lambda} _ {x} ^ {2}}), \frac {7 2 \sqrt {\lambda}}{\alpha \gamma^ {2} \tilde {\lambda} _ {x}}, \frac {7 2 \alpha C ^ {2}}{\gamma^ {2} \sqrt {\lambda} \tilde {\lambda} _ {x}} \right\} \\ + \left(2 \sqrt {\lambda} + \sqrt {2 \log (T) + d \log \left(1 + \frac {T}{\lambda d}\right)} + \alpha C\right) \times \left(\sqrt {2 m d T \log \left(1 + \frac {T}{\lambda d}\right)} \right. \\ \left. + \frac {2 m d \log (1 + \frac {T}{\lambda d})}{\alpha}\right). \\ \end{array}
$$
Picking  $\alpha = \frac{\sqrt{\lambda} + \sqrt{d}}{C}$ , we can get
$$
R (T) \leq O \left(\left(\frac {C \sqrt {d}}{\gamma^ {2} \tilde {\lambda} _ {x}} + \frac {1}{\tilde {\lambda} _ {x} ^ {2}}\right) u \log (T)\right) + O \left(d \sqrt {m T} \log (T)\right) + O \left(m C d \log^ {1. 5} (T)\right). \tag {55}
$$
Thus we complete the proof of Theorem 3.
D Proof and Discussions of Theorem 4
Table 1 of the work [12] gives a lower bound for linear bandits with adversarial corruption for a single user. The lower bound of  $R(T)$  is given by:  $R(T) \geq \Omega(d\sqrt{T} + dC)$ . Therefore, suppose our problem with multiple users and  $m$  underlying clusters where the arrival times are  $T_i$  for each cluster, then for any algorithms, even if they know the underlying clustering structure and keep  $m$  independent linear bandit algorithms to leverage the common information of clusters, the best they can get is  $R(T) \geq dC + \sum_{i \in [m]} d\sqrt{T_i}$ . For a special case where  $T_i = \frac{T}{m}, \forall i \in [m]$ , we can get  $R(T) \geq dC + \sum_{i \in [m]} d\sqrt{\frac{T}{m}} = d\sqrt{mT} + dC$ , which gives a lower bound of  $\Omega(d\sqrt{mT} + dC)$  for the LOCUD problem.
Recall that the regret upper bound of RCLUB-WCU shown in Theorem 3 is of  $O\left(\left(\frac{C\sqrt{d}}{\gamma^2\tilde{\lambda}_x} + \frac{1}{\tilde{\lambda}_x^2}\right)u\log (T)\right) + O\big(d\sqrt{mT}\log (T)\big) + O\big(mCd\log^{1.5}(T)\big)$ , asymptotically matching this lower bound with respect to  $T$  up to logarithmic factors and with respect to  $C$  up to  $O(\sqrt{m})$  factors, showing the tightness of our theoretical results (where  $m$  are typically very small for real applications).
We conjecture that the gap for the  $m$  factor in the  $mC$  term of the lower bound is due to the strong assumption that cluster structures are known to prove our lower bound, and whether there exists a tighter lower bound will be left for future work.
We prove the theorem using the proof by contrapositive. Specifically, in Theorem 5, we need to prove that for any  $t \geq T_0$ , if the detection condition in Line 7 of Algo.2 for user  $i$ , then with probability at least  $1 - 5\delta$ , user  $i$  is indeed a corrupted user. By the proof by contrapositive, we can prove Theorem 5 by showing that: for any  $t \geq T_0$ , if user  $i$  is a normal user, then with probability at least  $1 - 5\delta$ , the detection condition in Line 7 of Algo.2 will not be satisfied for user  $i$ .
If the clustering structure is correct at  $t$ , then for any normal user  $i$
$$
\bar {\boldsymbol {\theta}} _ {i, t} - \bar {\boldsymbol {\theta}} _ {V _ {i, t}, t} = \bar {\boldsymbol {\theta}} _ {i, t} - \boldsymbol {\theta} _ {i} + \boldsymbol {\theta} _ {i} - \bar {\boldsymbol {\theta}} _ {V _ {i, t}, t}, \tag {56}
$$
where  $\tilde{\theta}_{i,t}$  is the non-robust estimation of the ground-truth  $\theta_{i}$ , and  $\hat{\pmb{\theta}}_{V_{i,t},t - 1}$  is the robust estimation of the inferred cluster  $V_{i,t}$  for user  $i$  at round  $t$ . Since the clustering structure is correct at  $t$ ,  $\hat{\pmb{\theta}}_{V_{i,t},t - 1}$  is the robust estimation of user  $i$ 's ground-truth cluster's preference vector  $\pmb{\theta}^{j(i)} = \pmb{\theta}_i$  at round  $t$ .
$$
\begin{array}{l} \tilde {\boldsymbol {\theta}} _ {i, t} - \boldsymbol {\theta} _ {i} = (\lambda \boldsymbol {I} + \tilde {\boldsymbol {M}} _ {i, t}) ^ {- 1} \tilde {\boldsymbol {b}} _ {i, t} - \boldsymbol {\theta} _ {i} \\ = (\lambda \boldsymbol {I} + \sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\boldsymbol{x}_{a_{s}}^{\top})^{-1}(\sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}r_{s}) - \boldsymbol{\theta}_{i} \\ = \left(\lambda I + \sum_ {\substack {s \in [ t ] \\ i _ {s} = i}} \boldsymbol {x} _ {a _ {s}} \boldsymbol {x} _ {a _ {s}} ^ {\top}\right) ^ {- 1} \left(\sum_ {\substack {s \in [ t ] \\ i _ {s} = i}} \boldsymbol {x} _ {a _ {s}} \left(\boldsymbol {x} _ {a _ {s}} ^ {\top} \boldsymbol {\theta} _ {i} + \eta_ {s}\right)\right) - \boldsymbol {\theta} _ {i} \tag{57} \\ = (\lambda I + \sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\boldsymbol{x}_{a_{s}}^{\top})^{-1}\bigl((\lambda I + \sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\boldsymbol{x}_{a_{s}}^{\top})\boldsymbol{\theta}_{i} - \lambda \boldsymbol{\theta}_{i} + \sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\eta_{s})\bigr) - \boldsymbol{\theta}_{i} \\ = -\lambda \tilde{M}^{\prime -1}_{i,t}\boldsymbol{\theta}_{i} + \tilde{M}^{\prime -1}_{i,t}\sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\eta_{s}  , \\ \end{array}
$$
where we denote  $\tilde{M}_{i,t}^{\prime}\triangleq \lambda I + \sum_{\substack{s\in [t]\\ i_s = i}}\boldsymbol{x}_{a_s}\boldsymbol{x}_{a_s}^\top$ , and Eq.(57) is because since user  $i$  is normal, we have  $c_{s} = 0,\forall s:i_{s} = i$
$$
\begin{array}{l} \left\| \tilde{\boldsymbol{\theta}}_{i,t} - \boldsymbol{\theta}_{i}\right\|_{2}\leq \left\| \lambda \tilde{\boldsymbol{M}}^{\prime -1}_{i,t}\boldsymbol{\theta}_{i}\right\|_{2} + \left\| \tilde{\boldsymbol{M}}^{\prime -1}_{i,t}\sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\eta_{s}\right\|_{2} \\ \leq \lambda \left\| \tilde {\boldsymbol {M}} _ {i, t} ^ {\prime - \frac {1}{2}} \right\| _ {2} ^ {2} \| \boldsymbol {\theta} _ {i} \| _ {2} + \left\| \tilde {\boldsymbol {M}} _ {i, t} ^ {\prime - \frac {1}{2}} \sum_ {\substack {s \in [ t ] \\ i _ {s} = i}} \boldsymbol {x} _ {a _ {s}} \eta_ {s} \right\| _ {2} \left\| \tilde {\boldsymbol {M}} _ {i, t} ^ {\prime - \frac {1}{2}} \right\| _ {2} (58) \\ \leq \frac {\sqrt {\lambda} + \left\| \sum_ {s \in [ t ]} x _ {a _ {s}} \eta_ {s} \right\| _ {\tilde {M} _ {i , t} ^ {\prime - 1}}}{\sqrt {\lambda_ {\min} \left(\tilde {M} _ {i , t} ^ {\prime}\right)}}, (59) \\ \end{array}
$$
where Eq.(58) follows by the Cauchy-Schwarz inequality and the inequality for the operator norm of matrices, and Eq.(59) follows by the Courant-Fischer theorem and the fact that  $\lambda_{\mathrm{min}}(\tilde{\boldsymbol{M}}_{i,t}^{\prime})\geq \lambda$  Following Theorem 1 in [1], for a fixed normal user  $i$ , with probability at least  $1 - \delta$  for some  $\delta \in (0,1)$  we have:
$$
\begin{array}{l} \left\| \sum_{\substack{s\in [t]\\ i_{s} = i}}\boldsymbol{x}_{a_{s}}\eta_{s}\right\|_{\tilde{\boldsymbol{M}}^{\prime}_{i,t}^{-1}}\leq \sqrt{2\log(\frac{1}{\delta}) + \log(\frac{\det(\tilde{\boldsymbol{M}}^{\prime}_{i,t})}{\det(\lambda\boldsymbol{I})})} \\ \leq \sqrt {2 \log (\frac {1}{\delta}) + d \log (1 + \frac {T _ {i , t}}{\lambda d})}, \tag {60} \\ \end{array}
$$
where Eq.(60) is because  $\operatorname*{det}(\tilde{\boldsymbol{M}}_{i,t}^{\prime})\leq \left(\frac{\operatorname{trace}(\lambda\boldsymbol{I} + \sum_{s\in[t]}\boldsymbol{x}_{a_s}\boldsymbol{x}_{a_s}^\top)}{d}\right)^d\leq \left(\frac{\lambda d + T_{i,t}}{d}\right)^d$  and  $\operatorname*{det}(\lambda I) = \lambda^d$
Plugging this into Eq.(59), we can get
$$
\left\| \tilde {\boldsymbol {\theta}} _ {i, t} - \boldsymbol {\theta} _ {i} \right\| _ {2} \leq \frac {\sqrt {\lambda} + \sqrt {2 \log (\frac {1}{\delta}) + d \log (1 + \frac {T _ {i , t}}{\lambda d})}}{\sqrt {\lambda_ {\operatorname* {m i n}} \left(\tilde {\boldsymbol {M}} _ {i , t} ^ {\prime}\right)}}. \tag {61}
$$
Then we need to bound  $\left\| \pmb{\theta}_i - \hat{\pmb{\theta}}_{V_{i,t},t}\right\|_2$ . With the correct clustering,  $V_{i,t} = V_{j(i)}$ , we have
$$
\begin{array}{l} \hat {\boldsymbol {\theta}} _ {V _ {i, t}, t} - \boldsymbol {\theta} _ {i} = \boldsymbol {M} _ {V _ {i, t}, t} ^ {- 1} \boldsymbol {b} _ {V _ {j, t}, t} \\ = (\lambda \boldsymbol {I} + \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\boldsymbol{x}^{\top}_{a_{s}})^{-1}(\sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}r_{s}) - \boldsymbol{\theta}_{i} \\ = \left(\lambda I + \sum_ {\substack {s \in [ t ] \\ i _ {s} \in V _ {j} (i)}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} \boldsymbol {x} _ {a _ {s}} ^ {\top}\right) ^ {- 1} \left(\sum_ {\substack {s \in [ t ] \\ i _ {s} \in V _ {j} (i)}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} \left(\boldsymbol {x} _ {a _ {s}} ^ {\top} \boldsymbol {\theta} _ {i} + \eta_ {s} + c _ {s}\right)\right) - \theta_ {i} (62) \\ = (\lambda \boldsymbol {I} + \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\boldsymbol{x}^{\top}_{a_{s}})^{-1}\bigl((\lambda \boldsymbol {I} + \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\boldsymbol{x}^{\top}_{a_{s}})\boldsymbol{\theta}_{i} - \lambda \boldsymbol{\theta}_{i} \\ \left. \right. + \sum_{\substack{s\in [t]\\i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\eta_{s} + \sum_{\substack{s\in [t]\\i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}c_{s})\right)\right) - \theta_{i} \\ = - \lambda M _ {V _ {i, t}, t} ^ {- 1} \boldsymbol {\theta} _ {i} + M _ {V _ {i, t}, t} ^ {- 1} \sum_ {\substack {s \in [ t ] \\ i _ {s} \in V _ {j (i)}}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} \eta_ {s} + M _ {V _ {i, t}, t} ^ {- 1} \sum_ {\substack {s \in [ t ] \\ i _ {s} \in V _ {j (i)}}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} c _ {s}. (63) \\ \end{array}
$$
$$
\left\| \boldsymbol{\theta}_{i} - \hat{\boldsymbol{\theta}}_{V_{i,t},t}\right\|_{2}\leq \lambda \left\| M_{V_{i,t},t}^{-1}\boldsymbol{\theta}_{i}\right\|_{2} + \left\| M_{V_{i,t},t}^{-1}\sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\eta_{s}\right\|_{2} + \left\| M_{V_{i,t},t}^{-1}\sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}c_{s}\right\|_{2}
$$
$$
\begin{array}{l} \leq \lambda \left\| M_{V_{i,t},t}^{-\frac{1}{2}}\right\|_{2}^{2}\| \boldsymbol{\theta}_{i}\|_{2} + \left\| M_{V_{i,t},t}^{-\frac{1}{2}}\sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\eta_{s}\right\|_{2}\left\| M_{V_{i,t},t}^{-\frac{1}{2}}\right\|_{2} \\ + \left\| M _ {V _ {i, t}, t} ^ {- \frac {1}{2}} \sum_ {\substack {s \in [ t ] \\ i _ {s} \in V _ {j (i)}}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} \eta_ {s} \right\| _ {2} \left\| M _ {V _ {i, t}, t} ^ {- \frac {1}{2}} \right\| _ {2} \tag{64} \\ \end{array}
$$
$$
\leq \frac {\sqrt {\lambda} + \left\| \sum_ {s \in [ t ] \atop i _ {s} \in V _ {j (i)}} w _ {i _ {s} , s} \boldsymbol {x} _ {a _ {s}} \eta_ {s} \right\| _ {M _ {V _ {i , t} , t} ^ {- 1}} + \left\| \sum_ {s \in [ t ] \atop i _ {s} \in V _ {j (i)}} w _ {i _ {s}, s} \boldsymbol {x} _ {a _ {s}} c _ {s} \right\| _ {M _ {V _ {i, t} , t} ^ {- 1}}}{\sqrt {\lambda_ {\operatorname* {m i n}} (M _ {V _ {i , t} , t})}} \tag {65}
$$
Let  $\tilde{\pmb{x}}_s\triangleq \sqrt{w_{i_s,s}}\pmb{x}_{a_s}$ ,  $\tilde{\eta}_s\triangleq \sqrt{w_{i_s,s}}\eta_s$ , then we have:  $\| \tilde{\pmb{x}}_s\| _2\leq \| \sqrt{w_{i_s,s}}\| _2\| \pmb{x}_{a_s}\| _2\leq 1$ ,  $\tilde{\eta}_s$  is still 1-sub-gaussian (since  $\eta_{s}$  is 1-sub-gaussian and  $\sqrt{w_{i_s,s}}\leq 1$ ),  $M_{V_{i,t},t} = \lambda I + \sum_{\substack{s\in [t]\\ i_s\in V_{j(i)}}}\tilde{\pmb{x}}_s\tilde{\pmb{x}}_s^\top$ , and  $\left\|\sum_{\substack{s\in [t]\\ i_s\in V_{j(i)}}}w_{i_s,s}\pmb{x}_{a_s}\eta_s\right\|_{M^{-1}_{V_{i,t},t}}$  becomes  $\left\|\sum_{\substack{s\in [t]\\ i_s\in V_{j(i)}}}\tilde{\pmb{x}}_s\tilde{\eta}_s\right\|_{M^{-1}_{V_{i,t},t}}$ . Then, following Theorem 1 in [1], with probability at least  $1 - \delta$  for some  $\delta \in (0,1)$ , for a fixed normal user  $i$ , we have
$$
\begin{array}{l} \left\| \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}\eta_{s}\right\|_{M^{-1}_{V_{i,t},t}}\leq \sqrt{2\log(\frac{1}{\delta}) + \log(\frac{\det(M_{V_{i,t},t})}{\det(\lambda I)})} \\ \leq \sqrt {2 \log (\frac {1}{\delta}) + d \log (1 + \frac {T _ {\tilde {V} _ {i , t} , t}}{\lambda d})}, \tag {66} \\ \end{array}
$$
where Eq.(60) is because  $\operatorname{det}(M_{V_{i,t},t}) \leq \left( \frac{\operatorname{trace}(\lambda I + \sum_{\substack{s \in [t] \\ i_s \in V_{j(i)}}} \boldsymbol{x}_{a_s} \boldsymbol{x}_{a_s}^\top)}{d} \right)^d \leq \left( \frac{\lambda d + T_{V_{i,t},t}}{d} \right)^d$ , and  $\operatorname{det}(\lambda I) = \lambda^d$ .
For  $\left\| \sum_{\substack{s\in [t]\\ i_s\in V_j(i)}}w_{i_s,s}\boldsymbol{x}_{a_s}c_s\right\|_{M_{V_i,t,t}^{-1}},$  we have
$$
\begin{array}{l} \left\| \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}w_{i_{s},s}\boldsymbol{x}_{a_{s}}c_{s}\right\|_{M^{-1}_{V_{i,t},t}}\leq \sum_{\substack{s\in [t]\\ i_{s}\in V_{j(i)}}}|c_{s}|  w_{i_{s},s}\left\| \boldsymbol{x}_{a_{s}}\right\|_{M^{-1}_{V_{i,t},t}} \\ \leq \alpha C, \tag {67} \\ \end{array}
$$
where Eq.(67) is because  $w_{i_s,s} \leq \frac{\alpha}{\|\pmb{x}_{a_s}\|_{M_{i_s,s}' - 1}} \leq \frac{\alpha}{\|\pmb{x}_{a_s}\|_{M_{i_s,t}' - 1}} \leq \frac{\alpha}{\|\pmb{x}_{a_s}\|_{M_{V_{i,t},t}' - 1}}$  (since  $M_{V_{i,t},t} \succeq M_{i_s,t}' \succeq M_{i_s,s}'$ ,  $M_{i_s,s}' \succeq M_{i_s,t}' \succeq M_{i_s,t}' \succeq M_{V_{i,t},t}'$ ,  $\| \pmb{x}_{a_s}\|_{M_{i_s,s}' - 1} \geq \| \pmb{x}_{a_s}\|_{M_{i_s,t}' - 1} \geq \| \pmb{x}_{a_s}\|_{M_{V_{i,t},t}' - 1})$ , and  $\sum_{s \in [t]} |c_s| \leq C$ ).
$$
\left\| \boldsymbol {\theta} _ {i} - \hat {\boldsymbol {\theta}} _ {V _ {i, t}, t} \right\| _ {2} \leq \frac {\sqrt {\lambda} + \sqrt {2 \log (\frac {1}{\delta}) + d \log (1 + \frac {T _ {V _ {i , t} , t}}{\lambda d})} + \alpha C}{\sqrt {\lambda_ {\min} \left(\boldsymbol {M} _ {V _ {i , t} , t}\right)}}. \tag {68}
$$
With Eq.(68), Eq.(61) and Eq.(56), together with Lemma 1, we have that for a normal user  $i$ , for any  $t \geq T_0$ , with probability at least  $1 - 5\delta$  for some  $\delta \in (0, \frac{1}{5})$
$$
\begin{array}{l} \left\| \tilde {\boldsymbol {\theta}} _ {i, t} - \hat {\boldsymbol {\theta}} _ {V _ {i, t}, t} \right\| \leq \left\| \tilde {\boldsymbol {\theta}} _ {i, t} - \boldsymbol {\theta} _ {i} \right\| _ {2} + \left\| \boldsymbol {\theta} _ {i} - \hat {\boldsymbol {\theta}} _ {V _ {i, t}, t} \right\| _ {2} \\ \leq \frac {\sqrt {\lambda} + \sqrt {2 \log \left(\frac {1}{\delta}\right) + d \log \left(1 + \frac {T _ {i , t}}{\lambda d}\right)}}{\sqrt {\lambda_ {\min} \left(\tilde {M} _ {i , t} ^ {\prime}\right)}} + \frac {\sqrt {\lambda} + \sqrt {2 \log \left(\frac {1}{\delta}\right) + d \log \left(1 + \frac {T _ {V _ {i , t} , t}}{\lambda d}\right)} + \alpha C}{\sqrt {\lambda_ {\min} \left(M _ {V _ {i , t} , t}\right)}}, \tag {69} \\ \end{array}
$$
which is exactly the detection condition in Line 7 of Algo.2.
Therefore, by the proof by contrapositive, we complete the proof of Theorem 5.
F Description of Baselines
We compare RCLUB-WCU to the following five baselines for recommendations.
- LinUCB[17]: A state-of-the-art bandit approach for a single user without corruption.
- LinUCB-Ind: Use a separate LinUCB for each user.
- CW-OFUL[12]: A state-of-the-art bandit approach for single user with corruption.
- CW-OFUL-Ind: Use a separate CW-OFUL for each user.
- CLUB[8]: A graph-based clustering of bandits approach for multiple users without corruption.
- SCLUB[21]: A set-based clustering of bandits approach for multiple users without corruption.
G More Experiments
G.1 Different Corruption Levels
To see our algorithm's performance under different corruption levels, we conduct the experiments under different corruption levels for RCLUB-WCU, CLUB, and SCLUB on Amazon and Yelp datasets. Recall the corruption mechanism in Section 6.1, we set  $k$  as 1,000; 10,000; 100,000. The results are shown in Fig.4. All the algorithms' performance becomes worse when the corruption level increases. But RCLUB-WCU is much robust than the baselines.
(a) Amazon Corruption Level
(b) Yelp Corruption Level
Figure 4: Cumulative regret in different corruption levels
(a) Amazon Cluster Number
Figure 5: Cumulative regret with different cluster numbers
(b) Yelp Cluster Number
G.2 Different Cluster numbers