SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Due to deviations from linearity, the estimation  $\hat{R}_{a,t}$  computed by a linear function is no longer accurate. To handle the estimation uncertainty of model misspecifications, we design an enlarged confidence radius  $C_{a,t}$ . The first term of  $C_{a,t}$  in Eq.(5) captures the uncertainty of online learning for the linear part, and the second term related to  $\epsilon_*$  reflects the additional uncertainty from deviations from linearity. The design of  $C_{a,t}$  theoretically relies on Lemma 5.6 which will be given in Section 5.
Update User Statistics. Based the feedback  $r_t$ , in Line 8 and 9, the agent updates the statistics for user  $i_t$ . Specifically, the agent estimates the preference vector  $\theta_{i_t}$  by
$$
\hat {\boldsymbol {\theta}} _ {i _ {t}, t} = \underset {\boldsymbol {\theta} \in \mathbb {R} ^ {d}} {\arg \min } \sum_ {\substack {s \in [ t ] \\ i _ {s} = i _ {t}}} (r _ {s} - \boldsymbol {x} _ {a _ {s}} ^ {\top} \boldsymbol {\theta}) ^ {2} + \lambda \| \boldsymbol {\theta} \| _ {2} ^ {2}, \tag{6}
$$
with solution  $\hat{\pmb{\theta}}_{i_t,t} = (\lambda \pmb {I} + \pmb{M}_{i_t,t})^{-1}\pmb{b}_{i_t,t}$ , where  $\pmb{M}_{i_t,t} = \sum_{\substack{s\in [t]\\ i_s = i_t}}\pmb{x}_{a_s}\pmb{x}_{a_s}^\top ,\pmb{b}_{i_t,t} = \sum_{\substack{s\in [t]\\ i_s = i_t}}r_{a_s}\pmb{x}_{a_s}$ .
Update the Graph  $G_{t}$ . Finally, in Line 10, the agent verifies whether the similarities between user  $i_{t}$  and other users are still true based on the updated estimation  $\hat{\theta}_{i_t,t}$ . For every user  $\ell \in \mathcal{U}$  connected with user  $i_{t}$  via edge  $(i_t,\ell)\in E_{t - 1}$ , if the gap between her estimated preference vector  $\hat{\theta}_{\ell ,t}$  and  $\hat{\theta}_{i_t,t}$  is larger than a threshold supported by Lemma H.1, the agent will delete the edge  $(i_t,\ell)$  to split them apart. The threshold in Line 10 is carefully designed, taking both estimation uncertainty in a linear model and deviations from linearity into consideration. As shown in the proof of Lemma H.1 (in Appendix H), using this threshold, with high probability, edges between users in the same ground-truth clusters will not be deleted, and edges between users that are not  $\zeta$ -close will always be deleted. Together with the filtering step in Line 5, with high probability, the algorithm will leverage all the collaborative information of similar users and avoid misusing the information of dissimilar users. The updated graph  $G_{t}$  will be used in the next round.
In this section, we theoretically analyze the performance of the RCLUMB algorithm by giving an upper bound of the expected regret defined in Eq.(2). Due to the space limitation, we only show the main result (Theorem 5.3), key lemmas, and a sketched proof for Theorem 5.3. Detailed proofs, other technical lemmas, and the regret analysis of the RSLUMB algorithm can be found in the Appendix.
To state our main result, we first give two definitions as follows. The first definition is about the minimum separable gap constant  $\gamma_{1}$  of a CBMUM problem instance.
Definition 5.1 (Minimum separable gap  $\gamma_{1}$ ). The minimum separable gap constant  $\gamma_{1}$  of a CBMUM problem instance is the minimum gap over the gaps among users that are greater than  $\zeta$  (Eq. (3))
$$
\gamma_ {1} = \min  \left\{\| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} _ {\ell} \| _ {2}: \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} _ {\ell} \| _ {2} > \zeta , \forall i, \ell \in \mathcal {U} \right\}, \text {w i t h} \min  \varnothing = \infty .
$$
Remark 2. In CBMUM, the role of  $\gamma_{1} - \zeta$  is similar to that of  $\gamma$  (given in Assumption 3.1) in the previous CB problem with perfectly linear models, quantifying the hardness of performing clustering on the problem instance. Intuitively, users are easier to cluster if  $\gamma_{1}$  is larger, and the deduction of  $\zeta$  shows the additional difficulty due to model diviations. If there are no misspecifications, i.e.,  $\zeta = 2\epsilon_*\sqrt{\frac{2}{\lambda_x}} = 0$ , then  $\gamma_{1} = \gamma$ , recovering the minimum separable gap between clusters in the classic CB problem [12, 25] without model misspecifications.
The second definition is about the number of "hard-to-cluster users"  $\tilde{u}$ .
Definition 5.2 (Number of "hard-to-cluster users"  $\tilde{u}$ ). The number of "hard-to-cluster users"  $\tilde{u}$  is the number of users in the ground-truth clusters which are  $\zeta$ -close to some other ground-truth clusters
$$
\tilde {u} = \sum_ {j \in [ m ]} | V _ {j} | \times \mathbb {I} \left\{\exists j ^ {\prime} \in [ m ], j ^ {\prime} \neq j: \left\| \boldsymbol {\theta} ^ {j ^ {\prime}} - \boldsymbol {\theta} ^ {j} \right\| _ {2} \leq \zeta \right\},
$$
where  $\mathbb{I}\{\cdot\}$  denotes the indicator function of the argument,  $|V_j|$  denotes the number of users in  $V_j$ .
Remark 3.  $\tilde{u}$  captures the number of users who belong to different ground-truth clusters but their gaps are less than  $\zeta$ . These users may be merged into one cluster by mistake and cause certain regret.
The following theorem gives an upper bound on the expected regret achieved by RCLUMB.
Theorem 5.3 (Main result on regret bound). Suppose that the assumptions in Section 3 are satisfied. Then the expected regret of the RCLUMB algorithm for  $T$  rounds satisfies
$$
\begin{array}{l} R (T) \leq O \left(u \left(\frac {d}{\tilde {\lambda} _ {x} (\gamma_ {1} - \zeta) ^ {2}} + \frac {1}{\tilde {\lambda} _ {x} ^ {2}}\right) \log T + \frac {\tilde {u}}{u} \frac {\epsilon_ {*} \sqrt {d} T}{\tilde {\lambda} _ {x} ^ {1 . 5}} + \epsilon_ {*} T \sqrt {m d \log T} + d \sqrt {m T} \log T\right) (7) \\ \leq O \left(\epsilon_ {*} T \sqrt {m d \log T} + d \sqrt {m T} \log T\right), (8) \\ \end{array}
$$
where  $\gamma_{1}$  is defined in Definition 5.1, and  $\tilde{u}$  is defined in Definition 5.2).
Discussion and Comparison. The bound in Eq.(7) has four terms. The first term is the time needed to gather enough information to assign appropriate clusters for users. The second term is the regret caused by misclustering  $\zeta$ -close but not precisely similar users together, which is unavoidable with model misspecifications. The third term is from the preference estimation errors caused by model deviations. The last term is the usual term in CB with perfectly linear models [12, 25, 27].
Let us discuss how the parameters affect this regret bound.
- If  $\gamma_{1} - \zeta$  is large, the gaps between clusters that are not "  $\zeta$ -close" are much greater than the minimum gap  $\zeta$  for the algorithm to distinguish, the first term in Eq.(7) will be small as it is easy to identify their dissimilarities. The role of  $\gamma_{1} - \zeta$  in CBMUM is similar to that of  $\gamma$  in the previous CB.
- If  $\tilde{u}$  is small, indicating that few ground-truth clusters are “ $\zeta$ -close”, RCLUMB will hardly miscluster different ground-truth clusters together thus the second term in Eq.(7) will be small.
- If the deviation level  $\epsilon_{*}$  is small, the user models are close to linearity and the misspecifications will not affect the estimations much, then both the second and third term in Eq.(7) will be small. The following theorem gives a regret lower bound of the CBMUM problem.
Theorem 5.4 (Regret lower bound for CBMUM). There exists a problem instance for the CBMUM problem such that for any algorithm  $R(T) \geq \Omega(\epsilon_* T \sqrt{d})$ .
The proof can be found in Appendix F. The upper bounds in Theorem 5.3 asymptotically match this lower bound with respect to  $T$  up to logarithmic factors (and a constant factor of  $\sqrt{m}$  where  $m$  is typically small in real-applications), showing the tightness of our theoretical results. Additionally, we conjecture the gap for the  $m$  factor is due to the strong assumption that cluster structures are known to prove this lower bound, and whether there exists a tighter lower bound is left for future work.
We then compare our results with two degenerate cases. First, when  $m = 1$  (indicating  $\tilde{u} = 0$ ), our setting degenerates to the MLB problem where all users share the same preference vector. In this case, our regret bound is  $O(\epsilon_{*}T\sqrt{d\log T} + d\sqrt{T}\log T)$ , exactly matching the current best bound of MLB [23]. Second, when  $\epsilon_{*} = 0$ , our setting reduces to the CB problem with perfectly linear user models and our bounds become  $O(d\sqrt{mT}\log T)$ , also perfectly match the existing best bound of the CB problem [25, 27]. The above discussions and comparisons show the tightness of our regret bounds. Additionally, we also provide detailed discussions on why trivially combining existing works on CB and MLB would not get any non-vacuous regret upper bound in Appendix D.
We define the following "good partition" for ease of interpretation.
Definition 5.5 (Good partition). RCLUMB does a "good partition" at  $t$ , if the cluster  $\overline{V}_t$  assigned to  $i_t$  is a  $\zeta$ -good cluster, and it contains all the users in the same ground-truth cluster as  $i_t$ , i.e.,
$$
\left\| \boldsymbol {\theta} _ {i _ {t}} - \boldsymbol {\theta} _ {\ell} \right\| _ {2} \leq \zeta , \forall \ell \in \bar {V} _ {t}, \text {a n d} V _ {j (i _ {t})} \subseteq \bar {V} _ {t}. \tag {9}
$$
Note that when the algorithm does a "good partition" at  $t$ ,  $\overline{V}_t$  will contain all the users in the same ground-truth cluster as  $i_t$  and may only contain some other  $\zeta$ -close users with respect to  $i_t$ , which means the gathered information associated with  $\overline{V}_t$  can be used to infer user  $i_t$ 's preference with high accuracy. Also, it is obvious that under a "good partition", if  $\overline{V}_t \in \mathcal{V}$ , then  $\overline{V}_t = V_{j(i_t)}$  by definition.
Next, we give a sketched proof for Theorem 5.3.
Proof. [Sketch for Theorem 5.3] The proof mainly contains two parts. First, we prove there is a sufficient time  $T_0$  for RCLUMB to get a "good partition" with high probability. Second, we prove the regret upper bound for RCLUMB after maintaining a "good partition". The most challenging part is to bound the regret caused by misclustering  $\zeta$ -close users after getting a "good partition".
1. Sufficient time to maintain a "good partition". With the item regularity (Assumption 3.3), we can prove after some  $T_0$  (defined in Lemma H.1 in Appendix H), RCLUMB will always have a
"good partition". Specifically, after  $t \geq O\left(u\left(\frac{d}{\tilde{\lambda}_x(\gamma_1 - \zeta)^2} + \frac{1}{\tilde{\lambda}_x^2}\right) \log T\right)$ , for any user  $i \in \mathcal{U}$ , the gap between the estimated  $\hat{\theta}_{i,t}$  and the ground-truth  $\pmb{\theta}^{j(i)}$  is less than  $\frac{\gamma_1}{4}$  with high probability. With this, we can get: for any two users  $i$  and  $\ell$ , if their gap is greater than  $\zeta$ , it will trigger the deletion of the edge  $(i,\ell)$  (Line 10 of Algo.1) with high probability; on the other hand, when the deletion condition of the edge  $(i,\ell)$  is satisfied, then  $\left\| \pmb{\theta}^{j(i)} - \pmb{\theta}^{j(\ell)} \right\|_2 > 0$ , which means user  $i$  and  $\ell$  belong to different ground-truth clusters by Assumption 3.1 with high probability. Therefore, we can get that with high probability, all those users in the same ground-truth cluster as  $i_t$  will be directly connected with  $i_t$ , and users directly connected with  $i_t$  must be  $\zeta$ -close to  $i_t$ . By filtering users directly linked with  $i_t$  as the cluster  $\overline{V}_t$  (Algo.1 Line 5) and the definition of "good partition", we can ensure that RCLUMB will keep a "good partition" afterward with high probability.
2. Bounding the regret after getting a "good partition". After  $T_0$ , with the "good partition", we can prove the following lemma that gives a bound of the difference between  $\hat{\theta}_{\overline{V}_t, t-1}$  and ground-truth  $\theta_{i_t}$  in direction of action vector  $x_a$ , and supports the design of the confidence radius  $C_{a,t}$  in Eq.(5).
Lemma 5.6. With probability at least  $1 - 5\delta$  for some  $\delta \in (0,\frac{1}{5}),\forall t\geq T_0$
$$
\left|\boldsymbol{x}_{a}^{\top}(\boldsymbol{\theta}_{it}- \hat{\boldsymbol{\theta}}_{\overline{V}_{t},t - 1})\right|\leq \frac{\epsilon_{*}\sqrt{2d}}{\tilde{\lambda}_{x}^{\frac{3}{2}}}\mathbb{I}\{\overline{V}_{t}\notin \mathcal{V}\} +\epsilon_{*}\sum_{\substack{s\in [t - 1]\\ i_{s}\in \overline{V}_{t}}}\left|\boldsymbol{x}_{a}^{\top}\overline{\boldsymbol{M}}_{\overline{V}_{t},t - 1}^{-1}\boldsymbol{x}_{a_{s}}\right| + \beta   \| \boldsymbol{x}_{a}\|_{\overline{\boldsymbol{M}}_{\overline{V}_{t},t - 1}^{-1}}  .
$$
To prove this lemma, we consider the following two situations.
(i) Assigning a perfect cluster for  $i_t$ . In this case,  $\overline{V}_t \in \mathcal{V}$ , meaning the cluster assigned for user  $i_t$  is the same as her ground-truth cluster, i.e.,  $\overline{V}_t = V_{j(i_t)}$ . Therefore, we have that  $\forall \ell \in \overline{V}_t, \theta_\ell = \theta_{i_t}$ . With careful analysis, we can bound  $\left| x_a^\top (\theta_{i_t} - \hat{\theta}_{\overline{V}_t, t-1}) \right|$  by  $C_{a,t}$  (defined in Eq.(5)).
(ii) Bounding the term of misclustering  $i_t$ 's  $\zeta$ -close users. In this case,  $\overline{V}_t \notin \mathcal{V}$ , meaning the algorithm misclusters user  $i_t$ , i.e.,  $\overline{V}_t \neq V_{j(i_t)}$ . Thus, we do not have  $\forall \ell \in \overline{V}_t$ ,  $\pmb{\theta}_{\ell} = \pmb{\theta}_{i_t}$  anymore, but we have all the users in  $\overline{V}_t$  are  $\zeta$ -close to  $i_t$  (by "good partition"), i.e.,  $\| \pmb{\theta}_{i_s} - \pmb{\theta}_{i_t} \|_2 \leq \zeta$ ,  $\forall \ell \in \overline{V}_t$ . Then an additional term can be caused by using the information of  $i_t$ 's  $\zeta$ -close users in  $\overline{V}_t$  lying in different ground-truth clusters from  $i_t$  to estimate  $\pmb{\theta}_{i_t}$ . It is highly challenging to bound this part.
We will get an extra term  $\left|\pmb{x}_a^\top \overline{\pmb{M}}_{\overline{V}_t,t - 1}^{-1}\sum_{\substack{s\in [t - 1]\\ i_s\in \overline{V}_t}}\pmb{x}_{a_s}\pmb{x}_{a_s}^\top (\pmb{\theta}_{i_s} - \pmb{\theta}_{i_t})\right|$  when bounding the regret in this case, where  $\| \pmb {\theta}_{\ell} - \pmb{\theta}_{i_t}\| _2\leq \zeta ,\forall \ell \in \overline{V}_t$ . It is an easy-to-be-made mistake to directly drag  $\| \pmb{\theta}_{i_s} - \pmb{\theta}_{i_t}\| _2$  out to bound it by  $\left\| \pmb{x}_a^\top \overline{\pmb{M}}_{\overline{V}_t,t - 1}^{-1}\sum_{\substack{s\in [t - 1]\\ i_s\in \overline{V}_t}}\pmb{x}_{a_s}\pmb{x}_{a_s}^\top \right\| _2\times \zeta$ . With subtle analysis, we propose the following lemma to bound the above term.
Lemma 5.7 (Bound of error caused by misclustering).  $\forall t\geq T_0$  , if the current partition by RCLUMB is a "good partition", and  $\overline{V}_t\notin \mathcal{V}$  , then for all  $\pmb {x}_a\in \mathbb{R}^d$ $\| \pmb {x}_a\| _2\leq 1$  , with probability at least  $1 - \delta$  ..
$$
\left|\boldsymbol{x}_{a}^{\top}\overline{\boldsymbol{M}}_{\overline{V}_{t},t - 1}^{-1}\sum_{\substack{s\in [t - 1]\\ i_{s}\in \overline{V}_{t}}}\boldsymbol{x}_{a_{s}}\boldsymbol{x}_{a_{s}}^{\top}(\boldsymbol{\theta}_{i_{s}} - \boldsymbol{\theta}_{i_{t}})\right|\leq \frac{\epsilon_{*}\sqrt{2d}}{\bar{\lambda}_{x}^{\frac{3}{2}}}.
$$
This lemma is quite general. Please see Appendix G for details about its proof.
The expected occurrences of  $\{\overline{V}_t \notin \mathcal{V}\}$  is bounded by  $\frac{\tilde{u}}{u} T$  with Assumption 3.2, Definition 5.2 and 5.5. The result follows by bounding the expected sum of the bounds for the instantaneous regret using Lemma 5.6 with delicate analysis due to the time-varying clustering structure kept by RCLUMB.
This section compares RCLUMB and RSCLUMB with CLUB [12], SCLUB [27], LinUCB with a single estimated vector for all users, LinUCB-Ind with separate estimated vectors for each user, and two modifications of LinUCB in [23] which we name as RLinUCB and RLinUCB-Ind. We use averaged reward as the evaluation metric, where the average is taken over ten independent trials.
(a) Synthetic
(b) Yelp Case 1
(c) Yelp Case 2
(d) Movielens Case 1
(e) Movielens Case 2
Figure 1: The figures compare RCLUMB and RSCLUMB with the baselines. (a) shows the result on synthetic data, (b) and (c) show the results on Yelp dataset, (d) and (e) show the results on Movielens dataset. All experiments are under the setting of  $u = 1,000$  users,  $m = 10$  clusters, and  $d = 50$ . All results are averaged under 10 random trials. The error bars are standard deviations divided by  $\sqrt{10}$ .
6.1 Synthetic Experiments
We consider a setting with  $u = 1,000$  users,  $m = 10$  clusters and  $T = 10^{6}$  rounds. The preference and feature vectors are in  $d = 50$  dimension with each entry drawn from a standard Gaussian distribution, and are normalized to vectors with  $\| .\|_{2} = 1$  [27]. We fix an arm set with  $|\mathcal{A}| = 1000$  items, at each round  $t$ , 20 items are randomly selected to form a set  $\mathcal{A}_t$  for the user to choose from. We construct a matrix  $\epsilon \in \mathbb{R}^{1,000\times 1,000}$  in which each element  $\epsilon (i,j)$  is drawn uniformly from the range  $(-0.2,0.2)$  to represent the deviation. At  $t$ , for user  $i_t$  and the item  $a_t$ ,  $\epsilon (i_t,a_t)$  will be added to the feedback as the deviation, which corresponds to the  $\epsilon_{a_t}^{i_t,t}$  defined in Eq.(1).
The result is provided in Figure 1(a), showing that our algorithms have clear advantages: RCLUMB improves over CLUB by  $21.9\%$ , LinUCB by  $194.8\%$ , LinUCB-Ind by  $20.1\%$ , SCLUB by  $12.0\%$ , RLinUCB by  $185.2\%$  and RLinUCB-Ind by  $10.6\%$ . The performance difference between RCLUMB and RSCLUMB is very small as expected. RLinUCB performs better than LinUCB; RLinUCB-Ind performs better than LinUCB-Ind and CLUB, showing that the modification of the recommendation policy is effective. The set-based RSCLUMB and SCLUB can separate clusters quicker and have advantages in the early period, but eventually RCLUMB catches up with RSCLUMB, and SCLUB is surpassed by RLinUCB-Ind because it does not consider misspecifications. RCLUMB and RSCLUMB perform better than RLinUCB-Ind, which shows the advantage of the clustering. So it can be concluded that both the modification for misspecification and the clustering structure are critical to improving the algorithm's performance. We also have done some ablation experiments on different scales of  $\epsilon^{*}$  in Appendix P, and we can notice that under different  $\epsilon^{*}$ , our algorithms always outperform the baselines, and some baselines will perform worse as  $\epsilon^{*}$  increases.
6.2 Experiments on Real-world Datasets
We conduct experiments on the Yelp data and the  $20m$  MovieLens data [17]. For both data, we have two cases due to the different methods for generating feedback. For case 1, we extract 1,000 items with most ratings and 1,000 users who rate most; then we construct a binary matrix  $H^{1,000 \times 1,000}$  based on the user rating [40, 42]: if the user rating is greater than 3, the feedback is 1; otherwise, the feedback is 0. Then we use this binary matrix to generate the preference and feature vectors by singular-value decomposition (SVD) [27, 25, 40]. Similar to the synthetic experiment, we construct a matrix  $\epsilon \in \mathbb{R}^{1,000 \times 1,000}$  in which each element is drawn uniformly from the range  $(-0.2, 0.2)$ . For case 2, we extract 1,100 users who rate most and 1000 items with most ratings. We construct a binary feedback matrix  $H^{1,100 \times 1,000}$  based on the same rule as case 1. Then we select the first 100 rows  $H_{1}^{100 \times 1,000}$  to generate the feature vectors by SVD. The remaining 1,000 rows  $F^{1,000 \times 1,000}$
is used as the feedback matrix, meaning user  $i$  receives  $F(i,j)$  as feedback while choosing item  $j$ . In both cases, at time  $t$ , we randomly select 20 items for the algorithms to choose from. In case 1, the feedback is computed by the preference and feature vector with misspecification, in case 2, the feedback is from the feedback matrix.
The results on Yelp are shown in Fig 1(b) and Fig 1(c). In case 1, RCLUMB improves CLUB by  $45.1\%$ , SCLUB by  $53.4\%$ , LinUCB-One by  $170.1\%$ , LinUCB-Ind by  $46.2\%$ , RLinUCB by  $171.0\%$  and RLinUCB-Ind by  $21.5\%$ . In case 2, RCLUMB improves over clubhouse by  $13.9\%$ , SCLUB by  $5.1\%$ , LinUCB-One by  $135.6\%$ , LinUCB-Ind by  $10.1\%$ , RLinUCB by  $138.6\%$  and RLinUCB by  $8.5\%$ . It is notable that our modeling assumption 3.4 is violated in case 2 since the misspecification range is unknown. We set  $\epsilon_{*} = 0.2$  following our synthetic dataset and it can still perform better than other algorithms. When the misspecification level is known as in case 1, our algorithms' improvement is significantly enlarged, e.g., RCLUMB improves over SCLUB from  $5.1\%$  to  $53.4\%$ .
The results on Movielens are shown in Fig 1(d) and 1(e). In case 1, RCLUMB improves CLUB by  $58.8\%$ , SCLUB by  $92.1\%$ , LinUCB-One by  $107.7\%$ , LinUCB-Ind by  $61.5\%$ , RLinUCB by  $109.5\%$ , and RLinUCB-Ind by  $21.3\%$ . In case 2, RCLUMB improves over Club by  $5.5\%$ , SCLUB by  $2.9\%$ , LinUCB-One by  $28.5\%$ , LinUCB-Ind by  $6.1\%$ , RLinUCB by  $29.3\%$  and RLinUCB-Ind by  $5.8\%$ . The results are consistent with the Yelp data, confirming our superior performance.
We present a new problem of clustering of bandits with misspecified user models (CBMUM), where the agent has to adaptively assign appropriate clusters for users under model misspecifications. We propose two robust CB algorithms, RCLUMB and RSCLUMB. Under milder assumptions than previous CB works, we prove the regret bounds of our algorithms, which match the lower bound asymptotically in  $T$  up to logarithmic factors, and match the state-of-the-art results in several degenerate cases. It is challenging to bound the regret caused by misclustering users with close but not the same preference vectors and use inaccurate cluster-based information to select arms. Our analysis to bound this part of the regret is quite general and may be of independent interest. Experiments on synthetic and real-world data demonstrate the advantage of our algorithms. We would like to state some interesting future works: (1) Prove a tighter regret lower bound for CBMUM, (2) Incorporate recent model selection methods into our fundamental framework to design robust algorithms for CBMUM with unknown exact maximum model misspecification level, and (3) Consider the setting with misspecifications in the underlying user clustering structure rather than user models.
The corresponding author Shuai Li is supported by National Key Research and Development Program of China (2022ZD0114804) and National Natural Science Foundation of China (62376154, 62006151, 62076161). The work of John C.S. Lui was supported in part by the RGC's GRF 14215722.