SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Pier Giuseppe Sessa*
piergiuseppe.sessa@inf.ethz.ch
Nicolò Cesa-Bianchi
Università degli Studi di Milano
nicolo.cesa-bianchi@unimi.it
Pierre Laforgue*
pierre.laforgue@unimi.it
Multitask learning is a powerful framework that enables one to simultaneously learn multiple related tasks by sharing information between them. Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. In this work, we provide novel confidence intervals for multitask regression in the challenging agnostic setting, i.e., when neither the similarity between tasks nor the tasks' features are available to the learner. The obtained intervals do not require i.i.d. data and can be directly applied to bound the regret in online learning. Through a refined analysis of the multitask information gain, we obtain new regret guarantees that, depending on a task similarity parameter, can significantly improve over treating tasks independently. We further propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance, i.e., automatically adapting to task similarity. As a second key application of our results, we introduce a novel multitask active learning setup where several tasks must be simultaneously optimized, but only one of them can be queried for feedback by the learner at each round. For this problem, we design a no-regret algorithm that uses our confidence intervals to decide which task should be queried. Finally, we empirically validate our bounds and algorithms on synthetic and real-world (drug discovery) data.
In many real-world applications, one often faces multiple related tasks to be solved sequentially or simultaneously. The goal of multitask learning (MTL) [4] is to leverage the similarities across the tasks to obtain more accurate and robust models. Indeed, by jointly learning multiple tasks, MTL can exploit their statistical dependencies, yielding better generalization and faster learning than treating each task independently. MTL has gained significant attention in recent years, as it has been shown to be effective in a wide range of applications, including natural language processing, computer vision, federated learning, and drug discovery, see e.g., [11, 19, 16, 27].
A very natural model for learning across multiple tasks is the agnostic multitask (MT) regression approach of [13]. This utilizes a multitask kernel that can interpolate between running  $N$  (number of tasks) independent regressions, and regressing all tasks to their common average, depending on a tunable parameter. Notably, such a kernel does not require any knowledge neither about tasks' features
Figure 1: Independent vs. Multitask (MT) regression. MT regression leverages data coming from multiple related tasks and can yield more accurate and more confident estimates. In this work, we show naive confidence intervals are overly conservative and provide improved ones (shaded in red).
nor about their similarity, thus finding good application in several domains. For instance, Cavallanti et al. [5] study it for online classification, and Cesa-Bianchi et al. [6] for online convex optimization.
However, it is much less understood how to quantify the uncertainty of such MT regression, i.e., assessing confidence in the estimated tasks. In particular, as also outlined by [13] as an open problem, it is important to assess their generalization error as a function of the kernel parameter. Appropriately characterizing these confidence intervals is indeed of crucial importance for a whole set of downstream applications. More concretely, multitask confidence intervals are used in online learning to inform the next decision to be made [6]. In active learning—as we show next—these intervals are pivotal to deciding the most informative task to query.
In this work, we study the agnostic MT regression setup of [13], and provide new multitask confidence intervals (see Figure 1 for a visualization) for the full range of the kernel parameter. Our intervals hold in the so-called adaptive setting, i.e., without requiring i.i.d. data, and are tighter up to a  $\sqrt{N}$  factor than the naive ones employed in [6]. Moreover, we provide the first bounds for the information gain of MT regression and utilize them—together with the derived intervals—to obtain tighter online learning guarantees. The latter depend on a task similarity parameter and can significantly improve over treating tasks independently. Additionally, we propose an adaptive no-regret algorithm that exploits task similarity without knowing this parameter in advance. Finally, we consider a novel multitask active learning setup, where tasks should be simultaneously optimized but only one of them can be queried at each round. We show that the newly derived intervals are also crucial in such a setting, and provide a new algorithm that ensures sublinear regret. We demonstrate the superiority of the derived intervals over previously proposed algorithms on synthetic as well as real-world drug discovery tasks.
Related work. The agnostic MT regression approach of [13] reduces the learning of  $N$  tasks to a single regression problem, as a function of the MT kernel parameter. When combined with support vector machines, it was shown effective in a series of classification problems [13, 24], and since then was studied in various further settings. Cavallanti et al. [5], e.g., analyze mistake bounds for online MT classification algorithms as a function of the kernel parameter. Cesa-Bianchi et al. [6], instead, utilize the MT kernel to prove regret bounds in online MT learning with bandit feedback. Inspired by this, [10] focuses on learning more general kernel structures from data. An important question not addressed by previous work, though, is how to properly quantify the uncertainty of the obtained task estimates. This problem is well-understood in single-task learning (e.g., [2, 25, 8]) but remains largely unexplored in MT domains. As shown in [6], MT confidence intervals can in principle be obtained by a naive application of the single-task guarantees of [2]. However, as we show in Section 2, the so-obtained intervals are extremely conservative and—as a result—can hamper the MT learning performance. Our intervals are tighter by a factor up to  $\sqrt{N}$  w.r.t. the naive ones from [6], yielding novel online learning regret guarantees which can provably improve over treating tasks independently.
Compared to MT online learning [5, 6], where a single task is revealed to the learner at each round, a series of works have considered learning multiple tasks simultaneously, i.e., taking a decision for each one of them. Dekel et al. [12], e.g., propose the use of a shared loss function to account for tasks' relatedness, Lugosi et al. [20] studies the computational tractability of taking multiple actions with joint constraints, while Cavallanti et al. [5] propose a matrix-based extension of the multitask Perceptron algorithm. In all of these works, however, the learner receives feedback from all the
tasks. In Section 4, instead, we focus on the challenging setup where only one task can be queried by the learner at each round. Hence, in addition to choosing good actions, the learner faces the active learning task of assessing the most informative feedback, in order to achieve sublinear regret. Perhaps most related to ours, is the offline contextual Bayesian optimization setup of [7, 18] where the goal is to compute the best strategy for each context (task) with minimal function interactions. However, unlike us, [7, 18] do not guarantee sublinear regret but provide only sample-complexity results.
Finally, we note that MT confidence intervals and regret guarantees were also recently derived by [9], albeit in a different setup and regression model. Indeed, the authors of [9] focus on multi-objective optimization where they ought to learn multi-output functions (each output corresponding to a task) using matrix-valued kernels. Although their setup can be related to ours, it crucially requires all tasks to be observed at each round, leading to different challenges than ours, see Appendix A.1 for details.
Notation. We use  $[N] := \{1, \ldots, N\}$ ,  $\mathbb{1}_N$  for the vector in  $\mathbb{R}^N$  full of ones. Norms of functions are always taken w.r.t. the natural RKHS norm, so that we drop the subscript for simplicity of writing.
2 Improved Confidence Intervals for Multitask Kernel Regression
In this section, we introduce the MT kernel regression setting, and prove our refined confidence intervals. Of independent interest, these results are then leveraged in Sections 3 and 4 to derive novel regret bounds for online and active multitask learning. All proofs are deferred to the Appendix.
2.1 Multitask Kernel Regression
Given an input space  $\mathcal{X}$ , equipped with a (single task) scalar kernel  $k_{\mathcal{X}} \colon \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ , the goal of MT kernel regression is to jointly learn  $N$  different functions  $f_1, \ldots, f_N$  from  $\mathcal{X}$  to  $\mathbb{R}$ , all belonging to  $\mathcal{H}_{k_{\mathcal{X}}}$ , the RKHS associated to  $k_{\mathcal{X}}$ . To do so, the learner is given a set of triplets  $\{(i_s, x_s), y_s\}_{s=1}^t$  consisting of a measured task index  $i_s \in [N]$ , a measured point  $x_s \in \mathcal{X}$ , and a noisy measurement  $y_s = f_{i_s}(x_s) + \xi_s$ , where  $\xi_s$  is an independent random variable to be specified later. We can further define the multitask function  $f^{\mathrm{mt}} \colon [N] \times \mathcal{X} \to \mathbb{R}$  such that  $f^{\mathrm{mt}}(i, \cdot) = f_i$ , and the multitask kernel
$$
k \left((i, x), \left(i ^ {\prime}, x ^ {\prime}\right)\right) = k _ {\mathcal {T}} \left(i, i ^ {\prime}\right) \cdot k _ {\mathcal {X}} \left(x, x ^ {\prime}\right), \tag {1}
$$
where  $k_{\mathcal{T}}$  is a kernel on the tasks. In certain cases, the latter might be given as input to the learner, either under the form of the task Gram matrix, or via task features and an assumed (e.g., linear) similarity [17]. However, in practice such information is usually not accessible to the learner. In such a case, a standard agnostic approach to MT regression [5, 6, 10, 13, 24] then consists in leveraging a parameterized task kernel of the form
$$
k _ {\mathcal {T}} \left(i, i ^ {\prime}\right) = \left[ K _ {\text {t a s k}} (b) \right] _ {i i ^ {\prime}}, \quad \text {w i t h} \quad K _ {\text {t a s k}} (b) = \frac {1}{1 + b} I _ {N} + \frac {b}{1 + b} \frac {\mathbb {1} _ {N} \mathbb {1} _ {N} ^ {\top}}{N} \in \mathbb {R} ^ {N \times N}. \tag {2}
$$
Intuitively, parameter  $b \geq 0$  governs how similar the tasks are thought to be. When  $b = 0$ , we have  $K_{\mathrm{task}}(b) = I_N$ , such that  $k_{\mathcal{T}}(i,i^{\prime}) = \delta_{ii^{\prime}}$ , and the tasks are considered to be independent. When  $b$  goes to  $+\infty$ , we have  $K_{\mathrm{task}}(b) = \mathbb{1}_N\mathbb{1}_N^\top / N$ , and all tasks are considered to be one single common task. Any choice of  $b \in (0, + \infty)$  corresponds to a tradeoff between these two regimes. We make this intuition explicit in Proposition 2 (Appendix A.1). Note that all quantities depending on the kernel do by definition depend on  $b$ . We use the notation  $|b|$  to make this dependence explicit when relevant.
Given a history of measurements  $\{(i_s, x_s), y_s\}_{s=1}^t$ , one may then estimate  $f^{\mathrm{mt}}$ , or equivalently the  $\{f_i\}_{i=1}^N$ , by standard kernel Ridge regression using the MT kernel  $k$ . One obtains the estimates
$$
\mu_ {t} (i, x \mid b) = \boldsymbol {k} _ {t} (i, x) ^ {\top} \left(K _ {t} + \lambda I _ {t}\right) ^ {- 1} \boldsymbol {y} _ {1: t}, \tag {3}
$$
$$
\sigma_ {t} ^ {2} (i, x \mid b) = k ((i, x), (i, x)) - \boldsymbol {k} _ {t} (i, x) ^ {\top} \left(K _ {t} + \lambda I _ {t}\right) ^ {- 1} \boldsymbol {k} _ {t} (i, x), \tag {4}
$$
where  $\pmb{k}_t(i,x) = \left[k\big((i_s,x_s),(i,x)\big)\right]_{s = 1}^t$ $K_{t} = \left[k\big((i_{s},x_{s}),(i_{s^{\prime}},x_{s^{\prime}})\big)\right]_{s,s^{\prime} = 1}^{t}$ ,  $\pmb{y}_{1:t} = \left[y_s\right]_{s = 1}^t$ , and  $\lambda >0$  is some regularization parameter. Functions  $\mu_t$  and  $\sigma_t^2$  can be interpreted as the posterior mean and variance of a corresponding Gaussian Process model, see [25, 8]. In the next section, we will utilize  $\mu_t$  and  $\sigma_t^2$  to construct high-probability confidence intervals for the multitask function  $f^{\mathrm{mt}}$ .
Figure 2: Novel multi-task confidence width  $\beta_{t}^{\mathrm{new}}(b)$  (see Theorem 1) visualized for large and small values of  $b$ . It improves over the naive width  $\beta_{t}^{\mathrm{naive}}(b)$  by a factor of  $\sqrt{N}$  at  $b = 0$  and as  $b \to +\infty$ . Problem parameters were set to  $B = 1, \epsilon = 0.4, N = 20, t = 4$ , and  $\gamma_{t}^{\mathrm{mt}}(b) = \gamma_{t}^{\mathrm{st}} = 0$  for all  $b$ .
Information gain. An important quantity when analyzing (multitask) kernel regression is the so-called (multitask) information gain:
$$
\gamma_ {T} ^ {\mathrm {m t}} (b) = \frac {1}{2} \ln \left| I _ {T} + \lambda^ {- 1} K _ {T} \right|.
$$
It can be interpreted as the reduction in uncertainty about  $f^{\mathrm{mt}}$  after having observed a given set of  $T$  datapoints. Similarly to single-task setups [25, 8], we use  $\gamma_T^{\mathrm{mt}}$  in the next sections to characterize our confidence intervals and regret bounds. Note that  $\gamma_T^{\mathrm{mt}}$  depends on the multitask kernel through  $K_{T}$ , and hence on  $b$ . In Section 3, we exploit the properties of our multitask kernel to obtain a sharper control over  $\gamma_T^{\mathrm{mt}}$ , which is then fundamental to derive improved regret bounds.
2.2 Improved Confidence Intervals
In this section, we utilize the regression estimates obtained in Equations (3) and (4) to construct high probability confidence intervals around the unknown multitask function  $f^{\mathrm{mt}}$ . First, we assume that  $\| f_i\| \leq B$  for all  $i \in [N]$ , as it is standard in single-task regression. Moreover, let  $f_{\mathrm{avg}} = (1 / N)\sum_{i = 1}^{N}f_{i}$  be the average task function, and define
$$
\epsilon = \max  _ {i} \| f _ {i} - f _ {\text {a v g}} \| / B. \tag {5}
$$
Note that by definition  $\epsilon \in [0,2]$ . The smaller  $\epsilon$ , the more similar the tasks are, the limit case being that all tasks are equal, attained at  $\epsilon = 0$ . At the other extreme, when  $\epsilon \gg 0$  tasks are highly distant and ought to be learned independently. The deviation  $\epsilon$  plays a crucial role in the subsequent analysis.
A naive confidence interval. As discussed in [6], it is possible to construct the multitask feature map  $\widetilde{\psi}$  associated to  $k$ . One may then rewrite  $f^{\mathrm{mt}}(i,x) = \langle \widetilde{f},\widetilde{\psi} (i,x)\rangle$ , where  $\widetilde{f}$  is a transformed version of  $f^{\mathrm{mt}}$  which satisfies  $\| \widetilde{f}\| \leq B\sqrt{N(1 + b\epsilon^2)}$ , see Appendices A.1 and A.2 for details. MT regression thus boils down to single-task regression, over the modified features  $\widetilde{\psi} (i,x)$ , and with target function  $\widetilde{f}$ . One can then employ well-known linear regression results to obtain confidence intervals for  $f^{\mathrm{mt}}$ . Using [1, Theorem 3.11, Remark 3.13] and the definition of  $\gamma_t^{\mathrm{mt}}(b)$ , with probability  $1 - \delta$  we have that for all  $t\in \mathbb{N}$ ,  $i\in [N]$ , and  $x\in \mathcal{X}$  it holds  $\left|\mu_t(i,x\mid b) - f^{\mathrm{mt}}(i,x)\right|\leq \beta_t^{\mathrm{naive}}(b)\cdot \sigma_t(i,x\mid b)$ , where
$$
\beta_ {t} ^ {\mathrm {n a i v e}} (b) = B \sqrt {N (1 + b \epsilon^ {2})} + \lambda^ {- 1 / 2} \sqrt {2 \left(\gamma_ {t} ^ {\mathrm {m t}} (b) + \ln (1 / \delta)\right)}.
$$
Note that the above confidence interval was already established in [6, Theorem 1]. As expected, it depends on  $B$ ,  $N$ ,  $b$ , and in a decreasing fashion with respect to  $\epsilon$ . However, we argue that the above naive choice can be extremely conservative. Indeed, when  $b = 0$ , MT regression treats tasks independently, see Proposition 2. Hence, a valid confidence width from [2, 1, 8] is  $\mathcal{O}\big(B + \sqrt{\gamma_t^{\mathrm{st}}}\big)$ , where  $\gamma_t^{\mathrm{st}}$  is the single-task maximum information gain. Instead, noting that  $\gamma_t^{\mathrm{mt}}(0) = \mathcal{O}\big(N\gamma_t^{\mathrm{st}}\big)$ , see Proposition 1, the naive choice provides  $\beta_t^{\mathrm{naive}}(0) = \sqrt{N} \cdot \mathcal{O}\big(B + \sqrt{\gamma_t^{\mathrm{st}}}\big)$ , which is larger by a factor  $\sqrt{N}$ . A similar suboptimality gap of  $\sqrt{N}$  can also be proven when  $b$  tends to  $+\infty$ . Motivated by the above observation, we derive a novel confidence width that is less conservative than  $\beta_t^{\mathrm{naive}}(b)$  for the whole range of possible kernel parameters  $b$ .
Theorem 1 (Multitask confidence intervals). Let  $f^{\mathrm{mt}} \colon [N] \times \mathcal{X} \to \mathbb{R}$  such that for all  $i \in [N]$ ,  $f_i \coloneqq f^{\mathrm{mt}}(i, \cdot)$  belongs to the RKHS associated to  $k_{\mathcal{X}}$  and  $\| f_i \| \leq B$ . Moreover, let  $\mu_t$  and  $\sigma_t$  be the
regression estimates of Equations (3) and (4) with task kernel  $k_{\mathcal{T}}(i,j) = [K_{\mathrm{task}}(b)]_{ij}$ , parameter  $\lambda \in [1/(1+b), 1]$ , and noise  $\{\xi_{\tau}\}_{\tau=1}^{t}$  i.i.d. 1-sub-Gaussian. Then, with probability at least  $1 - 2\delta$ ,
$$
\left| \mu_ {t} (i, x \mid b) - f ^ {\mathrm {m t}} (i, x) \right| \leq \beta_ {t} ^ {\mathrm {n e w}} (b) \cdot \sigma_ {t} (i, x \mid b), \quad \forall t \in \mathbb {N}, i \in [ N ], x \in \mathcal {X},
$$
where  $\beta_t^{\mathrm{new}}(b) = \min \left\{\beta_t^{\mathrm{naive}}(b),\beta_t^{\mathrm{small - }b}(b),\beta_t^{\mathrm{large - }b}(b)\right\} ,$
$$
\beta_ {t} ^ {\mathrm {s m a l l -} b} (b) = B (1 + b \epsilon) \sqrt {\frac {1 + b N}{1 + b}} + \lambda^ {- 1 / 2} \sqrt {2 (1 + b N) \left(\gamma_ {t} ^ {\mathrm {s t}} + \ln (N / \delta)\right)},
$$
$$
\beta_ {t} ^ {\mathrm {l a r g e -} b} (b) = B \sqrt {\frac {(1 + b \epsilon) ^ {2}}{1 + b} + \frac {2 b N}{1 + b} + \frac {2 b (1 + b \epsilon) ^ {2}}{N \lambda^ {2} (1 + b) ^ {3}} t ^ {2}} + \lambda^ {- 1 / 2} \sqrt {2 \big (\gamma_ {t} ^ {\mathrm {m t}} (b) + \ln (1 / \delta) \big)}.
$$
The obtained improved confidence width  $\beta_{t}^{\mathrm{new}}(b)$  is the minimum between three confidence widths, see Figure 2. The first one is the naive one  $\beta_{t}^{\mathrm{naive}}(b)$ , obtained by standard arguments as outlined above, while  $\beta_{t}^{\mathrm{small - }b}(b)$  and  $\beta_{t}^{\mathrm{large - }b}(b)$  (dashed and dotted lines in Figure 2) are novel and useful for small and large values of  $b$ , respectively. Indeed, note that we have  $\beta_{t}^{\mathrm{small - }b}(b) \xrightarrow{b \to 0} \mathcal{O}\left(B + \sqrt{\gamma_{t}^{\mathrm{st}}}\right)$ , which is the expected single-task confidence width and  $\sqrt{N}$  smaller than  $\beta_{t}^{\mathrm{naive}}(0)$ . Similarly, as  $b$  goes to  $+\infty$  we have  $\beta_{t}^{\mathrm{large - }b}(b) \stackrel{b \to +\infty}{\sim} \mathcal{O}\left(B \sqrt{b\epsilon^{2} + 2N + 2\epsilon^{2}t^{2}/N}\right) \stackrel{b \to +\infty}{\sim} \mathcal{O}\left(\epsilon B \sqrt{b}\right)$ , while  $\beta_{t}^{\mathrm{naive}}(b) \stackrel{b \to +\infty}{\sim} \mathcal{O}\left(\epsilon B \sqrt{Nb}\right)$ . The obtained confidence width is therefore always smaller than the naive one, but also tighter by a factor  $\sqrt{N}$  for the extreme choices  $b = 0$  and  $b = +\infty$ .
From a technical viewpoint,  $\beta_{t}^{\mathrm{small - b}}$  and  $\beta_{t}^{\mathrm{large - b}}$  are obtained by viewing MT regression as a single-task regression over the inflated features  $\widetilde{\psi} (i,x)$ , as also done in [6]. However, unlike [6], we explicitly leverage the expressions of  $\widetilde{\psi} (i,x\mid b)$  and  $K_{\mathrm{task}}(b)$  as functions of  $b$ . In particular, because of the structure of  $K_{\mathrm{task}}$ , the regression kernel matrix is a rank-one perturbation of a block diagonal matrix, a fact that we exploit, e.g., via Lemma 2. Moreover, we note that refined widths can be obtained if one has access to task-specific constants  $B_{i}$  and  $\epsilon_{i}$ . For simplicity of exposition, we focus on uniform (over tasks)  $B$  and  $\epsilon$ . Also, a tighter data-dependent  $\beta_{t}^{\mathrm{large - b}}$  can be utilized as outlined in Appendix A.2.2. Finally, we remark that the obtained multitask intervals do not require i.i.d. data and thus apply to the adaptive design setting where data are, e.g., sequentially acquired by the learner, as shown in the next section.
3 New Guarantees for Multitask Online Learning
In this section, we show how the improved confidence interval established in Theorem 1 can be used to derive sharp regret guarantees for multitask online learning. To do so, we also prove novel bounds for the multitask information gain  $\gamma_T^{\mathrm{mt}}(b)$ . For  $t = 1,2,\ldots$  the learning protocol is as follows: nature reveals task index  $i_t\in [N]$ ; the learner chooses strategy  $x_{t}\in \mathcal{X}$  and pays  $f^{\mathrm{mt}}(i_t,x_t)$ ; the learner observes the noisy feedback  $y_{t} = f^{\mathrm{mt}}(i_{t},x_{t}) + \xi_{t}$ . The goal is to minimize for any horizon  $T$  the multitask regret
$$
R ^ {\mathrm {m t}} (T) = \sum_ {t = 1} ^ {T} \max  _ {x \in \mathcal {X}} f ^ {\mathrm {m t}} \left(i _ {t}, x\right) - \sum_ {t = 1} ^ {T} f ^ {\mathrm {m t}} \left(i _ {t}, x _ {t}\right). \tag {6}
$$
In the next subsection, we provide a generic algorithm to minimize (6). In particular, we show that naive choices of parameters allow to recover previous approaches with their guarantees, while using the refined confidence width  $\beta_t^{\mathrm{new}}(b)$  derived in Theorem 1 yields significant improvements.
3.1 Algorithm and regret guarantees
In line with the online learning literature, our approach is based on the multitask Upper Confidence Bound, defined for any  $t \in \mathbb{N}$  as
$$
\operatorname {u c b} _ {t} (i, x \mid b) = \mu_ {t} (i, x \mid b) + \beta_ {t} (b) \cdot \sigma_ {t} (i, x \mid b). \tag {7}
$$
Here  $\beta_{t}\colon \mathbb{R}_{+}\to \mathbb{R}_{+}$  is a function which assigns a confidence width  $\beta_t(b)$  to each kernel parameter  $b$ . We consider the general strategy MT-UCB (see Algorithm 1) which, at each round  $t$  selects
Algorithm 1 MT-UCB
Require: Domain  $\mathcal{X}$ , kernel  $k_{\mathcal{X}}$ , number of tasks  $N$ , width functions  $\{\beta_t\}_{t\in \mathbb{N}}$ , parameters  $b,\lambda$
1: for  $t = 1, 2, \ldots$  do
2: Observe  $i_t$ ,
3: Play  $x_{t} = \arg \max_{x\in \mathcal{X}}\operatorname {ucb}_{t - 1}(i_{t},x\mid b)$ ,
4: Observe  $y_{t} = f^{\mathrm{mt}}(i_{t},x_{t}) + \epsilon_{t}$
5: Update  $\mathbf{ucb}_t(\cdot ,\cdot |b)$  based on (3),(4), and (7).
Algorithm	βt	b	λ
IGP-UCB [8]	βtsmall-b	0	1
GoB.Lin [6]	βtnaive	b	1
This work	βtnew	b	N+b/N+bN
Table 1: Recovering previous works by appropriate choices of  ${\beta }_{t},b$  ,and  $\lambda$  .
$x_{t} = \arg \max_{x\in \mathcal{X}}\mathrm{ucb}_{t - 1}(i_{t},x\mid b)$ . As summarized in Table 1, both the strategy that runs  $N$  independent instances of IGP-UCB (one for each task), and GoB. Lin from [6] are particular cases of MT-UCB. Importantly, whenever  $\beta_{t}(b)$  is set such that  $[\mu_t(\cdot ,\cdot |b)\pm \beta_t(b)\cdot \sigma_t(\cdot ,\cdot |b)]$  is a valid confidence interval for  $f^{\mathrm{mt}}(\cdot ,\cdot)$ , the regret of MT-UCB can be controlled through the following lemma.
Lemma 1. Suppose that  $\lambda \geq (N + b) / (N + bN)$ , and that for all tasks  $i$ , point  $x$ , and time  $t$ , we have  $f^{\mathrm{mt}}(i, x \mid b) \in [\mu_t(i, x \mid b) \pm \beta_t(b) \cdot \sigma_t(i, x \mid b)]$ . Then, the multitask regret of MT-UCB satisfies
$$
R ^ {\mathrm {m t}} (T) \leq 4 \beta_ {T} (b) \sqrt {\lambda T \gamma_ {T} ^ {\mathrm {m t}} (b)}.
$$