Title: Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: multitask learning

confidence intervals

online learning theory

active learning

regret
Abstract: Multitask learning is a powerful framework that enables one to simultaneously learn multiple related tasks by sharing information between them. Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. In this work, we provide novel confidence intervals for multitask regression in the challenging agnostic setting, i.e., when neither the similarity between tasks nor the tasks' features are available to the learner. The obtained intervals do not require i.i.d. data and can be directly applied to bound the regret in online learning. Through a refined analysis of the multitask information gain, we obtain new regret guarantees that, depending on a task similarity parameter, can significantly improve over treating tasks independently. We further propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance, i.e., automatically adapting to task similarity. As a second key application of our results, we introduce a novel multitask active learning setup where several tasks must be simultaneously optimized, but only one of them can be queried for feedback by the learner at each round. For this problem, we design a no-regret algorithm that uses our confidence intervals to decide which task should be queried. Finally, we empirically validate our bounds and algorithms on synthetic and real-world (drug discovery) data.

Multitask Learning with No Regret:

Abstract

Multitask learning is a powerful framework that enables one to simultaneously learn multiple related tasks by sharing information between them. Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. In this work, we provide novel confidence intervals for multitask regression in the challenging agnostic setting, i.e., when neither the similarity between tasks nor the tasks' features are available to the learner. The obtained intervals do not require i.i.d. data and can be directly applied to bound the regret in online learning. Through a refined analysis of the multitask information gain, we obtain new regret guarantees that, depending on a task similarity parameter, can significantly improve over treating tasks independently. We further propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance, i.e., automatically adapting to task similarity. As a second key application of our results, we introduce a novel multitask active learning setup where several tasks must be simultaneously optimized, but only one of them can be queried for feedback by the learner at each round. For this problem, we design a no-regret algorithm that uses our confidence intervals to decide which task should be queried. Finally, we empirically validate our bounds and algorithms on synthetic and real-world (drug discovery) data.

1 Introduction

In many real-world applications, one often faces multiple related tasks to be solved sequentially or simultaneously. The goal of multitask learning (MTL) [4] is to leverage the similarities across the tasks to obtain more accurate and robust models. Indeed, by jointly learning multiple tasks, MTL can exploit their statistical dependencies, yielding better generalization and faster learning than treating each task independently. MTL has gained significant attention in recent years, as it has been shown to be effective in a wide range of applications, including natural language processing, computer vision, federated learning, and drug discovery, see e.g., [11, 19, 16, 27].

A very natural model for learning across multiple tasks is the agnostic multitask (MT) regression approach of [13]. This utilizes a multitask kernel that can interpolate between running N (number of tasks) independent regressions, and regressing all tasks to their common average, depending on a tunable parameter. Notably, such a kernel does not require any knowledge neither about tasks' features

Pierre Laforgue ∗

Andreas Krause

task 1

task 2

task3

2

true fi(-)

Independent

×

data

regression:

0

0

post. mean μt(-)

confidence

-1

0

1

-1

0

1

-1

0

1

task 1

task 2

task 3

2

2

2

true fi(.)

Multitask

×

data

regression:

9

0

0

0

post. mean μt(-)

naive

improvedconfidence(thiswork)

-2

-1

0

1

-1

0

1

1

0

1

nor about their similarity, thus finding good application in several domains. For instance, Cavallanti et al. [5] study it for online classification, and Cesa-Bianchi et al. [6] for online convex optimization.

However, it is much less understood how to quantify the uncertainty of such MT regression, i.e., assessing confidence in the estimated tasks. In particular, as also outlined by [13] as an open problem, it is important to assess their generalization error as a function of the kernel parameter. Appropriately characterizing these confidence intervals is indeed of crucial importance for a whole set of downstream applications. More concretely, multitask confidence intervals are used in online learning to inform the next decision to be made [6]. In active learning-as we show next-these intervals are pivotal to deciding the most informative task to query.

In this work, we study the agnostic MT regression setup of [13], and provide new multitask confidence intervals (see Figure 1 for a visualization) for the full range of the kernel parameter. Our intervals hold in the so-called adaptive setting, i.e., without requiring i.i.d. data, and are tighter up to a √ N factor than the naive ones employed in [6]. Moreover, we provide the first bounds for the information gain of MTregression and utilize them-together with the derived intervals-to obtain tighter online learning guarantees . The latter depend on a task similarity parameter and can significantly improve over treating tasks independently. Additionally, we propose an adaptive no-regret algorithm that exploits task similarity without knowing this parameter in advance. Finally, we consider a novel multitask active learning setup, where tasks should be simultaneously optimized but only one of them can be queried at each round. We show that the newly derived intervals are also crucial in such a setting, and provide a new algorithm that ensures sublinear regret. We demonstrate the superiority of the derived intervals over previously proposed algorithms on synthetic as well as real-world drug discovery tasks.

Related work. The agnostic MT regression approach of [13] reduces the learning of N tasks to a single regression problem, as a function of the MT kernel parameter. When combined with support vector machines, it was shown effective in a series of classification problems [13, 24], and since then was studied in various further settings. Cavallanti et al. [5], e.g., analyze mistake bounds for online MTclassification algorithms as a function of the kernel parameter. Cesa-Bianchi et al. [6], instead, utilize the MT kernel to prove regret bounds in online MT learning with bandit feedback. Inspired by this, [10] focuses on learning more general kernel structures from data. An important question not addressed by previous work, though, is how to properly quantify the uncertainty of the obtained task estimates. This problem is well-understood in single-task learning (e.g., [2, 25, 8]) but remains largely unexplored in MT domains. As shown in [6], MT confidence intervals can in principle be obtained by a naive application of the single-task guarantees of [2]. However, as we show in Section 2, the so-obtained intervals are extremely conservative and-as a result-can hamper the MT learning performance. Our intervals are tighter by a factor up to √ N w.r.t. the naive ones from [6], yielding novel online learning regret guarantees which can provably improve over treating tasks independently.

Compared to MT online learning [5, 6], where a single task is revealed to the learner at each round, a series of works have considered learning multiple tasks simultaneously , i.e., taking a decision for each one of them. Dekel et al. [12], e.g., propose the use of a shared loss function to account for tasks' relatedness, Lugosi et al. [20] studies the computational tractability of taking multiple actions with joint constraints, while Cavallanti et al. [5] propose a matrix-based extension of the multitask Perceptron algorithm. In all of these works, however, the learner receives feedback from all the

tasks. In Section 4, instead, we focus on the challenging setup where only one task can be queried by the learner at each round. Hence, in addition to choosing good actions, the learner faces the active learning task of assessing the most informative feedback, in order to achieve sublinear regret. Perhaps most related to ours, is the offline contextual Bayesian optimization setup of [7, 18] where the goal is to compute the best strategy for each context (task) with minimal function interactions. However, unlike us, [7, 18] do not guarantee sublinear regret but provide only sample-complexity results.

Finally, we note that MT confidence intervals and regret guarantees were also recently derived by [9], albeit in a different setup and regression model. Indeed, the authors of [9] focus on multi-objective optimization where they ought to learn multi-output functions (each output corresponding to a task) using matrix-valued kernels. Although their setup can be related to ours, it crucially requires all tasks to be observed at each round, leading to different challenges than ours, see Appendix A.1 for details.

Notation. We use [ N ] := { 1 , . . . , N } , N for the vector in R N full of ones. Norms of functions are always taken w.r.t. the natural RKHS norm, so that we drop the subscript for simplicity of writing.

2 Improved Confidence Intervals for Multitask Kernel Regression

In this section, we introduce the MT kernel regression setting, and prove our refined confidence intervals. Of independent interest, these results are then leveraged in Sections 3 and 4 to derive novel regret bounds for online and active multitask learning. All proofs are deferred to the Appendix.

2.1 Multitask Kernel Regression

Given an input space X , equipped with a (single task) scalar kernel k X : X × X → R , the goal of MT kernel regression is to jointly learn N different functions f 1 , . . . , f N from X to R , all belonging to H k X , the RKHS associated to k X . To do so, the learner is given a set of triplets { ( i s , x s ) , y s } t s =1 consisting of a measured task index i s ∈ [ N ] , a measured point x s ∈ X , and a noisy measurement y s = f i s ( x s ) + ξ s , where ξ s is an independent random variable to be specified later. We can further define the multitask function f mt : [ N ] × X → R such that f mt ( i, · ) = f i , and the multitask kernel where k T is a kernel on the tasks. In certain cases, the latter might be given as input to the learner, either under the form of the task Gram matrix, or via task features and an assumed (e.g., linear) similarity [17]. However, in practice such information is usually not accessible to the learner. In such a case, a standard agnostic approach to MT regression [5, 6, 10, 13, 24] then consists in leveraging a parameterized task kernel of the form

Given a history of measurements { ( i s , x s ) , y s } t s =1 , one may then estimate f mt , or equivalently the { f i } N i =1 , by standard kernel Ridge regression using the MT kernel k . One obtains the estimates

Intuitively, parameter b ≥ 0 governs how similar the tasks are thought to be. When b = 0 , we have K task ( b ) = I N , such that k T ( i, i ′ ) = δ ii ′ , and the tasks are considered to be independent. When b goes to + ∞ , we have K task ( b ) = N /latticetop N /N , and all tasks are considered to be one single common task. Any choice of b ∈ (0 , + ∞ ) corresponds to a tradeoff between these two regimes. We make this intuition explicit in Proposition 2 (Appendix A.1). Note that all quantities depending on the kernel do by definition depend on b . We use the notation | b to make this dependence explicit when relevant.

where k t ( i, x ) = [ k ( ( i s , x s ) , ( i, x ) )] t s =1 , K t = [ k ( ( i s , x s ) , ( i s ′ , x s ′ ) )] t s,s ′ =1 , y 1: t = [ y s ] t s =1 , and λ > 0 is some regularization parameter. Functions µ t and σ 2 t can be interpreted as the posterior mean and variance of a corresponding Gaussian Process model, see [25, 8]. In the next section, we will utilize µ t and σ 2 t to construct high-probability confidence intervals for the multitask function f mt .

0

20

40

60

80

1 +

O

(√

γ

st

t

)

√

N

+

O

(√

N

γ

st

t

)

O

(

/epsilon1

√

b

)

O

(

/epsilon1

√

bN

)

b

β

t

(

b

)

naive

this work

0

1

2

3

b

Information gain. An important quantity when analyzing (multitask) kernel regression is the so-called (multitask) information gain :

It can be interpreted as the reduction in uncertainty about f mt after having observed a given set of T datapoints. Similarly to single-task setups [25, 8], we use γ mt T in the next sections to characterize our confidence intervals and regret bounds. Note that γ mt T depends on the multitask kernel through K T , and hence on b . In Section 3, we exploit the properties of our multitask kernel to obtain a sharper control over γ mt T , which is then fundamental to derive improved regret bounds.

2.2 Improved Confidence Intervals

In this section, we utilize the regression estimates obtained in Equations (3) and (4) to construct high probability confidence intervals around the unknown multitask function f mt . First, we assume that ‖ f i ‖ ≤ B for all i ∈ [ N ] , as it is standard in single-task regression. Moreover, let f avg = (1 /N ) ∑ N i =1 f i be the average task function, and define

Note that by definition /epsilon1 ∈ [0 , 2] . The smaller /epsilon1 , the more similar the tasks are, the limit case being that all tasks are equal, attained at /epsilon1 = 0 . At the other extreme, when /epsilon1 /greatermuch 0 tasks are highly distant and ought to be learned independently. The deviation /epsilon1 plays a crucial role in the subsequent analysis.

A naive confidence interval. As discussed in [6], it is possible to construct the multitask feature map ˜ ψ associated to k . One may then rewrite f mt ( i, x ) = 〈 ˜ f, ˜ ψ ( i, x ) 〉 , where ˜ f is a transformed version of f mt which satisfies ∥ ∥ ˜ f ∥ ∥ ≤ B √ N (1 + b /epsilon1 2 ) , see Appendices A.1 and A.2 for details. MT regression thus boils down to single-task regression, over the modified features ˜ ψ ( i, x ) , and with target function ˜ f . One can then employ well-known linear regression results to obtain confidence intervals for f mt . Using [1, Theorem 3.11, Remark 3.13] and the definition of γ mt t ( b ) , with probability 1 -δ we have that for all t ∈ N , i ∈ [ N ] , and x ∈ X it holds ∣ µ t ( i, x | b ) -f mt ( i, x ) ∣ ≤ β naive t ( b ) · σ t ( i, x | b ) , where

Note that the above confidence interval was already established in [6, Theorem 1]. As expected, it depends on B , N , b , and in a decreasing fashion with respect to /epsilon1 . However, we argue that the above naive choice can be extremely conservative . Indeed, when b = 0 , MT regression treats tasks independently, see Proposition 2. Hence, a valid confidence width from [2, 1, 8] is O ( B + √ γ st t ) , where γ st t is the single-task maximum information gain. Instead, noting that γ mt t (0) = O ( N γ st t ) , see Proposition 1, the naive choice provides β naive t (0) = √ N · O ( B + √ γ st t ) , which is larger by a factor √ N . A similar suboptimality gap of √ N can also be proven when b tends to + ∞ . Motivated by the above observation, we derive a novel confidence width that is less conservative than β naive t ( b ) for the whole range of possible kernel parameters b .

Theorem 1 (Multitask confidence intervals) . Let f mt : [ N ] × X → R such that for all i ∈ [ N ] , f i := f mt ( i, · ) belongs to the RKHS associated to k X and ‖ f i ‖ ≤ B . Moreover, let µ t and σ t be the

regression estimates of Equations (3) and (4) with task kernel k T ( i, j ) = [ K task ( b )] ij , parameter λ ∈ [1 / (1 + b ) , 1] , and noise { ξ τ } t τ =1 i.i.d. 1 -sub-Gaussian. Then, with probability at least 1 -2 δ ,

The obtained improved confidence width β new t ( b ) is the minimum between three confidence widths, see Figure 2. The first one is the naive one β naive t ( b ) , obtained by standard arguments as outlined above, while β smallb t ( b ) and β largeb t ( b ) (dashed and dotted lines in Figure 2) are novel and useful for small and large values of b , respectively. Indeed, note that we have β smallb t ( b ) b → 0 - - - → O ( B + √ γ st t ) , which is the expected single-task confidence width and √ N smaller than β naive t (0) . Similarly, as b goes to + ∞ we have β largeb t ( b ) b → + ∞ ∼ O ( B √ b /epsilon1 2 +2 N +2 /epsilon1 2 t 2 /N ) b → + ∞ ∼ O ( /epsilon1 B √ b ) , while β naive t ( b ) b → + ∞ ∼ O ( /epsilon1 B √ Nb ) . The obtained confidence width is therefore always smaller than the naive one, but also tighter by a factor √ N for the extreme choices b = 0 and b = + ∞ .

From a technical viewpoint, β smallb t and β largeb t are obtained by viewing MT regression as a single-task regression over the inflated features ˜ ψ ( i, x ) , as also done in [6]. However, unlike [6], we explicitly leverage the expressions of ˜ ψ ( i, x | b ) and K task ( b ) as functions of b . In particular, because of the structure of K task, the regression kernel matrix is a rank-one perturbation of a block diagonal matrix, a fact that we exploit, e.g., via Lemma 2. Moreover, we note that refined widths can be obtained if one has access to task-specific constants B i and /epsilon1 i . For simplicity of exposition, we focus on uniform (over tasks) B and /epsilon1 . Also, a tighter data-dependent β largeb t can be utilized as outlined in Appendix A.2.2. Finally, we remark that the obtained multitask intervals do not require i.i.d. data and thus apply to the adaptive design setting where data are, e.g., sequentially acquired by the learner, as shown in the next section.

3 New Guarantees for Multitask Online Learning

In this section, we show how the improved confidence interval established in Theorem 1 can be used to derive sharp regret guarantees for multitask online learning. To do so, we also prove novel bounds for the multitask information gain γ mt T ( b ) . For t = 1 , 2 , . . . the learning protocol is as follows: nature reveals task index i t ∈ [ N ] ; the learner chooses strategy x t ∈ X and pays f mt ( i t , x t ) ; the learner observes the noisy feedback y t = f mt ( i t , x t ) + ξ t . The goal is to minimize for any horizon T the multitask regret

In the next subsection, we provide a generic algorithm to minimize (6). In particular, we show that naive choices of parameters allow to recover previous approaches with their guarantees, while using the refined confidence width β new t ( b ) derived in Theorem 1 yields significant improvements.

3.1 Algorithm and regret guarantees

In line with the online learning literature, our approach is based on the multitask Upper Confidence Bound, defined for any t ∈ N as

Here β t : R + → R + is a function which assigns a confidence width β t ( b ) to each kernel parameter b . We consider the general strategy MT-UCB (see Algorithm 1) which, at each round t selects

x t = arg max x ∈ X ucb t -1 ( i t , x | b ) . As summarized in Table 1, both the strategy that runs N independent instances of IGP-UCB (one for each task), and GoB.Lin from [6] are particular cases of MT-UCB . Importantly, whenever β t ( b ) is set such that [ µ t ( · , · | b ) ± β t ( b ) · σ t ( · , · | b )] is a valid confidence interval for f mt ( · , · ) , the regret of MT-UCB can be controlled through the following lemma.

Lemma 1. Suppose that λ ≥ ( N + b ) / ( N + bN ) , and that for all tasks i , point x , and time t , we have f mt ( i, x | b ) ∈ [ µ t ( i, x | b ) ± β t ( b ) · σ t ( i, x | b ) ] . Then, the multitask regret of MT-UCB satisfies

The main novelty of Lemma 1 is that the right-hand side scales with λ 1 / 2 , which might be chosen smaller than 1 . This improvement is due to the fact that multitask posterior variances are smaller than ( N + b ) / ( N + bN ) ≤ 1 . The right-hand side also depends on the multitask information gain γ mt T ( b ) , which is nontrivial to compute or upper bound. In the next proposition, we provide practical upper bounds of γ mt T ( b ) , in terms of the kernel parameter b and the single-task information gain γ st T .

Proposition 1. Let λ ≤ 1 , N ≥ 2 , and T i ≥ 1 for all i ∈ [ N ] . Then, for any b ≥ 0 , we have

We can now combine Theorem 1, Lemma 1, and Proposition 1 to obtain our main result: a bound on the multitask regret of MT-UCB run with the confidence width β new t from Theorem 1 and a specific λ .

Theorem 2. Assume that B ≥ 1 , and that MT-UCB is run with β t = β new t from Theorem 1, and λ = ( N + b ) / ( N + bN ) . Let b = N/ /epsilon1 2 if T ≤ N , b = 1 / /epsilon1 2 if T ≥ N and /epsilon1 ≤ N -1 / 4 T -1 / 2 , and b = 0 otherwise. Let R st ( T ) = B √ T γ st T + √ T γ st T ( γ st T +ln(1 / δ ) ) be the single task regret bound achieved by IGP-UCB (up to constant factors). Then, there exists a universal constant C such that with probability 1 -2 δ we have (up to log N factors)

The regret bound of Theorem 2 is the minimum between three bounds, obtained exploiting the three different regimes of the confidence width β new t derived in Theorem 1 (see Figure 2). The first bound is obtained using β new t ≤ β smallb t , and shows that our approach cannot be worse than independent learning. Indeed, it can be checked that, when facing N tasks, the regret of running N independent instances of IGP-UCB can be bounded by √ N times the single-task regret bound of IGP-UCB , that we denoted by R st ( T ) . Note however that our analysis slightly differs, insofar as we leverage the multitask information gain, while the independent analysis uses Jensen's inequality to aggregate the individual bounds, see Appendix B for details. Note finally that we are able to recover this bound as β smallb t is tight at b = 0 , unlike β naive t . The second bound uses β new t ≤ β largeb t and consists of two terms: the single task regret bound and an additional term that scales with the task deviation /epsilon1 . When the latter is small, i.e., when tasks are similar, the dominant term is R st ( T ) , as if only one task were solved. The third bound is similar, but obtained using β new t ≤ β naive t and is useful when T ≥ N . In

contrast with the independent bound, which does not exploit the task structure, the last two bounds show that multitask learning is always beneficial when the horizon T (and thus the additional /epsilon1 -related term) is small. As expected, this is particularly true when the number of tasks N is large: while the independent bound increases, the second bound does not depend on N . On the other hand, one can note that the condition on /epsilon1 to improve over independent becomes more constraining as the horizon T increases. This suggests that the benefit of multitask may vanish with the number of available points per task, an observation which is well-known by practitioners, see e.g. [21]. As far as we know, this work is the first one to provide theoretical evidence of such a phenomenon in online MT learning.

We conclude this section by comparing Theorem 2 to existing results. As already mentioned in the above discussion, independent IGP-UCB is a particular case of MT-UCB , such that we cannot be worse than the independent approach. We incidentally recover its regret bound as the first bound in the minimum of Theorem 2. Regarding GoB.Lin , since it is also a specific instance of MT-UCB (for β t = β naive t and λ = 1 ), Lemma 1 allows to recover its regret bound [6, Theorem 1].

Corollary 1 (Regret of GoB.Lin [6]) . For any b , the multitask regret of GoB.Lin using parameter b satisfies with probability 1 -δ

If tasks are similar, i.e., when /epsilon1 /lessmuch 1 , bound (8) suggests to choose b > 0 ; this does not impact too much the first term, but makes γ mt T ( b ) smaller. However, we recall that the above bound instantiated with b = 0 does not recover the independent bound. It is instead √ N bigger, since β naive t is not tight at b = 0 . Hence, the Gob.Lin analysis is not sufficient to show that multitask learning improves over independent learning. Our refined analysis, which uses instead β new t , closes this gap.

3.2 Adapting to unknown task similarity

In this section, we consider the case where parameter /epsilon1 (i.e., a bound on the task deviation from the average, see (5)) is a-priori unknown. Despite this challenge, we show that the regret bound of Theorem 2 can be approximately attained using an adaptive procedure, AdaMT-UCB (Algorithm 3), relegated to Appendix B.4 due to space limitations. The proposed approach is inspired by the model selection scheme of [22, Section 7] with a few important modifications that we will outline at the end of this section. AdaMT-UCB considers a plausible set of parameters E = { e 1 , . . . , e |E| } ⊂ (0 , 2] and, for each e ∈ E , initializes an instance of the MT-UCB algorithm with parameters set according to Theorem 2 assuming /epsilon1 = e . We denote such an instance as MT-UCB ( e ) . Moreover, we use the notation ucb e t to denote the upper confidence bounds constructed by MT-UCB ( e ) . We assume the existence of some e ∈ E such that e ≥ /epsilon1 , so that at least one of the learners is well-specified (i.e., its confidence bounds contain f mt with high probability). Our goal is to incur a regret which grows as the regret of the learner with the smallest e such that e ≥ /epsilon1 , since the smaller the e the smallest the regret bound (see Theorem 2), as long as e is a valid upper bound for /epsilon1 . Let us identify with e /star such learner.

At each round t , AdaMT-UCB uses learner e t = min E , and plays the action x t suggested by it, i.e., the maximizer of ucb e t t ( i t , · ) . Then, all MT-UCB ( e ) learners are updated based on the observed reward. In the meantime, a misspecification test is carried out to check whether learner e t is well-specified. Such a test compares the obtained cumulative reward, a lower confidence estimate on such reward according to the other learners, and the believed regret of learner e t . As long as the test does not trigger, the regret of learner e t is controlled by the believed one. Instead, if the test triggers, learner e t can be considered misspecified with high probability. As a result, it gets removed from E and a new epoch starts with the new set E . Let R mt /star ( T ) denote the regret bound (Theorem 2) of learner e /star had it been chosen from round 0 . We can state the following.

Theorem 3. Assume that there exists e ∈ E such that e ≥ /epsilon1 , and let M be the number of learners e ∈ E such that e < /epsilon1 (i.e., the number of misspecified learners in E ). The regret of AdaMT-UCB satisfies with high probability R mt ( T ) = O ( √ M +1 · R mt /star ( T ) ) .

Clearly, the number M of misspecified learners is not known in advance but is always less than |E| . Note that when /epsilon1 = 0 , we have M = 0 and we recover the single task regret bound. Moreover, given ρ ≤ 1 , we show in Appendix B.4 that one can attain a multiplicative accuracy ρ over /epsilon1 , assuming that /epsilon1 ≥ /epsilon1 min > 0 , through an exponential grid with M being polylogarithmic in 1 / ρ and 1 / /epsilon1 min.

Relation with the approach of [22]. Compared to [22, Section 7]-where the goal is to adapt to an unknown features' dimension-the set of learners considered in AdaMT-UCB share the same dimension d . This allows us to exploit the following two novelties with respect to [22]: (1) all learners are updated from the data gathered from learner i t (Line 6 in Algorithm 3), and (2) the lower confidence bounds L e in the misspecification test (Line 8) are all computed using action x t (i.e., the action recommended by learner i t ), as opposed to using the actions recommended by each learner e . Both these points are only applicable to our setting, leading to a simpler regret analysis.

4 Multitask Active Learning

The goal of the online learning setup of Section 3 is to optimize the tasks sequentially revealed by nature. In some situations (e.g., in [20] or the drug discovery problem considered in Section 5), however, we care about the performance of multiple tasks simultaneously , to eventually learn the best strategy for each one of them. Moreover, we ought to do so with minimal interactions T , i.e., minimizing the queries of the function f mt . We capture this by the following active learning protocol.

Learning protocol and regret. At each round t , the learner: chooses a strategy { x i t , i ∈ [ N ] } for each task , chooses which task i t ∈ [ N ] to query, and observes the noisy feedback y t = f mt ( i t , x t )+ ξ t . The learner's goal is to minimize the active learning regret:

Compared to the online learning regret of Equation (6), the learner's performance at each round is here measured by the average reward coming from each task (as opposed to just the task presented by nature). Moreover, compared to online learning, the learner faces the additional challenge of choosing-at each round-from which task information should be gathered. Intuitively, more difficult (or informative) tasks should be queried more often to ensure R mt AL ( T ) grows sublinearly. To the best of our knowledge, the above protocol and regret notion are novel in the multitask literature.

In Algorithm 2 we present MT-AL , an efficient strategy that ensures sublinear active learning regret. Like in MT-UCB , MT-AL constructs confidence intervals around f mt and, at each round, select strategy x i t = argmax x ∈ X ucb t -1 ( i, x ) for each task i ∈ [ N ] . When it comes to selecting which task to query, MT-AL selects i t ∈ arg max i ∈ [ N ] β i t -1 σ t -1 ( i, x i t ) , i.e., the task for which the believed optimizer x i t is subject to max-

Algorithm 2 MT-AL

for t=1,. . . , T do x i t = arg max x ∈ X ucb t -1 ( i, x ) , ∀ i ∈ [ N ] i t = arg max i ∈ [ N ] β i t -1 σ t -1 ( i, x i t ) Observe: y t = f mt ( i t , x i t t ) + ξ t Update ucb t ( · , · ) and σ t ( · , · ) based on observations.

imal uncertainty (we use generic task-dependent widths β i t for completeness). This rule, also known as uncertainty sampling in the literature [23], intuitively makes sure the learner can control the regrets for the tasks not queried and leads to the following theorem.

Theorem 4. Suppose that for all tasks i , point x , and time t , we have that f mt ( i, x ) ∈ [ µ t ( i, x ) ± β i t · σ t ( i, x ) ] . Then, the MT-AL algorithm ensures the active learning regret is bounded by

where { i t } is the sequence of queried tasks and { x i t t } the strategies selected for each of them.

The above bound only relies on MT-AL utilizing valid intervals around f mt and thus applies more broadly than our agnostic MT regression, e.g., when such intervals are constructed using a known multitask kernel k ( ( i, x ) , ( i ′ , x ′ ) ) ). However, Theorem 4 shows the active learning regret heavily depends on the constructed intervals, similar to online learning. In MT-AL , these are additionally utilized for deciding which task to query at each round. When specialized to our agnostic MT kernel and improved confidence, we obtain the following.

Corollary 2. Let MT-AL utilize the MT regression estimates of Eq. (3) -(4) with parameters set according to Theorem 2. Moreover, let R mt ( T ) be the bound on the online learning regret obtained in Theorem 2. Then, with high probability, we have R mt AL ( T ) ≤ R mt ( T ) .

Regret

0

20

40

60

80

100

T

20

40

60

80

100

Synthetic

0

200

400

600

T

200

400

600

800

1000

Drug discovery

Independent

Single

MT-UCB, naive (GoB.Lin)

MT-UCB, improved confidence

AdaMT-UCB

0

20

40

60

80

100

T

20

40

60

80

100

Synthetic

0

200

400

600

T

200

400

600

800

Drug discovery

MT-AL, naive

Unif., improved confidence

MT-AL, improved confidence

MTS

AE-LSVI

(a) Online learning

(b) Active learning

Thus, MT-AL ensures the active learning regret is always bounded by its online learning counterpart. Moreover, the same considerations as in Theorem 2 apply also here, regarding the benefit of multitask learning over independent single-task regression for instance.

5 Experiments

The goal of our experiments is to evaluate the effectiveness of the studied MT regression, and in particular of the improved confidence intervals obtained in Section 2, both in online learning and active learning setups. We utilize the following synthetic and real-world data 2 .

Synthetic data: We generate tasks of the form f i = (1 -δ ) · ¯ f + δ · f i dev , i ∈ [ N ] , where ¯ f, f i dev are random unit vectors representing a common model and individual deviations, respectively. Moreover, actions consist of 10 4 vectors x ∈ R d from the sphere of radius 10. Observation noise is unit normal.

Drug discovery MHC-I data [27]: The goal is to discover the peptides with maximal binding affinity to each Major Histocompatibility Complex class-I (MHC-I) allele. The dataset from [27] contains the standardized binding affinities (IC 50 values) of different peptide candidates to the MHC-I alleles (tasks). For each allele, the dataset contains ∼ 1000 peptides represented as x ∈ R 45 feature vectors. For our experiments, we utilize the 5 alleles A-{ 0201 , 0202 , 0203 , 2301 , 2402 } , since they were shown in [27] to share binding similarity. Note that such a problem falls into our multitask active learning setup, since we would like to retrieve the best peptide for each allele minimizing the number of interactions (i.e., lab experiments). Nevertheless, we also consider its online learning analog where we care about finding the best peptides for each revealed allele.

Online learning. At each round t , a random task i t ∈ [ N ] is observed and point x t is selected according to the following baselines: (1) Independent , which runs N independent IGP-UCB [8] algorithms (corresponding to MT-UCB with b = 0 ), (2) Single , which treats all tasks to be the same and runs a unique single-task IGP-UCB (corresponding to MT-UCB with b = + ∞ ), (3) MT-UCB which utilizes an appropriate parameter 0 < b < ∞ as well as a bound on the tasks similarity /epsilon1 (for synthetic data this can be exactly computed, while for MHC-I data we use /epsilon1 = 0 . 3 ) and utilizes the naive (i.e., Gob.Lin ) or improved confidence bounds, and (4) AdaMT-UCB which is run with the same b but uses the set of plausible deviations E = { . 1 , . 2 , . . . , 1 } instead of knowing the true /epsilon1 . For choosing b , we sweep over possible values and select the best-performing one, keeping it fixed for all the baselines.

Active learning. We follow the multitask active learning setup of Section 4. All baselines utilize confidence intervals from the agnostic MT regression of Section 2, where /epsilon1 and b are chosen as for online learning. Moreover, they all utilize the improved confidence intervals, unless otherwise specified. We compare: (1) Unif. which chooses the task i t to be queried uniformly at random (but still selects x i t ∈ arg max x ucb i t ( i, x ) ) and employs the naive or the improved confidence intervals, the offline contextual Bayesian optimization baselines (2) MTS [7] and (3) AE-LSVI [18], and (4) MT-AL which utilizes the naive or the improved confidence intervals.

We report the cumulative regret (online and active learning, respectively) of the considered baselines in Figure 3, averaged over 5 runs. For the synthetic data, we report results for d = 4 , N = 5 , δ = 0 . 4 , but provide a full set of experiments for different parameters in Appendix D. In Appendix D we also report the frequencies of each task being queried in our active learning experiments. In Figure 3 (a), both MT-UCB and AdaMT-UCB lead to superior performance compared to the Independent and Single baselines, demonstrating the benefits of MT regression. In addition, the improved confidence intervals significantly outperform the naive ones. Moreover, we observe AdaMT-UCB achieves comparable (sometimes even better, see Appendix D) performance to MT-UCB . Indeed, instead of using a conservative choice of /epsilon1 , the misspecification test (Line 8 of Algorithm 3) of AdaMT-UCB allows to use a smaller /epsilon1 and only increase it when there is evidence that the constructed intervals do not contain the true tasks. In active learning (Figure 3 (b)), we observe MT-AL has a significant advantage over the uniform sampling baselines and MTS , while performing comparably to AE-LSVI (both methods are similar as discussed in Appendix C.3). Moreover, its regret is bounded by the online learning regret of MT-UCB , conforming with Theorem 4. Importantly, the improved confidence intervals play a crucial role also here and enable a drastic performance improvement compared to the naive ones.

6 Future Directions

We believe this paper opens up several future research directions. The derived confidence intervals, as well as our analysis of the multitask information gain, heavily exploit the structure of the task Gram matrix K task ( b ) , see Equation (2). However, it remains unclear whether these can be extended to more general kernels. According to the graph perspective of [13], K task ( b ) can be seen as K task ( b ) = I N + L ( b ) , where L ( b ) ∈ R N × N is the Laplacian matrix of a clique graph with vertices [ N ] and edge weight b . Hence, it would be interesting to extend our results to different graph structures. Furthermore, we believe the proposed multitask confidence intervals hold potential for various related domains., e.g., to assess uncertainty in safety-critical systems [3], or to balance exploration-exploitation in multitask reinforcement learning [26], spam filtering [15], or personalized health [14]. In such applications, the introduced notion of active learning regret can serve as a measure of the overall sample efficiency.

References

Abbasi-Yadkori, Y. (2012). Online learning for linearly parametrized control problems. PhD thesis, Edmonton, Alberta, Canada .

Abbasi-Yadkori, Y., Pál, D., and Szepesvári, C. (2011). Improved algorithms for linear stochastic bandits. Advances in Neural Information Processing Systems (NeurIPS) , 24.

Camilleri, R., Wagenmaker, A., Morgenstern, J. H., Jain, L., and Jamieson, K. G. (2022). Active learning with safety constraints. Advances in Neural Information Processing Systems (NeurIPS) .

Caruana, R. (1997). Multitask learning. Machine learning , 28:41-75.

Cavallanti, G., Cesa-Bianchi, N., and Gentile, C. (2010). Linear algorithms for online multitask classification. Journal of Machine Learning Research , 11:2901-2934.

Cesa-Bianchi, N., Gentile, C., and Zappella, G. (2013). A gang of bandits. Advances in Neural Information Processing Systems (NeurIPS) , 26.

Char, I., Chung, Y., Neiswanger, W., Kandasamy, K., Nelson, A. O., Boyer, M., Kolemen, E., and Schneider, J. (2019). Offline contextual bayesian optimization. Advances in Neural Information Processing Systems (NeurIPS) , 32.

Chowdhury, S. R. and Gopalan, A. (2017). On kernelized multi-armed bandits. In International Conference on Machine Learning (ICML) , page 844-853.

Chowdhury, S. R. and Gopalan, A. (2021). No-regret algorithms for multi-task bayesian optimization. In International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 1873-1881.

Ciliberto, C., Rosasco, L., and Villa, S. (2015). Learning multiple visual tasks while discovering their structure. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 131-139.

Collobert, R. and Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. In International Conference on Machine learning (ICML) , pages 160-167.

Dekel, O., Long, P. M., and Singer, Y. (2007). Online learning of multiple tasks with a shared loss. Journal of Machine Learning Research , 8(10).

Evgeniou, T., Micchelli, C. A., Pontil, M., and Shawe-Taylor, J. (2005). Learning multiple tasks with kernel methods. Journal of Machine Learning Research , 6(4).

Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G., and Galstyan, A. (2019). Multitask learning and benchmarking with clinical time series data. Scientific data , 6(1):96.

Jain, G., Sharma, M., and Agarwal, B. (2019). Spam detection in social media using convolutional and long short term memory neural network. Annals of Mathematics and Artificial Intelligence , 85(1):21-44.

Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning , 14(1-2):1-210.

Krause, A. and Ong, C. (2011). Contextual Gaussian process bandit optimization. Advances in Neural Information Processing Systems (NeurIPS) , 24.

Li, X., Mehta, V., Kirschner, J., Char, I., Neiswanger, W., Schneider, J., Krause, A., and Bogunovic, I. (2023). Near-optimal policy identification in active reinforcement learning. International Conference on Learning Representations (ICRL) .

Liu, S., Johns, E., and Davison, A. J. (2019). End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1871-1880.

Lugosi, G., Papaspiliopoulos, O., and Stoltz, G. (2009). Online multi-task learning with hard constraints. In Conference on Learning Theory (COLT) .

Maurer, A., Pontil, M., and Romera-Paredes, B. (2016). The benefit of multitask representation learning. Journal of Machine Learning Research , 17(81):1-32.

Pacchiano, A., Dann, C., Gentile, C., and Bartlett, P. L. (2020). Regret bound balancing and elimination for model selection in bandits and rl. ArXiv , abs/2012.13045.

Settles, B. (2009). Active learning literature survey. Technical report, University of WisconsinMadison Department of Computer Sciences.

Sheldon, D. (2008). Graphical multi-task learning. Technical report, Cornell University, Ithaca, NY.

Srinivas, N., Krause, A., Kakade, S., and Seeger, M. (2010). Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference on Machine Learning (ICML) .

Vithayathil Varghese, N. and Mahmoud, Q. H. (2020). A survey of multi-task deep reinforcement learning. Electronics , 9(9):1363.

Widmer, C., Toussaint, N. C., Altun, Y., and Rätsch, G. (2010). Inferring latent task structure for multitask learning by multiple kernel learning. BMC bioinformatics , 11(8):1-8.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes refined confidence intervals for multitask kernel regression and applies this to the multitask online learning setting and the multitask active learning setting. The thrust of the idea is to further exploit the structure of the task kernel (Eq (2)) where the refined confidence bound is always smaller than the existing confidence bound and tighter by a factor of sqrt of the number of tasks for the extreme cases of b=0 and b=infty, corresponding to the independent tasks or identical tasks, respectively. \n\nTwo important parameters: $b$ is the task-relatedness in the definition of the multitask kernel and $\\eps$ is the task discrepancy from the average task. \n\nThe paper studies both cases of known and unknown $\\eps$. When $\\eps$ is unknown, they use a minibatch learning idea and misspecification task to determine the smallest upper bound of $\\eps$ from the predefined search space. This only incurs an extra term of the square root of the search space cardinality. \n\nThe paper also provides empirical validations to justify their results. \n\n---------- \nUPDATE: I am raising my rating from 5 to 7 to champion this paper. I think this is a solid contribution to the multitask learning setting and I would view this paper as an (interesting) theoretical contribution, rather than a strong algorithm paper in the deep learning setting. On this ground, I think the experimental results the paper presented are sufficient for validating the usefulness of multitask learning in the settings they considered. I believe the points raised by Reviewer ckqH are important but can be largely addressable without changing the scientific merit claimed in the reviewed paper. In particular, I agree with Reviewer ckqH in that the paper should make it clear from the title and the abstract that the paper focuses on a multitask kernel setting, not a general multitask learning setting.\n\nSTRENGTHS:\n- a new refined confidence bound that is tighter than the naive bound by exploiting the structure of multitask kernel. This result is neat. \n- a comprehensive study where they apply to various learning settings and provide empirical results\n\nWEAKNESSES:\n- some of the writing is not clear but improbable \n- the notion of regret in section 4 looks weird\n\nQUESTIONS:\n- An implicit assumption is that each $f_t$ is a sample of the known $GP(k_X)$ that I feel the paper does not make it explicit enough. \n- Also the task kernel in Eq (2) driven by $b$ is a modeling choice, not an assumption because the only task relatedness assumption comes from $\\epsilon$. The author can correct me on this point if I'm wrong. Either way, whether the task kernel is an assumption or a modeling choice should be made explicitly to the readers. \n- Line 211: $N >= T$ instead? \n- Line 248 - 250 about the misspecification test: the intuition behind why this test works should be explained further (I know what the test means but its current explanation is not very helpful to all readers). \n- The regret notion in Section 4 looks weird to me. Isn't $i_t$ instead of $i$ in the second term of the regret notion? Because the learner actively queries a task, thus, the goodness of the queried tasks should be included in the regret. \n- The second and third bound in Theorem 2 is definitely helpful when $\\epsilon = 0$. In which other regimes they are also helpful? and which are not?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on multi-task regression problems where they provide tighter confidence bounds that can be used to establish regret bounds in multi-task online learning and multi-task active learning. They also propose an algorithm that reduces the regret compared to previous bounds by adapting a task similarity parameter automatically. They perform regression experiments on a synthetic dataset and a drug discovery dataset (MHC-I) to show that their algorithms yield better (lower) regret than the reported baselines.\n\nSTRENGTHS:\nThe proposed confidence and regret bounds are general in the sense that they can be applied in online and active learning in the multi-task setting. At times, the paper prepares the reader what a section with a summary sentence, e.g. on L86-87 “In this section, we introduce the MT kernel regression setting, and prove our refined confidence intervals.” The multi-task active learning setting where only one of the tasks can be queried per time step is interesting.\n\nWEAKNESSES:\n**W1:** I am not a multi-task learning expert, however, I have to say that I find the paper hard to follow. There is a lot of inline math which makes reading less smooth. Also, most of the notation is introduced within the sections, while a separate paragraph that thoroughly describes all the notation that will come would have been of great help for the reader to go back to if necessary. Furthermore, I think the paper needs re-writing to emphasize what the main messages are to the reader in each section and paragraph. \n\n**W2:** One of the main contributions, Algorithm 3, described in Section 3.2 is placed in the Appendix, probably due to lack of space. I suggest that the Section 2 and 3 are re-written such that Algorithm 3 can fit into the main paper.   \n\n**W3:** Experiment section is very short (<1.5 pages). I believe that the paper would benefit from adding more experimental results to justify the contributions, as well as adding discussions around the results and emphasizing on the take-home messages to bring insights to the reader. \n\n**W4:** In my opinion, the scope of the paper is narrow as it only covers regression tasks in multi-task learning without touching any classification tasks. Furthermore, there are no experiments using deep neural networks which makes me doubt how relevant this paper is for the general audience at Neurips.\n\nQUESTIONS:\n**Q1:** Can you elaborate on the interpretation of the expression for calculating “regret” in Eq. 6? And can you elaborate on what a “no-regret algorithm” is?\n\n**Q2:** Can the proposed confidence intervals and regret bounds be used for classification tasks? Can they be applied for deep neural networks in multi-task learning problems for both regression and classification?\n\n**Q3:** Are there any metrics other than regret  that are relevant to report in the experiments, e.g., RMSE?\n\n**Q4:** Since you are focusing in this agnostic setting where the task similarity is unknown, I am wondering if you can mention any examples or applications where the task similarity parameter $b$ is actually known? I get the feeling that $b$ is unknown in most multi-task learning problems.\n\nLIMITATIONS:\nNo limitations are addressed as this is a theoretical work according to the authors. See Weaknesses and Questions above for limitations that I am concerned over.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies uncertainty quantification of estimated tasks in agnostic multitask regression. Through exploiting the structure of the task Gram matrix, This work proposes an improved bound on the multitask confidence interval that is tighter than a previous confidence interval [1] from by at most $\\sqrt{N}$.  With the derived interval, this work designed an online learning algorithm based on the upper confidence bound which selects the input with the largest upper confidence. This work also derives a regret bound for the online learning algorithm, showing that multitask learning is always beneficial. Moreover, this work designs a no-regret algorithm in an active learning setting, which uses confidence intervals to decide which task should be queried. This work proves that under the algorithm the active learning regret is always bounded by its online learning counterpart. To evaluate the proposed approach, this work conducts experiments on a synthetic dataset and a drug discovery dataset. In online learning, the proposed algorithm (MT-UCB) with the improved confidence interval outperforms independent learning, single-task learning, and MT-UCB with a previous bound [1]. In the setting of active learning, the proposed algorithm (MT-AL) improves over previous active learning algorithms, including MTS [2], AE-LSVI [3], and MT-AL with a previous bound [1]. \n\n [1] Cesa-Bianchi, N., Gentile, C., and Zappella, G. (2013). A gang of bandits. *Advances in Neural*\n\n*Information Processing Systems (NeurIPS)*. \n\n[2] Char, I., Chung, Y., Neiswanger, W., Kandasamy, K., Nelson, A. O., Boyer, M., Kolemen, E., and Schneider, J. (2019). Offline contextual bayesian optimization. *Advances in Neural Information Processing Systems (NeurIPS)*. \n\n[3] Li, X., Mehta, V., Kirschner, J., Char, I., Neiswanger, W., Schneider, J., Krause, A., and Bogunovic, I. (2023). Near-optimal policy identification in active reinforcement learning. *Interna- tional Conference on Learning Representations (ICRL)*.\n\nSTRENGTHS:\n- This paper derives an improved confidence interval that is tighter than the naive confidence interval in multitask regression. \n- This paper designs two effective algorithms using the largest upper confidence based on the derived confidence interval for online multitask learning and active multitask learning. Moreover, this paper provides regret analysis for the proposed algorithm which is shown to be not worse than a previous regret bound.\n- This paper empirically evaluates the proposed method in synthetic and drug discovery datasets and shows the advantages of the proposed methods in terms of regrets.\n\nWEAKNESSES:\nIt would be better to provide more intuition of deriving the improved confidence interval. For example, provide a proof sketch of Theorem 1 in terms of leveraging the expression of $K_{task}(b)$. Moreover, it would be better to provide an explanation of notations from the previous work, such as $\\tilde{f}$ and $\\tilde{\\phi}(i,x|b)$.\n\nQUESTIONS:\n- What would be the effect for different hyper-parameters in the algorithm, such as $b$, $N$, and $\\delta$? \n\n- What would be the interpretation for better results of AdaMT-UCB compared to MT-UCB as shown in experiments?\n\nLIMITATIONS:\nThis work has discussed the limitations of the proposed methods.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

comments: SUMMARY:
This paper focuses on multitask learning in the agnostic setting. It provides multitask confidence intervals and novel online learning algorithm that achieves such improved regret. The paper also introduces a novel multitask active learning setup where several tasks must be simultaneously optimized but only one of them can be queried for feedback by the learner at each round. They design a no-regret algorithm that uses the confidence intervals to decide which task should be queried.

STRENGTHS:
- Theoretical analysis in this field of MTL is valuable
- The proposed active learning problem in MTL is interesting and can be extended to other real-world applications.
- The proposed method is thoroughly evaluated on synthetic and real-world drug discovery data.

WEAKNESSES:
- Although the paper is well written, I feel that it is densely pack with multiple goals. It would be a easier to understand if the paper focus more on a single concept rather than expanding it to several applications and add empirical results for each of them.

QUESTIONS:
- Can you please also add other real-world applications where multi-task active learning can be extended? This can be a part of the future work section/paragraph.

LIMITATIONS:
- I think that the main limitation is that the proposed method isn't compared directly with existing single-task methods as baselines.
- Is it possible to have an active learning method built for single-task and apply it to all tasks in an MTL problem? How does that compare? For instance, techniques like Margin sampling or Least confidence can be easily applied to each task in an MTL problem with minimum overhead.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-17

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe primary objective of this paper is to analyze the confidence bound of the multi-task kernel regression model and explore its application in online learning and active learning scenarios. Initially, the authors introduce a novel confidence bound, revealing distinct regimes based on task similarity. Notably, their bound proves to be tighter than a naive alternative. Leveraging this improved confidence interval, they proceed to devise online learning and active learning algorithms, providing comprehensive analyses of the confidence intervals in these contexts. In conclusion, the authors conduct empirical comparisons of their methods against existing approaches and baselines, utilizing synthetic data and drug discovery data to validate their findings.\n\nSTRENGTHS:\n- The analysis on the confidence interval of the multi-task kernel regression seems to be a decent addition to the related literature for adapting multi-task kernel regression to online and active learning.\n- Their analysis can provide insight into how training on different tasks can be used to help out on other relevant tasks, which seems to help understanding multi-tasks learning in general.\n\nWEAKNESSES:\nI think the weakness of this work lies in the empirical results.\n- The empirical results are quite limited with only one real dataset and a synthetic dataset. \n- It is mentioned that for online learning, the parameter b is selected based on sweeping over all possible values and choose the best performing one. However, I am not sure if this is possible in practice. At least, a separated validation set should be used for parameter selection. (this problem seems to be also existing in the active learning setting)\n- The empirical results for active learning are also a bit weak. According to Figure 3, it seems that the proposed algorithm could be slightly worse in some T and at best on par with existing algorithms. Are there any other benefits for adapting the proposed algorithm instead of other competitors in practice?\n- For active leaning, I believe this work could also benefit from a more detailed analysis on the how are the examples being selected with different algorithms differs from each other.\n\nminor:\n- The y-axis in Figure 3 should have labels.\n\nQUESTIONS:\n- Line 217 and 218: \"This suggests that the benefit of multitask may vanish with the number of available points per task, an observation which is well-known by practitioners\". Are there any relevant citations?\n\nLIMITATIONS:\nI think the empirical results are quite limited. But this limitation is not mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}