L IFELONG L EARNING WITH O UTPUT K ERNELS
Anonymous authors
Paper under double-blind review
A BSTRACT
Lifelong learning poses considerable challenges in terms of effectiveness (minimizing prediction errors for all tasks) and overall computational tractability for real-time performance.
This paper addresses continuous lifelong multitask learning by jointly re-estimating the inter-task relations ( output kernel) and the per-task model parameters at each round, assuming data arrives in a streaming fashion.
We propose a novel algorithm called Online Output Kernel Learning Algorithm (OOKLA) for lifelong learning setting.
To avoid the memory explosion, we propose a robust budget-limited versions of the proposed algorithm that efﬁciently utilize the relationship between the tasks to bound the total number of representative examples in the support set.
In addition, we propose a two-stage budgeted scheme for efﬁciently tackling the task-speciﬁc budget constraints in lifelong learning.
Our empirical results over three datasets indicate superior AUC performance for OOKLA and its budget-limited cousins over strong baselines.
I NTRODUCTION
Instead of learning individual models, learning from multiple tasks leverage the relationships between tasks to jointly build a better model for each task and thereby improve the transfer of relevant knowledge between the tasks, especially from information-rich tasks to information-poor ones.
Unlike in the traditional multitask learning, where the tasks are presented simultaneously and an entire training set is available to the learner (Caruana (1998)), in lifelong learning the tasks arrives sequentially (Thrun (1996)).
This paper considers a continuous lifelong learning setting in which both the tasks and the examples of the tasks arrives in an online fashion, without any speciﬁc order.
Following the online setting, particularly from Saha et al.
; Cavallanti et al. (2010), at each round t , the learner receives an example from a task, along with the task identiﬁer and predicts the output label for the example.
Subsequently, the learner receives the true label and updates the model(s) as necessary.
This process is repeated as we receive additional data from a given dataset.
Our approach follows an error-driven update rule in which the model for a given task is updated only when the prediction for that task is in error.
Lifelong learning poses considerable challenges in terms of effectiveness (minimizing prediction errors for all tasks) and overall computational tractability for real-time performance.
A lifelong learning agent must provide an efﬁcient way to learn new tasks faster by utilizing the knowledge learned from the previous tasks and at the same time, without forgetting the old tasks.
The goal of a lifelong learner is to minimize errors compared to the full hindsight learner.
This paper addresses lifelong multitask learning by jointly re-estimating the inter-task relations from the data and the per-task model parameters at each round, assuming data arrives in a streaming fashion.
We call the task relationship matrix as output kernels deﬁned using Reproducing Kernel Hilbert Space (RKHS) on multitask examples.
We propose a novel algorithm called Online Output Kernel Learning Algorithm (OOKLA) for lifelong learning setting.
For a successful lifelong learning with kernels, we need to address two key challenges: (1) learn the relationships between the tasks (output kernel) efﬁciently from the data stream and (2) bound the size of the knowledge to avoid memory explosion.
The key challenge in learning with a large number of tasks is to adaptively learn the model parameters and the task relationships, which potentially change over time.
Without manageability-efﬁcient updates at each round, learning the task relationship matrix automatically may impose a severe computational burden.
In other words, we need to make predictions and update the models in an efﬁcient real time manner.
We propose simple and quite intuitive update rules for learning the task relationship matrix. When we receive a new example, the algorithm updates the output kernel (when the learner made a mistake) by computing the similarity between the new example and the set of representative examples (stored in the memory) that belongs to a speciﬁc task.
If the two examples have similar (different) labels and high similarity, then the relationship between the tasks is increased to reﬂect the positive (negative) correlation and vice versa. To avoid the memory explosion associated with the lifelong learning setting, we propose a robust budget-limited versions of the proposed algorithm that efﬁciently utilize the relationship between the tasks to bound the total number of representative examples in the support set. In addition, we propose a two-stage budgeted scheme for efﬁciently tackling the task-speciﬁc budget constraints in lifelong learning.
It is worth noting that the problem of lifelong multitask learning is closely related to online multitask learning. Although the objectives of both online multitask learning and lifelong learning are similar, one key difference is that the online multitask learning, unlike in the lifelong learning, may require the number of tasks beforehand. In recent years, online multitask learning has attracted extensive research attention Abernethy et al. ; Dekel et al.
; Lugosi et al. ; Cavallanti et al. (2010); Saha et al. ; Murugesan et al.
We evaluate our proposed methods with several state-of-the-art online learning algorithms for multiple tasks.
Throughout this paper, we refer to our proposed method as online multitask learning or lifelong learning.
There are many useful application areas for lifelong learning, including optimizing ﬁnancial trading, email prioritization, personalized news, and spam ﬁltering.
Consider the latter, where some spam is universal to all users (e.g. ﬁnancial scams), some messages might be useful to certain afﬁnity groups, but spam to most others (e.g. announcements of meditation classes or other special interest activities), and some may depend on evolving user interests.
In spam ﬁltering each user is a "task," and shared interests and dis-interests formulate the inter-task relationship matrix.
If we can learn the matrix as well as improving models from speciﬁc spam/not-spam decisions, we can perform mass customization of spam ﬁltering, borrowing from spam/not-spam feedback from users with similar preferences.
The primary contribution of this paper is precisely the joint learning of inter-task relationships and its use in estimating per-task model parameters in a lifelong learning setting.
1. R ELATED W ORK
Most existing work in online learning of multiple task focuses on how to take advantage of task relationships.
To achieve this, Lugosi et al.
(2009) imposed a hard constraint on the K simultaneous actions taken by the learner in the expert setting, Agarwal et al.
(2008) used matrix regularization, and Dekel et al.
(2007) proposed a global loss function, as an absolute norm, to tie together the loss values of the individual tasks.
Different from existing online multitask learning models, our paper proposes an intuitive and efﬁcient way to learn the task relationship matrix automatically from the data, and to explicitly take into account the learned relationships during model updates.
Cavallanti et al.
(2010) assumes that task relationships are available a priori .
However such task-relation prior knowledge is either unavailable or infeasible to obtain for many applications especially when the number of tasks K is large (Weinberger et al.
and/or when the manual annotation of task relationships is expensive (Kshirsagar et al.
Saha et al.
(2011) formulated the learning of task relationship matrix as a Bregman-divergence minimization problem w.r.t.
positive deﬁnite matrices.
The model suffers from high computational complexity as semi-deﬁnite programming is required when updating the task relationship matrix at each online round.
We show that with a different formulation, we can obtain a similar but much cheaper updating rule for learning the inter-task weights.
Murugesan et al.
(2016) proposed an efﬁcient method for learning the task relationship matrix using the cross-task performance measure, but their approach learns only the positive correlation between the tasks.
Our proposed approach learns positive and negative correlations between the tasks for robust transfer of knowledge from the previously learned tasks.
Recent works in output kernel learning learn task covariance matrix in RKHS space, inferred directly from the data (Dinuzzo et al.
; Sindhwani et al.
; Jawanpuria et al.
The task covariance matrix is called the output kernel deﬁned on the tasks, similar to the scalar kernel on the inputs.
Most recently, Jawanpuria et al.
(2015) showed that for a class of regularization functions, we can efﬁciently learn this output kernel.
Unfortunately most of the proposed methods for learning output kernels require access to the entire data for the learning algorithm, a luxury unavailable in the lifelong learning setting.
Unlike in online multitask learning, most lifelong learning approaches use a single model for all the tasks or reuse the models from the previous tasks to build a model for the new task (Alquier et al.
; Pentina & Urner (2016);
Pentina & Ben-David (2015); Balcan et al.
; Rusu et al.
; Hinton et al.
; Rebufﬁ et al.
; Ruvolo & Eaton (2013)). These approaches either increases the computation time on iterations where we encounter a novel task or reduces the prediction power of the model learned from the previous tasks due to catastrophic forgetting.
To the best of our knowledge, relationship between the tasks has not been exploited in the lifelong learning setting due to the difﬁculty in learning a positive semi-deﬁnite task relationship matrix in large-scale applications.
This paper provides an efﬁcient way to learn the task relationship matrix in the lifelong learning setting.
L IFELONG L EARNING OF M ULTIPLE T ASKS
Let (( x t , i t ) , y t ) be the example received by the learner from the task i t (at the time step t ) where we assume that the x t ∈ X and y t is its corresponding true label.
The task i t can be a new task or seen by the learner in the previous iterations.
We denote by [ N ] the consecutive integers ranging from 1 to N .
In this paper, we do not assume that the number of tasks is known to the learner ahead of time, an important constraint in lifelong learning problems.
Let K be the number of tasks seen so far until the current iteration t .
For brevity, we consider a binary classiﬁcation problem for each task y t ∈ {− 1 , +1 } , but the methods generalize to multi-class and are also applicable to regression tasks.
We assume that the learner made a mistake if y t (cid:54) =
ˆ y t where ˆ y t is the predicted label.
Our approach follows a mistake-driven update rule in which the model for a given task is updated only on rounds where the learner made a mistake.
Let K : X × X
→ R (kernel on input space) and Ω : N × N → R ( output kernel) be symmetric, positive semi-deﬁnite (p.s.d) multitask kernel functions and denote H as their corresponding RKHS of functions with the norm (cid:107) · (cid:107) H on multitask examples (Smola & Schölkopf (1998)
; Dinuzzo et al.
; Jawanpuria et al.
Using the above notation, we can deﬁne a kernel representation of an example based on a set of representative examples collected on the previous iterations ( prototypes ).
Formally, given an example x ∈ X , its kernel representation can be written using this set:
[Equation]
S is the set of stored examples for which the learner made a mistake in the past.
The set S is called the support set .
The online classiﬁcation function is then deﬁned as the weighted sum of the kernel combination of the examples in the support set.
To account for the examples from the different tasks, we consider both the kernel on the input spaces K and the output kernel Ω in our classiﬁcation function.
[Equation]
We set α s = y s .
The predicted label for a new example is computed from the linear combination of the labels of the examples from the support set S weighted by their input similarity K and the task similarity Ω to the new example.
Using the kernel trick , one can write:
[Equation]
Note that, in the above representation, we need learn both the support set S and the output kernel Ω from the data.
As explained in the previous section, for a successful lifelong learning with kernels, we need to address two key challenges: (1) learn the relationships between the tasks (output kernel) efﬁciently from the
data
arriving in an online fashion and (2) bound the
size of the support set
S to avoid memory explosion.
We address these two challenges in the following sections.
2. O NLINE O UTPUT K ERNEL L EARNING
Our objective function for the lifelong learning problem is given as follows:
[Equation]
(4) where (cid:96) ( · ) is some loss function such as hinge loss or logistic loss, R ( · ) is the regularization on the task relationship matrix Ω and λ is the regularization parameter.
Note that f in the above equation depends on Ω .
In order to reduce the time taken for each time-step, we require an efﬁcient update to the task relationship matrix Ω .
Following the work of Jawanpuria et al.
(2015) in the batch setting, we consider a subset of regularization functions R for which we can efﬁciently learn the task covariance matrix.
Consider the dual function of the above equation, at time-step t (see Barzilai & Crammer (2015); Jawanpuria et al. (2015)):
[Equation]
When we consider entry-wise l p norm between Ω and Ω ( t − 1) from the previous iteration as our regularization i.e.,
R (Ω , Ω ( t − 1) )
= (cid:80) k
| Ω i t k − (Ω ( t − 1) )
i t k | p for p ≥ 1 1 , we get the update function in Equation 2.
Similarly, if we Initialize: S = ∅ , Ω For t = 1 , 2 , · · · Receive new example ( x t , i t )
Let f i t = (cid:80) ( x s ,i s ) ∈S
y s Ω
i s i t φ ( x s )
Predict ˆ y t = sign ( f i t ( x t ))
Receive label y t
If y t (cid:54) =
ˆ y t
S ← S ∪ ( x t , i t )
Update Ω i t k ∀ k ∈
[ K ] Ω i t k ← Ω
i t k + 1 λ (cid:88) ( x s ,i s ) ∈S i s =
k y t K ( x t , x s )
y s (2) (or) Ω i t k ← Ω i t k exp (cid:26)λ (cid:88) ( x s ,i s ) ∈S i s =
k y t K ( x t , x s )
y s (cid:27) (3) end end consider generalized KL-divergence between Ω and Ω ( t − 1) i.e., R (Ω , Ω ( t − 1) )
= (cid:80) k
Ω i t k log Ω itk (Ω ( [Equation] − 1) )
[Equation] − [Equation] i t k
+ (Ω ( t − 1) )
i t k , we get the update function in Equation 3.
Unlike in the previous works, we update only the row (and the corresponding column) of the task relationship matrix Ω speciﬁc to the task i t , which signiﬁcantly reduces the time taken per example.
We can see that the update equations are simple and quite intuitive.
For a given new example ( x t , i t ) at round t , the algorithm updates Ω i t k
(for some k ∈ [ K ] ) by computing the similarity between the new example and the examples in the support set S that belongs to the task k .
If the two examples have similar (different) labels and high similarity
K ( x t , x s ) , then the Ω i t k is increased to reﬂect the positive (negative) correlation and vice versa.
The value closer to 0 implies no signiﬁcant relationship between the tasks.
The update to the Ω i t k is normalized by the regularization parameter λ for scaling.
It is worth noting that our update equations does not violate the p.s.d constraints on Ω in Equation 5.
If Ω from the previous iteration is a p.s.d matrix and the update is a p.s.d matrix (as it is computed using the Gram matrix of the example from the previous iteration), the sum and Hadamard product of two p.s.d matrices satisfy the p.s.d constraint (using Schur Product Theorem).
Algorithm 1 outlines the key steps in our proposed method.
We write f (( x t , i t ))
as f i t ( x t ) for notational convenience.
At each time-step t , the learner receives an example x t ∈ X and predicts the output label y t using ˆ y t = sign ( f i t ( x t )) .
We update both the support set S and the output kernel Ω i t · when the learner makes a mistake.
2. B UDGETED L EARNING
In Algorithm 1, we can see that both the classiﬁcation function f and the update equations for Ω uses the support set S .
When the target function changes over time, the support set S grows unboundedly.
This leads to serious computational and runtime issues especially in the lifelong learning setting.
The most common solution to this problem is to impose a bound on the number of examples in the support set S .
There are several budget maintenance strategies proposed recently (Cavallanti et al. ; Dekel et al. ; Orabona et al. Unfortunately these schemes cannot be directly used in our setting due to the output kernels in our learning formulation.
Cavallanti & Cesa-Bianchi (2012) proposed the multitask variant of these schemes but they are impractical for the lifelong learning setting.
We follow a simple support set removal schemes based on Crammer et al.
In single-task setting, when the number of examples in the support set S exceeds the limit (say B ), a simple removal scheme chooses an example x r with the highest conﬁdence from S
The conﬁdence of an example x r is measured using y r f ( x r ) after removing x r from the
support set S .
[Equation]
We can extend the above approach to the multitask and lifelong learning settings.
Since the support set S is shared by all the tasks, we choose an example x r with high conﬁdence to remove from each task function f k , weighted by the relationship between the tasks.
The objective function to choose the example is shown in Equation 7.
We show in the experiment section, this simple approach is efﬁcient and performs signiﬁcantly better than the state-of-the-art budget maintenance strategies.
Algorithm 2 shows pseudocode of the proposed budgeted learning algorithm.
Initialize: S = ∅ , Ω
For t = 1 , 2 , · · ·
Receive new example ( x t , i t )
Let f i t = (cid:80) ( x s ,i s ) ∈S
y s Ω
i s i t φ ( x s )
Predict ˆ y t = sign ( f i t ( x t ))
Receive label y t
If y t (cid:54) =
ˆ y t
If |S| < B
S ← S ∪ ( x t , i t )
Else
Find an example to remove arg
[Equation]
(cid:88)
[Equation]
S ← S ∪ ( x t , i t ) \ ( x r , i r ) end
Update Ω i t k ∀ k ∈
[ K ] as in Algorithm 1.
end end
Initialize: S = ∅ , T k = ∅ , k ∈
[ K ] , Ω
For t = 1 , 2 , · · ·
Receive new example ( x t , i t )
Let f i t = (cid:80) ( x s ,i s ) ∈T it
y s Ω
i s i t φ ( x s )
Predict ˆ y t = sign ( f i t ( x t ))
Receive label y t
If y t (cid:54) =
ˆ y t
S ← S ∪ { t }\{ r } /* r = ∅ if |S| < B , else choose r using Equation*/
For k ∈ [ K ]
T k ← T k ∪ { l }\{ r } /* remove r from T k ; choose l from S − T k */ end
Update Ω i t k ∀ k ∈
[ K ] as in Algorithm 1.
end end
2.2. T WO -S TAGE B UDGETED L EARNING
In lifelong learning setting, the number of tasks is typically large.
The support set S has hundreds of examples from all the tasks.
Each task does not use all the examples from the support set S .
For example, in movie recommendations task, recommendation for each user (task) can be characterized by just a few movies (subset of examples) in the support set
S .
Motivated by this observation, we propose a two-stage budgeted learning algorithm for the lifelong learning setting.
Algorithm 3 shows pseudocode of the proposed two-stage budgeted learning algorithm.
In addition to the support set S , we maintain task-speciﬁc support set T k .
We choose the budget for each task (say L ) where L <<< B .
Similar to the removal strategies for S , we remove an example from T k when |T k | >
L
and replace with an example from the set
S − T k .
The proposed two-stage approach provides better runtime complexity compared to the budgeted algorithm in
Algorithm 2.
Since only a subset of tasks may hold an example from S , removal step in Equation 7 requires only a subset of tasks for choosing an example.
This improves the runtime per iteration signiﬁcantly when the number of tasks is large.
One may consider a different budget size for each task L k based on the complexity of the task.
In addition, the proposed two-stage budgeted learning algorithm provides an alternative approach to using state-of-theart budget maintenance strategies.
For example, it is easier to use the Projectron algorithm (Orabona et al. (2008)) on
T k , rather than on S .
We will explore this line of research in our future work.
E XPERIMENTS
In this section, we evaluate the performance of our algorithms.
All reported results in this section are averaged over 10 random runs or permutations of the training data.
Unless otherwise speciﬁed, all model parameters are chosen via 5-fold cross validation. [Table]
3. B ENCHMARK D ATASETS
We use three benchmark datasets, commonly used for evaluating online multitask learning.
Details are given below:
Newsgroups Dataset 2 consists of 20 tasks generated from two subject groups: comp and talk.politics .
We paired two newsgroups, one from each subject (e.g.,comp.graphics vs talk.politics.guns), for each task.
In order to account for positive/negative correlation between the tasks, we randomly choose one of the newsgroups as positive (+) or negative ( − ) class.
Each post in a newsgroup is represented by a vocabulary of approximately 60 K unique features.
Spam Dataset
We use the dataset obtained from ECML PAKDD 2006 Discovery challenge for the spam detection task.
We used the task B challenge dataset which consists of labeled training data from the inboxes of 15 users.
We consider each user as a single task and the goal is to build a personalized spam ﬁlter for each user.
Each task is a binary classiﬁcation problem: spam (+) or non-spam ( − ) and each example consists of approximately 150 K features representing term frequency of the word occurrences.
Since some spam is universal to all users (e.g. ﬁnancial scams), some messages might be useful to certain afﬁnity groups, but spam to most others.
Such adaptive behavior of user’s interests and dis-interests can be modeled efﬁciently by utilizing the data from other users to learn per-user model parameters.
Sentiment Dataset 4 We evaluated our algorithm on product reviews from amazon.
The dataset contains product reviews from 25 domains.
We consider each domain as a binary classiﬁcation task.
Reviews with rating > 3 were labeled positive (+) , those with rating < 3 were labeled negative ( − ) , reviews with rating = 3 are discarded as the sentiments were ambiguous and hard to predict.
Similar to the previous datasets, each example consists of approximately 350 K features representing term frequency of the word occurrences.
We choose 2000 examples ( 100 posts per task) for 20 Newsgroups , 1500 emails for spam ( 100 emails per user inbox) and 2500 reviews for sentiment ( 100 reviews per domain) as training set for our experiments.
Note that we intentionally kept the size of the training data small to simulate the lifelong learning setting and drive the need for learning from previous tasks, which diminishes as the training sets per task become large.
Since these datasets have a class-imbalance issue (with few (+) examples as compared to ( − ) examples), we use average Area Under the ROC Curve ( AUC ) as the performance measure on test set.
3. R ESULTS
To evaluate the performance of our proposed algorithm ( OOKLA ), we use the three datasets ( Newsgroups , Spam and Sentiment ) for evaluation and compare our proposed methods to 5 baselines.
We implemented Perceptron and
Passive-Aggressive algorithm ( PA ) Crammer et al.
(2006) for online multitask learning.
Both Perceptron and PA learns [Table] .5s 0. (0.02) 2804.5s
Budget -OKL 0.
(0.02) 6.3s 0. (0.03) 10.2s 0. (0.03) 13.4s 0. (0.03) 18.8s 0.
(0.03) 22.6s 0.
(0.04) 24.7s 0.
(0.02) 1708.8s 0.
(0.01) 1674.4s 0.
(0.01) 1935.2s
Budget -OOKLA 0.
(0.04) 9.9s 0.
(0.04) 11.7s 0.
(0.03) 14.1s 0.
(0.08) 20.1s 0.
(0.04) 24.3s 0.
(0.02) 26.9s 0.
(0.06) 2250.5s 0.
(0.01) 2073.0s 0.
(0.04) 2334.3.3s independent model for each task.
These two baselines do not exploit the task-relationship or the data from other tasks during model update.
Next, we implemented two online multitask learning related to our approach: FOML – initializes Ω with ﬁxed weights Cavallanti et al.
(2010), Online Multitask Relationship Learning ( OMTRL )
Saha et al.
(2011) – learns a task covariance matrix along with task parameters.
Since OMTRL requires expensive call to SVD routines, we update the task-relationship matrix every 10 iterations.
In addition, we compare our proposed methods against the performance of Online Smooth Multitask Learning ( OSMTL ) which learn a probabilistic distribution over all tasks, and adaptively reﬁning such distribution over time Murugesan et al.
We implement two versions of our proposed algorithm with different updates for the task-relationship matrix: OOKLA-sum (Equation 2 OOKLA with sum update) OOKLA-exp (Equation 3 OOKLA with exponential update) as shown in Algorithm 1.
Table 1 summarizes the performance of all the above algorithms on the three datasets.
In addition to the AUC scores, we report the average total number of support vectors ( nSV ) and the CPU time taken for learning from one instance ( Time ).
From the table, it is evident that both OOKLA- sum and OOKLA- exp outperform all the baselines in terms of both
AUC and nSV .
This is expected for the two default baselines ( Perceptron and PA ).
The update rule for FOML is similar to ours but using ﬁxed weights.
The results justify our claim that learning the task-relationship matrix adaptively leads to improved performance.
As expected, both OOKLA and OSMTL consume less or comparable CPU time than the baselines which take into account for learning inter-task relationships.
Unlike in the OMTRL algorithm that recomputes the task covariance matrix every ( 10 ) iteration using expensive SVD routines, the task-relationship matrix in our proposed methods (and OSMTL ) are updated independently for each task.
We implement the OSMTL with exponential update for our experiments as it have shown to perform better than the other baselines.
One of the major drawbacks of
OSMTL is that it learn only the positive correlations between the tasks.
The performance of OSMTL worsens when the tasks are negatively correlated.
As we can see from the table, our proposed methods outperform OSMTL signiﬁcantly in the Newsgroup dataset.
Table 2 compares the proposed methods with different budget schemes and budget sizes in terms of test set AUC scores and the runtime.
We use OOKLA-sum for this experiment.
We set the value of B to { 50 , 100 , 150 } for all the datasets.
We compare our proposed budgeted learning algorithm (Algorithm 2) with the following state-of-the-art algorithms for online budgeted learning: (1) Random Budgeted Perceptron (RBP)
Cavallanti et al.
(2007)- randomly chooses an example to remove (2) (Self-tuned) Forgetron Dekel et al.
(2008)- forgets the oldest example (3) Projectron ++
Orabona et al.
(2008)- projects the new example on the others (4) Budgeted Online Kernel learning (Budget-OKL)- removes an example using equation 6 (5) Budgeted Online Output Kernel Learning (Budget-OOKLA)- removes an example using equation 7.
Following Cavallanti & Cesa-Bianchi (2012), we implement the online multitask version of
RBP, Forgetron and Projectron for our experiments.
Table 2 shows both the test set AUC scores (ﬁrst line) and time taken for learning from one instance (including the removal step).
It is evident from the table, our proposed budgeted learning algorithm for online multitask learning signiﬁcantly outperforms the other state-of-the-art budget schemes on most settings.
Our proposed algorithm uses the relationship between the tasks efﬁciently to choose the next example for removal.
It is worth noting that our proposed budgeted learning algorithm consumes less CPU time per example, comparable to that of RBP and Forgetron but better than Projectron which requires the computation of matrix inverse at each budget maintenance step
[Figure]
Finally, we evaluate the performance of the proposed two-stage budgeted scheme compared to the Algorithm 2.
To study the effect of different budget sizes L , we compute the cumulative mistake rate (cid:80) t I { y t (cid:54)
= ˆ y t } t measured at each time-step t . Figures 1 show average rate of mistakes against the number of training examples seen so far on newsgroup , spam and sentiment datasets for different values of L .
We ﬁx the value of B = 100 .
As we can see from the ﬁgures, even for the small values of L , the average mistake rate of the two-stage budgeted learning algorithm is comparable to the model which uses all the examples from the support set S .
We observe similar trend in the test set AUC scores.
On average, we achieved over 16% improvement in running time compared to the budget maintenance scheme in Algorithm 2.
We believe that the time consumption and the performance improvement will be even better for applications with large number of tasks.
C ONCLUSIONS
We proposed a novel lifelong learning algorithm using output kernels.
The proposed method efﬁciently learns both the model and the inter-task relationships at each iteration.
Our update rules for learning the task relationship matrix, at each iteration, were motivated by the recent work in output kernel learning.
In order to handle the memory explosion from the unbounded support set in lifelong learning setting, we proposed a novel budget maintenance scheme that utilize the task relationship matrix to remove the least-useful (high conﬁdence) example from the support set.
In addition, we proposed two-stage budget learning scheme based on the intuition that each task only requires a subset of the representative examples in the support set for efﬁcient learning.
It provides a competitive and efﬁcient approach to handle large number of tasks in many real-life applications.
Effectiveness of our algorithm is empirically veriﬁed over several benchmark datasets, outperforming several competitive baselines both in the unconstrained case and the budget-limited case, where selective forgetting was required.
Future research includes other forms of budget limits (e.g. time or reprocessing capacity), more complex inter-task relationships (e.g. hierarchical), and additional factors such as task reliability (e.g. prefer transferring from a well-trained related task, vs from a relatively data-sparse one, whose model is less reliable).