Transfer learning for text classification

Chuong B. Do
Computer Science Department
Stanford University
Stanford, CA 94305

Andrew Y. Ng
Computer Science Department
Stanford University
Stanford, CA 94305

Abstract
Linear text classification algorithms work by computing an inner product between a test document vector and a parameter vector. In many suchalgorithms, including naive Bayes and most TFIDF variants, the parameters are determined by some simple, closed-form, function of training set
statistics; we call this mapping mapping from statistics to parameters, theparameter function. Much research in text classification over the last few
decades has consisted of manual efforts to identify better parameter functions. In this paper, we propose an algorithm for automatically learning
this function from related classification problems. The parameter function found by our algorithm then defines a new learning algorithm fortext classification, which we can apply to novel classification tasks. We
find that our learned classifier outperforms existing methods on a variety
of multiclass text classification tasks.

1

Introduction

In the multiclass text classification task, we are given a training set of documents, eachlabeled as belonging to one of K disjoint classes, and a new unlabeled test document.
Using the training set as a guide, we must predict the most likely class for the test document. “Bag-of-words” linear text classifiers represent a document as a vector x ofword counts, and predict
Pn the class whose score (a linear function of x) is highest, i.e.,arg maxk∈{1,...,K} i=1 θki xi . Choosing parameters {θki } which give high classification
accuracy on test data, thus, is the main challenge for linear text classification algorithms.
In this paper, we focus on linear text classification algorithms in which the parameters are
pre-specified functions of training set statistics; that is, each θki is a function θki := g(uki )of some fixed statistics uki of the training set. Unlike discriminative learning methods, suchas logistic regression [1] or support vector machines (SVMs) [2], which use numerical optimization to pick parameters, the learners we consider perform no optimization. Rather, in
our technique, parameter learning involves tabulating statistics vectors {u ki } and applying
the closed-form function g to obtain parameters. We refer to g, this mapping from statistics
to parameters, as the parameter function.
Many common text classification methods—including the multinomial and multivariate
Bernoulli event models for naive Bayes [3], the vector space-based TFIDF classifier [4],and its probabilistic variant, PrTFIDF [5]—belong to this class of algorithms. Here, picking
a good text classifier from this class is equivalent to finding the right parameter function
for the available statistics.
In practice, researchers often develop text classification algorithms by trial-and-error,
guided by empirical testing on real-world classification tasks (cf. [6, 7]). Indeed, one couldargue that much of the 30-year history of information retrieval has consisted of manuallytrying TFIDF formula variants (i.e. adjusting the parameter function g) to optimize performance [8]. Even though this heuristic process can often lead to good parameter functions,such a laborious task requires much human ingenuity, and risks failing to find algorithm
variations not considered by the designer.
In this paper, we consider the task of automatically learning a parameter function g fortext classification. Given a set of example text classification problems, we wish to “metalearn” a new learning algorithm (as specified by the parameter function g), which may thenbe applied new classification problems. The meta-learning technique we propose, whichleverages data from a variety of related classification tasks to obtain a good classifier for
new tasks, is thus an instance of transfer learning; specifically, our framework automates
the process of finding a good parameter function for text classifiers, replacing hours ofhand-tweaking with a straightforward, globally-convergent, convex optimization problem.
Our experiments demonstrate the effectiveness of learning classifier forms. In low training
data classification tasks, the learning algorithm given by our automatically learned parameter function consistently outperforms human-designed parameter functions based on naive
Bayes and TFIDF, as well as existing discriminative learning approaches.

2

Preliminaries

Let V = {w1 , . . . , wn } be a fixed vocabulary of words, and let X = Zn and Y ={1, . . . , K} be the input and output spaces for our classification problem. A labeled document is a pair (x, y) ∈ X × Y, where x is an n-dimensional vector with xi indicating thenumber of occurrences of word wi in the document, and y is the document’s class label. A
classification problem is a tuple hD, S, (xtest , ytest )i, where D is a distribution over X × Y,
S = {(xi , yi )}M
i=1 is a set of M training examples, (xtest , ytest ) is a single test example, andall M + 1 examples are drawn iid from D. Given a training set S and a test input vector
xtest , we must predict the value of the test class label ytest .
P
In linear classification algorithms, we evaluate the score fk (xtest ) := i θki xtest i for assigning xtest to each class k ∈ {1, . . . , K} and pick the class y = arg maxk fk (xtest ) withthe highest score. In our meta-learning setting, we define each θki as the component-wiseevaluation of the parameter function g on some vector of training set statistics u ki :
 


θk1
g(uk1 )
 θk2  g(uk2 )  .  :=  .  .
(1)
 .. 
 .. 
θkn
g(ukn )
Here, each uki ∈ Rq (k = 1, . . . , K, i = 1, . . . , n) is a vector whose components arecomputed from the training set S (we will provide specific examples later). Furthermore,g : Rq → R is the parameter function mapping from uki to its corresponding parameterθki . To illustrate these definitions, we show that two specific cases of the naive Bayes and
TFIDF classification methods belong to the class of algorithms described above.

Naive Bayes: In the multinomial variant of the naive Bayes classification algorithm, 1 thescore for assigning a document x to class k isPn
(2)
fkNB (x) := log p̂(y = k) + i=1 xi log p̂(wi | y = k).
The first term, p̂(y = k), corresponds to a “prior” over document classes, andthe second term, p̂(wi | y = k), is the (smoothed) relative frequency of word
1
Despite naive Bayes’ overly strong independence assumptions and thus its shortcomings as aprobabilistic model for text documents, we can nonetheless view naive Bayes as simply an algorithm
which makes predictions by computing certain functions of the training set. This view has proved
useful for analysis of naive Bayes even when none of its probabilistic assumptions hold [9]; here, we
adopt this view, without attaching any particular probabilistic meaning to the empirical frequenciesp̂(·) that happen to be computed by the algorithm.

wi in training documents of class k. For P
balanced training sets, the first term isirrelevant. Therefore, we have fkNB (x) = i θki xi where θki = gNB (uki ),

 

uki1
number of times wi appears in documents of class kuki2   number of documents of class k containing wi  uki := uki3  =  total number of words in documents of class k  , (3)u  

total number of documents of class k
ki4
uki5
total number of documents
anduki1 + ε(4)gNB (uki ) := log
uki3 + nε
where ε is a smoothing parameter. (ε = 1 gives Laplace smoothing.)
TFIDF: In the unnormalized TFIDF classifier, the score for assigning x to class k is
´³
´
Pn ³
fkTFIDF (x) := i=1 xi |y=k · log p̂(xi1>0) xi · log p̂(xi1>0) ,(5)

where xi |y=k (sometimes called the average term frequency of wi ) is the average
ith component of all document vectors of class k, and p̂(xi > 0) (sometimes
called the document frequency of wi ) is the
Pproportion of all documents containing
wi .2 As before, we write fkTFIDF (x) = i θki xi with θki = gTFIDF (uki ). The
statistics vector is again defined as in (3), but this time,
µ
¶2
uki5
uki1
log
gTFIDF (uki ) :=
.
(6)
uki4
uki2

Space constraints preclude a detailed discussion, but many other classification algorithms
can similarly be expressed in this framework, using other definitions of the statistics vectors {uki }. These include most other variants of TFIDF based on different TF and IDF
terms [7], PrTFIDF [5], and various heuristically modified versions of naive Bayes [6].

3

Learning the parameter function

In the last section, we gave two examples of algorithms that obtain their parameters θ ki
by applying a function g to a statistics vector uki . In each case, the parameter function
was hand-designed, either from probabilistic (in the case of naive Bayes [3]) or geometric(in the case of TFIDF [4]) considerations. We now consider the problem of automaticallylearning a parameter function from example classification tasks. In the sequel, we assume
fixed statistics vectors {uki } and focus on finding an optimal parameter function g.
In the standard supervised learning setting, we are given a training set of examples sampled from some unknown distribution D, and our goal is to use the training set to make a
prediction on a new test example also sampled from D. By using the training examples tounderstand the statistical regularities in D, we hope to predict ytest from xtest with low error.
Analogously, the problem of meta-learning g is again a supervised learning task; here, however, the training “examples” are now classification problems sampled from a distribution
D over classification problems.3 By seeing many instances of text classification problems
2

Note that (5) implicitly defines fkTFIDF (x) as a dot product of two vectors, each of whose components consist of a product of two terms. In the normalized TFIDF classifier, both vectors arenormalized to unit length before computing the dot product, a modification that makes the algorithm
more stable for documents of varying length. This too can be represented within our framework byconsidering appropriately normalized statistics vectors.
3
Note that in our meta-learning problem, the output of our algorithm is a parameter function
g mapping statistics to parameters. Our training data, however, do not explicitly indicate the bestparameter function g ∗ for each example classification problem. Effectively then, in the meta-learningtask, the central problem is to fit g to some unseen g∗ , based on test examples in each trainingclassification problem.

drawn from D, we hope to learn a parameter function g that exploits the statistical regularities in problems from D. Formally, let S = {hD (j) , S (j) , (x(j) , y (j) )i}m
j=1 be a collectionof m classification problems sampled iid from D. For a new, test classification problem
hDtest , Stest , (xtest , ytest )i sampled independently from D, we desire that our learned g correctly classify xtest with high probability.
To achieve our goal, we first restrict our attention to parameter functions g that are linear
in their inputs. Using the linearity assumption, we pose a convex optimization problem
for finding a parameter function g that achieves small loss on test examples in the training
collection. Finally, we generalize our method to the non-parametric setting via the “kernel
trick,” thus allowing us to learn complex, highly non-linear functions of the input statistics.
3.1

Softmax learning

Recall that in softmax regression, the class probabilities p(y | x) are modeled asP
exp( i θki xi )Pp(y = k | x; {θki }) := P,
k = 1, . . . , K,0k0 exp(i θk i xi )(7)

where the parameters {θki } are learned from the training data S by maximizing the conditional log likelihood of the data. In this approach, a total of Kn parameters are trained
jointly using numerical optimization. Here, we consider an alternative approach in which
each of the Kn parameters is some function of the prespecified statistics vectors; in particular, θki := g(uki ). Our goal is to learn an appropriate g.
To pose our optimization problem, we start by learning the linear form g(uki ) = β T uki .
Under this parameterization, the conditional likelihood of an example (x, y) isP
exp( i β T uki xi ),k = 1, . . . , K.
(8)p(y = k | x; β) = P
P T
k0 exp(i β uk 0 i xi )In this setup, one natural approach for learning a linear function g is to maximize the(regularized) conditional log likelihood `(β : S ) for the entire collection S :Pm`(β : S ) := j=1 log p(y (j) | x(j) ; β) − C||β||2
³ P
´ 

(j)
(j)
m
exp β T i uy(j) i xi
X
³ P´  − C||β||2 .
(9)=log  P(j) (j)
T
exp
β
u
x
j=1
k
i ki i

In (9), the latter term corresponds to a Gaussian prior on the parameters β, which provides
a means for controlling the complexity of the learned parameter function g. The maximization of (9) is similar to softmax regression training except that here, instead of optimizing
over the parameters {θki } directly, we optimize over the choice of β.
3.2

Nonparametric function learning

In this section, we generalize the technique of the previous section to nonlinear g. By the
Representer Theorem [10], there exists a maximizing solution to (9) for which the optimal
parameter vector β ∗ is a linear combination of training set statistics:
Pm P ∗ P (j) (j)(10)β ∗ = j=1 k αjki uki xi .

From this, we reparameterize the original optimization over β in (9) as an equivalent optimization over training example weights {αjk }. For notational convenience, let
P P (j) (j 0 ) (j)(j 0 )
K(j, j 0 , k, k 0 ) := i i0 xi xi0 (uki )T uk0 i0 .
(11)

1

0
−5

0.8

uk2

exp(uk2)

−10
0.6
−15

0.4
−20
0.2
0

(a)

−25

0

0.2

0.4
0.6
exp(u )

0.8

1

k1

(b)

−30
−35

−30

−25

−20

−15

−10

−5

0

u

k1

Figure 1: Distribution of unnormalized uki vectors in dmoz data (a) with and (b) without
applying the log transformation in (15). In principle, one could alternatively use a feature
vector representation using these frequencies directly, as in (a). However, applying the logtransformation yields a feature space with fewer isolated points in R2 , as in (b). When using
the Gaussian kernel, a feature space with few isolated points is important as the topologyof the feature space establishes locality of influence for support vectors.
Substituting (10) and (11) into (9), we obtain
³P
´ 

m P
0
(j 0 )
m
exp
)
X
j=1
k αjk K(j, j , k, y³P´log  P`({αjk } : S ) :=m P
00
j 0 =1
k0 expj=1
k αjk K(j, j , k, k )
−C

m XX
m X
X
j=1 j 0 =1 kαjk αj 0 k0 K(j, j 0 , k, k 0 ).

(12)

k0

Note that (12) is concave and differentiable, so we can train the model using any standard
numerical gradient optimization procedure, such as conjugate gradient or L-BFGS [11].
The assumption that g is a linear function of uki , however, places a severe restriction on
the class of learnable parameter functions. Noting that the statistics vectors appear only as
an inner product in (11), we apply the “kernel trick” to obtain
P P (j) (j 0 )(j)(j 0 )(13)
K(j, j 0 , k, k 0 ) := i i0 xi xi0 K(uki , uk0 i0 ),­®
where the kernel function K(u, v) = Φ(u), Φ(v) defines the inner product of somehigh-dimensional mapping Φ(·) of its inputs.4 In particular, choosing a Gaussian (RBF)
kernel, K(u, v) := exp(−γ||u − v||2 ), gives a non-parametric representation for g:
Pm P P(j)(j)
g(uki ) = β T Φ(uki ) = j=1 k i αjk xi exp(−γ||uki − uki ||2 ).
(14)
(j)

Thus, g(uki ) is a weighted combination of the values {αjk xi }, where the weights depend(j)exponentially on the squared `2 -distance of uki to each of the statistics vectors {uki }. As aresult, we can approximate any sufficiently smooth bounded function of u arbitrarily well,
given sufficiently many training classification problems.

4

Experiments

To validate our method, we evaluated its ability to learn parameter functions on a varietyof email and webpage classification tasks in which the number of classes, K, was large(K = 10), and the number of number of training examples per class, m/K, was small(m/K = 2). We used the dmoz Open Directory Project hierarchy,5 the 20 Newsgroups
dataset,6 the Reuters-21578 dataset,7 and the Industry Sector dataset8 .
4Note also that as a consequence of our kernelization, K itself can be considered a “kernel”
between all statistics vectors from two entire documents.
5
http://www.dmoz.org
6
http://kdd.ics.uci.edu/databases/20newsgroups/20newsgroups.tar.gz
7
http://www.daviddlewis.com/resources/testcollections/reuters21578/reuters21578.tar.gz
8
http://www.cs.umass.edu/˜mccallum/data/sector.tar.gz

Table 1: Test set accuracy on dmoz categories. Columns 2-4 give the proportion of correctclassifications using non-discriminative methods: the learned g, Naive Bayes, and TFIDF,
respectively. Columns 5-7 give the corresponding values for the discriminative methods:
softmax regression, 1-vs-all SVMs, and multiclass SVMs. The best accuracy in each row
is shown in bold.
Category
Arts
Business
Computers
Games
Health
Home
Kids and Teens
News
Recreation
Reference
Regional
Science
Shopping
SocietySportsWorld
Average

g
0.421
0.456
0.467
0.411
0.479
0.640
0.252
0.349
0.663
0.635
0.438
0.363
0.612
0.435
0.619
0.531
0.486

gNB
0.296
0.283
0.304
0.288
0.282
0.470
0.205
0.222
0.487
0.415
0.268
0.256
0.456
0.308
0.432
0.491
0.341

gTFIDF
0.286
0.286
0.327
0.240
0.337
0.454
0.142
0.212
0.529
0.458
0.258
0.246
0.556
0.285
0.285
0.352
0.328

softmax
0.352
0.336
0.344
0.279
0.382
0.501
0.202
0.382
0.477
0.602
0.329
0.353
0.483
0.379
0.507
0.329
0.390

1VA-SVM
0.203
0.233
0.217
0.240
0.213
0.333
0.173
0.270
0.353
0.383
0.260
0.223
0.373
0.213
0.267
0.277
0.264

MC-SVM
0.367
0.340
0.387
0.330
0.337
0.440
0.167
0.397
0.590
0.543
0.357
0.340
0.550
0.377
0.527
0.303
0.397

The dmoz project is a hierarchical collection of webpage links organized by subject matter.
The top level of the hierarchy consists of 16 major categories, each of which contains several subcategories. To perform cross-validated testing, we obtained classification problems
from each of the top-level categories by retrieving webpages from each of their respective subcategories. For the 20 Newsgroups, Reuters-21578, and Industry Sector datasets,
we performed similar preprocessing.9 Given a dataset of documents, we sampled 10-class2-training-examples-per-class classification problems by randomly selecting 10 different
classes within the dataset, picking 2 training examples within each class, and choosing one
test example from a randomly chosen class.
4.1

Choice of features

Theoretically, for the method described in this paper, any sufficiently rich set of features
could be used to learn a parameter function for classification. For simplicity, we reduced
the feature vector in (3) to the following two-dimensional representation,10¸·
log(proportion of wi among words from documents of class k)
.
(15)
uki =
log(proportion of documents containing wi )Note that up to the log transformation, the components of uki correspond to the relative
term frequency and document frequency of a word relative to class k (see Figure 1).
4.2

Generalization performance

We tested our meta-learning algorithm on classification problems taken from each of the
16 top-level dmoz categories. For each top-level category, we built a collection of 300
classification problems from that category; results reported here are averages over these9

For the Reuters data, we associated each article with its hand-annotated “topic” label and discarded any articles with more than one topic annotation. For each dataset, we discarded all categories
with fewer than 50 examples, and selected a 500-word vocabulary based on information gain.
10
Features were rescaled to have zero mean and unit variance over the training set.

Table 2: Cross corpora classification accuracy, using classifiers trained on each of the four
corpora. The best accuracy in each row is shown in bold.
Datasetgdmoz gnews greut
gindu
gNB gTFIDF softmax 1VA-SVM MC-SVM
dmoz
n/a 0.471 0.475 0.473 0.365 0.352 0.381
0.283
0.412
0.206
0.248
20 Newsgroups 0.369 n/a 0.371 0.369 0.223 0.184 0.217
Reuters-21578 0.567 0.567 n/a 0.619 0.463 0.475 0.463
0.308
0.481
0.271
0.375
Industry Sector 0.438 0.459 0.446 n/a 0.374 0.274 0.376

problems. To assess the accuracy of our meta-learning algorithm for a particular test category, we used the g learned from a set of 450 classification problems drawn from the other15 top-level categories.11 This ensured no overlap of training and testing data. In 15 out
of 16 categories, the learned parameter function g outperforms naive Bayes and TFIDF in
addition to the discriminative methods we tested (softmax regression, 1-vs-all SVMs [12],and multiclass SVMs [13]12 ; see Table 1).13
Next, we assessed the ability of g to transfer across even more dissimilar corpora. Here, foreach of the four corpora (dmoz, 20 Newsgroups, Reuters-21578, Industry Sector), we constructed independent training and testing datasets of 480 random classification problems.
After training separate classifiers (gdmoz , gnews , greut , and gindu ) using data from each of thefour corpora, we tested the performance of each learned classifier on the remaining three
corpora (see Table 2). Again, the learned parameter functions compare favorably to theother methods. Moreover, these tests show that a single parameter function may give an accurate classification algorithm for many different corpora, demonstrating the effectivenessof our approach for achieving transfer across related learning tasks.

5

Discussion and Related WorkIn this paper, we presented an algorithm based on softmax regression for learning a parameter function g from example classification problems. Once learned, g defines a newlearning algorithm that can be applied to novel classification tasks.
Another approach for learning g is to modify the multiclass support vector machine formulation of Crammer and Singer [13] in a manner analagous to the modification of softmax
regression in Section 3.1, giving the following quadratic program:P
minimize 12 ||β||2 + C j ξjβ∈Rn ,ξ∈Rm
P (j) (j)(j)
subject to β T i xi (uy(j) i − uki ) ≥ I{k6=y(j) } − ξj , ∀k, ∀j.
As usual, taking the dual leads naturally to an SMO-like procedure for optimization. We
implemented this method and found that the learned g, like in the softmax formulation,outperforms naive Bayes, TFIDF, and the other discriminative methods.
The techniques described in this paper give one approach for achieving inductive transfer inclassifier design—using labeled data from related example classification problems to solvea particular classification problem [16, 17]. Bennett et al. [18] also consider the issue ofknowledge transfer in text classification in the context of ensemble classifiers, and proposea system for using related classification problems to learn the reliability of individual classifiers within the ensemble. Unlike their approach, which attempts to meta-learn properties
11
For each execution of the learning algorithm, (C, γ) parameters were determined via grid searchusing a small holdout set of 160 classification problems. The same holdout set was used to selectregularization parameters for the discriminative learning algorithms.
12
We used LIBSVM [14] to assess 1VA-SVMs and SVM-Light [15] for multiclass SVMs.
13
For larger values of m/K (e.g. m/K = 10), softmax and multiclass SVMs consistently outperform naive Bayes and TFIDF; nevertheless, the learned g achieves a performance on par with discriminative methods, despite being constrained to parameters which are explicit functions of training
data statistics. This result is consistent with a previous study in which a heuristically hand-tuned
version of Naive Bayes attained near-SVM text classification performance for large datasets [6].

of algorithms, our method uses meta-learning to construct a new classification algorithm.
Though not directly applied to text classification, Teevan and Karger [19] consider the
problem of automatically learning term distributions for use in information retrieval.
Finally, Thrun and O’Sullivan [20] consider the task of classification in a mobile robot domain. In this work, the authors describe a task-clustering (TC) algorithm in which learning
tasks are grouped via a nearest neighbors algorithm, as a means of facilitating knowledgetransfer. A similar concept is implicit in the kernelized parameter function learned by ouralgorithm, where the Gaussian kernel facilitates transfer between similar statistics vectors.

Acknowledgments
We thank David Vickrey and Pieter Abbeel for useful discussions, and the anonymousreferees for helpful comments. CBD was supported by an NDSEG fellowship. This work
was supported by DARPA under contract number FA8750-05-2-0249.

References[1] K. Nigam, J. Lafferty, and A. McCallum. Using maximum entropy for text classification. In
IJCAI-99 Workshop on Machine Learning for Information Filtering, pages 61–67, 1999.
[2] T. Joachims. Text categorization with support vector machines: Learning with many relevant
features. In Machine Learning: ECML-98, pages 137–142, 1998.
[3] A. McCallum and K. Nigam. A comparison of event models for Naive Bayes text classification.
In AAAI-98 Workshop on Learning for Text Categorization, 1998.
[4] G. Salton and C. Buckley. Term weighting approaches in automatic text retrieval. Information
Processing and Management, 29(5):513–523, 1988.
[5] T. Joachims. A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization. In Proceedings of ICML-97, pages 143–151, 1997.
[6] J. D. Rennie, L. Shih, J. Teevan, and D. R. Karger. Tackling the poor assumptions of naive
Bayes text classifiers. In ICML, pages 616–623, 2003.
[7] A. Moffat and J. Zobel. Exploring the similarity space. In ACM SIGIR Forum 32, 1998.
[8] C. Manning and H. Schutze. Foundations of statistical natural language processing, 1999.
[9] A. Ng and M. Jordan. On discriminative vs. generative classifiers: a comparison of logistic
regression and naive Bayes. In NIPS 14, 2002.
[10] G. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions. J. Math. Anal.
Appl., 33:82–95, 1971.
[11] J. Nocedal and S. J. Wright. Numerical Optimization. Springer, 1999.
[12] R. Rifkin and A. Klautau. In defense of one-vs-all classification. J. Mach. Learn. Res., 5:101–
141, 2004.
[13] K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based
vector machines. J. Mach. Learn. Res., 2:265–292, 2001.
[14] C-C. Chang and C-J. Lin. LIBSVM: a library for support vector machines, 2001. Software
available at http://www.csie.ntu.edu.tw/˜cjlin/libsvm.
[15] T. Joachims. Making large-scale support vector machine learning practical. In Advances in
Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA, 1998.
[16] S. Thrun. Lifelong learning: A case study. CMU tech report CS-95-208, 1995.
[17] R. Caruana. Multitask learning. Machine Learning, 28(1):41–75, 1997.
[18] P. N. Bennett, S. T. Dumais, and E. Horvitz. Inductive transfer for text classification using
generalized reliability indicators. In Proceedings of ICML Workshop on The Continuum fromLabeled to Unlabeled Data in Machine Learning and Data Mining, 2003.
[19] J. Teevan and D. R. Karger. Empirical development of an exponential probabilistic model for
text retrieval: Using textual analysis to build a better model. In SIGIR ’03, 2003.
[20] S. Thrun and J. O’Sullivan. Discovering structure in multiple learning tasks: The TC algorithm.
In International Conference on Machine Learning, pages 489–497, 1996.