Information Retrieval as Statistical Translation
Adam Berger and John La erty
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213<aberger,lafferty>@cs.cmu.edu

Abstract

we can formulate tractable mathematical models of thequery generation process, models that naturally accountfor many of the features that are critical to modern highperformance retrieval systems, such as term weighting
and query expansion. Moreover, statistical translation
models for information retrieval can be implemented ina quite straightforward way, and our experiments withthese models demonstrate very promising empirical behavior.
We begin by detailing a conceptual model of information retrieval. In formulating a query to a retrieval
system, we imagine that a user begins with an information need. This information need is then represented
as a fragment of an \ideal document"|a portion of thetype of document that the user hopes to receive fromthe system. The user then translates or \distills" thisideal document fragment into a succinct query, selecting key terms and replacing some terms with related
terms: replacing pontiff with pope, for instance.
Summarizing the model of query generation,
1. The user has an information need =.
2. From this need, he generates an ideal document
fragment d= .
3. He selects a set of key terms from d= , and generates a query q from this set.
One can view this imaginary process of query formulation as a corruption of the ideal document. In thissetting, the task of a retrieval system is to nd those
documents most similar to d= . In other words, retrieval
is the task of nding, among the documents comprising
the collection, likely preimages of the user's query. Figure 1 depicts this model of retrieval in a block diagram.
We have drawn Figure 1 in a way that suggests an
information-theoretic perspective. One can view the
information need = as a signal that gets corrupted as
the user U distills it into a query q. That is, the queryformulation process represents a noisy channel, corrupting the information need just as a telephone cable corrupts the data transmitted by a modem. Given q anda model of the channel|how an information need getscorrupted into a query|the retrieval system's task is
to identify those documents d that best satisfy the information need of the user.

We propose a new probabilistic approach to information
retrieval based upon the ideas and methods of statistical machine translation. The central ingredient in thisapproach is a statistical model of how a user might distill or \translate" a given document into a query. To
assess the relevance of a document to a user's query, we
estimate the probability that the query would have been
generated as a translation of the document, and factor
in the user's general preferences in the form of a priordistribution over documents. We propose a simple, wellmotivated model of the document-to-query translation
process, and describe an algorithm for learning the parameters of this model in an unsupervised manner froma collection of documents. As we show, one can view
this approach as a generalization and justi cation of
the \language modeling" strategy recently proposed byPonte and Croft. In a series of experiments on TREC
data, a simple translation-based retrieval system performs well in comparison to conventional retrieval techniques. This prototype system only begins to tap thefull potential of translation-based retrieval.

1 Introduction
When a person formulates a query to a retrieval system,
what he is really doing is distilling an information need
into a succinct query. In this work, we take the viewthat this distillation is a form of translation from one
language to another: from documents, which containthe normal super uence of textual fat and connective
tissue such as prepositions, commas and so forth, toqueries, comprised of just the skeletal index terms thatcharacterize the document.
We take this view not because it is an accurate modelof how a user decides what to ask of an information
retrieval system, but because it turns out to be a useful expedient. By thinking about retrieval in this way,

ACM SIGIR Forum

219

Vol. 51 No. 2, July 2017

=
information
need

=

d

Document
generation model

Document-querytranslation model

ideal document
fragment

fdg

q
query

q
Retrieval SearchEngineretrieved
documents

Figure 1.

related, and exploit these relations to better rank documents by relevance to a query.
The rest of the paper proceeds as follows. Section 2
outlines some antecedents to this work, and describes ingreater detail the relationship between information retrieval and statistical translation. Section 3 introduces
two speci c models of translation from documents toqueries. Section 4 explains how to estimate the parameters of such models automatically from a collectionof documents. Section 5 presents results of several experiments on widely-used benchmarks in information
retrieval. These experimental results demonstrate thecompetitiveness of translation-based retrieval, compared
to standard t df vector space techniques. We conclude
in Section 6 with some further discussion of this approach and directions for future research.

user’squery

Model of query generation and retrieval

More precisely, the retrieval system's task is to nd
the a posteriori most likely documents given the query;that is, those d for which p (d q; ) is highest. By
Bayes' law,p (q d; ) p (d ):(1)
p (d q; ) =p (q )
j

j

j

U

U

2 Background and Related WorkU

There is a large literature on probabilistic approaches
to information retrieval, and we will not attempt tosurvey it here. Instead, we focus on the language modeling approach introduced recently by Ponte and Croft[10, 9], which is closest in spirit to the present work.
To each document in the collection, this approach associates a probability distribution p ( d) over terms;
using the terminology developed in the eld of speech
recognition, Ponte and Croft call this distribution a\language model." The probability of a term t givena document is related to the frequency of t in the document. The probability of a query q = q1 ; q2 ; : : : qm
is just the Q
product of the individual term probabilities,p (q d) = i p (qi d). The relevance of a document d
to a query q is presumed to be monotonically related
to p (q d).
The language modeling approach represents a novel
and theoretically motivated approach to retrieval, which
Ponte and Croft demonstrate to be e ective. However,this framework does not allow the capability to model
di erent forms or styles of queries, nor does it directly
address the important issues of synonymy and polysemy :multiple terms sharing similar meanings and the same
term having multiple meanings. In formulating a statistical translation-based approach to IR, we aim to develop a general statistical framework for handling theseissues.
The inspiration and foundation for the present work
comes from statistical machine translation|an area of
research that until now has existed independently of
information retrieval. It would take us too far a eld
to present an overview of the motivation and methodsof statistical translation. Instead, we refer the reader
to two articles [3, 4] outlining the IBM work that rst
developed the idea of statistical machine translation.
We will, however, brie y describe the general form of
the IBM statistical translation models, illustrated forthe case of translating from French into English. Thesemodels are comprised of translation probabilities t (f e)for each English word e translating to each French wordf , fertility probabilities to model the number of FrenchjU

jU

Since the denominator p (q ) is xed for a given queryand user, we can ignore it for the purpose of ranking
documents, and de ne the relevance q (d) of a document to a query asq (d) = p (q d; )(2)
| {z } p| (d{z }) :
jU

j

U

j

jU

query-dependent query-independent

Equation (2) highlights the decomposition of relevanceinto two terms: rst, a query-dependent term measuring the proximity of d to q, and second, a queryindependent or \prior" term, measuring the quality ofthe document according to the user's general preferences and information needs. Though in this work wetake the prior term to be uniform over all documents,
we imagine that in real-world retrieval systems the priorwill be crucial for improved performance, and for adapting to the user's needs and interests. At the very least,the document prior can be used to discount short documents, or perhaps documents in a foreign language.
High-performance document retrieval systems must
be sophisticated enough to handle polysemy andsynonymy|to know, for instance, that pontiff andpope are related terms. The eld of statistical translation concerns itself with how to mine large text
databases to automatically discover such semantic relations. Brown et al. [3, 4] showed, for instance, how asystem can \learn" to associate French terms with theirEnglish translations, given only a collection of bilingual
French/English sentences. We shall demonstrate how,in a similar fashion, an IR system can, from a collection of documents, automatically learn which terms arej

We use the convention that boldface roman letters refer to collections of words such as documents or queries, while italic roman lettersrefer to individual terms. Thus p (q j d) refers to the probability ofgenerating a single query word from an entire document d.

ACM SIGIR Forum

j

j

j

220

Vol. 51 No. 2, July 2017

words that an English word can generate, and distortion
probabilities for introducing a dependence on relative
word order in the two languages. These parameters areused to form a model p (f ; a e) for generating a Frenchsentence f = f1 ; f2 : : : ; fm from an English sentencee = e1 ; e2 ; : : : ; en in terms of a hidden alignment a
between the words in the two sentences.
Brown et al. propose a series of increasingly complex and powerful statistical models of translation, theparameters of which are estimated by a bootstrapping
procedure. We refer to [4] for a detailed presentationof these models, including the mathematical details of
training them using the EM algorithm. In the next section we discuss analogous models for query generation.

all possible alignments, given by
p

j

f

f

g

g

Suppose that an information analyst is given a news
article and asked to quickly generate a list of a few
words to serve as a rough summary of the article's topic.
As the analyst rapidly skims the story, he encountersa collection of words and phrases. Many of these arerejected as irrelevant, but his eyes rest on certain key
terms as he decides how to render them in the summary.
For example, when presented with an article about PopeJohn Paul II's visit to Cuba in 1998, the analyst decides
that the words pontiff and vatican can simply be
represented by the word pope, and that cuba, castro
and island can be collectively referred to as cuba.
In this section we present two statistical models of
this query formation process, making speci c independence assumptions to derive computationally and statistically ecient algorithms. While our simple query
generation models are mathematically similar to thoseused for statistical translation of natural language, theduties of the models are qualitatively di erent in thetwo settings. Document-query translation requires adistillation of the document, while translation of natural language will tolerate little being thrown away.

mn YX
am =0 j =1t

(qj daj ):
j

j

j

where
p

(3)

(qj d) =
j

X
w

t

j

(q j w ) l (w d ) ;jjwith the document language model l (w d) given by relative counts. Thus, we see that the query terms are generated using a mixture model|the document language
model provides the mixing weights for the translation
model, which has parameters t (q w). An alternative
view (and terminology) for this model is to describe
it as a Hidden Markov Model, where the states correspond to the words in the vocabulary, and the transition probabilities between states are proportional to theword frequencies.
The simplest version of Model 1, which we will distinguish as Model 0, is the one for which each word w
can be translated only as itself; that is, the translation
probabilities are \diagonal":
n 1 if q = w
t (q w ) =0 otherwise .
In this case, the query generation model is given by1np (q d ) =n + 1 l (q d) + n + 1 t (w <null>) ;a linear interpolation of the document language modeland the background model associated with the null word.
jjj

We rst consider the simplest of the IBM translation
models for the document-to-query mapping. This model
produces a query according to the following generative
procedure. First we choose a length m for the query,according to the distribution (m d). Then, for eachposition j [1 : : : m] in the query, we choose a position iin the document from which to generate qj , and generate the query word by \translating" di according to thetranslation model t ( di ). We include in position zero of
the document an arti cial \null word," written <null>.
The purpose of the null word is to generate spurious orcontent-free terms in the query (consider, for example,
a query q = Find all of the documents: : :).
Let's now denote the length of the document by
d = n. The probability p (q d) is then the sum over

j

j

j

2

j

j

3.2. Model 1 : A binomial model0

Our imaginary information analyst, when asked to generate a brief list of descriptive terms for a document, isunlikely to list multiple occurrences of the same word.
To account for this assumption in terms of a statistical model, we assume that a list of words is generated
by making several independent translations of the document d into a single query term q, in the following
manner. First, the analyst chooses a word w at randomj

ACM SIGIR Forum

a1 =0



j

3.1. Model 1: A mixture model

j

j

j

Just as the most primitive version of IBM's translation model takes no account of the subtler aspectsof language translation, including the way word ordertends to di er across languages, so our basic IR translation approach is but an impressionistic model of the relation between queries and documents relevant to them.
Since IBM called their most basic scheme Model 1, weshall do the same for this rudimentary retrieval model.
A little algebraic manipulation shows that the probability of generating query q according to Model 1 canbe rewritten asp (q d ) =

m  n
Y
1 t (w <null>)p (qj d) +(m d )
n+1
j =1 n + 1

3 Two Models of Document-Query Translationjnm d) X(q d) = (n(+
1)m

221

Vol. 51 No. 2, July 2017

Model 10 also has an interpretation in the degenerate case of diagonal translation probabilities. To see
this, let us make a further simpli cation by xing the
average number of samples to be a constant  independent of the document d, and suppose that the expected
number of times a query word is drawn is less than one,so that maxi  (qi j d) < 1. Then to rst order, theprobability assigned to the query according to Model 10
is a constant times the product of the language model
probabilities:

from the document. He chooses this word according to
the document language model (w j d). Next, he translates w into the word or phrase q according to the translation model (q j w). Thus, the probability of choosing
q as a representative of the document d isltp

(q j d ) =

X

w2d

l

(w j d) (q j w):lt

We assume that the analyst repeats this process n times,
where n is chosen according to the sample size model (n j d), and that the resulting list of words is ltered
to remove duplicates before it is presented as the summary, or query, q = q1 ; q2 ; : : : qm .
In order to calculate the probability that a particularquery q is generated in this way, we need to sum overall sample sizes n, and consider that each of the terms
qi may have been generated multiple times. Thus, theprocess described above assigns to q a total probability(q j d) =pXn (n j d)

X

n1 >0



X 

nm >0

Y
m

n

n1    nm i=1pp4.1. Synthetic training data

(4)

Lacking a large corpus of queries and their associatedrelevant documents, we decided to tease out the semantic relationships among words by generating synthetic queries for a large collection of documents andestimating the translation probabilities from this synthetic data. To explain the rationale for this scheme, we
return to our ctitious information analyst, and recall
that when presented with a document d, he will tend
to select terms that are suggestive of the content of the
document. Suppose now that he himself selects an arbitrary document d from a database D, and asks us toguess, based only upon his summary q, which document
he chose. The amount by which we are able to do better,on average, than randomly guessing a document fromD is the mutual information I (D; Q) = H (D) , H (D j Q)
between the random variables representing his choice of
document D and query Q. Here H (D) is the entropy in
the analyst's choice of document, and H (D j Q) is theconditional entropy of the document given the query. If
he is playing this game cooperatively, he will generatequeries for which this mutual information is large.
With this game in mind, we took a collection of
TREC documents D, and for each document d in the

not

not

p

Y

w2q

(1 , w )Y

w 2= qw:

Our use of this model was inspired by another IBM
statistical translation model, one that was designed formodeling a bilingual dictionary [5].

ACM SIGIR Forum(5)tThis formula shows that the probability of the queryis given as a product of terms. Yet the query termtranslations are independent, due to the process ofltering out the generated list to remove duplicates. We
refer to the model expressed in equation (4) as Model 10 .
Model 10 has an interpretation in terms of binomial
random variables. Suppose that a word w does
belong to the query with probability w = e,(d)p(w j d) .
Then Model 10 amounts to ipping independent w biased coins to determine which set of words comprisethe query. That is, we can reexpress the probability(q j d) of equation (4) as(q j d) =(qi j d) :We now describe an implementation of the models described in the previous section, and their application to
TREC data. The key ingredient in these models is thecollection of translation probabilities (q j w). But how
are we to obtain these probabilities? The statisticaltranslation strategy is to learn these probabilities froman aligned bilingual corpus of translated sentences, using the likelihood criterion. Ideally, we should have acollection of query/document pairs to learn from, obtained by human relevance judgments. But we know of
no publicly-available collection of data suciently large
to estimate parameters for general queries.

Under this assumption, the above sum takes on a muchfriendlier appearance:pl

4 Building a Translation-Based IR System

(d)n
:
n!

i=1

i=1

l

(qi j d)ni

m

Y
,

(d)
p (q j d) = ee(d) p(qi j d) , 1 :mYSince the mean  is xed for all documents, the document that maximizes the righthandQside of the above
expression is that which maximizes m
i=1 (qi j d). Thisis precisely the value assigned to the query in what
Ponte and Croft (1998) call the \language modeling approach."

In spite of its intimidating appearance, this expression
can be calculated eciently using simple combinatorial
identities and dynamic programming techniques. Instead of pursuing this path, we will assume that the
number of samples n is chosen according to a Poisson
distribution with mean (d): (n j d) = e,(d)(q = q1 ; : : : ; qm j d)  e, m222

Vol. 51 No. 2, July 2017qt (q j w )solzhenitsyn
0.319
citizenship
0.049
exile
0.044
archipelago
0.030
alexander
0.025
soviet
0.023
union
0.018
komsomolskaya
0.017
treason0.015
vishnevskaya
0.015
w = solzhenitsynqt (q j w )pontiff
0.502
pope
0.169
paul
0.065
john
0.035
vatican
0.033
ii
0.028
visit
0.017
papal
0.010
church0.005
flight0.004
w = pontiff

Figure 2.

t (q j w )qcarcinogen
0.667
cancer0.032
scientific0.024
science0.014
environment
0.013
chemical
0.012
exposure
0.012
pesticide
0.010
agent
0.009
protect
0.008
w = carcinogenqt (q j w )everest
0.439
climb
0.057
climber
0.045
whittaker
0.039
expedition
0.036
float
0.024
mountain
0.024
summit0.021
highest
0.018
reach
0.015
w = everestt (q j w )q

t (q j w )wildlife
0.705
fish
0.038
acre
0.012
species
0.010
forest0.010
environment
0.009
habitat0.008
endangered
0.007
protected
0.007
bird
0.007
w = wildlife

Sample translation probabilities after EM training on synthetic data.

collection, computed the mutual information statistic
I (w; d) for each of its words according toI (w; d) = p (w; d) log pp((ww jj Dd)) :Here p (w j d) is the probability of the word in the document, and p (w j D) is the probability of the word in
the collection at large. By scaling these I (w; d) values
appropriately, we constructed an arti cial cumulative
distribution function over words in each document. We
then drew m   ( j d) random samples from the document according to this distribution, forming a queryq = q1 ; : : : ; qm , and several such queries were generated
for each document.

from a corpus obtained by generating ve synthetic mutual information queries for each of the 78,325 documents in the collection.
For statistical models of this form, smoothing or interpolating the parameters away from their maximum
likelihood estimates is important. We used a linear interpolation of the background unigram model and the
EM-trained translation model:p (q j d) = p (q j D) + (1 , ) p (q j d)
= p (q j D) + (1 , )l (w j d) t (q j w) :X

w2d

The weight was empirically set to = 0:05 on heldout
data. The models for the baseline language modelingapproach, or Model 0, were also smoothed using linear
interpolation:l (w j d) = p (w j D) + (1 , ) l (w j d) :This interpolation weight was simply xed at = 0:1.
The Poisson parameter for the sample size distribution
was xed at  = 15, independent of the document. No
adjustment of any parameters, other than those determined by unsupervised EM training of the translation
probabilities, was carried out on the TREC volume 3data that we ran our evaluation on.

4.2. EM training

The resulting corpus f(d; q)g of documents and synthetic queries was used to t the translation probabilities of Models 1 and 1 with the EM algorithm [7], run
for only three iterations as a means to avoid over tting. Space limitations prevent us from explaining thedetails of the training process, but enough detail to enable implementation can be found in the papers [4, 5],which describe very similar models. A sample of theresulting translation probabilities, when trained on the
Associated Press (AP) portion of the TREC volume 3
corpus, is shown in Figure 2. In this gure, a document word is shown together with the ten most probable query words that it will translate to according to themodel. The probabilities in these tables are among the47,065,200 translation probabilities that were trainedfor our 132,625 word vocabulary. They were estimated0

ACM SIGIR Forumqzubin mehta
0.248
zubin
0.139
mehta
0.134
philharmonic
0.103
orchestra
0.046
music
0.036
bernstein
0.029
york
0.026
end
0.018
sir
0.016
w = zubin

5 Experimental Results on TREC Data
In this section we summarize the quantitative resultsof using the models described in the previous two sections to rank documents for a set of queries. Though223

Vol. 51 No. 2, July 2017

AP, concept fields

SJMN, concept fields
Model 1Model 1’Model 0
TFIDF

0.7

Model 1Model 1’Model 0
TFIDF

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0
0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

AP, concept fields

0.8

1AP, title fields
Model 1’, 3 EM iterations
Model 1’, 2 EM iterations
TFIDF

0.7

0.6

Model 1Model 1’Model 0
TFIDF

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0
0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

Spoken Document Retrieval Track

0.6

0.8

1

Language Model vs. Model 0
Model 1Model 1’Model 0
TFIDF

0.7

Model 0Language model

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0
0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

Figure 3. Precision-recall curves on TREC data. The two plots in the top row compare the performance ofModels 1 and 1 to the baseline t df and Model 0 performance on AP data (left) and SJMN data (right) whenranking documents for queries formulated from the concept elds for topics 51{100. The middle row shows thediscrepancy between two and three EM iterations of training for Model 1 (left) and the performance of models
on the short (average 2.8 words/query) queries obtained from the title eld of topics 51{100 (right). The bottom
row compares Models 1 and 1 to Model 0 and
SDR data (left) and the same language model scoredQ t ldf(qiond)theaccording to Model 0 and using the product m, demonstrating that the approximation in equation (5) isi=1
very good (right).
0

0

0

j

ACM SIGIR Forum

224

Vol. 51 No. 2, July 2017

not a real-time computation, calculating the relevance
score q(d) for each query can be done quickly enoughto obviate the need for a \fast match" to throw out
documents that are clearly irrelevant.
Our experiments fall into three categories. First,
we report on a series of experiments carried out on
the AP portion of the TREC data from volume 3 forboth the concept and title elds in topics 51{100. The
concept eld queries comprise a set of roughly 20 keywords, while the title elds are much more succinct|
typically not more than four words. Next, we tabulatea corresponding series of experiments carried out on
the 90,250 document San Jose Mercury News (SJMN)
portion of the data, also evaluated on topics 51{100.
Finally, we present results on the Spoken Document
Retrieval (SDR) track of the 1998 TREC evaluation,a small collection of 2,866 broadcast news transcripts.
All of the data is preprocessed by converting to upper
case, stemming, and ltering with a list of 571 stopwords from the SMART system.
Precision-recall curves for the AP and SJMN data,
generated from the output of the TREC evaluation software, appear in Figure 3. The baseline curves in theseplots show the performance of the t df measure using
Robertson's tf score, as described by Ponte in [9]. They
also show the result of using Model 0 to score the documents, using only word-for-word translations. In thisapproach, documents receive high relevance scores ifthey contain terms appearing in the query. Model 1 improves the average precision over the t df baseline by
19.4% on the AP data, and by 27.3% on the SJMN data.
The R-precision improves by 10.0% on the AP data andby 22.8% on the SJMN data. The actual numbers for
AP are collected in Table 1.
As discussed earlier, over tting during EM training is a concern. Figure 3 shows the performance ofModel 1 on the AP data when the probabilities aretrained for two and three EM iterations. To study thee ects of query length on the models, we also scored the
documents for the title elds of topics 51{100, where
the average query length is only 2.8 words. As the middle right plot of Figure 3 reveals, the precision-recall
curves are qualitatively di erent, showing a degradationin performance in the high-precision range. Overall,
Model 1 achieves an improvement over the t df baselineof 30.2% in average precision and 17.8% in R-precisionon these short queries. The marginal improvement ofModel 1 over Model 0 is smaller here|6.3% in average
precision and 4.9% in R-precision.
Two precision-recall plots for the SDR task are given.
The bottom left plot of Figure 3 shows the improvementof Model 1 over the baseline. There is an improvementof 22.2% in average precision and 18.4% in R-precision.
The bottom right plot compares Model 0 to the redocuments according to the probability
Qsultm ofl (ranking
q
d
)
for
the same language model, as in Ponteii=1and Croft's method. There is very little di erence inthe results, showing that equation (5) is indeed a good
approximation.

Relevant:
Rel.ret.:Precision:
at 0.00
at 0.10at 0.20
at 0.30
at 0.40
at 0.50at 0.60
at 0.70
at 0.80
at 0.90at 1.00
Avg.:
Precision at:
5 docs:
10 docs:
15 docs:
20 docs:
30 docs:
100 docs:
200 docs:
500 docs:
1000 docs:
R-Precision:

Model 15845
5845
0.71250.5916
0.52170.45540.41190.36360.31480.26980.22210.15800.04620.35750.55740.51700.51350.48510.45390.34190.26530.16100.09800.3578

%
|
|
+13.9
+13.1
+14.2
+17.1
+20.3
+19.8
+23.5
+27.4
+30.8
+40.7
+70.5
+19.4
+15.9
+10.0
+18.7
+15.1
+16.6
+15.5
+12.9
+9.8
+9.0
+10.0

1. Model 1 compared to the baseline systemfor queries constructed from the concept elds. These
numbers correspond to the upper left plot in Figure 3.

Table

6 Discussion

Viewing a user's interaction with an information retrieval system as translation of an information need into
a query is natural. Exactly this formulation is made ina recent overview of issues in information science presented to the theoretical computer science community[2]. In this paper we have attempted to lay the groundwork for building practical IR systems that exploit thisview, by demonstrating how statistical translation models can be built and used to advantage.
When designing a statistical model for language processing tasks, often the most natural route is to apply
a generative model which builds up the output stepby-step. Yet to be e ective, such models need to liberally distribute probability mass over a huge space ofpossible outcomes. This probability can be dicult tocontrol, making an accurate direct model of the distribution of interest dicult to construct. The sourcechannel perspective suggests a di erent approach: turnthe search problem around to predict the input. Far
more than a simple application of Bayes' law, there arecompelling reasons why reformulating the problem inthis way should be rewarding. In speech recognition,
natural language processing, and machine translation,
researchers have time and again found that predicting
what is already known (i.e., the query) from competing
hypotheses can be more e ective than directly predicting all of the hypotheses.
Our simple models only begin to tap the potentialof the translation-based approach. More powerful models of the query generation process, even along the linesof IBM's more complex models of translation, should

0

0

j

ACM SIGIR Forum

t df

5845
5845
0.62570.52310.45690.38900.34250.30350.25490.21170.16980.11230.02710.29930.48090.47020.4326
0.42130.3894
0.29600.23500.14660.08990.3254225

Vol. 51 No. 2, July 2017

Referenceso er performance gains. For example, one of the fundamental notions of statistical translation is the idea of
fertility, where a source word can generate zero or more
words in the target sentence. While there appears to
be no good reason why a word selected from the document should generate more than a single query term
per trial, we should allow for infertility probabilities,
where a word generates no terms at all. The use of stop
word lists mitigates but does not eliminate the need forthis improvement. The use of distortion probabilities
could be important for discounting relevant words thatappear toward the end of a document, and rewarding
those that appear at the beginning. Many other naturalextensions to the model are possible.
Our discussion has made the usual \bag of words"
assumption about documents, ignoring word order forthe sake of simplicity and computational ease. But therelative ordering of words is informative in almost all
applications, and crucial in some. The sense of a word isoften revealed by nearby words, and so by heeding contextual clues, one might hope to obtain a more accuratetranslation from document words to query words.
The retrieval-as-translation approach applies quite
naturally to the problem of multilingual retrieval, wherea user issues queries in a language T to retrieve documents in a source language S . The most direct approach
to multilingual IR expands the query q T to q T andthen translates directly into a source language queryq S , which is then issued to the database. The retrievalas-translation approach would learn translation models
which capture how words in S translate to words in
T . The source-channel framework should also be wellsuited to \higher-order" IR tasks, such as fact extraction and question answering from a database.

[1] A. Bookstein and D. Swanson (1974). \Probabilistic models for automatic indexing," Journal of
the American Society for Information Science, 25,
pp. 312{318.
[2] A. Broder and M. Henzinger (1998). \Information
retrieval on the web: Tools and algorithmic issues,"
Invited tutorial at Foundations of Computer Science (FOCS).
[3] P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra,
F. Jelinek, J. La erty, R. Mercer, and P. Roossin(1990). \A statistical approach to machine translation," Computational Linguistics, 16(2), pp. 79{85.
[4] P. Brown, S. Della Pietra, V. Della Pietra, and
R. Mercer (1993). \The mathematics of statisticalmachine translation: Parameter estimation," Computational Linguistics, 19(2), pp. 263{311.
[5] P. Brown, S. Della Pietra, V. Della Pietra, M.
Goldsmith, J. Hajic, R. Mercer, and S. Mohanty(1993). \But dictionaries are data too," In Proceedings of the ARPA Human Language Technology
Workshop, Plainsborough, New Jersey.
[6] W. B. Croft and D.J. Harper (1979). \Using probabilistic models of document retrieval without relevance information," Journal of Documentation, 35,
pp. 285{295.
[7] A. Dempster, N. Laird, and D. Rubin (1977).
\Maximum likelihood from incomplete data via the
EM algorithm," Journal of the Royal Statistical Society, 39(B), pp. 1{38.
[8] W. Gale and K. Church (1991). \Identifying
word correspondences in parallel texts," in Fourth
DARPA Workshop on Speech and Natural Language, Morgan Kaufmann Publishers, pp. 152{157.
[9] J. Ponte (1998). A language modeling approach to
information retrieval. Ph.D. thesis, University of
Massachusetts at Amherst.
[10] J. Ponte and W. B. Croft (1998). \A language modeling approach to information retrieval," Proceedings of the ACM SIGIR, pp. 275{281.
[11] S.E. Robertson and K. Sparck Jones (1976). \Relevance weighting of search terms," Journal of theAmerican Society for Information Science, 27,
pp. 129{146.
[12] S. Robertson, S. Walker, M. Hancock-Beaulieu, A.
Gull, and M. Lau (1992). \Okapi at TREC," In
Proceedings of the rst Text REtrieval Conference(TREC-1), Gaithersburg, Maryland.
[13] G. Salton and C. Buckley (1988). \Term-weighting
approaches in automatic text retrieval," Information Processing and Management, 24, pp. 513{523.
[14] H. Turtle and W. B. Croft (1991). \Ecient probabilistic inference for text retrieval," Proceedings of
RIAO 3.
[15] W. Weaver (1955). \Translation (1949)," In Machine Translation of Languages, MIT Press.

00

7 Conclusions
We have presented an approach to information retrievalthat exploits ideas and methods of statistical machinetranslation. After outlining the approach, we presented
two simple, motivated models of the document-querytranslation process. With the EM algorithm, the parameters of these models can be trained in an unsupervised fashion from a collection of documents. Experiments on TREC datasets demonstrate that eventhese simple methods can yield substantial improvements over standard baseline vector space methods, but
without explicit query expansion and term weighting
schemes, which are inherent in the translation approach
itself.

Acknowledgements
We would like to thank Jamie Callan, Jaime Carbonell,Ramesh Gopinath, Yiming Yang, and the anonymous
reviewers for helpful comments on this work. This research was supported in part by NSF KDI grant IIS9873009, an IBM University Partnership Award, an
IBM Cooperative Fellowship, and a grant from Justsystem Corporation.

ACM SIGIR Forum

226

Vol. 51 No. 2, July 2017