IV•

Snvironmont Of The Reported Bxporiments

The present study of relevance feedback compares several related formulas for query modification*

The following

general formula is available to the experimenter:
mln(n a , n r )^•1 - n HL + « qQ •o£min(n blaQ)

r± + \i^s±(D)

where n^r • n s (see formula A» Section 1) equals N f the number of documents retrieved for feedback*The experimental variables are oC% m$ •*, \i+ n t n,.f andaoN*

Tne parameter &£ is positive, and weights all incoming

relevant documents relative to the other contributors tothe query (previous query, initial query, non-relevantdocuments)•

The parameternpermits the previous query

to be increased in weight relative to the incoming documents,vw

is the initial query, as opposed to the queryof the previous iteration;«permits the initial query to

be used as part of the new query (see Section 3B) # The
parameter\ishould theoretically be negative, as it per-mits some significance to be attached to the non-relevant
documents retrieved.

The parametern(nh^ P***mi*s somespecific number of relevant (non-relevant) documents to beused in the query even if n^ (n„) is larger©
rs

It is as-sumed that the r, and a. are indexed in order of clecreasing relevance (as determined by the system) to thequery; that is, thenrelevant documents (ornfc

non-relov&nt documents) used in the new query will be those19

20
closest in the descriptor space to the previous query*
The flexibility of this formula permits the investigationof several feedback strategies'*
The system also provides the following formula tosimulate Hocchiof8 algorithm*min(nrf n a )*i*r n n r V i * " % • n s

2rmin(nB* nfe)

i~ n r]>8i <fi)

Formula S does not normalize the vector lengths as isd>ne in
Rocchio*6 algorithm (formula A)*
The document collection used in this study (the"Cranfield" collection) contains 200 documents from thefield of aerodynamics, chosen from a library of 1400 documents*

For this collection, there are 42 queries, con-structed by some of the authors of the 1400 documents;
these requestors are also responsible for the relevancejudgments*
The concept vectors describing document ana queries
are quite sparse for the "Cranfield" collection.

The maxi-mum number of concepts used to describe one document is &5%
out of a possible 5i?2 concepts*

The largest weight given

to any concept in any document descriptor is 288*

The querydescription vectors are sparser by one order of magnitude
and shorter than the document descriptors*

The maximum

number of concepts used in a single query vector is 12;
the largest weight in any query vector is 24*

The largest

number of documents relevant to a single query is 12, or
six percent of the collection*

The comparative brevity of

21
the query vectors in this collection is typical in technical document retrievalf because document abstracts generally contain more detailed information than user queries*
The characteristics of an experimental document collection determine the extent to which experimental results
typify retrieval behavior in 'real1 document collections*
The three collections described in this study are much
too small to require sophisticated retrieval techniques*
However, the Cranfield 200 document collection is morerealistic than the ADI and IRE collections for three reasons*First, more queries are available for the Cranfield
200 collection•

The number of queries available is im-portant in judging the statistical significance of experimental results*
Second*, the documents in the Cranfield 200 collection
were chosen from a more typical environment*

The ADIcollection consists of short papers all presented at thesame conference*

The papers in the IRS collection were

all published in the same magazine within a seven-month
period*By contrast, the 1400 documents in the full Cran-field collection were in effect selected by knowledgableauthors from the field of aerodynamics, and the 200 documents in the small collection were chosen to represent thelarger collection*
Third, the queries end relevance judgments in the ADIand IRJS collections were constructed by a small number ofInformation retrieval experts, while those in the full
Cranfield collection were constructed by 162 authors of22

recent papers in aerodynamics•
Concept vector** for both the ADI and Cranfield collections were constructed auton&tically using a regularsubject-area theaaurus#In all experiments reported here,

the co&lne correlation (Section I) is used as the distancefunction.