II*

User Interaction With An On-Line Retrieval System

The full search retrieval algorithm returns to theuser thendocument** with concept vectors "closest" tothe query vector as measured by the angle between vectors(cosine correlation)*If the U6erfs original query is an

accurate and complete description (in "concepts") of hisneed, and if the documents relevant to the user are clustered "close" together in the space of concept vectors, thisalgorithm can isolate these few relevant items from a large
collection of irrelevant material*

Howeverf neither ofthese conditions is common in practice*

It is evident fromexperiments with the SMART system that a user familiar withthe subject area but unaware of vocabulary and word frequency effects on the search process is unlikely to formulate an initial query that provides optimum retrieval *• K
It is unreasonable, however, to expect each user to understand the fine details of the document classification system*
Further| there is evidence in the experimental docuf

ment collection used here that the documents judged relevantby the users are not always clustered neatly in the conceptvector space*

Sven with full knowledge of the document

collection it is often impossible to formulate a single
query that will rank all relevant documents above all nonrelevant documents*

This may indicate flaws in the text-to-

vector mapping used for this study*

However, the needs ofthe human users of document collections are so diverse thata subject classification system appropriate for all queries
may not exist, or may be impractical to implement*
89Since the user's original query is often inadequatef
some sort of user interaction with the retrieval operation
is desirable*The user of a manual retrieval system suchas a library might at first ask a general and unclear question.

The librarian, using his knowledge of the document

collectionf might then ask the user a few questions andshow him a few books in an attempt to pinpoint his needs*
Recent technological developments encourage the investigation of similar types of user feedback in automatic retrieval systems*

Large capacity random access memory

devices allow the storage of natural language document
titles and abstracts*

On-line low speed terminals andtime-sharing techniques may be used to provide real timeinteraction with many users at once, at several convenient
locations*Two major considerations arise in such an on-linesystem*

In the present batch-processing systemsf such asNASA and Medlars L J immediate response to the user is notnecessary*

In an interactive system the computer time re-quired to process a single query takes on a new importance*
The low imput-output speeds of those terminals appropriate
for interactive applications introduce a second limitation*
For example0 typing out a single document abstract on atypewriter terminal could easily consume more time than the10computer retrieval operation*

An interactive document

retrieval system therefore requires an efficient retrieval
algorithm and a minimum of necessary interactive ityput andoutput,
Several methods of user interaction have been tested
in the SKAKT system using the document collection employedin this study (the •Cranfield 200f collection described inSection IV)• Results of this investigation^ ^are summarized
below*The interactive strategies tested can be divided intopre-search and post-search algorithms*

In pre-searchinteraction, information is presented to the user and a
new query is constructed by him before the search operation
takes place*

The "repeated concepts" algorithm asks theuser to choose one or more of his query terms to be repeated for emphasis*

The "word frequency1' technique dis-

plays for the user the frequencies with which his query
terms occur in the document collection*

The user is theninvited to eliminate or change query terms that are too
comron or too rare to be useful for retrieval.

Both of these

displays help the uninformed user to take advantage of theeffects of word frequency in a retrieval system using frequency-weighted vectors for document classification^The"thesaurus display" supplies synonyms and terms related tothe terms of the initial query from a stored thesaurus
appropriate to the subject area*

The thesaurus used for11this display in reference 6 is the "regular thesaurus"
described in Section I of this report*

Since the same

thesaurus can be incorporated automatically into the SMART
system, manual and automatic thesaurus procedures are compared in reference 6*

The automatic application of thethesaurus to document and query vectors gives better retrieval results than the manual thesaurus displayf except
at low recall levolso

The "source document display" ex-hibits concepts assigned to a relevant document known tothe user before retrieval*

When this display is used inaddition to the automatic thesaurus* results are better
than with automatic thesaurus alone*
Post-search techniques display the partial results ofan initial search operation so that the user can reformulate
his query and request another search*

These algorithmsmay be iterated as often as the user desires*

All post-seerch algorithms share a common disadvantage* the computer
time required for several search operations*

The time iswell spent* however* for all post-search techniques investigated give better retrieval than automatic thesaurus
display*"Title display" which displays the titles of thefirst(in this reference n«5)% documents retrieved byn

the initial queryt provides better retrieval than.thesaurus
display except at high recall*displaysn"Abstract display"f whichfull abstracts, requires more output time andmore time for user thoughtf but gives consistently better

12performance than title display.

A variation of "relevance

feedback111 the technique investigated in this study, gives
retrieval results nearly comparable to abstract display*^:oreoverf this report gives more effective variations of
the relevance feedback algorithm than the version used by
Lesk and Salton*

When pre-search and post-search informa-tion is combined, manual thesaurus display followed by
abstract display gives better retrieval than either method
alone*

Adding word frequency information to the combina-tion is helpful when the null thesaurus is used*
3stimates of the search cost per query show that abstract display% which gives the best overall performanceof the methods tested* is the most expensive*

The otherpost-search algorithms, title display and relevance feedbackt are more costly than any pre-search method*

Relevance

feedback requires the least user effort of any post-searchstrategy*

Lesk and Salton "• * recommend the following

algorithms:a)

For normal users needing high recall* automaticthesaurus followed by automatic relevance feedback*b)

Por highest precision when high recall is not

requiredf word stem matching followed by title display.
c)

For experienced and patient users needing maximum

performancef thesaurus display plus frequency infdrmation
followed by abstract display*•Lesk and Salton use the £} Strategy with N equal to 5*
See Sections VI-C and VI-D for more effective algorithms*

1?
The Leak and Salton study shows that relevance feedback is one of the most effective user interaction techniques*
In relevance feedback9 the user is given a email set ofiteixis retrieved using his original query*

He ia then asked

to judge which items of this set are relevant to his needs*
This information is used to automatically produce a newquery for another search*

This feedback process can be

iterated as often as desired.

Relevance feedback has adefinite psychological advantage over abstract display; theuser is not required to make sophisticated decisions in rephrasing his own query*

Instead* he can supply much infor-

mation to the retrieval system at little effort by snying
in effect "I want documents on the same subject as thisdocument"*

The stored abstract of a chosen relevant docu-ment contains a more detailed description of the subject
than a user would care to type as a query*

In the experi-mental collection, the document vectors have approximately
ten times as xrmy concepts as the query vectors, so the user
submits a ten times more detailed "query" simply by typing
a document identifying number.
The disadvantages of relevnnce feedback should belrfeiffirai^t*

Like abstract display, relevance feedback re-quires the system output of document abstracts or Informationof comparable detail*Alsof multiple searches of the doc-ument collection are made*Designers of retrieval systems

must decide whether the extra output time and computer time
is justified by the retrieval improvements obtained*