Reaction to SIGIR 99 Panel on User Interface Issues
Gene Golovchinsky
FX Palo Alto Laboratory
Palo Alto, CAA panel held at the SIGIR99 conference,reflecting the mandate of the conference tofocus on user interface issues, raised someinteresting questions regarding applying
results of user studies and observations to
the design of information retrieval systems
and interfaces. In part, the panel echoed aproblem well-known to the IR community:
how can we respect the tradition andmethodology of search engine evaluation
and at the same time recognize and adapt tothe emerging role of interactive information
retrieval? The community, it seems, has afew choices:algorithms to the design and evaluation ofinteractive systems, systems that include thepeople using them in their scope.

Retain focus on search engines. Should
SIGIR, as a community, reject the emphasis
on interactive use and focus on theunderlying indexing and retrieval
technology? Surely there are communities
such as CHI and CSCW that can pick up thetechnology and apply their expertise inunderstanding the larger social issued
introduced by wide-scale use to produce
useful and usable tools. Combining SIGIR
expertise in core technology with design
expertise from CHI should in the long run
produce powerful and easy to use systems.

Embrace heterogeneity. We have madesuch progress in our understanding, in ourdesign, and in our ability to manufacture
software that esoteric tools of a few years
ago are commodities today. Continued
innovation requires us to look broader rather
than deeper, to think in terms of systems
rather than components, and to find value insynergy. By systems I mean man-machine
systems rather than computer software; IRsystems include the software, thedocuments, the users, and, importantly, therelationships among them. A great searchalgorithm isn't terribly useful if its interface
isn't designed for its users or if it isn't
sensitive to the peculiarities of thedocuments being retrieved. Similarly, awonderful interface is useless unless it canbe used to retrieve documents that people
need. Thus, an important question for thefield, if it is to remain relevant, is tounderstand how to provide people withsearch engines that are powerful and suited
to their needs and tasks.

Shift focus to user issues. Althoughindexing and retrieval algorithms and theory
continue to evolve, the marginal returns ineffectiveness and efficiency appear to bedeclining. It is certainly premature toannounce the problem solved, but we appear
to be converging on a class of solutions, and
it is unlikely that innovation in algorithms
will result in order-of-magnitudeimprovements. So perhaps the field will
benefit more from a shift in focus from

I would like to argue that the question facing
us is not whether we should change, butrather how can we, as researchers, retain ourindividual expertise (in indexing, inretrieval, in librarianship, in evaluation, etc.)
and yet adopt a heterogeneous approach thatwill produce systems rather thancomponents? Currently we are able todesign and build sophisticated tools, and weare able to assess the needs and behaviors ofreal people, but we are unable to map thoseneeds to concrete system design decisions.
The skills required to study a group ofpeople and to understand how they use
information are very different from the skills
required to design and build a distributed
retrieval system. So different are they that it
is often difficult to find a common language:a paper that describes work practice in areference library may not offer muchspecific advice to a system builder
concerned with query throughput or with theintegration of thesaurus-based query
expansion. Similarly, researchers runninguser studies or experiments may not be able
to take advantage of the algorithms
described in the latest conference
proceedings because they do not haveappropriate system-building skills.
These problems are not new. They aremerely an indication that our field hasreached a level of maturity that requires
some changes in the way we see ourselves.
Other research communities (e.g., CSCW)
have encountered similar divergences
between the researchers who understandpeople and those who understandtechnology. But such differences exist
outside the research world also: for example,
mechanical engineers, electronics engineers,
software engineers and optics engineers
need to work together to build heads-up
displays for airplanes. None of thedisciplines speaks the others' language, buttheir efforts must be coordinated to build thesystem. Thus systems engineers mediate thecommunication among the other groups,
translate requirements from one discipline toanother, and resolve constraints andconflicts that may arise. We need to learn
from such organizations.
If we are to produce useful tools suited to
the way people work, we cannot concentratesolely on the tool building, we cannot
simply delegate the UI work to theamorphous CHI community and hope thatsomeone will get it right. We must create
within our ranks a group of people who canunderstand the needs of the engine builders,of the user interface designers and of theexperimenters and ethnographers, and whoare able to derive systems requirements fromobservations of users, who are able totranslate experimental results to implications
for user interface design, who are able tosuggest avenues of exploration that would
benefit research in search engines and inunderstanding user behavior.
Such people already exist today. Some ofthem build software and others run
experiments, but their roles as integratorsand mediators are not recognized. We needto recognize and to reward such efforts, andwe need to change the way we educate
students to reflect the growing importance ofsuch mediation. This effort does not merely
increase the "greater good" of SIGIR, butalso benefits the individual researchers: forexample, our application of some of thesetechniques at FXPAL has produced a varietyof patents and publications that would nothave been possible had we focused solely on
software creation or on studies andobservation of people.