Salton Award lecture: on theoretical argument in information retrieval (summary only): on theoretical argument in information retrieval
Authors: Stephen Robertson
Venue: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval (2000)

The last winner of the Salton Award, Tefko Saracevic, gave an acceptance address at SIGIR in Philadelphia in 1997. Previous winners were William Cooper (1994), Cyril Cleverdon (1991), Karen Sparck Jones (1988) and Gerard Salton himself (1985). In this talk, I plan to follow the tradition of acceptance addresses, and present a personal view of and retrospective on some of the areas in which I work. However, I will not be saying much about what are perhaps the two most obvious parts of my work: the probabilistic approach to retrieval and evaluation of retrieval systems. Rather I will attempt to get under the skin of my take on IR, by discussing the nature of theoretical argument in the field, partly through examples. This talk is about the place of theory in the study of information retrieval (in some sense following Bill Cooper's 1994 topic), but not so much Theory with a capital T &mdash; rather what might be described as small-t theory. The field has a very strong pragmatic orientation, reflected both in the attitudes of the commercial participants and in the emphasis on formal evaluation in the academic environment. Nevertheless, there are many theoretical ideas buried in, or implied by, the ways we talk about the field &mdash; the language we use to discuss it. I will be discussing two areas to illustrate these low-level theoretical ideas: precision devices, and the apparent symmetry between retrieval and filtering. The phrase `precision device' used to have a rather clear meaning in IR, in the days of set-based retrieval systems. In that context, a precision device was a device to enable the restriction of the retrieved set to those most likely (out of the documents originally included) to be useful. These days, with the ubiquitous scoring and ranking methods largely replacing set-based retrieval, the idea has lost its meaning It is worth exploring the formal relationships involved to understand the change a little better. My second area is to do with the relation between filtering and the more traditional type of adhoc information retrieval. There is a tendency and a temptation to see these as the same kind of thing, sometimes with a more specific assumption of duality, based on the inversion of the roles of documents and queries. It is important to see how far this parallel extends, and where it breaks down. I explore the nature of the duality and the kinds of reasons why it does break down. These examples reflect my interest in the basic logical structure of information retrieval systems and the situations in which such systems may be found. I argue for a certain level of logical argument in information retrieval, which might be taken as small-t theory, though not as capital-T Theory. I believe there are reasons to think a Grand Theory of IR to be an unattainable goal &mdash; such a theory would have to encompass so many different aspects of retrieval, having to do for example with human cognition and behaviour and the structure of knowledge, as well as with the statistical concepts that inform the probabilistic approach. However, accepting the unattainability of a Grand Theory does not preclude the development of further and more useful models based on particular aspects and lower-level logic The low-level logic is important not only in its own right, but as the basis for linking together more sophisticated theories concerned with more restricted domains. The most elaborate and complete theory of (say) user behaviour is of no use at all without a strong linkage between the parts of that theory and the entities relevant to IR that fall outside its scope. The glue that provides that linkage has to be low-level logic.