Intelligent Information Retrieval: Whose Intelligence?

The concept of 'intelligent' information retrieval was first mooted in the late 1970s, but had lost currency within the information retrieval community by at least the early 1990s. With the popularity of the concept of 'intelligent agents', it appears that the idea of intelligent information retrieval is again in general vogue. In this paper, I attempt to show that the naive concept of intelligent information retrieval, based on the idea of agency, misses the essence of intelligence in the information retrieval system, and will inevitably lead to dysfunctional information retrieval. As a counter-proposal, I suggest that true intelligence in information retrieval resides in appropriate allocation of responsibility amongst all the actors in the information retrieval system, and that intelligent information retrieval will be achieved through effective support of people in their various interactions with information. 1. What could we mean by intelligent information retrieval? Intelligent information retrieval (IR) has been variously defined by different people, but a consistent theme has been one of the machine (or program) doing something for the user, or the machine (or program) taking over some functions that previously had to be performed by humans (either user or intermediary). So, for instance, for Belkin et al. 87, an intelligent IR system was one in which the functions of the human intermediary were performed by a program, interacting with the human user. For Maes 94, on the other hand, intelligent IR is performed by a computer program (a so-called intelligent agent), which, acting on (perhaps minimal or even no explicit) instructions from a human user, retrieves and presents information to the user without any other interaction. Croft 87, introducing the Information Processing and Management Special Issue on Artificial Intelligence and Information Retrieval, suggests that intelligent IR is just good IR, meaning that it is inappropriate to ascribe intelligence to computer programs, and also meaning that good IR is that in which the programs (i.e. the representation, comparison and interaction methods implemented in the system) result in effective performance. In all of these constructions, there is some idea that the intelligence (or goodness) of an intelligent IR system resides in the built system. Indeed, it is assumed by almost all people who have commented on intelligent IR, that the IR system is only the built system. In this paper, following Belkin et al. 83 and Belkin 93, I will argue that this assumption is, in itself, erroneous, that it has led to an inappropriate conception of what constitutes IR, and in particular, for * Currently Fulbright Fellow, Department of Information Studies, University of Tampere, P.O. Box 607, FIN3101 Tampere, tel: +358 31 215 7039, email lonibe@kielo.uta. fi Intelligent Information Retrieval 2 our purposes, has misconstrued the nature of intelligence in the IR system. Furthermore, following Bates 90, Belkin and Vickery 85, and Ingwersen 92 I will suggest both that interaction with information is the key to understanding intelligence in IR, and also that a fundamental problem for intelligent IR is how much, and what kind of support to offer users for such interaction. 2. What constitutes an information retrieval system? IR systems are often construed as some collection of components and processes which take input, as a query to the system and as texts or information collected by the system, represent those inputs, compare them, and produce, as output, some set of texts or information objects predicted to be responsive to the query. I, and many others, have argued that it is a mistake to consider the IR system as only the built system. For extended argument on this issue, see Belkin 84. The crux of the issue is that, in order properly to consider all that happens in IR, and in order properly to evaluate the performance of IR systems, the boundary of the IR system must be outside the user, including the user within the IR system. From this point of view, the IR system consists of three major components: the user in the system; the knowledge resource to which the user has access and with which s/he interacts; and, some person(s) and/or device(s) which supports and mediates the user's interactions with the knowledge resource (the intermediary). The processes which are then considered to be significant within most conceptions of the IR system are: representation (of user’s information problem, of texts in the knowledge resource: e.g. indexing); comparison (of representations of information problem and texts: e.g. retrieval techniques); interaction (between user and intermediary: e.g. reference interview or human-computer interaction); and, sometimes, judgment t (of appropriateness of text to information problem, by the user: e.g. relevance judgments); and modification (of representation of information problem: e.g. relevance feedback or query reformulation). Belkin 84, Belkin et al. 83 and Ingwersen 92, for instance, have elaborated this general model of the IR system in several ways, primarily through being more explicit and detailed about the components, by extending the range of the different processes, by specifying how the processes are carried out, and especially by considering the nature of interaction in the IR system in much more detail. In particular, Ingwersen and Wormell 86 have suggested that it is appropriate to consider the direct interaction between user and text as a separate form of interaction, and this idea has been taken up in a more fundamental sense by Belkin 93, who suggests that IR should be considered explicitly as a form of interaction with information. But in general, such elaborations have tended merely to extend somewhat the range of the IR system, without significantly changing the concept of the IR system itself. Thus, under this type of model, we understand the IR system to be composed of the components of user, intermediary and knowledge resource, related to one another by the processes of representation, comparison, interaction, judgment and modification. The significant point in this model, for Intelligent Information Retrieval 3 our purposes here, is that interaction, judgment and modification are inherent aspects of the IR system, and that the user is an inherent component of that system, not just some entity outside it, giving input and evaluating output. 3. Where could intelligence be manifest in an information retrieval system? The inclusion of the user in the IR system, and the incorporation of interaction as a major process in IR, have some significant implications for how we might consider what would constitute intelligence in an IR system. For instance, under this view, the idea of the ‘intelligent agent’ seems untenable, at least in its most straightforward sense. That is, a program which takes a query as input, and returns documents as output, without affording the opportunity for judgment, modification and especially interaction with text, or with the program, is one which would not qualify as an IR system at all. In particular, such a program would fail to know about the user’s information problem (relying only upon the query, some poor representation of that problem), and would fail to incorporate that one process which is known to improve retrieval performance significantly, interaction (especially, but not exclusively, through relevance feedback). So, although we might say that the representation and comparison processes might be performed well, and even ‘intelligently’, the system as a whole would not perform intelligently (if by that, we mean well, or effectively). Another point which this view of the IR system raises is that there are some processes in the IR system which cannot be performed by any other component than the user. In particular, interaction is a joint process of user with the other components (also of the other components with one another), and judgment is a process which can only be performed by the user. Furthermore, although modification is something that can be done by the other components of the system with reference to modifying query or text representation, modification of understanding of the information problem is something that can realistically be done only by the user. Thus, the idea of the ‘intelligent intermediary’ as being the basis of intelligent IR, although perhaps necessary, is not sufficient to characterize the complete intelligent IR system. Similarly, the idea of good IR as being effective IR fails if all the intelligence is concentrated in only the built system, since it thereby excludes the most significant aspect of effectiveness, the user’s judgment of the comparison performance. These arguments lead us to the position that intelligence in intelligent IR perhaps resides not only in that which is built to support the user, but also in the user her/himself, and in particular in proper assignment of roles and responsibilities to all of the various components of the IR system. Clearly, some of the processes we have mentioned must be carried out by the components in concert, and some knowledge on which they are based (for instance, knowledge of database structure and contents) might be privileged to only one of the built components. Thus, there is some sense in which we might say that doing these processes (or applying such knowledge) well is important to intelligent IR. However, this view does not incorporate the undoubted intelligence of the user, in particular in respect of the judgment, modification and interaction processes. Thus, we might say that a role of the user is to judge or evaluate the texts with respect to the information problem; a role of the intermediary is to offer to the user interesting texts with which to interact; and a role of the knowledge resource is to be organized in such a way as to promote effective interaction. These are only examples of the roles and res