SIGIR WORKSHOP REPORTWorkshop on Web Information Seeking and Interaction
Kerry Rodden
Ian Ruthven
Ryen W. WhiteGoogle
University of Strathclyde
Microsoft Research
Mountain View, CA 94043, USA Glasgow, G1 1XH, UK Redmond, WA 98052, USA
kerryr@google.com
ir@cis.strath.ac.uk
ryenw@microsoft.com

1Introduction

The World Wide Web has provided access to a diverse range of information sources and systems.
People engaging with this rich network of information may need to interact with different
technologies, interfaces, and information providers in the course of a single search task. Thesesystems may offer different interaction affordances and require users to adapt their informationseeking strategies. Not only is this challenging for users, but it also presents challenges for thedesigners of interactive systems, who need to make their own system useful and usable to broad usergroups. The popularity of Web browsing and Web search engines has given rise to distinct forms of
information-seeking behaviour, and new interaction styles, but we do not yet fully understand theseor their implications for the development of new systems.
Web Information Seeking and Interaction (WISI) (i.e., the interaction of users with Web-based
content and applications during information-seeking activities) is a topic that unites many strands ofacademic and commercial research, from studies of information-seeking behaviour to the design andconstruction of large-scale interactive systems. Designing components to support this interaction(and evaluating these components) is particularly challenging given the scale of the Web, thediversity of the user population, the diversity in tasks being undertaken, and the dynamic nature of
the information. This workshop acted as a focal point for researchers and practitioners whose work is
related to Web information seeking and interaction, to enable them to share experiences andcollaborate.
Papers submitted to the workshop were reviewed by an international programme committee and 16
short papers and position papers were accepted for the workshop proceedings. These papers reflect
the diversity of interest and contributions to the area of Web information seeking and interaction,
describing research on evaluation, interaction modelling, information behaviour, personalisation,
document surrogates, and novel interface development, amongst other topics. The proceedings areavailable at http://research.microsoft.com/~ryenw/proceedings/WISI2007.pdf.

2PanelThe workshop began with a panel on “Challenges and Opportunities in Supporting Web SearchInteraction”. We were fortunate to have three interesting speakers: Peter Anick (Yahoo!), Diane
Kelly (University of North Carolina), and Maria Stone (Google). What follows is our personal
interpretation of the panel session and discussion.
ACM SIGIR Forum63

Vol. 41 No. 2 December 2007

Maria Stone has led the user experience research team at Google for six years. Maria covered
usability and evaluation issues from a Web search engine perspective. Web search engine developersand evaluators face similar problems to other practitioners with a generally unknown, and diverse
user population, and an extremely diverse set of data. Data quality itself is the biggest issue for searchengine developers. In her panel address Maria started off by reminding us that many Google users are
actually infrequent users who, although they may have complex information needs, interact in very
simple ways with the tools available. Major challenges are, therefore, understanding what users wantwhen searching and also how to best to support their searching through new search features. A
separate, related, challenge is how to provide these features in a way that does not reduce theefficiency of current Web search tools.
Peter Anick presented three main challenges to developing new methods of interaction:
discoverability, content, and learnability. He presented a fascinating characterisation of typical searchengine users, characterising them as generally:•••

Risk averse - do not use interface features they do not understandCognitive misers - do not want to think deeply when specifying queries
Having tunnel vision - focus specifically on a portion of the results page near thefirst result•Reluctant to use help offered - unwilling to utilise textual descriptions of interface
featuresThis should not be seen as a negative characterisation, but rather a set of generalisations about
typical interactions with a search engine. Such behaviour raises interesting challenges, including how
to empower users to have more control and input into their interaction, how to cope with different
types of searches (such as exploratory search), and what tools are appropriate for investigation ofbehaviour and evaluation. Peter also gave an interesting demo on the some of the to-be-releasedinteraction features from Yahoo’s “search assist” tool which led to important discussion on how toencourage searchers to provide more information to the search system.
Diane Kelly then turned the discussion to the fundamentals of Web information seeking andinteraction by presenting a series of questions about the direction and nature of the field. In the WISI
area, for example, what kind of activities are we (and should we) be engaged in? Are we, forexample, really trying to learn about users and their needs or just treating users as black boxes andexamining only their output? Are any lessons we learn from user studies generalisable into solid
principles that can be applied in different situations? A core question posed is whether the field isonly an applied science - or whether it is even a science at all. Certainly the area of search systems
itself has many applications, but what about the methodologies and models we develop – are thesegood science? These questions have many implications for the community, how it develops, and how
it evaluates its own progress.

3

Breakout sessions

The panel was followed by breakout sessions where participants were asked to form small groups andpresent solutions to problems set by the panelists and organisers. Each group was asked to discuss
the problem and report.
Two main issues arose from the discussions: (i) tasks and (ii) improving the search experience.
ACM SIGIR Forum

64

Vol. 41 No. 2 December 2007

Tasks: Many search engines perform well for some types of tasks. For example, Google is generally
excellent for fact-finding searches. However other types of task require increased support. Knowing
more about which tasks require such support and what type of support would be useful was a central
issue across the discussions.
Improving the search experience: If we are to improve search engine effectiveness across a range oftasks, how might this be achieved in practice? Are better systems or better interfaces more fruitful
areas of investigation?
Tackling the overall search experience was an important issue. One group, whose conclusions werepresented by Mark Smucker of UMass, identified a spectrum of possible solutions, from manuallyled approaches (based on improved information literacy and teaching) through to automatic, systembased, approaches (based on more intelligent systems). We coarsely present this useful spectrum
below.

ManualTeaching increased searchand information literacy

Focussed teaching- System help
- Feedback on query- Expectation management- Expose system model

Automatic- Improved system adaptation- Task modelling

One implication of this spectrum is that not all advances in search engine effectiveness need be thosearising from technical developments. That is, education and awareness of existing tools can be
crucial to successful searching.

4Presented papers

The first afternoon session consisted of short paper presentations. We selected 4 papers from those
appearing in the proceedings that reflected common themes within the papers submitted for theworkshop.
Shady Elbassuoni presented his paper Adaptive Personalization of Web Search, co-authored with
Julia Luxenburger and Gerhard Weikum, which proposed a system architecture for adapting search
results to a user’s search tasks. This client-side model captures user interactions, specifically HTTP
traffic, so that the personalisation system can capture user queries, result clicks and page visits.
Information on previous searches is used to predict what type of search a user is carrying out and reorder search results to adapt to that type of search. A repeat query, for example, is seen an attempt tore-find previously seen information and are prioritised in the results list. A comparison with a nonadaptive search engine shows promising results.
Kerry Rodden presented Exploring How Mouse Movements Relate to Eye Movements on Web Search
Results Pages, co-authored with Xin Fu. The study presented in this paper attempted to correlatemouse movements (as opposed to simple mouse clicks) with eye-tracking data. In this exploratory
study eye movements and mouse movements did show a reasonably good correlation. The study also
showed interesting behaviours with the mouse. For example a small number of participants used themouse pointer as a reading aid, moving the mouse across the text they were reading. OtherACM SIGIR Forum

65

Vol. 41 No. 2 December 2007

participants left the mouse pointer at an interesting result in order to ‘mark’ the result whilst theyscanned other results. Work such as this open up new ways of measuring interactive searchbehaviour and giving new insights into online searching.
Elaine Toms presented Evaluating Engagement in Interactive Search co-authored with Heather
O’Brien in which they propose user engagement as a new metric for evaluating interactive searchsystems. Engagement here represents a higher-order construct that expresses the level of positive
involvement in a task or process. Engagement is a compound construct and Toms and O’Brien
discuss the methodological issues involved in estimating such a complex, but useful, metric both interms of measuring the outcome – how engaged were users in a search – and the process of searching– how does engagement change and manifest itself during a period of interaction?
The final paper titled "Clickthrough-based Measures of Search Engine Performance" which was coauthored by Craig Macdonald, and Iadh Ounis was presented by Erik Graf. Within this paper theyexamined an important issue in interactive information seeking and search, namely how to modelinteractive search behaviour. They present a framework for modelling interactive search behaviour
based on a taxonomy that arranges loggable interaction features such as the query, user ID, and query
submission timestamp into user related, request related, or result related groups. This framework,instantiated within an intranet retrieval system, was used to test various hypotheses about userinteraction. One of which was for example whether clickthrough behaviour can be used to infer thesuccess of search sessions? A particular example presented to outline the potential of their approachlay in the successful identification of a period of system failure via analysis of interaction patterns.

5Discussion

Our workshop ended with a plenary discussion on the themes arising from the discussions throughout
the event, the background papers, and breakout sessions. Here we summarise some of these themes
and the discussions arising.
• How to get more from users about what they want and how do we obtain this information in a
way that that involves the least intrusion? Web search engines have provided simple models ofinteraction and users have learnt a particular model of interaction. Going beyond this, by asking
users to provide more information themselves, may be difficult unless we can, at the same time,
make clear the benefits of providing more information.
• Diane Kelly, in her panel address, raised the question of what type of science(s) we are engagedin. One point raised in response was that our field has an increasingly heavy industrial
component and both sides of the academic/industrial fence need guidance on what the other side
wants. From the audience, the academics were clear that they wanted data whereas practioners
wanted general, coherent principles of interaction.
• What concepts are (most) useful for investigation? A recurring concept was that of task, reflecting
the increased interest in this concept within the community. This led into a discussion of thegranularity of investigation appropriate for learning about searchers and search behaviour of theindividual characteristics that are important in search interaction such as age, information literacy,and information poverty. The issue of information literacy led back to a discussion on theeducation of users – most Web users have little training in online searching – and the possible
dangers of users passively accepting search results.
• The issue of success and user satisfaction was another recurring theme throughout the day.
Participants from a background in more traditional system-based evaluation were keen for aACM SIGIR Forum66

Vol. 41 No. 2 December 2007

dialogue with interactive practitioners and for user models and tasks that can be used within testcollection or system-based evaluations. A concern was raised that the core metric of relevance isbeing stretched too far and we need additional measures to discuss interactive searching. The
work of Toms and O’Brien was signaled as a way of defining new metrics from careful userstudies. In addition, we should not isolate online searching to search engines alone, but also
include other sources of information with different models, such as those used by Amazon.com.
There was also recognition that the process of searching is as important as the end-product ofsearching.
Throughout the workshop the discussion demonstrated that Web Information Seeking andInteraction is both a complex area and one that requires input from many sources. We are,therefore, grateful for the input from our panelists, attendees, and contributors to the proceedings
who gave us a lively day of discussion and insights from different parts of the community.

ACM SIGIR Forum67

Vol. 41 No. 2 December 2007