WORKSHOPSSIGIR 2009 Workshop Program Overview
Diane Kelly, Workshop Chair
University of North Carolina at Chapel Hilldianek@email.unc.edu
SIGIR workshops provide a platform for presenting novel ideas in a less formal and more focused way
than the main conference. The SIGIR Workshop Program is a venue where discussion, collaborationand the sharing of ideas can take place. In the 2009 call for workshop proposals, workshops that fosteredcollaboration, discussion, and group problem-solving were encouraged.
SIGIR Workshops have spanned a range of topics over the years and this year was no exception. Topics
of past workshops include those that represent novel IR research areas and areas from allied fields thathave drifted into the IR community. Past workshops have also addressed new (or ongoing) challenges
and research problems in the community. Many workshops have developed into successful series andhave appeared at consecutive SIGIR Conferences or rotated among various related conferences to draw
more diverse participation. This year’s program covered a range of topics including evaluation,
multilingual retrieval, search in social media, ranking, and advertising. The program contained a mix of
new workshops, as well as follow-ups from successful past workshops. Organizers were fromacademia, industry and government and a range of geographic locations.
We received many excellent submissions and had a difficult decision of selecting a smaller set for theprogram. Fourteen workshops proposals were submitted and eight were selected for inclusion in theprogram. (For comparison, 15 were submitted last year and 9 were held.) Table 1 displays the titles andorganizers of the workshops. Workshops are described in more detail by the organizers in the pages that
follow this overview. In total, 290 people registered for a workshop. Individual workshop sizes ranged
from 16 to 56, with an average size of 36.
Table 1. 2009 SIGIR Workshops (ordered alphabetically by title)
Title
Future of IR Evaluation

Organizers
Jaap Kamps, Shlomo Geva, Carol Peters, Tetsuya
Sakai, Andrew Trotman, Ellen Voorhees
Fredric Gey, Noriko Kando, Jussi Karlgren

Information Access in a Multilingual World:
Transitioning from Research to Real-World
Applications
Information Retrieval and Advertising

Misha Bilenko, Evgeniy Gabrilovich, Matthew
Richardson, Yi ZhangClaudio Lucchese, Gleb Skobeltsyn, Wai Gen Yee

Large-Scale Distributed Systems for Information
Retrieval (LSDS-IR)
Learning to Rank for Information Retrieval
Redundancy, Diversity and Interdependent Document
Relevance
Search in Social Media
Understanding the User: Logging and Interpreting
User Interactions in Information Search and Retrieval

ACM SIGIR Forum

Hang Li, Tie-Yan Liu, ChengXiang Zhai
Filip Radlinski, Paul N. Bennett, Ben Carterette,
Thorsten Joachims
Eugene Agichtein, Marti Hearst, Ian Soboroff
Nicholas J. Belkin, Ralf Bierig, Georg Buscher, Ludger
van Elst, Jacek Gwizdka, Joemon Jose, Jaime Teevan

10Vol. 43 No. 2 December 2009

Schedule
One important difference in the way things were conducted this year was that the submission deadline
for workshops was moved forward and notifications for workshops were circulated before notifications
for papers and posters for the main conference. The primary reason for this was to allow workshop
organizers a longer period of time to solicit and review submissions. The schedule was as follows (alldates in 2009 unless otherwise specified):
December 22, 2008
February 02
February 03-22
February 23-25
February 27
March 01
March 05July 23

Initial call for workshop proposals
Workshop proposals dueWorkshop review period
Discussion with workshop reviewers
Report and recommendations sent to conference and program chairs
Report and recommendations sent to senior program committee
Notifications sent to workshop organizers
SIGIR Workshops

Program Committee
The workshop Program Committee (PC) is listed below. The goal in assembling the PC was to include
a mix of expertise, as well as senior and junior researchers. Efforts were also made to include people
from a broad geographic range and from industry and academia.
Peter Anick, Yahoo! Labs, USA (Last Year’s SIGIR Workshop Co-Chair)
Pia Borlund, Royal School of Library and Information Science, Denmark
Charlie Clarke, University of Waterloo, Canada
David A. Evans, JustSystems Evans Research, USA
Jaana Kekäläinen, University of Tampere, Finland
Falk Scholer, RMIT University, Australia
Alan Smeaton, Dublin City University, Ireland
John Tait, Information Retrieval Facility, Austria
Ryen W. White, Microsoft Research, USA
Feedback from Participants
An online workshop feedback questionnaire was created and made available to organizers andparticipants on the day of the workshop. Workshop organizers were asked to post the URL of the form
at the end of the workshop. Workshop attendees were also emailed several days following the workshop
with another invitation to provide feedback.
In total, 92 people provided feedback about the workshops. Overall, there was a 32% response rate andfor most individual workshops the response rate was around 31%. The specific results are not presented
in this paper, as they were primarily used for internal purposes. However, a brief, qualitative summaryof the feedback is provided.
In general, most respondents indicated that the content of the workshop was what they expected and thatthey enjoyed the workshops. Many respondents indicated that they would attend a workshop on thesame topic next year. Respondents were generally satisfied with the amount of discussion that took

ACM SIGIR Forum

11Vol. 43 No. 2 December 2009

place in the workshop and about 65% of them indicated that they would pursue the research themes
discussed in the workshop during the upcoming year. Most respondents stated that they enjoyed theinteractivity of the workshops and the amount of discussion that took place. Many respondents alsocommented on the high quality keynotes and presentations. Many others indicated that they liked themix of people from academia and industry, and the opportunity to chat informally with others. Thesecomments were observed for almost all of the workshops, so it seems that all the organizers did a greatjob designing a workshop format that supported interaction and discussion amongst participants.
Respondents were asked what they liked least about the workshops. About half of the respondents
provided a response to this question. One set of comments was about the general environment of theworkshops. Comments about the content of the workshops were mostly about the formation andfacilitation of discussion groups. Some felt that these groups could have been better formed in order toprovide some balance in their composition and content. Others noted topic drift. Some observed that a
lot of people left throughout the day which they believed impacted the quality of discussions thatoccurred during latter portions of the workshops.
Summary
Overall, the SIGIR 2009 Workshop Program was a success. We received many excellent submissions
and the workshops that were selected were well-attended and received positive reviews from attendees.
A change in schedule gave organizers a greater length of time to solicit and review submissions and theindividual workshop programs, which will be discussed in detail in the following pages, were varied andof high quality.
We look forward to receiving your workshop proposal next year and seeing you at a 2010 SIGIR
workshop!
Acknowledgment
I wish to thank the conference and workshops program committee, to all those who submitted proposals
and organized workshops, to those who participated in the workshops, and to the staff of SIGIR 2009.

ACM SIGIR Forum

12Vol. 43 No. 2 December 2009