EVENT REPORT

Report on the 1st Workshop on Information
Retrieval for Understudied Users (IR4U2 2024)
at ECIR 2024
Maria Soledad Pera

Emiliana Murgia

Monica Landoni

TU Delft
Netherlands

University of Genova
Italy

Università della Svizzera Italiana
Switzerland

m.s.pera@tudelft.nl

emilianamurgia@gmail.com

monica.landoni@usi.chFederica Cena

Noemi Mauro

Theo Huibers

University of Turin
Italy

University of Turin
Italy

University of Twente
Netherlands

federica.cena@unito.it

noemi.mauro@unito.it

theo.huibers@twch.nl

Abstract
In this report, we summarise the program and takeaways from the 1st edition of the Workshop
on Information Retrieval for Understudied Users (IR4U2), co-located with the 46th European
Conference on Information Retrieval (ECIR).
Date: 24 March 2024.
Website: https://ir4u2workshop.wixsite.com/ir4u2.

1The Purpose

With the 1st Workshop on Information Retrieval for Understudied Users (IR4U2), which was colocated with the ECIR conference,1 we aimed to raise awareness about ongoing efforts centred
on Information Retrieval (IR) technologies tailored for often overlooked user groups. Additionally, we aspired to establish a community of professionals interested in exploring this topic andincorporating diverse perspectives [Pera et al., 2024].
The main objectives for IR4U2 included: (i) identifying challenges and open issues impacting
this area of research, (2) igniting discussions to identify common frameworks for future research,and (3) enabling cross-fertilisation and community-building by sharing lessons learned from research catering to different audiences by researchers and (industry) practitioners across various
disciplines. In doing so, we hoped to spotlight the various user groups frequently sidelined inmainstream research contributions, enhancing their visibility.
1

https://www.ecir2024.org/

ACM SIGIR Forum

1Vol. 58 No. 1 – June 2024

2The People

Considering the emphasis on IR4U2, we initially worried about drawing the interest of numerous
researchers and (industry) practitioners; especially with the broad range of workshop choices
available at ECIR ’24. However, we were pleasantly surprised to welcome up to 20 attendees.
Furthermore, we were thrilled with the diverse backgrounds of workshop participants, ranging
from early PhD students and researchers to experienced academics and industry professionals.
Their interests and expertise spanned various areas of study, such as natural language processing,information retrieval, conversational search, recommender systems, fairness, ethics, child-computerinteraction, participatory design, and education. This diversity sparked engaging and thoughtprovoking discussions.

3The Program

We envisioned a highly interactive, hands-off workshop. Therefore, we proposed scheduling it as
an in-person-only event. During the first half of our workshop, we focused on knowledge andinspiration, in the form of a keynote address and presentations of accepted contributions. During
the second half, we focused on distilling insights and drafting a vision for the IR4U2 community.

3.1

Keynote Address

We welcomed Dr. Michael Ekstrand, an assistant professor of information science at Drexel
University, as our keynote speaker. His research blends IR, computer interaction, machine learning,
and algorithmic fairness to try to make information access systems, such as recommender systems
and search engines, good for everyone they affect. In 2018, he received the NSF CAREER award
to study how recommender systems respond to biases in input data and experimental protocols
and predict their future response under various technical and sociological conditions and is coPI on the NSF-funded POPROX project to develop infrastructure for user-facing recommendersystems research. Previously he was faculty at Boise State University, where he co-led the People
and Information Research Team, and earned his Ph.D. from the University of Minnesota. He
leads the LensKit open-source software project for enabling high-velocity reproducible researchin recommender systems and co-created the Recommender Systems specialisation on Coursera.
He is currently working to develop and support communities studying fairness and accountability,
within information access through the FATREC and FACTS-IR workshops and the Fair Ranking
track at TREC, and through the ACM FAccT community in various roles.
During his address, entitled “To Serve Whom and How? Provocations on the Political andEthical Foundations of Access to Information Access”, Michael, encouraged us to think about
why enabling broader access to information access technologies and their resulting benefits foracquiring, updating, and disseminating knowledge is crucial for both research and industrial development. More so, he elaborated on what makes this a complex and multifaceted question andurged us to think about What is the ultimate purpose of enabling such access; Which ethical principles may motivate that goal or specific efforts in its pursuit, what theory of change or politicalvision is assumed by such efforts? What is the role of ’the users’ relative to other actors in thesocio-technical milieu in which information retrieval operates? Throughout his discourse, Michael
ACM SIGIR Forum

2Vol. 58 No. 1 – June 2024

urged us to contemplate the significant implications for when and how we work to enable more
universal information access, and how we define and evaluate success in that quest.
The keynote address can be found at: https://md.ekstrandom.net/talks/2024/ir4u2.

3.2Presentations

We had two presentations of accepted contributions; each bringing attention to specific understudied users and IR applications. Note that submitted contributions underwent peer reviews by
at least 2 reviewers, who were experts in the theme of the workshop.
Zheng et al. [2024], presented ‘Inclusive Design Insights from a Preliminary Image-Based Conversational Search Systems Evaluation’. The authors explored an image-based conversationalsearch system alongside text-based and mixed modalities. Although text-based systems reduce
confusion, image-based ones pose interpretation challenges. However, the mixed system achieves
the highest engagement, blending visual and textual information. Outcomes from this work highlighted the potential of these systems for aiding individuals with intellectual disabilities and aimed
to encourage researchers and practitioners to integrate an image-based modality into a mixed system to offer both clarity and engagement.
During their presentation of ‘Vulnerable by Design: Reconsidering User Vulnerability and Recommender Systems,’ Nyhan et al. [2024] discussed the ethical challenges surrounding recommendersystems and the need to safeguard vulnerable users, primarily children and adults with specific attributes. They emphasised in the discussion that vulnerability is not solely due to user attributes
but also stems from the design of recommender systems and interactions with them; identifying
three threads of vulnerability: user attributes, flaws in recommender systems, and interactions
with recommendation algorithms. Consequently, they suggested that while protecting vulnerable
users is crucial, all users could be vulnerable due to issues within recommendation algorithms.
This reevaluation of vulnerability emphasises the need to research the impacts of recommender
algorithms on currently understudied user groups.

3.3

Reflecting Together

To start the interactive discussion, we moved to a World Cafe format [Estacio and Karic, 2016;Löhr et al., 2020], inspired by a similar activity proposed during SIGCHI Futures Summit ’24
to create a CHANGE Manifesto for the wider community.2 In our case, we engaged workshop
attendees in a small-group discussion. We created four groups and four stations–each in a cornerof the room–and asked each to reflect and contribute their thoughts on the prompt assigned totheir station:“To best respond to Understudied Users, in the future, the Information Retrieval (research) community must [be . . . /have . . . / achieve . . . /value. . . ].”After 4 minutes, each group would move to the next station to focus on the next prompt. Here,they would read their colleagues’ words and underline those already shared that also resonatedwith the group; they could also add more words as needed. Each group did so until reaching their2

https://medium.com/sigchi/about-the-sigchi-futures-summit-4e6717a406ce

ACM SIGIR Forum3

Vol. 58 No. 1 – June 2024

original station (the one with the first prompt they interacted with). At this point, each group
was asked to distil the opinions provided, i.e., the contributions recorded by all attendees.

4The Path Forward

Following the small-group discussions, we joined a round table to report on emerging insights.
The need to foster an inherently diverse, multidisciplinary community, one that is open-minded,willing to listen to different (and even contradicting) perspectives and learn from mistakes, resonated with all workshop attendees. They recognised this as a necessary step to continue to growwhile bringing attention to understudied users.
This immediately opened the discussion for how to reach a consensus on the right terminology
and the importance of undertaking participatory methodologies moving forward to advance, notonly the IR4U2 community but the IR community at large. Equally important was the focus
on resources—from European project proposals to datasets and frameworks to enable further
research at scale as well as attracting and supporting incoming researchers into this importantarea of study.
Ultimately, it became evident from the round table discussions that visibility is essential.
Increased visibility not only inspires and attracts researchers to contribute to this area but also
facilitates the practical impact of research outcomes, both on a small scale within specific user
communities and on a larger scale within the field of IR as a whole.
In the end, we collectively came up with the following vision for our IR4U2 community:
To best respond to Understudied Users, in the future, the Information Retrieval(research) community must:• be ambitious and creative• have resources• achieve positive and practical impact• value pluralistic access based on fairness
We aim to continue to discuss open issues, dilemmas, and possibilities related to understudied
users and information access. Encouraged by the attendees’ support, we hope to continue to growthe IR4U2 community. In the short term, hosting other iterations of this workshop at venues like
CHIIR, UMAP, and RecSys to broaden the interest in this topic. In the longer term, we would
like to seek a planning grant such as US’ NSF Emerging Frontiers in Research and Innovation
Program3 to sustain IR4U2 growth, as well as organise a special issue at an IR-related journal toenhance the visibility of IR4U2 among this community.
Archival material can be found on the IR4U2 website. Moreover, you can join the conversationor share your interest in future workshop editions by reaching out to the workshop organisers at:
ir4u2.research@gmail.com
3

https://new.nsf.gov/funding/opportunities/planning-grants-broaden-participation-emerging

ACM SIGIR Forum4Vol. 58 No. 1 – June 2024

References
Emee Vida Estacio and Toni Karic. The world café: An innovative method to facilitate reflections
on internationalisation in higher education. Journal of Further and Higher Education, 40(6):731–745, 2016.
Katharina Löhr, Michael Weinhardt, and Stefan Sieber. The “world café” as a participatory method for collecting qualitative data. International journal of qualitative methods, 19:
1609406920916976, 2020.
Megan Nyhan, Josephine Griffith, Susan Leavy, Qin Ruan, Tai Tan Mai, and Ruihai Dong. Vulnerable by design: Reconsidering user vulnerability and recommender systems. In 1st workshop
on Information Retrieval for Understudied Users (IR4U2), page To appear, 2024.
Maria Soledad Pera, Federica Cena, Theo Huibers, Monica Landoni, Noemi Mauro, and Emiliana
Murgia. 1st workshop on information retrieval for understudied users ((IR4U2). In European
Conference on Information Retrieval, pages 409–414. Springer, 2024.
Yue Zheng, Lei Yu, Junmian Chen, Tianyu Xia, Shan Wang, and Haiming Liu. Iclusive design insights from a preliminary image-based conversational search systems evaluation. In 1st workshop
on Information Retrieval for Understudied Users (IR4U2), page To appear, 2024.

ACM SIGIR Forum5Vol. 58 No. 1 – June 2024