EVENT REPORT

Report on the 4th Workshop on Reducing
Online Misinformation through Credible
Information Retrieval (ROMCIR 2024)
at ECIR 2024
Marinella Petrocchi

Marco Viviani

IIT-CNR, IMT School for Advanced Studies
Italy

University of Milano-Bicocca, DISCo
Italy

marinella.petrocchi@iit.cnr.it

marco.viviani@unimib.it

Abstract
The 4th Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2024) was part of the Satellite Events of the 46th European Conferenceon Information Retrieval (ECIR 2024). ROMCIR served as a platform for discussions onaccessing accurate information and addressing the issue of information disorder prevalent in
the online landscape at that time. In general, the challenge is multifaceted, encompassing
various types of information sources (e.g., websites, social media posts, ...) across different
platforms and domains (e.g., fake news detection, health-related information retrieval, propaganda reduction). Additionally, in this edition, the critical need to assess the impact ofgenerative models like Large Language Models (LLMs) on inadvertently amplifying misinformation and explore their potential role in supporting Information Retrieval Systems (IRSs)
began to emerge. In this context, diverse approaches to the problem of access to truthful
information found their place. Keynote speech and articles in this year’s workshop mainly
focused on themes such as health misinformation, multimedia and multimodal fact-checking,and information filtering to combat misinformation.
Date: 24 March 2024.
Website: https://romcir.disco.unimib.it/.

1

Introduction

In the July 1983 issue of The New York Times, American historian Daniel J. Boorstin remarked
on the computerization of libraries as follows:
Technology is so much fun but we can drown in our technology. The fog of information
can drive out knowledge.
Approximately four decades later, its relevance may arguably surpass its initial significance.
Indeed, the advent of Web 2.0 technologies has ushered in a process of disintermediation in the
ACM SIGIR Forum

1

Vol. 58 No. 1 – June 2024

creation and dissemination of online content within the Social Web, resulting in well-documented
challenges such as information overload [Bawden et al., 1999; Khaleel et al., 2020] and the proliferation of misinformation [Chen et al., 2023]. These issues hamper users’ ability to access truly
valuable information for their needs [Pasi and Viviani, 2020; Viviani and Pasi, 2017]. Furthermore,
recent advances in generative models such as Large Language Models (LLMs) pose a new threat,as they can generate text that mimics human writing but may lack accuracy and truthfulness[Kojima et al., 2022; Monteith et al., 2023; Xu et al., 2023].
Therefore, the ROMCIR Workshop focuses on studying and developing Information Retrieval(IR) solutions aimed at providing users with access to relevant and truthful information, while also
addressing the phenomenon of information disorder across various domains. Information disorderencompasses a spectrum of issues, from unintentional misinformation rooted in ignorance or bias
to deliberate dissemination of false content, both manually and through automated means [Wardle
and Derakhshan, 2017; Pratelli et al., 2024]. This challenge is exacerbated by filter bubbles andecho chambers prevalent in the digital ecosystem [Bozdag and Van Den Hoven, 2015; Del Vicario
et al., 2016; Villa et al., 2021; Pratelli et al., 2023; Mattei et al., 2022].
The resolution of the information disorder issue is inherently complex, involving various typesof content, web platforms, and user objectives. Furthermore, emerging AI-related concerns suchas the explainability of search results [Anand et al., 2023; Upadhyay et al., 2023], assessmentof truthfulness in user-generated content [Viviani and Pasi, 2017; Soprano et al., 2021], and theuse of generative models to support IR systems require attention [Cabitza et al., 2022; Najork,2023]. In addition, ensuring data confidentiality, especially in unstructured data, is paramount[Livraga and Viviani, 2019; Livraga et al., 2023]. In this context, the development of appropriate
experimental evaluation paradigms for IR systems is crucial [Lioma et al., 2017; Suominen et al.,2021].

2

Aim and Topics of Interest

Within the ECIR conference, the key goal of the Workshop was to explore the challenges surrounding online information disorder within the realm of Information Retrieval, while also delving
into associated domains of Artificial Intelligence such as Natural Language Processing (NLP), Natural Language Understanding (NLU), Computer Vision, Machine Learning, and Deep Learning.
Therefore, the focal points of interest for ROMCIR 2024 encompassed, yet are not confined to: Artificial Intelligence and information truthfulness assessment.
 Bot/spam/troll detection.
 Computational fact-checking/truthfulness assessment.
 Crowdsourcing for information truthfulness assessment.
 Disinformation/misinformation and bias detection.
 Generative models and information truthfulness assessment.
 Harassment/bullying/hate speech detection.
 Information polarization in online communities, echo chambers.
 Propaganda identification/analysis.
 Retrieval and evaluation of truthful information.
 Security, privacy, and information truthfulness.

ACM SIGIR Forum2

Vol. 58 No. 1 – June 2024 Sentiment/emotional analysis and stance detection.
 Societal reaction to misinformation.
 Trust, reputation, and misinformation.

3

Keynote Speech

As part of the Workshop, a Keynote Speech was given by Prof. David E. Losada, on the current
open issues that concern health misinformation detection:
 Title. Health Misinformation Detection: Search Challenges, Annotation Issues and Reliability of LLMs.
 Abstract. In this presentation, we will share insights from our work at CiTIUS (University of Santiago de Compostela, Spain) on the development of technological and scientificsolutions for detecting health misinformation. We will delve into the complexities of developing a multi-faceted retrieval system for misinformation detection that integrates multiple
content-based features. The challenges of creating robust credibility benchmarks, given thesubjective nature of credibility, will also be discussed. Lastly, we will share our recent efforts
to evaluate the quality of LLMs’ responses to health-related queries.
 Short Bio. David E. Losada1 is a full professor in Computer Science and Artificial Intelligence at the University of Santiago de Compostela (Spain). He received his BS in Computer
Science in 1997 and his PhD in Computer Science in 2001, both of them with honors, fromthe University of A Coruña (Spain). From 2001 to 2002, he was a Lecturer in the San
Pablo-CEU University (Spain) and in 2003 joined the University of Santiago de Compostela
as a senior research fellow (“Ramón y Cajal” R&D programme). His current research interests include a wide range of Information Retrieval (IR) and related areas such as: IRprobabilistic models, summarization, novelty detection, sentence retrieval, patent search,and opinion mining. He is an active member of the IR community. He participated in the
Programme Committee of prestigious international conferences such as SIGIR or ECIR. He
also led several R&D projects and contracts in the area of search technologies. In 2011 he
was recognized with an ACM senior member award.

4

Submissions

The ROMCIR 2024 workshop received 14 submissions, of which 6 were accepted, resulting in an
acceptance rate of approximately 43%. The submissions came from five different countries, including Germany (2), Italy (1), The Netherlands (1), Poland (1), and Spain (1). This year, submissions
have particularly focused on the issues of: (i) health misinformation, (ii) fact-checking (including multimedia and multimodal approaches), and (iii) information filtering and misinformation[Petrocchi and Viviani, 2024].
Concerning research issue (i), Fernández-Pichel et al. [2024] presented a work focused on assessing the credibility of information in the context of health information seeking. In the study,1

https://citius.gal/team/david-enrique-losada-carril/

ACM SIGIR Forum

3

Vol. 58 No. 1 – June 2024

the authors acknowledge the subjectivity and bias susceptibility of this process and emphasize
the importance of defining robust guidelines for credibility assessment. Through a study involving 1,000 participants, they demonstrate a correlation between participants’ judgments and thereference values established following such guidelines. Further data analyses reveal concerning
insights into people’s ability to evaluate the credibility of online medical content, posing the riskof personal harm.
Concerning research issue (ii), Mongelli et al. [2024] presented a work focused on enhancing
deepfake detection by simultaneously analyzing audio and visual cues, proposing the Convolutional
Multimodal deepfake detection model (CMDD). This approach improves detection accuracy byleveraging the power of Convolutional Neural Networks (CNNs) to extract spatial and temporalfeatures concurrently. Frick and Steinebach [2024] presented a work addressing the challenge of
combating false information on social media by proposing a method to assess the checkworthinessof tweets. Their approach incorporates analysis of image content, captions, and text obtained fromoptical character recognition to outperform existing recognition techniques. The work presented
by Vogel et al. [2024] focuses on detecting check-worthy statements to prioritize claims for factchecking. They propose an adapter fusion model combining task and Named Entity Recognition(NER) adapters, achieving state-of-the-art results in checkworthiness benchmarks.
In relation to research issue (iii), the work presented by Hornig et al. [2024] delve into theissue of propagation of misinformation in the domain of video recommendation. They evaluate
a range of top-n recommendation algorithms to assess their effectiveness in minimizing misinformation while optimizing overall performance. Their empirical exploration highlights the potentialof certain algorithms, including neighborhood-based, neural, and advanced collaborative filtering
approaches, in combating misinformation and promoting responsible recommender systems. Finally, the work by Hasimi and Poniszewska-Marańda [2024] focuses on the broader implications offake news and disinformation on human rights, particularly freedom of speech. They explore theuse of Artificial Intelligence (AI) in detecting and filtering disinformation, highlighting the risks
to freedom of expression posed by censorship and the suppression of critical thinking.

5

Past Editions

The first three editions of the ROMCIR Workshop, all co-located with the ECIR conference, led tofervent discussion and presentation of innovative work concerning a variety of open issues related
to information disorder and IR. The first edition took place in online mode on April 1, 2021. The
second edition took place both in presence in Stavanger, Norway, and online, on April 10, 2022.
The third edition took place in presence in Dublin, Ireland, on April 2, 2023. The papers accepted
for the first three editions of ROMCIR are collected in CEUR Proceedings [Saracco and Viviani,
2021; Petrocchi and Viviani, 2022, 2023], which are freely accessible. Updated information on
past and current ROMCIR editions can be found on the official website.22

https://romcir.disco.unimib.it/

ACM SIGIR Forum

4

Vol. 58 No. 1 – June 2024

6

Workshop Organization

The ROMCIR 2022 Organizing Team was composed of the following people with respect to their
distinct roles.

6.1

Co-chairs

 Marinella Petrocchi.3 She is a Senior Researcher at the Institute of Informatics andTelematics of the National Research Council (IIT-CNR) in Pisa, Italy, under the Trust,
Security, and Privacy research unit. She also collaborates with the Sysma unit at IMT
School for Advanced Studies, in Lucca, Italy. Her field of research lies between Cybersecurity,
Artificial Intelligence, and Data Science. Specifically, she studies novel techniques for online
fake news/fake accounts detection and automated methods to rank the reputability of online
news media. She is the author of several international publications on these themes and she
usually gives talks and lectures on the topic. She is CNR lead and WP leader of Humane:4
Holistic sUpports to inforMAtioN disordErs, sub-project of SERICS (PE00000014), NRRP
MUR program funded by the EU - NGEU.
 Marco Viviani.5 He is an Associate Professor at the Department of Informatics Systems,
and Communication of the University of Milano-Bicocca, Italy. He received his M.Sc. and
Ph.D. in Computer Science from the University of Milan (La Statale), Italy. He was latera postdoctoral fellow at both Italian (University of Insubria) and foreign institutions (University of Burgundy and INSA Lyon, France). He is involved in organizing several research
initiatives at the international level. He was the General Co-chair of MDAI 2019 and organized several Workshops and Special Tracks at International Conferences. He is an Associate
Editor of “Social Network Analysis and Mining” (SNAM) and “Frontiers in Artificial Intelligence - Natural Language Processing”, an Area Editor (Web Intelligence and E-Services) of
the “International Journal of Computational Intelligence Systems” (IJCIS), and an Editorial
Board Member of “Online Social Networks and Media”. His main research interests include
Social Computing, Information Retrieval, Natural Language Processing, Privacy, and Trust.
On these topics, he has published more than 90 research works in International Journals, atInternational Conferences, as Monographs and Book Chapters.

6.2

Program Committee Members

 John Bianchi, IMT Scuola Alti Studi Lucca, Italy.
 Edoardo Di Paolo, Università degli Studi di Roma “La Sapienza”, Italy.
 Tiziano Fagni, Istituto di Informatica e Telematica – CNR, Italy.
 Carlos A. Iglesias, Universidad Politécnica de Madrid, Spain.
 Udo Kruschwitz, Universität Regensburg, Germany.
 David Losada, Universidad de Santiago de Compostela, Spain.
 Lorenzo Mannocci, Università degli Studi di Pisa, Italy.
3

https://www.iit.cnr.it/en/marinella.petrocchi/
https://serics.eu/en/5
https://ikr3.disco.unimib.it/people/marco-viviani/
4

ACM SIGIR Forum

5

Vol. 58 No. 1 – June 2024

 Gabriella Pasi, Università degli Studi di Milano-Bicocca, Italy.
 Marinella Petrocchi, Istituto di Informatica e Telematica – CNR, Italy.
 Manuel Pratelli, IMT Scuola Alti Studi Lucca, Italy.
 Daisy Romanini, Istituto di Informatica e Telematica – CNR, Italy.
 Paolo Rosso, Universitat Politècnica de València, Spain.
 Irene Sánchez Rodrı́guez, IMT Scuola Alti Studi Lucca, Italy.
 Fabio Saracco, Centro Ricerche Enrico Fermi, Italy.
 Serena Tardelli, Istituto di Informatica e Telematica – CNR, Italy.
 Marco Viviani, Università degli Studi di Milano-Bicocca, Italy.

Acknowledgements
ROMCIR 2024 was partially supported by re-DESIRE: DissEmination of ScIentific REsults 2.0,
funded by IIT–CNR; by SERICS (PE00000014) under the NRRP MUR program funded by the
EU - NGEU; by the PNRR ICSC National Research Centre for High Performance Computing, Big
Data and Quantum Computing (CN00000013), under the NRRP MUR program funded by the
NextGenerationEU; by KURAMi:6 Knowledge-based, explainable User empowerment in Releasing
private data and Assessing Misinformation in online environments, under the PRIN MUR 2022
program (20225WTRFN).

References
Avishek Anand, Procheta Sen, Sourav Saha, Manisha Verma, and Mandar Mitra. Explainable
information retrieval. In Proceedings of the 46th International ACM SIGIR Conference on
Research and Development in Information Retrieval, pages 3448–3451, 2023.
David Bawden, Clive Holtham, and Nigel Courtney. Perspectives on information overload. In
Aslib proceedings, volume 51, pages 249–255. MCB UP Ltd, 1999.
Engin Bozdag and Jeroen Van Den Hoven. Breaking the filter bubble: democracy and design.
Ethics and information technology, 17:249–265, 2015.
Federico Cabitza, Davide Ciucci, Gabriella Pasi, and Marco Viviani. Responsible ai in healthcare.
arXiv preprint arXiv:2203.03616, 2022.
Sijing Chen, Lu Xiao, and Akit Kumar. Spread of misinformation on social media: What contributes to it and how to combat it. Computers in Human Behavior, 141:107643, 2023.
Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, H Eugene Stanley, and Walter Quattrociocchi. The spreading of misinformation online.
Proceedings of the National Academy of Sciences, 113(3):554–559, 2016.
6https://kurami.disco.unimib.it/

ACM SIGIR Forum

6

Vol. 58 No. 1 – June 2024

Jane Doe and John Doe. Sample title of a SIGIR Forum publication. SIGIR Forum, 54(2):1–2,
2020.
Marcos Fernández-Pichel, Markus Bink, David E Losada, and David Elsweiler. A user study on
people’s perception to the credibility of online health information. In Proceedings of ROMCIR
2024: The 4th Workshop on Reducing Online Misinformation through Credible Information
Retrieval (held as part of ECIR 2024: the 46th European Conference on Information Retrieval),
March 24, 2024, Glasgow, UK, pages 2–16. CEUR-WS, 2024.
Raphael Antonius Frick and Martin Steinebach. Improved identification of check-worthiness insocial media data through multimodal analyses. In Proceedings of ROMCIR 2024: The 4thWorkshop on Reducing Online Misinformation through Credible Information Retrieval (held aspart of ECIR 2024: the 46th European Conference on Information Retrieval), March 24, 2024,
Glasgow, UK, pages 31–40. CEUR-WS, 2024.
Lumbardha Hasimi and Aneta Poniszewska-Marańda. Detection of disinformation and content
filtering using machine learning: implications to human rights and freedom of speech. In Proceedings of ROMCIR 2024: The 4th Workshop on Reducing Online Misinformation throughCredible Information Retrieval (held as part of ECIR 2024: the 46th European Conference on
Information Retrieval), March 24, 2024, Glasgow, UK, pages 68–77. CEUR-WS, 2024.
Benedikt Hornig, Maria Soledad Pera, and Jan Scholtes. Misinformation in video recommendations: an exploration of top-n recommendation algorithms. In Proceedings of ROMCIR 2024:
The 4th Workshop on Reducing Online Misinformation through Credible Information Retrieval(held as part of ECIR 2024: the 46th European Conference on Information Retrieval), March24, 2024, Glasgow, UK, pages 54–67. CEUR-WS, 2024.
Israa Khaleel, Barbara C Wimmer, Gregory M Peterson, Syed Tabish Razi Zaidi, Erin Roehrer,
Elizabeth Cummings, and Kenneth Lee. Health information overload among health consumers:
A scoping review. Patient education and counseling, 103(1):15–32, 2020.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large
language models are zero-shot reasoners. Advances in neural information processing systems,
35:22199–22213, 2022.
Christina Lioma, Jakob Grue Simonsen, and Birger Larsen. Evaluation measures for relevanceand credibility in ranked lists. In Proceedings of the ACM SIGIR International Conference on
Theory of Information Retrieval, pages 91–98, 2017.
Giovanni Livraga and Marco Viviani. Data confidentiality and information credibility in online ecosystems. In Proceedings of the 11th international conference on management of digital
ecosystems, pages 191–198, 2019.
Giovanni Livraga, Artjoms Olzojevs, and Marco Viviani. Unveiling the privacy risk: A trade-off
between user behavior and information propagation in social media. In International Conference
on Complex Networks and Their Applications, pages 277–290. Springer, 2023.

ACM SIGIR Forum

7

Vol. 58 No. 1 – June 2024

Mattia Mattei et al. Bow-tie structures of twitter discursive communities. Scientific Reports, 12,2022. URL https://doi.org/10.1038/s41598-022-16603-7.
Leonardo Mongelli, Luca Maiano, and Irene Amerini. Cmdd: A novel multimodal two-stream
CNN deepfakes detector. In Proceedings of ROMCIR 2024: The 4th Workshop on Reducing
Online Misinformation through Credible Information Retrieval (held as part of ECIR 2024: the46th European Conference on Information Retrieval), March 24, 2024, Glasgow, UK, pages
17–30. CEUR-WS, 2024.
Scott Monteith, Tasha Glenn, John R Geddes, Peter C Whybrow, Eric Achtyes, and Michael
Bauer. Artificial intelligence and increasing misinformation. The British Journal of Psychiatry,
pages 1–3, 2023.
Marc Najork. Generative information retrieval. In Proceedings of the 46th International ACM
SIGIR Conference on Research and Development in Information Retrieval, pages 1–1, 2023.
Gabriella Pasi and Marco Viviani. Information credibility in the social web: Contexts, approaches,and open issues. arXiv preprint arXiv:2001.09473, 2020.
Marinella Petrocchi and Marco Viviani. Overview of ROMCIR 2022: The 2nd Workshop on
Reducing Online Misinformation through Credible Information Retrieval. In ROMCIR 2022
CEUR Workshop Proc, volume 3138, pages i–vii, 2022.
Marinella Petrocchi and Marco Viviani. Overview of ROMCIR 2023: The 3rd Workshop on
Reducing Online Misinformation through Credible Information Retrieval. In ROMCIR 2023
CEUR Workshop Proc, volume 3406, pages i–ix, 2023.
Marinella Petrocchi and Marco Viviani. Overview of romcir 2024: The 4th workshop on reducing
online misinformation through credible information retrieval. In Proceedings of ROMCIR 2024:
The 4th Workshop on Reducing Online Misinformation through Credible Information Retrieval(held as part of ECIR 2024: the 46th European Conference on Information Retrieval), March24, 2024, Glasgow, UK, pages i–vii. CEUR-WS, 2024.
Manuel Pratelli, Fabio Saracco, and Marinella Petrocchi. Entropy-based detection of twitter echochambers. CoRR, abs/2308.01750, 2023. URL https://doi.org/10.48550/arXiv.2308.01
750.
Manuel Pratelli, Marinella Petrocchi, Fabio Saracco, and Rocco De Nicola. Online disinformationin the 2020 U.S. election: swing vs. safe states. EPJ Data Sci., 13(1):25, 2024. URL https://doi.org/10.1140/epjds/s13688-024-00461-6.
Fabio Saracco and Marco Viviani. Overview of ROMCIR 2021: Workshop on Reducing Online
Misinformation through Credible Information Retrieval. In ROMCIR 2021 CEUR Workshop
Proc, volume 2838, pages i–vii, 2021.
Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin, Damiano Spina, Stefano
Mizzaro, and Gianluca Demartini. The many dimensions of truthfulness: Crowdsourcing misinformation assessments on a multidimensional scale. Information Processing & Management,
58(6):102710, 2021.
ACM SIGIR Forum

8

Vol. 58 No. 1 – June 2024

Hanna Suominen, Lorraine Goeuriot, Liadh Kelly, Laura Alonso Alemany, Elias Bassani, Nicola
Brew-Sam, Viviana Cotik, Darı́o Filippo, Gabriela González-Sáez, Franco Luque, Philippe Mulhem, Gabriella Pasi, Roland Roller, Sandaru Seneviratne, Rishabh Upadhyay, Jorge Vivaldi,
Marco Viviani, and Chenchen Xu. Overview of the CLEF eHealth Evaluation Lab 2021. In
K. Selçuk Candan, Bogdan Ionescu, Lorraine Goeuriot, Birger Larsen, Henning Müller, Alexis
Joly, Maria Maistro, Florina Piroi, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental
IR Meets Multilinguality, Multimodality, and Interaction, pages 308–323, Cham, 2021. Springer
International Publishing. ISBN 978-3-030-85251-1.
Rishabh Upadhyay, Petr Knoth, Gabriella Pasi, and Marco Viviani. Explainable online health information truthfulness in consumer health search. Frontiers in Artificial Intelligence, 6:1184851,
2023.
Giacomo Villa, Gabriella Pasi, and Marco Viviani. Echo chamber detection and analysis: a
topology-and content-based approach in the covid-19 scenario. Social Network Analysis andMining, 11(1):78, 2021.
Marco Viviani and Gabriella Pasi. Credibility in social media: opinions, news, and health information—a survey. Wiley interdisciplinary reviews: Data mining and knowledge discovery, 7(5):e1209, 2017.
Inna Vogel, Pauline Möhle, Meghana Meghana, and Martin Steinebach. Adapter fusion for checkworthiness detection – combining a task adapter with a NER adapter. In Proceedings of ROMCIR 2024: The 4th Workshop on Reducing Online Misinformation through Credible Information
Retrieval (held as part of ECIR 2024: the 46th European Conference on Information Retrieval),
March 24, 2024, Glasgow, UK, pages 41–53. CEUR-WS, 2024.
Claire Wardle and Hossein Derakhshan. Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe, 27, 2017.
Danni Xu, Shaojing Fan, and Mohan Kankanhalli. Combating Misinformation in the Era ofGenerative AI Models. In Proceedings of the 31st ACM International Conference on Multimedia,
pages 9291–9298, 2023.

ACM SIGIR Forum

9

Vol. 58 No. 1 – June 2024