EVENT REPORT

Report on the 11th International Workshop on
News Recommendation and Analytics(INRA 2023) at ACM RecSys 2023
Benjamin Kille

Andreas Lommatzsch

Özlem Özgöbek

NTNU
NorwayTU Berlin
Germany

NTNU
Norwaybenjamin.u.kille@ntnu.no

andreas.lommatzsch@dai-labor.deozlem.ozgobek@ntnu.no

Peng Liu

Lemei Zhang

Simen Eide

NTNU
NorwayNTNU
NorwaySchibsted
Norwaypeng.liu@ntnu.no

lemei.zhang@ntnu.no

simen.eide@schibsted.com

Abstract
News remains a challenging domain for personalization. INRA’23 allowed experts and practitioners to get together and discuss recent trends and future directions. Peer review selected
six contributions for presentation covering a wide array of topics including beyond accuracy
evaluation, the use of large language models, misinformation, and emotions in news.
Date: 18 September 2023.
Website: https://research.idi.ntnu.no/NewsTech/INRA/.

1

Motivation

Democratic decision making relies on a functioning news eco-system. Citizens need to have accessto information about recent events and their implications. News organizations used to print
newspapers covering a set of stories and features other information. In the modern, digital age,
digital social network sites, mobile devices, and AI shape news consumption. News organizations
have invested into personalization with the vision to present readers with exactly the content they
are looking for, thus reducing the time necessary to stay informed. These data-driven systems
complement the editorial work.
On the other hand, AI has opened up the opportunities for accidental misinformation anddeliberate disinformation. This raises a multitude of ethical questions. How should the ownershipof data be regulated? Should AI use be subject to legal restrictions? How much transparency isrequired to maintain public’s trust in media?
Researchers and practitioners need a venue to discuss the technical progress and these questions. The eleventh International Workshop on News Recommendation and Analytics provides
ACM SIGIR Forum

1Vol. 57 No. 2 – December 2023

the venue. INRA’23 was co-located with ACM RecSys in Singapore. The venue enabled manyexperts to attend and engage in discussions with speakers and each other.

2Theme and Topics

Personalization and AI have started affecting many aspects of the modern news eco-system. INRAserves as a venue to exchange ideas and discuss different facets of this eco-system. Recently, Large
Language Models have allowed publishers to further automate news publication. Topics of interestfor this year’s edition of INRA included but were not limited to: News Personalization– Context-awareness– Social Media– Multimodality– User behavior and modeling– Data mining– User Interfaces– Diversity and beyond accuracy News Analytics– Semantics and ontologies– Summarization– Opinion mining– Social Graphs– Big data technologies
 Psychological, Societal, and Ethical Aspects– Privacy– Clickbait, fake news, and misinformation– Bias
– Transparency and explainability
 Generative AI– Large Language Models– Automated Content Generation

3Contributions

INRA’23 received nine submissions. After a thorough peer review, six submissions were acceptedfor oral presentation:
 Alaqabawy et al. [2023] conducted an interview study with journalists. The study explored
how journalists perceive the current news eco-system features a set of data-driven elements
such as news recommender systems. The authors argue for a multi-stakeholder perspectiveof news personalization, which would increase transparency and allow for more control innews ranking.

ACM SIGIR Forum

2Vol. 57 No. 2 – December 2023

 Li et al. [2023] investigate the potential of ChatGPT as news ranker. They check how well
the generative AI ranks news, how fairly it treats different providers, and how much potential
it has to generate misinformation. The evaluation reports that using JSON-formatted input
improves the ranking over plain text. The authors observe a publisher bias.
 Iana et al. [2023] present a bilingual collection of news about migration. They emphasize
the need for beyond accuracy evaluation in the news domain. The news collection features
sentiment labels and the authors disambiguated entities with the help of Wikidata. Inaddition, the data resource features domain-specific knowledge graphs.
 Knudsen et al. [2023] explore the value of different features in determining users’ preferences toward news. They define seven features—headline, reading time, recency, geographic
distance, topic match, demographics, and popularity—and distribute the study to 1,664
Norwegians. The authors found that topic match has the highest predictive power.
 Sertkan and Neidhardt [2023] examine the emotional dimension of news articles. They argue
that authors inevitably leave some emotional cues in the articles despite efforts to keep thetone objective. The authors check whether news recommender systems reduce diversity inemotions and find some evidence for it.
 Kolb et al. [2023] advocate for beyond accuracy optimization of news recommender systems.
Their survey explores different dimensions including diversity, serendipity, and novelty. Theauthors interview a set of experts and summarize their views. Results indicate that serendipity represents the most common objective beyond accuracy.

Overall, the presented works illuminated the complexities and challenges in the news personalization and the presentation of news. One major line of research focused the necessity for a
multi-stakeholder perspective and the pursuit of objectives beyond mere accuracy in news recommender systems, advocating for greater transparency and content diversity. Another dominantresearch topic explores the technical efficacy of AI systems, such as ChatGPT, in news ranking
and the impact of various user-preference features. Additional insights are provided into the emotional and multicultural aspects of news, highlighting the significance of sentiment and cultural
diversity in news reporting and consumption.

4Future Directions

Discussions during the workshop emphasized the need to involve experts from different fields. News
represents an interdisciplinary domain that involves not only Computer Science, but also Social
Sciences, Media Science, Political Science, and Legal Science. INRA has been attached to mainly
Computer Science conferences in the past. Attendees also suggested to better involve legislatorsand news organizations. While legislators often express the wish to better understand technology,
there have been very few attending events such as INRA. ChatGPT’s release in November 2022
marks an incision making (generative) AI available to the general public. We expect that thiswill have long-lasting effects which will be subject to further research. News organizations alreadystruggle to maintain public’s trust and they will likely continue to struggle. Future editions ofINRA can help to address some of the pressing issues and devise strategies to better understandthem.

ACM SIGIR Forum

3Vol. 57 No. 2 – December 2023

Acknowledgements
We recruited a set of experts to assure the quality of the accepted papers. We would like to thank: Michael Beam, Kent State University, OH, U.S.A.
 Simen Eide, Schibsted ASA, Norway
 Mehdi Elahi, University Bergen, Norway
 Dietmar Jannach, Adria-Alpen Universität Klagenfurt, Austria
 Christoph Trattner, University Bergen, Norway

References
Nihal Alaqabawy, Aishwarya Satwani, Amy Voida, and Robin Burke. “It’s just a robot that looks
at numbers”: Restoring Journalistic Voice in News Recommendation. In Proceedings of theEleventh International Workshop on News Recommendation and Analytics. CEUR WorkshopProceedings, 2023.
Andreea Iana, Mehwish Alam, Alexander Grote, Nevena Nikolajevic, Katharina Ludwig, PhilippMüller, Christof Weinhardt, and Heiko Paulheim. NeMig—A Bilingual News Collection andKnowledge Graph about Migration. In Proceedings of the Eleventh International Workshop onNews Recommendation and Analytics. CEUR Workshop Proceedings, 2023.
Erik Knudsen, Alain D. Starke, and Christoph Trattner. Topical Preference Trumps Other Features in News Recommendation: A Conjoint Analysis on a Representative Sample from Norway.
In Proceedings of the Eleventh International Workshop on News Recommendation and Analytics.
CEUR Workshop Proceedings, 2023.
Thomas E. Kolb, Irina Nalis, and Julia Neidhardt. Like a Skilled DJ—an expert Study on News
Recommendation Beyond Accuracy. In Proceedings of the Eleventh International Workshop on
News Recommendation and Analytics. CEUR Workshop Proceedings, 2023.
Xinyi Li, Yongfeng Zhang, and Edward C. Malthouse. A Preliminary Study of ChatGPT on
NewsRecommendation: Personalization, Provider Fairness, and Fake News. In Proceedings ofthe Eleventh International Workshop on News Recommendation and Analytics. CEUR Workshop Proceedings, 2023.
Mete Sertkan and Julia Neidhardt. On the Effect of Incorporating Expressed Emotions in News
Articles on Diversity within Recommendation Models. In Proceedings of the Eleventh International Workshop on News Recommendation and Analytics. CEUR Workshop Proceedings,2023.

ACM SIGIR Forum

4Vol. 57 No. 2 – December 2023