EVENT REPORT

Report on the 3rd Workshop of Knowledge-aware
and Conversational Recommender Systems(KARS/ComplexRec) at RecSys 2021
Vito Walter Anelli
Polytechnic University of Bari
vitowalter.anelli@poliba.it

Pierpaolo Basile
University of Bari Aldo Moropierpaolo.basile@poliba.it

Toine Bogers
Aalborg University Copenhagentoine@hum.aau.dk

Tommaso Di Noia
Polytechnic University of Bari
tommaso.dinoia@poliba.it

Francesco Maria Donini
University of Tuscia
donini@unitus.it

Bamshad Mobasher
DePaul University
mobasher@cs.depaul.edu

Cataldo Musto
University of Bari Aldo Morocataldo.musto@uniba.it

Fedelucio Narducci
Polytechnic University of Bari
fedelucio.narducci@poliba.it

Casper Petersen
Sampension
casper.ptrsn@gmail.com

Maria Soledad Pera
Boise State University
solepera@boisestate.edu

Markus Zanker
Free University of Bozen-Bolzano andUniversity Klagenfurt
mzanker@unibz.it
Abstract
In this report, we offer a brief overview of the contributions and takeaways from the Joint
KaRS & ComplexRec Workshop, co-located with the 15th edition of the ACM RecSys in
Amsterdam, The Netherlands. With this workshop, we aimed to merge the main objectives
envisioned for the 3rd Edition of the Workshop of Knowledge-aware and Conversational
Recommender Systems and the 5th Edition of the Workshop on Recommendation in Complex
Environments. This joint workshop adopted a hybrid format aligned with the goal of thisACM SIGIR Forum

1

Vol. 55 No. 2 December 2021

year’s main conference congregating to continue to build community around recommendersystems research and development.
Date: 27th September–1st October, 2021.
Website: https://kars-workshop.github.io/2021/program/.

1

Introduction

Recommender systems are embedded in the daily lives of users worldwide, providing them withsuggestions of movies to watch, books to read, points of interests to visit, or items to purchase.
Indeed recommender systems are prominent in e-commerce and other entertainment-related online
platforms. Nevertheless, the applicability of recommendation technologies is far-reaching.
To continue to advance knowledge of researchers and industry practitioners on the challenges
and open problems pertaining to recommender systems that exploit external and explicit knowledge
sources to feed and build recommendation engines and adopt interactions based on the conversational
paradigm, particularly applied to more complex domains, we merged two of the workshops originallyaccepted to take place during the 15th edition of the ACM Conference on Recommender Systems(RecSys): 3rd Edition of Knowledge-aware and Conversational Recommender Systems (KaRS)[Anelli et al., 2021a] & 5th Edition of Recommendation in Complex Environments (ComplexRec)[Abdollahpouri et al., 2021].
Aligning with ACM RecSys 2021, ours was a hybrid workshop that took place over 2 days: one
fully online hosted on Zoom and one in-person hosted at the main conference venue, but welcoming
remote participation via Zoom. In the rest of this report, we offer an overview of keynote addresses
and accepted contributions. We also summarize main takeaways from the workshop.

2

Workshop Focus

For this edition of the Joint KaRS & ComplexRec Workshop, we narrowed the focus of the expected
contributions to topics related to one of the following main themes: complex inputs, complex
outputs, conversational recommender systems, explainable recommendations, and knowledge-aware
recommendations for complex scenarios.

2.1

Complex Inputs

An important source of complexity for recommenders comes from the various types of inputs tothe system beyond users and items, such as features, queries and constraints. There are active
user inputs (interaction), implicit user inputs (task, context, preferences), item inputs (features or
attributes) and domain inputs (eligibility, availability). In group-based recommendation, the user
input can be a combination of inputs for multiple individual users as well as group aspects suchas the composition of the group and how well they know each other. In the context of education
for instance, inputs range from the knowledge and skill level of the learner, their preferences formodes of learning, the amount of time available for studying and rehearsing, and the constraints of
the location, resources and physical space. These constraints may even conflict with each other orwith preferences of different stakeholders.
ACM SIGIR Forum2

Vol. 55 No. 2 December 2021

An additional challenge is providing users with ways to have control over the inputs. For
instance by selecting and weighting or ranking user and item features, or providing interactive
queries to steer the recommendation. In recommender systems designed to assist customers andtroubleshoot problems, one of the inputs is the problem statement, which can be provided in
the form of a few keywords, natural language questions, or longer narrative statements. These
require increasing levels of natural language understanding to transform into relevant features andconstraints that have to be combined with many other types of inputs, including the knowledge
and skill level of the person for whom the advice or report is intended.
There are many examples of other domains where recommenders have to deal with a complex setof inputs, such as: Food, where there may be dietary requirements, constraints on the availabilityof ingredients, cooking utilities, time to cook and budget, paired accompaniments such as wine, as
well as taste preferences and requirements for cuisine and number of courses; Tourism, with similar
constraints on time and budget, distance, preferences for types of locations, accommodation andmode of transport, as well as inputs related to the travel group; and Transportation, which has totake into account different numbers and types of packages, available vehicles and drivers, as well as
constraints from customers being available to receive a package, as well as different priority levels.

2.2

Complex Outputs

Another type of complexity that is that associated with the complexity of the outputs of arecommender system to move away from a straightforward ranked list of items as output. An
example of such complex output is package recommendation: suggesting a set or combination of
items that go well together and are complementary on dimensions that matter to the user. In
many domains the sequence in which items are recommended is also important. Moreover, different
users may want different information about items, so the output complexity goes beyond ranking
and also manifests itself in how the interface should allow the user to view the type of informationthat is most relevant to them. Another example of complexity in recommender systems output areenvironments where the system’s goal is to create new, composite items that must satisfy certain
constraints (such as menu recommendation, or recommendations for product designs).
There are many examples of domains where the outputs of the recommender system are complex
in one or more of the above-mentioned ways, including Food, where a recommender system could
help the users by suggesting a balanced diet by recommending a set of dishes over a longer period,or in composing entirely new dishes based on available ingredients; Tourism, where travel packages
should consist of an optimized, complementary set of locations, accommodations, and activities;
and Fashion, where complete matching outfits could be recommended to the user.

2.3

Conversational Recommender Systems

Conversational Recommender Systems (CRSs) are gaining a lot of attention in last few years.
CRSs are characterized by a multi-turn dialog between the user and the system. A CRS might
converse in natural language, but it may allow more constrained modes of user interaction too.
This kind of interaction introduces new challenges, since it makes even less sharp the difference
between recommendation and retrieval. A CRS should be able to exploit both short- and long-term
preferences, for example. Furthermore, a CRS should be able to adapt its behaviour timely when
ACM SIGIR Forum

3

Vol. 55 No. 2 December 2021

a user feedback is given. These are just some peculiarities of this kind of interaction. As we canimagine, another sensitive issue is the evaluation of CRSs, since also in this case we need to go
beyond simple accuracy metrics. The low availability of datasets is an additional obstacle to theevaluation of these systems. While research and development into CRSs has never gone away, it
has certainly been less prominent for a while. Recently, the literature on this topic is growing
again. Prior KaRS workshop editions had been a good place for discussing challenging ideas forthe exploitation of background and domain-specific knowledge for generating more effective andappealing conversational recommendations. Very interesting approaches have been presented also
for addressing complementary, but crucial tasks for conversational recommendations such as theentity recognition.

2.4

Explainable Recommendations

Explanation has a fundamental role in the recommendation pipeline. In the context of thisworkshop, explanation has been analyzed as a mean for improving the fairness of machine learningmodel and for helping the user in reaching her goals. In particular, the attention has been posedon the capability of exploiting the output of post-hoc models such as Lime1 , Shap2 as input for
generating natural language or more generally user friendly explanations.

2.5

Knowledge-aware recommendation for complex scenarios

Knowledge-based approaches began to appear two decades ago. Nonetheless, they have become
widely used with the advent of the Linking Open Data3 initiative when a huge number of knowledgegraphs have been released and made freely available as RDF triples. These include encyclopedic
datasets such as DBpedia4 or, more recently, Wikidata5 , where semantics-aware information isavailable on different knowledge domains and applications. The exploitation of such datasets
together with their ontologies is at the basis of many approaches to recommendation and challenges
proposed in the last years. In this workshop, we turned the spotlight on the exploitation ofknowledge in complex scenarios such as cars or software configurations, job calls recommendations,and police lineups. Furthermore, analysis and processing of knowledge coming from complex
sources such as user reviews had a relevant role in the workshop discussions.

3

Workshop ProgramThis year, we were fortunate to have two keynote speakers who fostered interesting discussions
among workshop attendees. We also heard from authors for each of the accepted contributions.
We summarize main takeaways from these presentations below.
1

https://github.com/marcotcr/lime
https://shap.readthedocs.io/en/latest/3
http://linkeddata.org
4
https://dbpedia.org
5
https://wikidata.org
2

ACM SIGIR Forum

4

Vol. 55 No. 2 December 2021

3.1

Keynotes

The workshop featured two keynote presentations: Edward C. Malthouse and Gerard de Melo.
Dr. Malthouse is the Erastus Otis Haven Professor at Northwestern University and a research
fellow at the Media Management Center, a partnership between Medill and Kellogg. He is also theresearch director of the Medill IMC Spiegel Research Center. During his talk, entitled Toward
the next generation of news Recommender Systems, Dr. Malthouse discussed the complexities
associated with designing new recommender systems outputs, such as newsletters or websites.
He emphasized the table d’hôte approach to create engaging RS outputs. This approach, whichbuilds on communication theories and stratified sampling to create automated bundled news
recommendations, takes into consideration the fact that in the news domain, the sequencing
of items is an important consideration—beyond more traditional perspectives associated withassessment of recommender systems, such coverage, novelty, diversity and serendipity.
Dr. De Melo is a professor at the Hasso Plattner Institute for Digital Engineering and the
University of Potsdam, Germany, where he holds the Chair for Artificial Intelligence and Intelligent
Systems. During his keynote address entitled How Can I Help You? Knowledge Graphs for
Explainable Recommendation, Dr. De Melo discussed several methods to bring together knowledge
about user preferences, items, and background by means of knowledge graphs and neuro-symbolic
explainable artificial intelligence (AI). These methods often rely on deep reinforcement learning or
neural logic reasoning to provide explanations that enable users to better understand why particular
items are recommended to them. More importantly, Dr. De Melo brought to our attention diverse
ongoing research efforts focused on how to mitigate biases and enable dialogue-based interactions
in conversational recommender systems. We also discussed datasets and code that are available for
the RecSys community to leverage in future research endeavours.

3.2

Accepted Contributions

As outlined in the call for papers, this year edition of KaRS and ComplexRec invited contributions
focused particularly on the inputs and outputs that are often associated with recommendation
systems embedded in complex environments, as well as design technologies and evaluation methodologies impacting knowledge-aware and conversational recommenders.
Overall, we accepted 17 contributions: 11 long papers, 3 short papers, and 3 position papers.
Each presentation was peer-reviewed by at least 3 program committee (PC) members, consistingof international experts in the field. Evaluation criteria for acceptance included novelty, diversity,
significance for theory/practice, quality of presentation, and the potential for sparking interesting
discussion at the workshop.
The accepted contributions, each invited to give a short presentation, covered a broad set ofcomplex and challenging recommendation scenarios.
3.2.1

Recommender Systems in Complex Scenarios

Kim et al. [2021] presented a unified framework based on neural networks embedded in a rewardpenalty structure that supports group prediction. Extensive empirical evaluations based on both
real and synthetic datasets demonstrate the robustness of the proposed framework, its ability to
bypass scalability issues and, more importantly, its superiority (in terms of accuracy and cross
ACM SIGIR Forum

5

Vol. 55 No. 2 December 2021

entropy) over a number of baseline and state-of-the-art strategies. Eichinger [2021] highlighted
an ongoing problem recommender strategies face: small user profiles for which ratings rarely
overlap with other users, preventing identification of preferred items. To address this concern,
he proposed a new similarity measure based on users’ items reviews. Results from in-depth
experiments conducted using Amazon Reviews-core dataset revealed that even using as little as
10 keywords extracted from users’ reviewers are sufficient to adequately model their preferences.
Segura-Tinoco and Cantador [2021] introduced a taxonomy of argumentative relations meant
to advance research on automatically extracting arguments from textual content that are thenemployed by recommender systems. The authors shared a lexicon in both English and Spanish
comprised of linguistic connectors. Via a case study on an e-participation platform the authors
illustrate the applicability of the proposed taxonomy and its implication for recommender systems.
3.2.2

Conversational Recommender Systems

Lin et al. [2021] proposed a dialogue generation system to incorporate background knowledge forcoherent response generation, and to recommend items with respect to dialogue context. They
employ pre-trained language models with multi-task learning to jointly learn response generationand goal prediction towards the target. Dingwall and Gao [2021] designed a framework for gazetteer
knowledge integration that incorporates external knowledge regarding entity popularity (e.g., asong’s play count) to reduce spurious entity matching and improve the robustness of gazetteer
features. Starke and Lee [2021] analyzed the adoption of voice-based technologies and show thatthey may put specific groups of users at a disadvantage, as they are likely to run into accessibility
issues. Biancofiore et al. [2021] developed a job recommendation system to match user skills andrequests with job positions available on the Gazzetta Ufficiale website, offering recommendation
services in a conversational setting. Guapp’s dialogues are modelled employing a domain-specificKnowledge Graph, which improves the users’ natural language interaction with the app.
3.2.3

Explainable Recommendation

Anelli et al. [2021b] showed how the characteristics of a classical post-hoc model (i.e., LIME-RS)
based on local surrogate models is strongly model-dependent and does not prove to be accountable
for the explanations generated. Cornacchia et al. [2021] focused on Recommender systems in
the Financial Services domain. Recently, the adoption of AI in this domain has shed light on
new ethical and legal risks. The authors proposed a model for generating natural language andcounterfactual explanations for a loan recommender system with the aim of providing fairer andmore transparent suggestions.
3.2.4

Knowledge-aware Recommendation for Complex Scenarios

Uta et al. [2021] showed how the determination of configurations of complex and configurable items(e.g., cars and software) can be supported by neural network based recommendation. Indeed, theusers, due to cognitive overloads, and missing domain knowledge, may in many cases not able
to completely specify their preferences with regard to all relevant component properties. The
authors introduced a semantic regularization approach that helps to take into account configuration
constraints within the scope of neural network learning. Rocco et al. [2021] aimed to define a
ACM SIGIR Forum

6

Vol. 55 No. 2 December 2021

precisely curated and organized core set of concepts and practices, i.e., a Body of Knowledge (BOK)
for Recommender Systems, as already done in other disciplines, including software engineering andmodel-driven engineering. Ivanova and Ricci [2021] aimed at developing a Climbing Recommender
System for suggesting routes that are suited for training and practicing sport climbing. They modela climber by relying on both explicit and implicit feedback. Implicit feedback is acquired by anautomatic activity recognition component (in climbing gyms), while explicit feedback is acquired bymeans of a mobile application. Sottocornola et al. [2021] designed a knowledge-based recommendersystem capable to diagnose post-harvest diseases of apples. It describes the process of knowledge
elicitation and construction of a Bayesian Network reasoning system as well as its evaluation withthree different types of studies involving diseased apples. Dokoupil and Peska [2021] developed
LiGAN, an application aiming on on-the-fly recommendation of artificial fillers for police photo
lineups. Police photo lineups are an important part of criminal proceedings, where the task is
to identify the perpetrator among photos of other persons (fillers). LiGAN utilizes StyleGAN2
architecture to generate images, identity-preserving autoencoder for suspect seeding and optionalmodel finetuning for individual lineups. It recommends fillers based on the semantic proximity to
the suspect, or as an interpolation between suspect and filler images. Pastore et al. [2021] exploredthe possibility of transferring knowledge across domains for automatically extracting aspects fromuser reviews. The authors develop a novel Deep Learning model for aspect extraction and evaluate
it in terms of recommendation accuracy. Harrando and Troncy [2021] studied the potential of using
off-the-shelf automatic annotation tools from the Information Extraction literature to improverecommendation performance without any extra cost of training, data collection or annotation.
They analyzed how these annotations can improve recommendations on two tasks: the traditionaluser history-based recommendation, as well as a purely content-based recommendation. Mentec
et al. [2021] proposed some conversational recommendation techniques that can support recruiters’
work and recommend candidates to a given job offer, based on relevant skills, that can provide an
explanation for the recommendation, so that the recruiter has specific information as to why acandidate is recommended.

Acknowledgments and Closing Remarks
We would like to thank ACM and RecSys for hosting this workshop, particularly the RecSys
workshop chairs Jen Golbeck, Marijn Koolen, and Denis Parra. We would also like to thank the
program committee for our joint workshop: Azzurra Ragone, Paolo Rosso, Andrea Iovine, Adir
Solomon, Maurizio Ferrari Dacrema, Diego Antognini, Marco Polignano, Iván Cantador, Marco deGemmis, Raffaele Perego, Claudio Gennaro, Gianmaria Silvello, Cataldo Musto, Davide Di Ruscio,
Nicola Tonellotto, Pierpaolo Basile, Alejandro Bellogin, Chiara Renso, Pablo Sánchez, Benjamin
Heitmann, Maria Maistro, Olga Marino, Francesco M. Donini, Dietmar Jannach, Cristina Gena,
Giorgio Maria Di Nunzio, Federica Cena, Markus Zanker, Ludovico Boratto, Franco Maria Nardini,
Alain Starke, Toine Bogers, Francesco Ricci, Giovanni Semeraro, Nourah Alrossais, Nicola Ferro,
Yashar Deldjoo, Claudio Pomo, Antonio Ferrara, Adamopoulos Panagotis, Pasquale Lops, Christine
Bauer, Fabio Gasparetti, Mehdi Elahi, Peter Dolog, Tommaso Di Noia, Fedelucio Narducci, Mirko
Marras, Markus Schedl, Hanna Schäfer, Bei Yu.
Finally, we would like to thank to the authors who submitted their contributions, our invited
speakers Edward C. Malthouse and Gerard de Melo, and all the participants for an interesting and
ACM SIGIR Forum

7

Vol. 55 No. 2 December 2021

lively workshop.
Throughout the two days of this joint workshop, we were able to advance knowledge, shareideas pertaining both KaRS and ComplexRec. More importantly, through the interactions among
attendees and discussions emerging as a result of keynote and paper presentations, we were able toidentify future research directions.
The workshop material (list of accepted papers, keynote, and the workshop schedule) can befound on the ComplexRec 2021 workshop website6 as well as KaRS 2021 website7 . The proceedings
have been published as a CEUR Workshop Proceedings volume8 . The recordings of workshop
presentations are available on the RecSys YouTube channel9 .

References
Himan Abdollahpouri, Toine Bogers, Bamshad Mobasher, Casper Petersen, and Maria
Soledad Soledad Pera. Complexrec 2021: Fifth workshop on recommendation in complex
environments. In Fifteenth ACM Conference on Recommender Systems, pages 775–777, 2021.
Vito Walter Anelli, Pierpaolo Basile, Tommaso Di Noia, Francesco M Donini, Cataldo Musto,
Fedelucio Narducci, and Markus Zanker. Third knowledge-aware and conversational recommendersystems workshop (kars). In Fifteenth ACM Conference on Recommender Systems, pages 806–809,
2021a.
Vito Walter Anelli, Alejandro Bellogin, Tommaso Di Noia, Francesco Maria Donini, Vincenzo
Paparella, and Claudio Pomo. Adherence and constancy in lime-rs explanations for recommendation. In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021b. URL
http://ceur-ws.org/Vol-2960/paper11.pdf.
Giovanni Maria Biancofiore, Tommaso Di Noia, Eugenio Di Sciascio, Fedelucio Narducci, andPaolo Pastore. Guapp: a knowledge-aware conversational agent for job recommendation.
In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper10.pdf.
Giandomenico Cornacchia, Fedelucio Narducci, and Azzurra Ragone. A general model for fairand explainable recommendation in the loan domain. In Proceedings of the Joint KaRS &ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper12.pdf.
Nicholas Dingwall and Vianne R. Gao. Enhancing gazetteers for named entity recognition inconversational recommender systems. In Proceedings of the Joint KaRS & ComplexRec Workshop.
CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper8.pdf.
Patrik Dokoupil and Ladislav Peska. Ligan: Recommending artificial fillers for police photo
lineups. In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL
http://ceur-ws.org/Vol-2960/paper14.pdf.
6https://complexrec2021.aau.dk/
https://kars-workshop.github.io/2021/8
http://ceur-ws.org/Vol-2960/
9https://www.youtube.com/channel/UC2nEn-yNA1BtdDNWziphPGA
7

ACM SIGIR Forum

8

Vol. 55 No. 2 December 2021

Tobias Eichinger. Reviews are gold!? on the link between item reviews and item preferences.
In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper2.pdf.
Ismail Harrando and Raphael Troncy. Improving media content recommendation with automatic
annotations. In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL
http://ceur-ws.org/Vol-2960/paper16.pdf.
Iustina Ivanova and Marina Andricand Francesco Ricci. Knowledge-based recommendations for
climbers. In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL
http://ceur-ws.org/Vol-2960/paper6.pdf.
Sunghyun Kim, Minje Jang, and Changho Suh. Group match prediction via neural networks.
In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper1.pdf.
Dongding Lin, Jian Wang, and Wenjie Li. Target-guided knowledge-aware recommendation dialoguesystem: An empirical investigation. In Proceedings of the Joint KaRS & ComplexRec Workshop.
CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper7.pdf.
Francois Mentec, Zoltan Miklos, Sebastien Hervieu, and Thierry Roger. Conversational recommendations for job recruiters. In Proceedings of the Joint KaRS & ComplexRec Workshop.
CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper17.pdf.
Paolo Pastore, Andrea Iovine, Fedelucio Narducci, and Giovanni Semeraro. A general aspectterm-extraction model for multi-criteria recommendations. In Proceedings of the Joint KaRS &ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper15.pdf.
Juri Di Rocco, Davide Di Ruscio, Claudio Di Sipio, Phuong T. Nguyen, and Claudio Pomo. On
the need for a body of knowledge on recommender systems. In Proceedings of the Joint KaRS &ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper5.pdf.
Andrés Segura-Tinoco and Iván Cantador. On the extraction and use of arguments in recommendersystems: A case study in the e-participation domain. In Proceedings of the Joint KaRS &ComplexRec Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper3.pdf.
Gabriele Sottocornola, Sanja Baric, Fabio Stella, and Markus Zanker. Case study on the developmentof a recommender for apple disease diagnosis with a knowledge-based bayesian network. In
Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL http://
ceur-ws.org/Vol-2960/paper13.pdf.
Alain D. Starke and Minha Lee. Voicing concerns: User-specific pitfalls of favoring voice overtext in conversational recommender systems. In Proceedings of the Joint KaRS & ComplexRec
Workshop. CEUR-WS, 2021. URL http://ceur-ws.org/Vol-2960/paper9.pdf.
Mathias Uta, Alexander Felfernig, and Denis Helic. Constraint-aware recommendation of complex
items. In Proceedings of the Joint KaRS & ComplexRec Workshop. CEUR-WS, 2021. URL
http://ceur-ws.org/Vol-2960/paper4.pdf.
ACM SIGIR Forum

9

Vol. 55 No. 2 December 2021