WORKSHOP REPORT

Building Community: Report on KidRec
Workshop on Children and Recommender
Systems at RecSys 2017
Maria Soledad Pera
People and Information Research Team
Dept. of Computer Science
Boise State University, Boise, ID, USA
solepera@boisestate.edu
Jerry Alan Fails
Dept. of Computer Science
Boise State University, Boise, ID, USA
jerryfails@boisestate.edu
Mirko Gelsomini
Computer Engineering at the Politecnico di Milano
Dept. of Electronics, Information, and Bioengineering, Milan, Italy
mirko.gelsomini@polimi.it
Franca Garzotto
Computer Engineering at the Politecnico di Milano
Dept. of Electronics, Information, and Bioengineering, Milan, Italy
franca.garzotto@polimi.it
Abstract
We present in this manuscript a report on KidRec 2017 workshop, the first edition of theinternational workshop on Children & Recommender Systems, co-located with the 11th ACM
Conference on Recommender Systems (RecSys) held in Como, Italy. While research focusedon recommender systems is prolific and addresses issues pertaining to multiple domains–frommusic, restaurant, and movie recommendations to scholarly publications or news–little has
been done to explore the constraints and limitations that existing strategies must face whenapplied to non-traditional stakeholders, such as children. The first edition of our workshop
was aimed at providing a venue dedicated to building community in this area and identify
future research paths to nurture and grow this emerging community.

ACM SIGIR Forum

153

Vol. 52 No. 1 June 2018

1

Introduction

Recommendation systems (RS) have traditionally been focused on serving adults [1, 7, 14].
Adults can directly and explicitly identifying their wants and needs and are often willing toprovide feedback in the form of ratings or reviews. As they can purchase items themselves,
this indirect type of feedback can be utilized to inform RS. While research for adult RS havetargeted items such as movies, books, and other forms of entertainment, RS for other areasand populations are still emerging. In particular, there is a need for researching the specificpopulation of children due to their increased use of technology [8, 11].
Popular systems such as YouTube and Netflix are utilized by children and have bothtaken forays into recommending content for children, however, their focus has primarily
been on filtering content so as not to include unsuitable content for children [12]. Whilefiltering content is part of the issue related to providing content, there are a plethora ofother considerations and factors that will influence the design of effective RS for children.
When focused on this particular audience, the role of a RS needs to be reformulated, as
it is not sufficient for recommenders to identify items that match users preferences andinterests. Instead, it is imperative that they also explicitly consider their needs from multiple
perspectives: educational, developmental, and engagement, to name a few. RS for education
have started to consider other aspects of recommendation [16, 17], but there remains a rich setof questions to answer: What are the benefits of child-specific recommendations? What role
does personal history, age, developmental stage, or even curricular standards have in RS forchildren? What are the goals of RS for children: persuade, educate, guide, and support theirwell-being, or something else entirely? What are the ethical or privacy challenges associatedwith this type of recommendation? What are the domains in which recommendations makesense for children? Are they the same as for adults?; How should we model personalization
given that traditional mechanisms of explicit feedback, writing reviews, and purchasing items(directly) do not match this particular population whose literacy levels are growing and who
generally do not directly have the ability to purchase items? Are their specifics concern,from the point of view of developing technology that should be considered?
In addition, the interests and needs of this population are rapidly evolving, as they arestill refining their interests as they get to know new things, and as their educational horizons broaden. Given the increasing use of technology by children, and the breadth of issues
already identified, we felt it was necessary to bring researchers from diverse backgrounds towork to build this community, collectively map out the field of RS for children, and to begin
to address these issues together.
Need. To the best of our knowledge, there are have been no workshops on Children &Recommender Systems. However, a number of workshops have been held in the past few
years in closely-related or complementary areas, which provide evidence of the interest of
the research community in such a topic. EdRecSys [19], attracts researchers focused on RS
tailored to education. Search as Learning (SAL) [9], held last year at ACM SIGIR, assembled
researchers interested in information retrieval, natural language processing, and education.
Neither of the aforementioned workshops focus on K-12 populations and focus primarily on
learning not the full range of RS that children are exposed to or use. The ACM-sponsoredconference Interaction Design and Children (IDC) offered a number of workshops last yearthat emphasized designing with and for children [2, 5], however, the focus was on how todesign technology for children, not RS for children.

ACM SIGIR Forum

154

Vol. 52 No. 1 June 2018

Goal. The main goal of this interactive workshop was to congregate researchers and experts
from multiple disciplines, in order to understand the ethical, pedagogical, and technical implications of designing and developing RS that can be of use by children, whether for leisureor educational activities.
Why at RecSys? RecSys is a venue that historically encouraged academia and industry
participation and there are academic and industry interests in RS for children. Furthermore,
due to the interdisciplinary nature of this domain, researchers from varied disciplines arebrought together, from machine learning and information retrieval to human computer interaction and privacy. Because diversity is welcomed at RecSys, we felt it was an ideal venue
to discuss and start to address the growing needs and issues related to child-centered RS.

2

Workshop Description

In this section we address the diversity of disciplines required to address the issues related
to KidRec, the interactive format of our workshop, brief descriptions and abstracts of theaccepted contributions, and perhaps most importantly the discussions and challenges thatwere raised during the workshop.

2.1

Audience of Interest & Format

Since workshop objectives included sharing and discussing research and projects that reachbeyond classic recommender techniques and discuss the many child related challenges of RS;
we reached out to researchers with diverse expertise: Technology for children; Human computer Interaction; Information Retrieval; Assessment; Personalization; and Gamification, toname a few. By reaching out to members from these communities and building a communityof KidRec researchers, we aimed to foster a multidisciplinary perspective that can initiate aconversation of the challenges, limitations, and needs related to the design and developmentof recommendation systems that can be directly and widely adopted by children.
KidRec was purposefully designed to not follow a traditional workshop pattern, but instead adopted a highly participatory approach meant to generate discussions and connections
among researchers interested in topics that applied to RS targeted towards young audiences.
With that in mind, we elicited submissions discussing novel work in progress and position
papers that focused on open challenges, promising research directions, or speculative or innovative work in progress.
In our program, we reserved around 45 minutes for a round of short “lightning”-stylepresentations for each of the accepted contributions (see a brief discussion of each of these
accepted publications in Section 2.2). The rest of our program was focused on open discussion
about current research related to KidRec, identifying problem areas, identifying community
needs and expectations, and agreeing on directions for future work.

2.2

Accepted Contributions

For this first meeting, we accepted six submissions. The topics of these submissions arevaried and come from researchers in informatics, human computer interaction, gaming, music,
design, and visualization. We outline below the major take-aways from the “lightning”-stylepresentations. This is followed by the abstracts of each of the accepted contributions.

ACM SIGIR Forum

155

Vol. 52 No. 1 June 2018

• Monica Landoni [13] discussed how aesthetic relevance influences the decision making
process when it comes to recommend multimedia items for children.
• Yashar Deldjoo [3] focused instead on the personalization aspect of recommendation toenhance children’s experience with these systems. He specially focused on how attention
patterns can bias the recommendation selection of items in the movie domain.
• Christine Bauer [18] brought up important points to be considered when it comes torecommending musing to adolescents; including the influence of music genre preference
and preference homogeneity.
• Min Zhang [15] lead the discussion into the educational aspects of recommendations,by presenting her research work on the use of RS for vocabulary development amongchildren learning a second language.
• Federica Delprino [4] focused her discussion on the importance of understanding children’s interest through game to then inform the recommendation process.
• Michael Ekstrand [6] opened up the discussion on how to evaluate RS tailored to younger
audiences, given the lack of existing benchmarks and what are the main challenges: fromgathering data for implicit feedback to conducting user studies for explicit feedback.
Aesthetic Relevance when Selecting Multimedia Stories - by M. Landoni and E.
Rubegni
We explore the concept of Aesthetic Relevance in order to study how children choose shortmultimedia stories to read for leisure. We gathered data via observations and questionnaires
in order to understand what are the main factors and dimensions to affect selection and here
compare our findings with those emerging from a seminal work looking at how children selectbooks in a digital library. We observed how the social dimension played a strong role on selection with few children, perceived as more knowledgeable, influencing others and their choices.
Enhancing Childrens Experience with Recommendation Systems - by Y. Deldjoo,
C. Fr, M. Valla, M. A. Tuncel, F. Garzotto, P. Cremonesi, A. Paladini, and D. Anghileri.
Recommender systems (RS) offer a personalized support in exploring large amounts of information, assisting users in decision making about products matching their taste and preferences. Most of the research to date on RS have focused on traditional users, i.e., adultindividuals. However, childrens patterns of attention and interaction are quite different fromthose of adults. This paper presents the first results of a research-in-progress that can besuited to bridge the barrier between children and RS by providing a child-friendly interaction paradigm. Specifically, a web application is developed that employs real-time object
recognition on movie thumbnails or DVD cover-photos in a real-time manner. The tangible
object can be manipulated by the user and provide input to the RS to generate movie recommendations.
Online Music Listening Culture of Kids and Adolescents: Listening Analysis andMusic Recommendation Tailored to the Young - by M. Schedl and C. Bauer.
Abstract: In this paper, we analyze a large dataset of user-generated music listening eventsfrom Last.fm, focusing on users aged 6 to 18 years. Our contribution is two-fold. First, westudy the music genre preferences of this young user group and analyze these preferences
for homogeneity within more fine-grained age groups and with respect to gender and countries. Second, we investigate the performance of a collaborative filtering recommender when

ACM SIGIR Forum

156

Vol. 52 No. 1 June 2018

tailoring music recommendations to different age groups. We find that doing so improves
performance for all user groups up to 18 years, but decreases performance for adult users
aged 19 years and older.
A Game-based Data Collecting Framework for the Recommendation of Kids
Second Language Learning - by W. Ma, M. Zhang, C. Zhang, Y. Chen, Q. Xie, W. Sun,
Y. Liu, and S. Ma.
Under the background of globalization, more and more people are learning a second language.
Word learning is essential in language learning. It is valuable and helpful if a recommendation system for the language word learning is proposed for learners. In previous work, theresearch of language cognition is often based on the dataset of kids rst language development,such as: Word Bank, to summarize and study the word learning process. However, there
lacks proper large-scale second language vocabulary development dataset, and it is very difficult to collect a big dataset with traditional collecting methods. is limits the study of second
language learning and the construction of words recommendation systems. In this paper, we
designed a data collecting framework for kids based on the idea of games with a purpose,
to collect kids vocabulary development status and his/her attributes. We have implemented
the second language vocabulary development collecting system for English. Moreover, we
proposed an idea of word learning recommendation system for kids.
Playing Outdoor, Recommending New Content: Stimulating Kids Learning
Through the ABBOT Smart Object - by F. Delprino, O. F. Bravo, M. Mariani, C.
Piva, N. Izzo, M. Matera, and R. Tassi.
This paper describes the design of a smart service that aims at stimulating kids learningby combining smart objects for outdoor playing with recommendations. A tangible object,
ABBOT, is proposed as a companion for kids in the first years of primary school. This
toy–equipped with a photo camera activated by an accelerometer that intercepts object
shaking–helps kids record what they find interesting in the physical environment where theyplay. In a following phase (typically occurring at home) kids can play with the digital materials collected during outdoor play and related contents using simple interactive games on atablet (with the support of caregivers, e.g., parents). The system exploits (i) an image recognition service to extract features from pictures (e.g., a leaf or wood of a specific plant) and(ii) a mixed-recommender technique (content-based and rule-based) to create associations
between the images collected during play and semantically-relevant visual contents (selectedfrom a pre-defined collection of material and from online sources).
Challenges in Evaluating Recommendations for Children - by M. Ekstrand.
Recommender systems research and development cannot advance without robust evaluation
strategies. While many evaluation strategies have proven effective for deploying and testing
recommenders for general audiences, child-oriented recommendations pose unique challenges,
and adult-oriented evaluation strategies do not necessarily translate. In this position paper, I
briefly describe several of the challenges I see in evaluating recommender systems for children,
how they relate to similar problems for general audiences, and why existing solutions from therecommender systems community are insufficient. Significant progress in building compelling,useful, and personalized information experiences for children will require new developmentsin evaluating their effectiveness.

ACM SIGIR Forum

157

Vol. 52 No. 1 June 2018

2.3

Discussion and Challenges

An initial break-the-ice session to present background and areas of expertise of attendees wasfollowed by an exchange of ideas resulting from presentations outlined in Section 2.2 and abrief presentation of research work currently being conducted at the Politecnico di Milano
focused on children with neuro-developmental disorders and the corresponding findings that
can inform design of RS for children. These activities lead to rich interchanges among attendees, the outcomes of which are summarized below.
Data Sets. There is a paucity of datasets for children. Unlike datasets like MovieLens[10] or Book-Crossing [20], which are available to the community at large and thus facilitate
assessment and comparison among recommendation strategies for a general audience; there
are no publicly available datasets that can be used to validate design and performance ofrecommendation strategies tailored to children. This lead us spotlight the need for this particular community to create and share a dataset that can encourage and facilitate research
discoveries. We discussed that gathering historical (i.e., logged) data from every day activities or collecting everyday routines–not just “usability” test data– might be the next step;
but at the same time large user studies can be time-consuming and difficult to carry out
when the studied population are children. In addition, surreptitious observations of children
can have ethical ramifications. In summary, we agreed that it is imperative that we createand share dataset for this population with regards to recommendation tasks, but, at the endof the day: What are the good ingredients for data sets for child recommendations? and,
What is the best way to gather this data?
Expertise. Building a community that can properly and holistically address the challengesof the task of interest for KidRec, requires looking at the problem from multiple perspectives.
Therefore, people with a diverse set of skills and areas of expertise must be a part of thiscommunity in order to address the breadth of issues related to children and RS. Among themost prominent areas discussed, we highlight: Human-computer Interaction (HCI); Information Retrieval (IR); user experience (UX); ethics; cognitive psychology; creativity, andeducational development. We discussed the need for interdisciplinary research among theseareas. For example, HCI and IR both strive to enable personalization, but from different
perspectives. Working together, these perspectives can strengthen the end result. The needto continue to grow this community is particularly important as there is such a wide arrayof expertise that can positively benefit RS for children.
Application Domains of Interest. Given that we are dealing with an emerging community, we focused some of our discussions on trying to identify the application domains ofinterest that could most benefit from algorithms, technologies, and strategies that facilitate
the decision making process for children. Immediately, RS that could complement educational tasks–specifically reading and learning–were at the top of the list. Other domains ofinterest included recommenders for entertainment (e.g. cartoons, comics, or movies), andpromoting healthy habits or the next “best” everyday activity to complete.
Future Challenges. Throughout the day we identified a number of challenges, which weoutline below, evidencing the need to continue to build community around RS for children.

ACM SIGIR Forum

158

Vol. 52 No. 1 June 2018• Which is the first target area or product we should focus on as a community? The
agreement in this case was that ideally, we should focus on a specific topic or item that
we could build upon. Books (and libraries) seemed like the best alternative as it crosses
into several domains (e.g., entertainment and education). The benefit of a shared target
area was that we could share data and information allowing us to collectively create
more effective solutions.
• How does the context of use effect RS usage? Adult and child expectations may change
based on the context of use. Are they using a RS at home or school? Are they using
them in a shared setting on a large display or individually such as on a mobile device?
• How does child development impact RS usage? Also, with regards to children, because
they are developing at a more rapid pace, preferences can change quickly. How can this
rapid change of preferences and needs be adequately addressed in the design of RS?
• How do we evaluate RS for children? With no datasets to compare, one alternative
is to conduct user studies. Unfortunately, these tend to involve small-size populations
and can sometimes not reflect real-world usage scenarios. Alternatively, if we indeed
focus on creating a dataset and conducting empirical analyses, then questions arise
pertaining to what type of assessment data to collect and how to properly collect it: do
we rely on the effectiveness of questionnaires or direct feedback? how reliable are these
options given that children might be less willing that mature counterparts in filling out
surveys or expressing their opinions? This phenomena is observed in adults, but how
is this magnified or skewed when dealing with children remains unanswered.
• In determining what is a relevant recommendation, are children’s opinions enough? Oneof the main challenges in creating RS for children pertains to the issue of “relevance”.
When it comes to more mature audiences past preference is one of the most influential factors in identifying items to recommend. However, when it comes to younger
audiences we wondered if that is still true: how do these recommenders balance what
adults (e.g., parents, educators, librarians) expect from recommended items for theirchildren versus the child’s wants and preferences?. In the end, how can these systems
generate “good” recommendations that simultaneously balance caregivers expectations
and children’s likes and dislikes?
• What are the ethical ramifications of RS for children? While listed here as the lastchallenge, ethics is definitely not an “add-on” issue. Ethical considerations are critically important, particularly when researching with children. RS for children raises
several challenging questions, including: Is monitoring children’s activities an invasionof privacy? What biases do RS have? How might that influence child behavior? How
ethical is it to persuade children? Whose educational, ethical, moral values do theserecommendations align with: the children, their parents or guardians, their teachers,
researchers, developers, companies, etc.? These questions and others must be addressed
as RS progressively target and can potentially alter children’s behaviors.

3

Future Workshop

Since such a broad diversity of perspectives are needed in considering recommendations forchildren, we discussed how to continue to build the community through future workshops at
appropriate venues where this kind of diversity would be embodied and embraced. Some of

ACM SIGIR Forum

159

Vol. 52 No. 1 June 2018

the possible venues included: ACM Recommender Systems (RecSys), Computer-Supported
Cooperative Work and Social Computing (CSCW), CHI-Play (the international and interdisciplinary conference), and Interaction Design and Children (IDC). This year, KidRec’s
second edition is set to occur at IDC 2018 in Trondheim, Norway. It is worth noting thatthe discussion presented in Section 2.3 was instrumental in helping us identify the “theme”
of KidRec 2018, which will be on RS in the educational domain.

4

Acknowledgments

We would like to thank the Program Committee members for their prompt and insightfulreviews, which directly impacted the level of the workshop discussions. We also thank theOrganizing Committee for RecSys 2017–especially the Workshop Chairs Marko Tkalcic andGiovanni Semeraro–for giving us the opportunity to host this workshop in conjunction withthe main RecSys 2017 conference. We would also thank the workshop presenters and attendees, who greatly contributed to discussion and helped build a foundation for future areasof work and topics of interests upon which this emerging community can continue to grow.
Information pertaining to 2017 KidRec, including list of accepted papers and the workshop
schedule, can be found at the KidRec 2017 workshop website: https://kidrec.github.io/2017/.

References[1] G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems:
A survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge
and data engineering, 17(6):734–749, 2005.
[2] A. N. Antle, J. L. Warren, E. S. Cramer, M. Fan, and B. Matkin. Designing tangibles
for children: One day hands-on workshop. In Proceedings of the 15th International
Conference on Interaction Design and Children, pages 726–730. ACM, 2016.
[3] Y. Deldjoo, C. Fr, M. Valla, M. A. Tuncel, F. Garzotto, P. Cremonesi, A. Paladini,
and D. Anghileri. Enhancing children’s experience with recommendation systems. In
International Workshop on Children & Recommender Systems. Available at: https://drive.google.com/open?id=0B8S_mL8gIjZweUp1aHIyQnViRVk, 2017.
[4] F. Delprino, O. F. Bravo, M. Mariani, C. Piva, N. Izzo, M. Matera, and
R. Tassi. Playing outdoor, recommending new content: Stimulating kids’ learning through the abbot smart object. In International Workshop on Children &Recommender Systems. Available at: https://drive.google.com/open?id=0B8S_
mL8gIjZwYnQ5UkF0N2l3dnM, 2017.
[5] I. Duytschaever and P. Conradie. Philosophy with children: Helping designers cooperatewith children. In Proceedings of the the 15th International Conference on Interaction
Design and Children, pages 736–741. ACM, 2016.
[6] M. Ekstrand. Challenges in evaluating recommendations for children. In International Workshop on Children & Recommender Systems. Available at: https://drive.
google.com/file/d/0B8S_mL8gIjZwVzZSLVZKN3Q4RG8/view, 2017.

ACM SIGIR Forum

160

Vol. 52 No. 1 June 2018

[7] M. D. Ekstrand, J. T. Riedl, J. A. Konstan, et al. Collaborative filtering recommendersystems. Foundations and Trends R in Human–Computer Interaction, 4(2):81–173,
2011.
[8] A. L. Gutnick, M. Robb, L. Takeuchi, and J. Kotler. Always Connected: The new
digital media habits of young children. Technical report, Joan Ganz Cooney Center.
[9] J. Gwizdka, P. Hansen, C. Hauff, J. He, and N. Kando. Search as learning (sal) workshop
2016. In Proceedings of the 39th International ACM SIGIR Conference on Research andDevelopment in Information Retrieval, pages 1249–1250. ACM, 2016.
[10] F. M. Harper and J. A. Konstan. The movielens datasets: History and context. ACM
Transactions on Interactive Intelligent Systems (TiiS), 5(4):19, 2016.
[11] D. Holloway, L. Green, and S. Livingstone. Zero to eight: Young children and theirinternet use. LSE London, EU Kids Online, 2013.
[12] H. K. Kabali, M. M. Irigoyen, R. Nunez-Davis, J. G. Budacki, S. H. Mohanty, K. P.
Leister, and R. L. Bonner. Exposure and Use of Mobile Media Devices by YoungChildren. Pediatrics, 136(6):1044–1050, Dec. 2015.
[13] M. Landoni and E. Rubegni. Aesthetic relevance when selecting multimedia stories. In
International Workshop on Children & Recommender Systems. Available at: https://drive.google.com/open?id=0B8S_mL8gIjZwdWNLN1R3a3pxUkU, 2017.
[14] P. Lops, M. De Gemmis, and G. Semeraro. Content-based recommender systems: Stateof the art and trends. In Recommender systems handbook, pages 73–105. Springer, 2011.
[15] W. Ma, M. Zhang, C. Zhang, Y. Chen, Q. Xie, W. Sun, Y. Liu, and S. Ma. A game-based
data collecting framework for the recommendation of kids’ second language learning. In
International Workshop on Children & Recommender Systems. Available at: https://drive.google.com/open?id=0B8S_mL8gIjZwcHlxUkdiMUZnYjA, 2017.
[16] N. Manouselis, H. Drachsler, R. Vuorikari, H. Hummel, and R. Koper. Recommendersystems in technology enhanced learning. In Recommender systems handbook, pages
387–415. Springer, 2011.
[17] M. S. Pera and Y.-K. Ng. Automating readers’ advisory to make book recommendationsfor k-12 readers. In Proceedings of the 8th ACM Conference on Recommender systems,
pages 9–16. ACM, 2014.
[18] M. Schedl and C. Bauer. Online music listening culture of kids and adolescents: Listening analysis and music recommendation tailored to the young. In International Workshop on Children & Recommender Systems. Available at: https://drive.google.com/
file/d/0B8S_mL8gIjZwd0ZwVTV5dy1PMWs/, 2017.
[19] Y. Zheng, O. Santos, M. S. Pera, R. Burke, and G. Guo. 3rd international workshop
on educational recommender systems (edrecsys). Workshop IEEE/WIC/ACM International Conference on Web Intelligence. Available at: https://edrecsys.wordpress.
com/organization/, 2016.
[20] C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen. Improving recommendation
lists through topic diversification. In Proceedings of the 14th international conferenceon World Wide Web, pages 22–32. ACM, 2005.

ACM SIGIR Forum

161

Vol. 52 No. 1 June 2018