The Elsagate Corpus: Characterising Commentary on Alarming Video Content (2024)

Abstract: Identifying disturbing online content being targeted at children is an important content moderation problem. However, previous approaches to this problem have focused on features of the content itself, and neglected potentially helpful insights from the reactions expressed by its online audience. To help remedy this, we present the Elsagate Corpus, a collection of over 22 million comments on more than 18,000 videos that have been associated with disturbing content. We describe the how we collected this corpus and present some insights from our initial explorations, including the surprisingly positive reactions from audiences to this content, some unusual non-linguistic commenting behavior of uncertain purpose and references to some concerning themes.

## introduction the topic of elsagate is one of the most important problems that has emerged recently in online content moderation. the term, which has attracted major media attention @xcite and research interest @xcite @xcite @xcite , refers to the widespread distribution of inappropriate and disturbing content aimed at children across multiple online channels, such as video-sharing websites and social networking platforms. these videos frequently incorporate popular children's characters, such as the titular elsa (from the disney movie frozen), but they juxtapose these child-friendly elements with disturbing or harmful themes such as violence, sexual innuendos, and graphic imagery. youtube's algorithm often recommends these forms of inappropriate content to children, since at a superficial content level the videos can be similar to otherwise appropriate content @xcite . however, the phenomenon is more than a misfiring of content recommendation systems. the makers of this content exploit popular keywords and tags to attract innocent young viewers @xcite , thereby potentially causing psychological and emotional harm @xcite . there are also more recent references to inappropriate content on youtube (tech transparency project, 2022; hern, 2022) which shows that content like this still exists in the platform @xcite . this paper discusses the creation and initial explorations of a corpus consisting of comments extracted from youtube videos that have been identified as elsagate content. while previous work on elsagate content has focused on its detection as a computer vision problem, youtube comments provide valuable linguistic insights into forms of user engagement with videos. our primary interest in this corpus is as a resource that could be used to help automated systems identify future inappropriate content, either on youtube or in similar online spaces, as we expect the pattern of reactions to elsagate content to be distinctive when compared to reactions to genuine child-appropriate content. however, this corpus may also provide valuable broader insight into the variable nature of user engagement with disturbing content, and later in this paper we detail several surprising features of our dataset, including unusual non-linguistic commenting behaviour which has not previously been described. ## related work while most research targeting youtube focuses on either sentiment analysis or hate speech detection, since the rise of the elsagate phenomenon in 2016, there has been a shift towards detecting disturbing content @xcite . previous attempts at identifying this content have employed image or video data for their analyses. the first attempts at categorisation of videos on youtube kids took place before the emergence of the elsagate phenomenon, using a combination of computer vision with deep learning @xcite to categorise videos as benign, explicit or violent. @xcite were the first to discuss the elsagate phenomenon as a distinctive online risk, proposing a deep learning detection mechanism derived from the pornography detection literature. @xcite presented the first characterisation of disturbing videos targeted at kids by developing a highly accurate deep learning classifier finding that 8.6% of the videos in their dataset were inappropriate but still recommended for toddlers. @xcite used a deep learning-based approach to detect inappropriate children's content from youtube. in later work, the same authors use a bilstm network for disturbing video content multiclass classification @xcite . @xcite extend the previous video-based approaches to building a detection mechanism for channels creating disturbing content. most recently, textual content been employed to assist detection mechanisms, with @xcite using subtitle features alongside image data and video metadata to assist in classification. however, no previous approach has considered the reaction expressed by commenters as a possible means of detecting or understanding elsagate material. ## corpus description our corpus collection was grounded in previous work that had identified specific youtube channels or videos as disturbing content fitting the description of elsagate material. @xcite provided a list of 33 channels that produce elsagate content, sourced from a subreddit devoted to tracking this material. after identifying content from the r/elsagate using specific keywords they also collected a random sample of the 500 most popular videos uploaded between 18/11/2018 and 2/11/2018 in united states, great britain, russia, india, and canada. @xcite separately provided a list of videos from 80 channels that produce ageinappropriate content, as determined by reference to youtube and ftc guidelines. their categorization encompassed a wide range of content either visual or linguistic that may be deemed inappropriate, including classic cartoons edited with in-appropriate text or visuals, adult gaming content, adult cartoons, toy destruction videos, deceptive channels targeting children and family channels demonstrating child abuse coming from four annotators. as many videos and channels examined in previous research have been removed due to previous reporting, and new content is still being created, we first gathered all still-accessible videos from these sources, and then updated our list using the methodology described by @xcite , collecting new video ids reported on the /r/elsagate subreddit. in total, our collection covers comments on videos from 53 active channels that have been associated with elsagate-style content. out of the 25,861 video ids identified from these channels, we extracted comment data from 18,324 (71%). the remainder reflects videos identified in previous research that have since been taken down, videos with comment sections disabled, and videos that had no comments. for these 18,324 videos, we used the youtube api to extract video metadata and all associated comments. to protect user privacy, we anonymised any personally identifiable information. in total, we acquired 22,849,726 comments produced by 7,591,907 unique users. ## conclusion our large dataset of comments on videos associated with disturbing content contains a variety of behaviours, with a range including highly positive audience engagement, spam, expressions of discomfort with content, and non-linguistic comments that serve no immediately evident purpose. our analysis to date covers only an initial exploration of this corpus, and we anticipate that it may prove useful to understanding and preventing the spread of disturbing content, both alone and in conjunction with other resources. of particular interest is the challenge posed by distinguishing content that is directed at children. it is crucial to assess the engagement of various groups including children, adults and threat actors in the comment sections of these videos. elsagate observers worry about many risks posed by this content, including psychological harm to young children. this language resource sets a foundation for further linguistic studies of reactions to elsagate content, and provides a first step towards developing language-related technologies that ensure a safer digital space.