To Block or not to Block: Experiments with Machine Learning for News Comment Moderation (2021)

Abstract: Today, news media organizations regularly engage with readers by enabling them to comment on news articles. This creates the need for comment moderation and removal of disallowed comments – a time-consuming task often performed by human moderators. In this paper we approach the problem of automatic news comment moderation as classification of comments into blocked and not blocked categories. We construct a novel dataset of annotated English comments, experiment with cross-lingual transfer of comment labels and evaluate several machine learning models on datasets of Croatian and Estonian news comments. Team name: SuperAdmin; Challenge: Detection of blocked comments; Tools/models: CroSloEn BERT, FinEst BERT, 24Sata comment dataset, Ekspress comment dataset.

## introduction comment sections are an important part of news sites, providing an opportunity for newsrooms to engage with their audience. comment moderation aims to safeguard respectful conversation by blocking comments that are uncivil, disruptive or potentially unlawful. this is a complex task that balances legal implications and editorial guidelines. common categories of blocked comments include: unsafe or illegal content (ex. defamation or hate speech), disruptive content (ex. trolling), advertisements, and copyrighted content @xcite . while newsrooms are becoming increasingly aware of the benefits provided by artificial intelligence and expect comment moderation to become more manageable, implementation of ai solutions is far from prevalent (society of @xcite . some newsrooms use custom automated comment moderation solutions developed inhouse or third-party plugins to complement human moderation. others rely on external companies that provide comment moderation performed by teams of contracted moderators @xcite @xcite . for most in-house and third-party solutions, the extent of use and details of the machine learning solutions are not publicly revealed. the stand-out third-party option is perspective, foot_0 a free api developed by google's jigsaw, available in seven highresourced languages @xcite . to the best of our knowledge, there are no machine learning solutions suitable for comment moderation for underresourced languages. in the academic literature, the problem of comment moderation is commonly approached as a binary classification of comments into blocked and not blocked categories @xcite @xcite . in this paper, which reports the work done during the embeddia hackashop hackathon foot_1 @xcite , we approach the problem in the same manner and perform experiments with comment classification on datasets of croatian and estonian news comments @xcite . motivated by the lack of an english dataset of comments labelled as either blocked or not blocked, we construct such a dataset from existing datasets of news and social media comments. we then experiment with the cross-lingual transfer of english labels to croatian and estonian comment datasets by means of a multilingual bert model @xcite . finally, we construct and evaluate several classification models trained on croatian and estonian datasets, analyze the results, and discuss the problem of automatic detection of blocked comments. we make the source code of the experiments freely available. 3 2 related work computational comment moderation includes tasks such as offensive language detection @xcite and blocked comment detection @xcite @xcite , which is the focus of our study. most of the prior studies on comment filtering tackle the problem using text from highresourced languages such as english @xcite and german @xcite . there are only a few studies that focus on low-resourced languages @xcite . the methods for comment filtering vary from classical machine learning methods to deep learning approaches. @xcite classify comments with a logistic regression classifier using features computed from comments, news articles, and users. deep neural networks such as rnn and cnn have also been applied @xcite . most recently, @xcite leverage multilingual bert (mbert) @xcite for the moderation of news comments in balto-slavic languages. ## english dataset for comment moderation there are multiple publicly available datasets in english with annotated comments that have been used in previous research about comment moderation. however, most of these datasets contain annotations of only a subset of the categories of blocked comments @xcite . we construct a large corpus of comments containing different categories of blocked comments by unifying different datasets and defining a new label. since comments in these datasets are not explicitly labeled as blocked, we created the flagged and not flagged labels instead. the idea is to identify comments that moderators should review and decide whether to block them or not. the flagged label therefore serves as an approximation of the blocking decision and classifiers that detect it automatically have the potential to save time and human effort. ## automatic comment moderation experiments next, we construct and evaluate classifiers that aim to detect blocked news comments. we experiment with embeddia multilingual bert models (ulcar and robnik-sikonja, 2020) fine-tuned for classification and with standard non-neural classifiers using n-gram features. ## discussion automatic detection of blocked comments of the croatian and the estonian dataset is a hard problem. this claim is supported by modest f 1 scores and sharp precision/recall tradeoffs observed both in our experiments and in the experiments of @xcite . while inclusion of non-textual comment features would probably lead to better results @xcite , we hypothesize that the main problem is the poor quality of comment labelling. the definition of sensible text categories and consistent annotation of texts with these categories falls within the domain of content analysis @xcite . ideally, the category definitions are discussed and fine-tuned, and the measure of inter-annotator agreement (iaa) is reported. in the case of the blocked comment detection, the precise process of category definition is unknown @xcite @xcite , while the iaa is either not available @xcite , or modest @xcite . moreover, there are indications of inconsistencies in the definition of a blocked comment class. @xcite report that the varying blocking rates are probably caused by changes in moderation policy. @xcite and @xcite report that a high influx of user comments, for example during high-interest events, causes more strict comment blocking. the mentioned problems should be tackled since the consistent labelling of the comments is key to building highquality classifiers. the binary classification approach might be in disconnect with the true needs of the comment moderators. an engineering perspective of a machine learning system can significantly differ from the end user's perspective @xcite . we believe that studies including comment moderators are essential in order to define and evaluate the appropriate solution. for example, the amount of moderators' time saved might prove as a useful metric, and the best application of classifiers might not be automatic blocking but flagging and pre-filtering of comments. additionally, moderators operate within boundaries set by in-house rules and practices and legal regulations. an investigation of the nature and impact of such restrictions would provide perspective on the role of automatic comment moderation. for example, in a scenario where the publisher can be held accountable for the comments containing hate speech, any automatic classifier would be required to achieve very high recall. ## conclusion and future work we plan to further develop the dataset of flagged english comments, experiment with other classification models and to improve the bert-based language transfer models. we also plan to examine multi-task learning approaches that can lead to state-of-art results on transferring knowledge among related tasks @xcite . we believe that more attention should be paid to the problem of comment labelling. this could lead to better classifiers, reliable inter-annotator agreement scores that can serve as upper bounds on performance, and to a better understanding of the semantics of the composite category of blocked comments. in our view, an essential future work direction is design and implementation of studies with comment moderators that examine real-world scenarios and user needs. we believe that such studies would be invaluable and would lead to more realistic and usable machine learning comment moderation tools.