Not All Comments Are Equal: Insights into Comment Moderation from a Topic-Aware Model (2021)

Abstract: Moderation of reader comments is a significant problem for online news platforms. Here, we experiment with models for automatic moderation, using a dataset of comments from a popular Croatian newspaper. Our analysis shows that while comments that violate the moderation rules mostly share common linguistic and thematic features, their content varies across the different sections of the newspaper. We therefore make our models topic-aware, incorporating semantic features from a topic model into the classification decision. Our results show that topic information improves the performance of the model, increases its confidence in correct outputs, and helps us understand the model’s outputs.

## introduction most newspapers publish their articles online, and allow readers to comment on those articles. this can increase user engagement and page views, and provides readers with an important route to public freedom of expression and opinion, with the ability to interact and discuss with others. comment sections usually provide some degree of anonymity; 1 while improving accessibility, this can also encourage inappropriate behaviour, and publishers therefore usually employ some moderation policy to regulate content and to ensure legal compliance (in some cases, publishers can be held responsible for user-contributed content on their sites). one possible approach is a 'moderate then publish' policy, in which comments must be approved by a moderator before they appear; this requires significant manpower and introduces delays and limitations into the user conversation (for example, the new york times only allows comments for 1 some newspapers allow completely anonymous posting; some require commenters to create an account with a username, but this does not usually reveal their true identity. detecting comments that need moderators' attention is usually approached as a text classification task (see e.g. @xcite ; but comments can be blocked for a range of reasons @xcite . one is the presence of offensive language, a well-studied nlp task (see section 2 below); however, others include advertising or spam, illegal content, spreading misinformation, trolling and incitement -all distinct categories which might be expected to show distinct features, and perhaps to vary according to the content being commented on. another aspect that distinguishes the comment moderation task from the usual text classification tasks in nlp is the need for interpretable or explainable models: if classifiers are to be used by human moderators within publishers' working practices, they must be able to understand the outputs @xcite . here, we therefore investigate models which can provide both an aspect of interpretability and the ability to take account of the topics being discussed, by incorporating topic information into the comment classifier. specifically, we incorporate semantic representations learned by the embedded topic model (etm) @xcite into a classifier pipeline based on long short-term memory (lstm) networks @xcite . our model improves performance by 4.4% over a text-only approach on the same dataset @xcite , and is more confident in the correct decisions it makes. inspection of the topic distributions reveals how different newspaper sections have different language and topic distributions, including differences in the kind of comments that need moderation. et al. (2017a) and švec et al. (2018) use neural networks, specifically rnns with an attention mechanism. recently, @xcite and @xcite apply a modified bert model @xcite while @xcite use a bag-of-words approach. some approaches go beyond the comment text itself: @xcite add information like user id and article headline into their rnn to make the model context-aware; @xcite incorporate user embeddings; schabus and skowron (2018) incorporate the news category metadata of the article. however, no work so far investigates automatic modelling of topics (rather than relying on categorical metadata), or applies this to the comments rather than just their parent articles. some steps towards model intepretability and output explanation have also been taken: both @xcite and @xcite use an attention saliency map to highlight possibly problematic words. however, we are not aware of any work using higher-level topic information as a route to understanding model outputs. available datasets several datasets have been created for the news comment moderation task. @xcite provide 1.43m comments posted on yahoo! finance and news over 1.5 years, in which 7% of the comments are labelled as abusive via a community moderation process. @xcite contains 1.5k comments from fox news, annotated with specific hateful/non-hateful labels as a post-hoc task, and having 28% hateful comments. however, both are relatively small, and their labelling methods mean that neither dataset is entirely representative of the moderation process performed by newspapers. @xcite provides 1.6m comments from gazzetta, a greek sports news portal, over c.1.5 years. here, 34% of comments are labelled as blocked, and the labels are derived from the newspaper's human moderators and journalists. @xcite and @xcite provide a dataset from a german-language austrian newspaper with 1m comments posted over 1 year, out of which 11,773 comments are annotated using seven different rules. more recently, @xcite present a dataset from 24sata, croatia's most widely read newspaper. 4 this dataset is significantly larger (10 years, c.20m comments); and moderator labels include not only a label for blocked comments, but also a record of the reason for the decision according to a 9-class moderation policy. however, their experiments show that classifier performance is limited, and transfers poorly across years. here, we therefore use this dataset (see section 3), with a view to improving performance and applying a topic-aware model to improve and better understand the robustness in the face of changing topics. related tasks more attention has been given to related tasks, most prominently the detection of offensive language, hate speech, and toxicity @xcite . a comprehensive survey of dataset collection is provided by @xcite and @xcite . 5 topic modelling topic models capture the latent themes (also known as topics) from a collection of documents through the co-occurence statistics of the words used in a document. latent dirichlet allocation (lda) @xcite , a popular method for capturing these topics, is a generative document model where a document is a mixture of topics expressed as a probability distribution over the topics and a topic is a distribution over the words in a vocabulary. the embedded topic model @xcite ) is an lda-like topic modelling method that exploits the semantic information captured in word embeddings during topic inference. the advantage of etm over lda is that it combines the advantages of word embeddings with the document-level dependencies captured by topic modelling and has been shown to produce more coherent topics than regular lda. ## dataset we use the 24sata comment dataset @xcite , introduced in section 2. this contains c.21m comments on 476k articles from the years 2007-2019 foot_4 , written in croatian. the dataset has details of comments blocked by the 24sata moderators, based on a set of moderation rules-these vary from hate speech to abuse to spam (see @xcite , for rule description). the dataset also identifies the article under which a comment was posted, together with the section/subsection of the newspaper the article appeared in. these sections/sub-sections relate to the content of the article: for example, the sport section contains sports-related news while the kolumne (columns) section contains opinion pieces. the largest section, vijesti (news), is further subdivided as shown in ## topic modelling we now apply a topic model to gain insight into what characterises a blocked comment and a nonblocked one, and whether this varies between different sections where different subjects are discussed. ## topic-aware classifier our aim is to improve comment moderation predictions by combining textual features with documentlevel semantic information in the form of topics. to this end, we test several model architectures that combine a language model with topic features. for the comment text representation, we use a bidirectional lstm (bilstm, schuster and paliwal, 1997). the comment text is given as input to an embedding layer then a bilstm layer where the output of the final hidden state is taken as the encoded representation of the comment. for the topic representations, we use the topic distributions (dtd) and topic embeddings (dte) discussed in section 4.1. we propose two fusion mechanisms to combine the text and topic representations: early and late fusion. in early fusion, topic features are concatenated with the output of the embedding layer and then passed to the bilstm layer. in earlyfusion1 (ef1), only dtd is concatenated with the word embeddings; earlyfusion2 (ef2) uses dte instead of dtd; and earlyfusion3 (ef3) uses both dte and dtd. in late fusion, topic features are concatenated with the output representation of the bilstm layer, and passed to the mlp for classification. again, latefusion1 (lf1) uses dtd; latefusion2 (lf2) uses dte; and latefusion3 (lf3) uses both. figure 3 shows the architectures. our model is inspired by the topic compositional neural language model (tcnlm, @xcite and the neural composite language model (nclm, chaudhary et al., 2020) that incorporate latent document-topic distributions with language models. both of these models simultaneously learn a topic model and a language model through a joint training approach. the nclm introduced the use of word embeddings to generate an explanatory topic representation for a document in addition to the document-topic proportions. in our work, instead of using the word embeddings of the top words of the latent topics of a document (where the number of top words is a hyperparameter), we leverage the topic embeddings learned by etm and combine them with the document-topic proportions to produce the document-topic embeddings (dte). also unlike the tcnlm and nclm, we use pre-trained topics in our model so as to easily de-couple and analyse the influence of topics in the classifier performance. another related work is @xcite , a model that uses topic proportions to re-score the words generated by the language model. the topics generated by this model, however, have been shown to have lower coherences compared to nclm @xcite . ## experimental setup dataset as discussed in section 3.1, we use the 2018 data as the training and validation sets of our topic-aware classifier and the 2019 data as the test set. details of the train and validation sets are shown in baseline models to assess how topic information improves comment classification, we use as baselines the following models trained only on text or topics: hyperparameters we use 300d word2vec embeddings, pretrained on the croatian web corpus (hrwac, @xcite , for training the etm and to initialize the embedding layer of the bilstm. the etm is trained for 500 epochs for 100 topics using the default hyperparameters from the original implementation foot_6 . the bilstm is composed of one hidden layer of size 128 with dropout set to 0.5. the mlp classifier is composed of one fully-connected layer, one hidden layer of size 64, a relu activation, and a sigmoid for classification with the classification threshold set to 0.5. we use adam optimizer with @xmath0.005. we train all classifiers for 20 epochs with early stopping based on the validation loss. ## conclusion in this work, we propose a model to combine document-level semantics in the form of topics with text for comment moderation. our analysis shows that blocked and non-blocked comments have different linguistic and thematic features, and that topics and language use vary considerably across news sections, including some variation in the comments that should be blocked. we also found that blocked comments tend to be more semantically coherent across sections than nonblocked ones. we therefore see that the use of topics in our model improves performance, and gives more confident outputs, over a model that only uses the comment text. the model also provides topic distributions, interpretable as keywords, as a form of an explanation of its prediction. as future work, we plan to incorporate comment, article, and user metadata into the model.