Detecting E ast A sian Prejudice on Social Media (2020)

Abstract: During COVID-19 concerns have heightened about the spread of aggressive and hateful language online, especially hostility directed against East Asia and East Asian people. We report on a new dataset and the creation of a machine learning classifier that categorizes social media posts from Twitter into four classes: Hostility against East Asia, Criticism of East Asia, Meta-discussions of East Asian prejudice, and a neutral class. The classifier achieves a macro-F1 score of 0.83. We then conduct an in-depth ground-up error analysis and show that the model struggles with edge cases and ambiguous content. We provide the 20,000 tweet training dataset (annotated by experienced analysts), which also contains several secondary categories and additional flags. We also provide the 40,000 original annotations (before adjudication), the full codebook, annotations for COVID-19 relevance and East Asian relevance and stance for 1,000 hashtags, and the final model.

## introduction the outbreak of covid-19 has raised concerns about the spread of sinophobia and other forms of east asian prejudice across the world, with reports of online and offline abuse directed against east asian people, including physical attacks @xcite @xcite @xcite . the united nations high commissioner for human rights has drawn attention to these issues, calling on un member states to fight the 'tsunami' of hate and xenophobia @xcite . as digital technologies become even more important for maintaining social connections, it is crucial that online spaces remain safe, accessible and free from abuse @xcite -and that people's fears and distress are not exploited during the pandemic. computational tools, including machine learning and natural language processing, offer powerful ways of creating scalable and robust systems for detecting and measuring prejudice. these, in turn, can assist with both online content moderation processes and further research into the dynamics, prevalence, causes, and impact of abuse. we report on the creation of a new dataset and classifier to detect east asian prejudice in social media data. the classifier distinguishes between four primary categories: hostility against east asia, criticism of east asia, meta-discussions of east asian prejudice, and non-related. it achieves a macro-f1 score of 0.83. the 20,000 tweet training dataset used to create the classifier and the annotation codebook are also provided. the dataset contains annotations for several secondary categories, including threatening language, interpersonal abuse, and dehumanization, which can be used for further research. in addition, we provide the 40,000 original annotations given by our experienced annotators, which can be used for further investigation of annotating prejudice. we also annotated 1,000 hashtags in our dataset for east asian relevance and stance, as well as other attributes. these are also provided for other researchers. 1 to provide insight into what types of content causes the model to fail, we conduct a ground-up qualitative error analysis. we show that 17% of errors are due to annotation mistakes and 83% due to the machine learning model. of these, 29% are clear errors (i.e. obvious mistakes) and 54% are edge cases (i.e. more complex and nuanced cases). in the machine learning edge cases, we show that the model struggles with lexically similar content (e.g. distinguishing hostility against east asia from criticism of east asia), as well as ambiguous content (i.e. where there is uncertainty among the annotators about the correct label). finally, we analyze the hashtags most closely associated with the primary categories in the training dataset, identifying several terms which could guide future work. ## background east asian prejudice, such as sinophobia, can be understood as fear or hatred of east asia and east asian people @xcite . this prejudice has a long history in the west: in the 19th century the term "yellow peril" was used to refer to chinese immigrants who were stereotyped as dirty and diseased and considered akin to a plague @xcite . often, the association of covid-19 with china plays into these old stereotypes, as shown by derogatory references to 'bats' and 'monkeys' online @xcite . in 2017 a study found that 21% of asian americans had received threats based on their asian identities, and 10% had been victims of violence @xcite . likewise, a 2009 report on the discrimination and experiences of east asian people in the uk, described sinophobia as a problem that was 'hidden from public view' @xcite . new research related to east asian prejudice during covid-19 has already provided insight into its nature, prevalence and dynamics with @xcite finding an increase in sinophobic language on some social media platforms such as 4chan. analysis by the company moonshot cve also suggests that the use of anti-chinese hashtags has increased substantially (the new statesman, 2020). they analysed more than 600 million tweets and found that 200,000 contained either sinophobic hate speech or conspiracy theories, and identified a 300% increase in hashtags that supported or encouraged violence against china during a single week in march 2020. similarly, @xcite show the existence of a sinophobic 'hate multiverse', with hateful content following contagion patterns and clusters which are similar to the epidemiological diffusion of covid-19 itself. @xcite argue that racism 'is a virus', and study a dataset of 30 million covid-19 tweets using a classifier trained on 2,400 tweets. they identify 900,000 hateful tweets and 200,000 counter-hate, and find that 10% of users who share hate speech are bot accounts. @xcite used the word embedding association test in the context of covid-19 to analyse anti-china sentiment on twitter, finding substantial biases in how asian people are viewed. east asian prejudice has also been linked to the spread of covid-19 healthrelated misinformation @xcite and in march 2020 the polling company yougov found that 1 in 5 brits believed the conspiracy theory that the coronavirus was developed in a chinese lab @xcite . research into computational tools for detecting, categorizing, and measuring online hate has received substantial attention in recent years @xcite . however, a systematic review of hate speech training datasets conducted by @xcite shows that classifiers and training datasets for east asian prejudice are not currently available. somewhat similar datasets are available, pertaining to racism @xcite islamophobia @xcite and 'hate' in general @xcite but they cannot easily be re-purposed for east asian prejudice detection. the absence of an appropriate training dataset (and automated detection tools) means that researchers have to rely instead on less precise ways of measuring east asian prejudice, such as using keyword searches for slurs and other pejorative terms. these methods create substantial errors @xcite as more covert prejudice is missed because the content does not contain the target keywords, and non-hateful content is misclassified simply because it does contain a keyword. developing new detection tools is a complex and lengthy process. the field of hate speech detection sits at the intersection of social science and computer science and is fraught with not only technical challenges but also deep-routed ethical and theoretical considerations @xcite . if machine learning tools are to be effective then they need to be developed with consideration of their social implications @xcite @xcite . ## dataset collection to create our 20,000 tweet training dataset, we collected tweets from twitter's streaming api using 14 hashtags that relate to both east asia and the novel coronavirus. 2 some of these hashtags ex-press anti-east asian sentiments (e.g. '#chinaflu') but others, such as '#wuhan' are more neutral, referring to the geographic origin of the virus. data collection ran initially from 11 to 17 march 2020, returning 769,763 tweets, of which 96,283 were unique entries in english. to minimize biases that could emerge from collecting data over a relatively short period of time, we then collected tweets from 1 january to 10 march 2020 using the same keywords from a 10% random sample of twitter (the 'decahose'), provided by a third party. we identified a further 63,037 unique tweets in english, which we added to our dataset. the full dataset comprises 159,320 unique tweets. to create a training dataset for annotation we sampled from the full dataset. to guide the sampling process, we extracted the 1,000 most used hashtags from the 159,320 tweets. three annotators independently marked them for: (1) whether they are east asian relevant and, if so, (2) what asian entity is discussed (e.g., china, xi jinping, south korea), ( 3 ) what the stance is towards the asian entity (very negative, negative, neutral, positive, or very positive) and also (4) whether they relate to covid-19. 97 hashtags were marked as either negative or very negative toward east asia by at least one annotator. all annotations for hashtags are available in our data repository. we then sampled 10,000 tweets at random from the full dataset and a further 10,000 tweets which used one of the 97 hashtags identified as negative or very negative towards east asia, thereby increasing the likelihood that prejudicial tweets would be identified and ensuring that our dataset is suitable for training a classifier @xcite . the training dataset comprises 20,000 tweets in total. ## classification results with discussion of east asian prejudice for classification. as such, the classification task was to distinguish between four primary categories: hostility, criticism, discussion of east asian prejudice and non-related. we implemented and fine-tuned several contextual embedding models as well as a one-hot lstm model with a linear input layer, tanh activation, and a softmax output layer. we expect contextual embeddings to perform best as they take into account the context surrounding a token when generating each embedding @xcite . we compared results against a one-hot lstm model to test this expectation. models were developed with a stratified 80/10/10 training, testing, and validation split (maintaining the class distribution of the whole dataset). we processed all tweets by removing urls and usernames, lower-casing, and replacing hashtags with either a generic hashtag-token or with the appropriate thematic hashtag-token from the annotation setup. training was conducted using the same hyper-parameter sweep identified in @xcite as most effective for the glue benchmark tasks. this included testing across learning @xmath0, 2e-5, 3e-5} and batch @xmath1, 64} with an early stopping regime. performance was optimized using the adamw algorithm @xcite and a scheduler that implements linear warmup and decay. for the lstm baseline, we conduct a hyper-parameter search over batch @xmath2, 32, 64} and learning rates {10 -i , @xmath3, 5 increments}. all of the contextual embedding models outperformed the baseline in terms of macro f1. roberta achieved the highest f1 score of the tested models (0.832), which is a 7-point improvement over the lstm (0.76). this model harnesses the underlying bidirectional transformer architecture of bert @xcite but alters the training hyperparameters and objectives to improve performance. unexpectedly, the lstm baseline outperforms all other models in terms of precision but has far lower recall. for the best performing model (roberta), misclassifications are shown in the confusion matrix. the model performs well across all categories, with strongest performance in detecting tweets in the non-related category (recall of 91.6% and precision of 93%). the model has few misclassifications between the most conceptually distinct categories (e.g. hostility versus non-related) but has far more errors between conceptually similar categories, such as criticism and hostility. ## error analysis to better understand classification errors, we conducted a qualitative analysis of misclassified content (from the best performing model, roberta), using a grounded theory approach @xcite . this qualitative methodology is entirely inductive and data-driven. it involves systematically exploring themes as they emerge from the data and organizing them into a taxonomyrefining and collapsing the categories until 'saturation' is reached and the data fits neatly into a set of mutually exclusive and collectively exhaustive categories. figure 2 shows the error categories within our sample of 340 misclassified tweets from the 2,000 (10%) validation split. the errors broadly fit within two branches: annotator errors (17%) and machine learning errors (83%). in future work, these errors could be addressed through creating a larger and more balanced dataset, more sophisticated machine learning architectures and reannotation of data. ## hashtag analysis we analysed the 20,000 tweet training dataset to understand which hashtags were associated with which primary categories. for each category, we filtered the data so only hashtags which appeared in at least ten tweets assigned to that category were included. then, we ranked the hashtags by the percentage of their uses which were in the primary category-our goal being to understand which hashtags are most closely associated with that category. for brevity, only the twenty hashtags most closely associated with the hostility category are shown in a small number of hashtags are highly likely to only appear in tweets that express hostility against east asian entities. these hashtags could be used to filter for prejudiced discourses online and, in some cases, their uses may intrinsically indicate prejudice. surprisingly, many seemingly hostile hashtags against east asia, such as #fuckchina and #blamechina are not always associated with hostile tweets (hostility accounts for 67.5% and 60.5% of their total use, respectively). this shows the importance of having a purpose-built machine learning classifier for detecting east asian hostility, rather than relying on hashtags and keywords alone. ## conclusion prejudice of all forms is a deeply concerning problem during covid-19, reflecting the huge social costs and difficulties that the pandemic has inflicted. in this paper we have reported on development of several research artefacts that we hope will enable further research into east asian prejudice, including a trained classifier, a training dataset (20,000 entries), annotations dataset (40,000 entries), 1,000 annotated hashtags, a list of hashtag replacements, a list of hashtags associated with hostility against east asia, and the full codebook (with extensive guidelines, information and examples). one concern with any model is that it will not generalize to new settings and platforms, limiting its utility for real-world applications. our hashtag replacement method was adopted to increase the generalizability of the final model, maximizing the likelihood that it can be applied to new contextsas it would pick up on the semantic features of hostile tweets rather than specific tokens. nonetheless, we caution that any re-use of the model should be accompanied by additional testing to understand its performance on different data.