Hostility Detection in UK Politics: A Dataset on Online Abuse Targeting MP s (2025)

Abstract: Social media platforms, particularly X, enable direct interaction between politicians and constituents but also expose politicians to hostile responses targetting both their governmental role and personal identity. This online hostility can undermine public trust and potentially incite offline violence. While general hostility detection models exist, they lack the specificity needed for political contexts and country-specific issues. We address this gap by creating a dataset of 3,320 English tweets directed at UK Members of Parliament (MPs) over two years, annotated for hostility and targeted identity characteristics (race, gender, religion). Through linguistic and topical analyses, we examine the unique features of UK political discourse and evaluate pre-trained language models and large language models on binary hostility detection and multi-class targeted identity type classification tasks. Our work provides essential data and insights for studying politics-related hostility in the UK.

## introduction with the rise of social media use among politicians, especially on x, there has been an increase in direct interaction with the public @xcite . this interaction, while beneficial for communication and feedback, also exposes politicians to a significant number of hostile replies due to the anonymity of online platforms @xcite . such hostility is considered a major concern as it erodes public trust in political processes and institutions, which disrupts constructive communication @xcite . furthermore, it affects the personal lives and mental health of politicians, with online abuse sometimes leading to real-world threats and violence @xcite . in extreme cases, sustained hostility has driven politicians to step down from their roles and retreat from public life altogether @xcite . hostility targeting politicians is a global phenomenon characterised by widespread misogyny, sexism, and racism. political and social science research indicates that all politicians receive hostility, but those from minority groups (e.g., black, female, lgbtq+) often face increased hostility based on their identity characteristics @xcite . in nlp, sentiment analysis tools have been used to identify negative posts and facilitate studies on abuse trends @xcite . although general hostility detection is prevalent, identifying political hostility requires specialised approaches as political discussions often reflect a country's unique linguistic and cultural characteristics, incorporating regional colloquialisms, profanity and prejudices. for example, hostility towards people of colour is more prevalent in the us @xcite , while the phenomenon of islamophobia is more severe in india @xcite . furthermore, hostile posts are frequently tied to trending issues. as the body of work on hate speech, abuse and hostility detection in nlp grows @xcite , there has been a move towards developing resources specifically for political hate speech detection across different countries @xcite . in the uk, members of parliament (mps) represent a wide range of backgrounds, and this diversity is mirrored in the nature of the abusive comments they receive @xcite . studies have compiled datasets to analyse abuse trends specific to uk politics, though these datasets are not publicly available @xcite . while existing political datasets are available, only two contain hostility-related labels potentially usable for automated detection: @xcite , with manual annotations, and @xcite , which relies entirely on automated labels without manual verification, limiting its reliability. a third dataset focuses solely on islamophobia in uk politics @xcite , a specific form of identity-based hostility, which restricts its broader applicability. none comprehensively capture identity-based hostility. we aim to bridge this gap by constructing a highquality hostility dataset spanning a two-year period to cover diverse political topics in the uk. our main contributions are: our work is distinctive in creating a dataset specifically for training models to detect identitybased political hostility towards uk mps. through topic analysis, we show how political hostility correlates with current events, which has crucial implications for model training @xcite . our analysis reveals that the governing party faces proportionally more hostility, with race-based attacks being most prevalent. the dataset's two-year span offers greater topic diversity and generalisability than existing datasets, while uniquely capturing intersectional hostility through identity characteristic labels-a particularly harmful form of online hostility @xcite . ## online hostility detection we finetune plms for political hostility detection to test how they perform on our dataset. we also evaluate the ability of llms to identify political hostility on our dataset, demonstrating its value. given a text snippet, we define online hostility detection as two classification tasks: (1) binary hostility classification (if a tweet contains hostility or not) and (2) multi-class classification to see if it contains one of the four identity-based hostility types (religion, gender, race, none) or no hostility at all. for multi-class classification, we use a two-level hierarchical classification method. 5 the first classifiers classify tweets as hostile or not, while the second classifiers then classify the identity types of those identified as hostile. ## conclusion this work focuses on the creation of data for investigating online hostility towards uk politicians. we developed an english dataset of 3,320 tweets, manually annotated with hostility as well as targeted identity characteristics: religion, gender, and race. we also conducted extensive linguistic and topical analyses to provide deeper insights into the specific content of these hostile interactions. by constructing and analysing such a dataset, we identify key patterns, such as the prevalence of race-based hostility, especially regarding immigration issues in the uk. our findings also suggest that there is a general lack of trust in mps in the uk. finally, we evaluated the performance of various plms and llms on binary hostility classification and multiclass targeted identity type classification using our dataset. this study not only offers valuable data but also lays the groundwork for future research aimed at understanding and mitigating the impact of online hostility in uk political contexts.