Revealing Hidden Mechanisms of Cross-Country Content Moderation with Natural Language Processing (2025)

Abstract: The ability of Natural Language Processing (NLP) methods to categorize text into multiple classes has motivated their use in online content moderation tasks, such as hate speech and fake news detection. However, there is limited understanding of how or why these methods make such decisions, or why certain content is moderated in the first place. To investigate the hidden mechanisms behind content moderation, we explore multiple directions: 1) training classifiers to reverse-engineer content moderation decisions across countries; 2) explaining content moderation decisions by analyzing Shapley values and LLM-guided explanations. Our primary focus is on content moderation decisions made across countries, using pre-existing corpora sampled from the Twitter Stream Grab. Our experiments reveal interesting patterns in censored posts, both across countries and over time. Through human evaluations of LLM-generated explanations across three LLMs, we assess the effectiveness of using LLMs in content moderation. Finally, we discuss potential future directions, as well as the limitations and ethical considerations of this work.

## introduction in an era where billions of users engage with digital platforms daily, content moderation serves as a critical governance mechanism that balances freedom of expression with the need to mitigate harm and maintain community standards @xcite . platforms enforce moderation policies to address a wide array of issues, including terrorism, graphic violence, hate speech, explicit content, child exploitation, and fraudulent activities such as spam and fake accounts @xcite @xcite . however, as mega-platforms scale, concerns persist regarding the transparency and accountability of moderation practices @xcite . the challenges manifest at two levels: (1) the opacity of moderation decisions, often hidden within proprietary algorithms, platform policies, and government interventions @xcite , and (2) the methodological limitations of nlp-based moderation systems, which predominantly rely on surfacelevel keyword detection without distinguishing be-tween mention and usage @xcite . moreover, the lack of interpretability of black-box models makes it difficult to verify whether content moderation decisions are justified or erroneous. while prior work has focused on improving moderation techniques, little attention has been paid to systematically analyzing the reasoning behind these decisions @xcite . @xcite is the closest to our motivation, however they look at platform-specific guidelines, whereas we intend to analyze guidelines across countries. to address these gaps, this work pioneers a novel investigation: deploying llms to audit and interpret cross-national moderation patterns, like automatic investigative journalists. we conduct a case study on twitter posts censored in five countries-germany, turkey, india, france, and russia-spanning from 2011 to 2020. we explore two research questions: rq1: can llms reverse-engineer moderation decisions across different countries? we model censorship behavior using neural classifiers of varying architectures and scales, leveraging the content moderation dataset from @xcite to predict whether a post was censored and to categorize it into one of six predefined censorship categories. ## related work online moderation content moderation is crucial for maintaining online discourse, encompassing hate speech detection @xcite @xcite , political censorship @xcite , and factual verification @xcite . while investigative journalists rely on nuanced decision-making, automated moderation systems predominantly use keyword-based filtering @xcite , which often lacks contextual depth. large datasets @xcite provide benchmarks, but moderation policies remain fluid. automated systems frequently misclassify critical content, such as calls to action in social movements @xcite , while users devise creative linguistic adaptations to bypass censorship @xcite . traditional language models (lms) also struggle to distinguish between the use and mention of keywords, sometimes incorrectly censoring counterspeech @xcite . with the rise of llms, research has examined their effectiveness in moderation. @xcite evaluate llms on broader metrics beyond accuracy, while @xcite report that llms outperform traditional classifiers in toxicity detection. however, moderation remains subjective, as @xcite show that llms' personabased attributes influence hate speech annotation. explainability explainability plays a crucial role in ai decision-making, particularly in content moderation and healthcare @xcite . techniques such as causal interventions analyze neuron behaviors @xcite , while intermediate representation translation reveals concept evolution across layers @xcite . influence functions, lime, and shap provide model explanations without modifying architectures @xcite , and @xcite propose an explainer model for interpreting lm decisions. activation patching enables self-interpretation of lm representations @xcite , while @xcite examine training dynamics to assess dataset impact. in content moderation, @xcite introduced hatexplain, the first benchmark for explainable hate speech detection. however, prior work does not address censorship explainability across multiple countries and time periods. our study takes a novel approach by treating llms as investigative journalists to analyze content moderation decisions across five countries. unlike hatexplain, our work (1) focuses on content moderation, which lacks rationale-specific datasets, and (2) generates full-length explanations for censored posts. to our knowledge, this is the first large-scale study using llms for content moderation pattern analysis and explanation. ## task formulation and preliminaries in this section, we formulate two key tasks based on our research questions: (1) reverse-engineering content moderation decisions and (2) leveraging llms to uncover hidden patterns behind these decisions. ## opening directions for future work our work opens several avenues for future research, particularly in using llms for content moderation, explaining the decisions behind censorship, and detecting censorship across both geographical and temporal scales. given the highly sensitive nature of this task, ensuring that llm-generated explanations remain faithful is crucial @xcite . with the growing focus on interpretability, we anticipate that future research can build upon our findings using advanced techniques such as activation patching @xcite and circuit discovery @xcite . furthermore, expanding censorship analysis to more countries while balancing dataset distributions across regions could provide deeper insights into how multilingualism influences censorship practices. ## conclusion this study analyzes content moderation mechanisms across multiple countries, using llms for classification and explainability. our findings highlight discrepancies in moderation policies, shaped by geopolitical, temporal, and event-specific factors. using explainability techniques like shapley values and llm-guided reasoning, we identify key censorship patterns, showing that while llms can replicate moderation practices, they struggle with nuanced, context-dependent rules. these insights emphasize the need for greater transparency in aidriven moderation. future research should focus on improving interpretability and ensuring the responsible deployment of llms in content governance.