Linking Transparency and Accountability: Analysing The Connection Between T ik T ok’s Terms of Service and Moderation Decisions (2025)

Abstract: The European Commission’s Digital Services Act (DSA) mandates that Very Large Online Platforms (VLOPs), like TikTok, provide Statements of Reason (SoRs) to justify their content moderation decisions in an attempt to enhance transparency and accountability for these platforms. However, we can often notice a gap between these automated decisions and the platform’s written policies. This leaves users unable to understand the specific rule they have violated. This paper addresses this gap by developing and evaluating a pipeline to link TikTok’s SoRs from the DSA transparency database to the most relevant clause from TikTok’s policy documents. We test multiple methods to perform the linking task and evaluate performance using a wide range of retrieval methods and metrics.We develop and deliver a gold-standard dataset where a team of legal research assistants annotated 100 SoRs based on four criteria: clarity, understanding, presence of unclear terms and level of detail, each rated on a 1–4 scale. In addition, a binary rating is assigned for redress clarity. Moreover, annotators determined the best link to the relevant TikTok policy clauses. Results show that both TikTok’s SoRs and policy clauses are often extremely broad, granting TikTok more freedom to decide how to apply the clauses, making it even less transparent for users. We also provide a demo that, for each SoR, provides a ranking of the most relevant clauses from TikTok’s written policies, a tool that can be useful for users, regulators and researchers to better understand content moderation decisions, assess compliance with transparency requirements, and support further analysis of platform accountability.

## introduction large online platforms have become a staple part of everyday life for sharing discourse, emotions and social interaction for billions of users. in 2025, it is projected that about 5.24 billion people use social media daily foot_0 and tiktok alone has 1.12 billion monthly users, spending an average of 95 minutes on the platform foot_1 . to manage the massive volume of user-generated content, these platforms increasingly rely on automated systems for content moderation @xcite . while this reliance is necessary to counter harmful content, these automated "black box" decisions lead to concerns about fairness, accountability, and transparency @xcite . to address this and enforce greater responsibility, the european commission introduced the digital services act (dsa) 3 . it is set out to increase the accountability and interpretability for these decisions by making vlops publish sors that explain why actions like removal or restrictions were taken and what means were used for their detection. in practice, this link between a specific statement of reasons (sor) and the exact policy clause is rarely clear to users or even legal teams. tik-tok's sors are often highly templated and vague, which makes it difficult for users and researchers to connect them to the governing rules @xcite . this paper addresses the gap in interpretability. using information retrieval techniques, we develop a pipeline to automatically link tik-tok's sors from the dsa transparency database to the most relevant clauses in its policy documents. the contributions of this paper are as follows: (a) we create a gold-standard dataset of 100 sors (tiktok-100) manually annotated by a team of legal research assistants. for each sor we evaluate its clarity, understanding, detail level, inclusion of unclear terms as well as the most relevant chunk (out of 124 in total) from the tiktok policy docu-ments, (b) we evaluate traditional sparse retrieval models, like tf-idf or bm25 and modern dense embedding models, like bert, openai's embeddings or cross-encoders, and generative models like gpt-4.1, as well as hybrid, fusion and finetuning strategies for the task of linking a sor with platform policy document clauses. (c) we provide a working demo that, for each sor, orders the most relevant platform policy clauses (for different retrieval models) and incorporates a twostage fairness assessment pipeline, combining the claudette model from @xcite with a custom model to flag policy clauses that may potentially be unfair or ambiguous. ## background content moderation is essential for social media platforms, which act as "new governors" of online speech by setting and enforcing rules @xcite . the dsa tried to make this governance more transparent by demanding that platforms publish sors. under the eu digital services act (dsa), providers of online platforms must issue statements of reasons when moderating content (art. 17), including the contractual or legal ground relied upon, whether automated tools were used, and available redress mechanisms. furthermore, platforms must submit these sors to a publicly available database (dsa transparency database (art. 24(5)), which exposes large-scale, near-real-time moderation rationales. 4 however, initial analyses of the dsa database reveal that platforms like tiktok often use repetitive, vague statements that undermine fairness and accountability @xcite . for instance, tiktok frequently cites generic "community guidelines" violations and relies heavily on automated tools for over 95% of decisions, leading to standardised explanations lacking specific detail. 5 early analyses of the dsa database already confirm this trend and show the differences within moderation practices. a study of over 156 million sors by (drolsbach and pröllochs, 2024) found that tiktok is by far the most active platform and performed over 350 times more moderation actions per user than x (twitter), with the majority of decisions being automated. the dsa transparency database was designed to counter exactly this kind of behaviour from platforms. these are more and more shifting their practices from removing content outright to new methods that hinder visibility for users. in fact, it can be argued that the dsa's requirement to provide a sor for every moderation action could function as a "prohibition on shadow banning". @xcite as we can see later, however, guidelines like these actually lead to the problem that is already discussed in the paper itself, in that platforms use visibility reduction for ambiguous content and, in that way, create a situation where the most sensitive cases are governed by the least transparent means. prior work has also highlighted discrepancies between platforms' stated policies and their practices. this tendency is reflected in platforms' official transparency reports, too. an analysis of these reports by @xcite found that companies are much more willing to talk about government removal requests than their own moderation decisions, which remain "largely obscure". the study argues that this can be seen as a form of "transparency-washing", which looks like they are open about their rule enforcement, while in reality, it stays mostly obscure to users. tiktok has faced criticism for unclear moderation practices, such as allegedly suppressing content from creators deemed "ugly, poor or disabled" (zeng and kaye, 2022). and while tiktok seems to provide more detail than some rival companies like facebook (or meta), which reportedly cited a generic "violation of our terms" for almost 100% of its removals, tik-tok still uses a "similar albeit shorter statement" across cases @xcite . this means that users usually get a standardised repetitive block of text, instead of an actually helpful reason. the discrepancy is also found in audits of the dsa database itself. for example, an analysis by @xcite of over 350 million sors found "striking inconsistencies" between the data that platforms submitted to the database and the information they stated in their own reports. the most significant contradictions were in the use of automation, where x (formerly twitter) reported using no automation at all in the database despite saying otherwise in their reports. nlp techniques have been used to improve the interpretability of legal documents, addressing a core reason that users perceive something as unfair, which is the lack of clear, consistent linking between a moderation decision and the platform's own rules. for instance, a study of youtube cre-ators by @xcite found that the perception of users and what they see as unfair relies heavily on the consistency of moderation and the equality when compared to other creators. if they feel like another user's content is not removed, even though they made the same content as them, they often deem the process unfair and arbitrary. the claudette system uses machine learning to automatically detect potentially unfair clauses in tos documents @xcite . more recently, aspromonte et al. (2024) used a multi-agent system with llms to link sors to tos clauses. this approach can be computationally expensive, however, and can lead to error propagation. our work builds on these findings by providing a broader comparative analysis of a number of different retrieval methods, including sparse, dense, and hybrid models, and integrating a fairness assessment pipeline specifically for the tiktok statements and clauses. ## data our work relies on three primary data sources: tik-tok's moderation decisions (sors), its policy documents, and a manually annotated gold-standard dataset that we use for the evaluation. ## results and analysis we evaluated our models against the tiktok-100 dataset ( §3.3). we use standard metrics for effectiveness, namely mean reciprocal rank (mrr) as a rank-aware metric and recall at various thresholds (r@k), which ignores rank but can be particularly useful for assessing performance in an ambiguous task like ours. more detailed descriptions of these methods can be found in appendix f. ## conclusion this paper presented a comprehensive pipeline for linking tiktok's moderation decisions to its policies. this is an important step towards enforcing the transparency that the digital services act originally mandated. our evaluation across a wide range of retrieval models showed several key insights. first, hybrid retrieval strategies that fuse sparse and dense methods are overall the most effective, outperforming any individual model. second, general-purpose models provide stronger outof-the-box performance than domain-specific ones like legalbert for this task. third, supervised fine-tuning provides significant performance gains. for further research, it would be interesting to ex-plore what effects it would have to create a larger, legally sound annotated dataset to avoid overfitting and get more meaningful insights. the same applies to assessing the fairness and clarity of the clauses. overall, our analysis confirms that tiktok -like many other platforms-relies on vague and repetitive explanations as well as overly broad "catchall" clauses that obscure the real reasoning behind moderation decisions. this then creates a gap regarding accountability. by automatically linking the moderation practice to the policies and flagging potentially unfair terms, our work provides a methodology and a practical toolkit for regulators, researchers, and users to better analyse and understand the moderation systems of online platforms.