Towards an Automated Framework to Audit Youth Safety on T ik T ok (2025)

Abstract: This paper investigates the effectiveness of TikTok’s enforcement mechanisms for limiting the exposure of harmful content to youth accounts. We collect over 7000 videos, classify them as harmful vs not-harmful, and then simulate interactions using age-specific sockpuppet accounts through both passive and active engagement strategies. We also evaluate the performance of large language (LLMs) and vision-language models (VLMs) in detecting harmful content, identifying key challenges in precision and scalability. Preliminary results show minimal differences in content exposure between adult and youth accounts, raising concerns about the platform’s age-based moderation. These findings suggest that the platform needs to strengthen youth safety measures and improve transparency in content moderation.

## introduction tiktok is a short-form video platform that has rapidly emerged as one of the world's most influential social media services. with over 1.6 billion monthly active users 1 and millions of videos uploaded daily @xcite , it now plays a central role in the global digital media landscape. children and adolescents increasingly rely on tiktok for both entertainment and everyday information @xcite @xcite . although tiktok enforces community guidelines through content removal and age-based restrictions 2 , concerns remain about the effectiveness of these moderation mechanisms in shielding young users from harmful content. these concerns are amplified by the introduction of the european 1 https://www.businessofapps.com/data/ tik-tok-statistics , accessed on august 5, 2025 2 https://www.tiktok.com/community-guidelines , accessed on august 8, 2025 digital services act foot_0 , which requires very large online platforms to assess and mitigate systemic risks-particularly those related to the protection of minors and the spread of harmful content. this paper reports preliminary findings from an ongoing work aiming to build an automated pipeline for auditing tiktok's safety enforcement mechanisms by systematically measuring the exposure to harmful content among youth and adult users, investigating different modes of interaction. specifically, we explore two main research questions: to this end, we created 10 youth and 10 adult sockpuppet accounts and simulated multiple sessions over several days, collecting over 7,000 videos across both "for you feed" (fyf) scrolling and using active search with harm-adjacent keywords. in addition, as a precise detection of harmful content is an heavy task for humans, we also consider a third research question: to this end, we employed both text-only and video-based llms to estimate the presence of harmful content that violates tiktok's community guidelines and evaluated their performance. ## related work tiktok's for you feed (fyf) algorithm personalizes content based on user language, location, posting time, and interactions such as likes and follows @xcite . this personalization engine rapidly amplifies interestaligned content-often within just 200 recommendations-thereby fostering echo chambers and limiting content diversity @xcite . these concerns are especially salient given tik-tok's young user base-over 60% of users were under 30 in 2021 @xcite . a large-scale audit using more than 100 automated accounts found that watch time plays a central role in shaping recommendations, reinforcing problematic content loops through prolonged exposure ( @xcite . recent studies further contextualize harm by examining user behavior. for instance, the median user consumes approximately 90 videos daily @xcite , while moderately addicted users average 7.86 minutes per session @xcite ). an experimental audit comparing tiktok, youtube, and instagram showed that accounts registered as 13-year-olds encountered harmful content more frequently and rapidly than accounts of older users @xcite . ## discussion we analyzed a dataset of over 7,000 tiktok videos collected from age-specific sockpuppet accounts using both passive and active interaction protocols. our findings indicate that adult and youth accounts were exposed to highly similar content, with fewer than 1 in 10 videos estimated to be harmful. this suggests that tiktok's moderation mechanisms may not meaningfully differentiate between age groups in practice. despite this, automated detection of harmful content remains challenging: experiments with vlms showed limited precision, with gpt-4o -operating on text only -outperforming videollama3, which had access to video content. our work presents promising results towards building a scalable and reproducible approach for auditing exposure to harmful content on recommender-driven platforms. by systematically comparing the content shown to adult and youth users, one can uncover tangible shortcomings in age-based content moderation, especially concerning the search function. our results suggest that platforms like tiktok should invest greater effort in strengthening youth safety protections and ensuring greater transparency in how content is being moderated for these specific audiences. future research will expand the scope of this study by including additional countries to assess cross-cultural differences in content moderation. we also plan to scale up the experiment with a larger and more diverse set of accounts, including personalized profiles that better simulate real user behavior. to improve ecological validity, future data collection will incorporate the mobile interface, reflecting the primary mode of user interaction on tiktok. finally, we aim to develop and integrate more accurate prediction models for detecting harmful content, leveraging multimodal signals more effectively. due to models such as gpt-4o's expensive costs, we plan to investigate and incorporate techniques such as the teacher and student model, which has been previously used for misinformation detection @xcite .