A Multi-Labeled Dataset for I ndonesian Discourse: Examining Toxicity, Polarization, and Demographics Information (2025)

Abstract: Online discourse is increasingly trapped in a vicious cycle where polarizing language fuelstoxicity and vice versa. Identity, one of the most divisive issues in modern politics, oftenincreases polarization. Yet, prior NLP research has mostly treated toxicity and polarization asseparate problems. In Indonesia, the world’s third-largest democracy, this dynamic threatens democratic discourse, particularly in online spaces. We argue that polarization and toxicity must be studied in relation to each other. To this end, we present a novel multi-label Indonesian dataset annotated for toxicity, polarization, and annotator demographic information. Benchmarking with BERT-base models and large language models (LLMs) reveals that polarization cues improve toxicity classification and vice versa. Including demographic context further enhances polarization classification performance.

## introduction while ideological differences are natural in a healthy democracy, extreme polarization deepens divisions, often escalating into hostility and societal fragmentation @xcite . in such cases, opposing groups begin to see each other as existential threats, making reconciliation increasingly difficult @xcite . at the same time, online toxicity disproportionately affects minority groups @xcite , leading to self-censorship @xcite and undermining public discourse, particularly in journalism @xcite . indonesia, the third-largest democracy in the world, is home to 277 million people from diverse backgrounds @xcite , making it an appropriate case study. the 2024 presidential election saw intense political competition alongside a sharp rise in divisive and toxic online discussions. for example, @xcite found that in 2019, 1.35% of 800,000 online texts contained toxic language, while in 2024, @xcite reported that 13.8% of 1.45 million texts were toxic, which is a tenfold increase. this surge highlights the growing toxicity in indonesian discourse. previous research has explored toxicity and polarization as separate problems extensively, yet their complex relationship remains largely unstudied. this gap limits our understanding of how hostile online environments evolve. while political polarization can intensify toxicity, not all polarized discourse is toxic, and not all toxic speech is politically polarized. a dataset that captures both distinctions allows for a clearer differentiation between divisive yet civil discussions and interactions that escalate into outright hostility. to address this, we introduce the first indonesian dataset with multiple labels that includes toxicity, polarization, and demographic information of the annotator foot_0 . this dataset serves as a foundation for investigating how these factors interact in online discourse, offering insights into the broader implications of digital polarization and toxicity. ## interplay of toxicity, political polarization, and identity online discourse is increasingly characterized by a vicious cycle in which polarization fuels toxic language and vice versa. social media platforms exacerbate these dynamics by allowing unopposed expression of opinions, thereby deepening societal divisions @xcite @xcite . ## available datasets prior polarization datasets are largely us-centric @xcite , although some have addressed other contexts, such as brexit @xcite and poland @xcite (see ## conclusion and future work we present a novel multi-labeled indonesian discourse dataset of 28,477 texts annotated for toxicity, polarization, and annotator demographics. our analysis yields the following findings: polarization detection is more subjective than toxicity detection. despite extensive training, coder agreement is significantly lower for polarization, reflecting its inherent subjectivity. however, polarization labels still enhance toxicity detection, even in non-ideal conditions (sections 5.2, 6.4). we hypothesize that jointly modeling polarization and toxicity detection through other means, such as using soft labels instead of binary labels, may lead to a better model on both tasks. demographic information aids classification but is less effective than cross-task features. while demographics improve both toxicity and polarization detection, using polarization as a feature for toxicity (and vice versa) has a greater impact. combining demographic and cross-task features further boosts performance. this hybrid approach (appendix f.5) improves precision@1 and recall@1, allowing polarization detection to reach performance levels comparable to toxicity detection (f1@1 = 0.748). gpt-4o-mini does not effectively utilize demographic information. likely due to its training data limitations, gpt-4o-mini ignores demographic attributes except in one setup, where recall improves at the cost of precision (appendix f.4). its inability to leverage polarization for toxicity detection (and vice versa) suggests selective feature prioritization, warranting further investigation (appendix m).