# Detecting Disinformation in Croatian Social Media Comments
## 4 Model Evaluation on Croatian Social Media Dataset
### Features of the Dataset Used
This issue will be addressed in future work using this framework, expanding the scope of training data by incorporating diverse thematic categories, thus enhancing the robustness and applicability of disinformation detection systems across different contexts. We acknowledge that the original annotation scheme of the FRENK 1.1 dataset was primarily designed to distinguish between \"offensive\" and \"acceptable\" speech. However, a closer qualitative analysis of the offensive category reveals a high incidence of speech acts that rely on or propagate misleading, false, or manipulative content--hallmarks of disinformation. Given the sociopolitical sensitivity of the dataset's themes, many of the offensive statements include narratives that are demonstrably false or misleading, designed to foster prejudice and shape public opinion through misinformation. Therefore, while the dataset was not explicitly labeled for disinformation, its content reflects a considerable overlap with known disinformation patterns, particularly in contexts where disinformation is used as a vehicle for hate speech or social manipulation, and in under-resourced languages like Croatian. In this study, we leverage this overlap by treating such instances as proxies for disinformation, while fully acknowledging the limitations of this approach. As part of future work, we intend to refine the annotation process and construct a more targeted dataset that includes explicit disinformation labeling based on factual verification and content analysis. This will allow for a clearer distinction between offensive, hateful, and misleading content, improving the precision of disinformation detection efforts and further strengthening the theoretical foundations of our framework. Although hate speech and disinformation serve different purposes (one targets individuals or groups, while the other manipulates beliefs), their textual patterns, social functions, and computational challenges overlap considerably. Therefore, insights from NLP research on hate speech detection can serve as a basis for methods, datasets, and interpretation techniques for disinformation research. Each comment was categorized either as offensive or acceptable. The prevalent topics of the comments were LGBTQ+ persons and immigrants. This dataset can be considered a disinformation dataset, as any hateful comment that objectively insults another person constitutes disinformation. The dataset was filled with emojis, and, therefore, all symbols that were not letters were ignored. After removing comments that did not provide any information (the empty comments), the number of comments in the dataset used for training was 7900, but this number was reduced again after removing the comments in the origin of the vector space (as we explained in the previous section). Disinformation on social media often manifests itself through inflammatory and derogatory comments, which can spread rapidly and influence public opinion. Disinformation, which involves deliberately spreading false information, can amplify hate speech by providing false narratives that fuel prejudice and discrimination. Both disinformation and hate speech can lead to the stigmatization and dehumanization of individuals or groups. Disinformation can spread unfounded rumors or conspiracy theories that portray certain groups as dangerous or inferior, which hate speech then exploits to justify discriminatory actions. Although the provided dataset consists of comments containing various forms of harmful speech, including insults, hate speech, and incitements to violence, we used it to train our model due to the interconnection of disinformation and hate speech. For instance, comments like \"Budale svasta majka rodi\" (eng. \"fools are born every day\") and \"cime se budala hvali pametan se srami\" (eng. \"what a fool boasts about, a wise person is ashamed of\") are derogatory and dismissive, contributing to a hostile online environment. Hate speech is evident in comments such as \"Jebem vam mater pedersku\" (eng.