# Limitations of Large Language Models in Propaganda Detection Task
## 2 Literature Review
### Media Bias, Fact Checking and Propaganda in Online News
Oversampling and BERT-based approaches yielded the best results for both tasks. In general, the fragment detection and technique-naming tasks were more difficult than the binary classification of propagandist content. The continuation and expansion of the previous problem was conducted during SemEval2020 Shared Task 4 on the detection of propaganda techniques in news articles [42; 43]. Due to limited examples of certain propaganda techniques, after deleting and merging some of them, the final number was limited to 14. The best results for the span identification task were obtained by employing a heterogeneous pre-trained model [44]. Propaganda technique classification was found to be the most successful when applying a RoBERTa-based model using a semi-supervised learning technique of self-training [45]. Later experiments with a fine-tuned RoBERTa model outperformed scores for the classification task [46]. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Organization** & **Description** & **Website** \\ \hline Ad Fontes Media & Creator of Interactive Media Bias Chart\({}^{\text{\textregistered}}\) & www.adfortesmedia.com (accessed on 20 March 2024) \\ \hline All Sides & Provides balanced news, media bias ratings and diverse perspectives—top news stories are presented from the left, center and right of the political spectrum. & [[https://www.allsides.com/unbiased-balanced-news](https://www.allsides.com/unbiased-balanced-news)]([https://www.allsides.com/unbiased-balanced-news](https://www.allsides.com/unbiased-balanced-news)) (accessed on 20 March 2024) \\ \hline FactCheck.org & Monitors the accuracy of information said by major US political figures in TV ads, debates, speeches, interviews and news releases. Receiver of a Pulitzer award. & www.factcheck.org (accessed on 20 March 2024) \\ \hline Media Bias/Fact Check & Promotes awareness of media bias and misinformation by rating the bias, factual accuracy and credibility of media sources. & www.medibiasfactcheck.com (accessed on 20 March 2024) \\ \hline PolitiFact & Receiver of a Pulitzer award. & www.politifact.com (accessed on 20 March 2024) \\ \hline Snopes & One of the first fact-checking services; initially investigated urban legends. & www.snopes.com (accessed on 20 March 2024) \\ \hline Swiss Policy Research (SPR) & Research group investigating geopolitical propaganda. Creators of \({}^{\sim}\)The Media Navigator\({}^{\sim}\)—leading media outlets classifier for English and German languages news providers based on their political stance and their establishment bias. & www.swprs.org/media-navigator (accessed on 20 March 2024) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Main organizations dealing with media bias and fact checking. Further development of a propaganda corpus and automatic propaganda detection field was part of SemEval2023 Task 3, which focused on category (opinion, reporting, or satire), framing (14 generic frames, including economic, morality and political) and persuasion (propaganda) technique detection in online news in different languages [47; 48; 49]. The list of techniques was enlarged to 23, forming six coarse-grained categories. Articles in six European languages, including Polish, were collected and annotated. The best yielding results for the third subtask on persuasion techniques classification were obtained with fine-tuned transformer models, such as XLNet, RoBERTa, XLM-RoBERTa-large or MarianMT [50; 51; 52; 53; 54; 55]. XGBoost and other classic methods were implemented only by two teams and performed poorly [56; 57]. Recently, in connection with the emergence and growing popularity of LLMs, there were several attempts to use commercial models for such a complex task, like propaganda detection. [PERSON] et al. tested two OpenAI models, namely, _gpt-4_ and _gpt-3_, which were fine-tuned with the _davinci_ model [58]. They reformulated the original task and used the annotated development set as a test set for easier evaluation of the results.