# Limitations of Large Language Models in Propaganda Detection Task
## 2 Literature Review
### Media Bias, Fact Checking and Propaganda in Online News  
Media bias analysis and fact-checking tasks are much broader and at the same time more popular topics than propaganda studies [20; 21; 22]. One of the connoted works performed automatic political fact checking (true or false) by analyzing linguistic characteristics of news excerpts with distant supervision [23]. The language used in news was compared with the one from satiric works, hoaxes and propagandist texts, which present untrustworthiness. The study showed that the analysis of stylistics can help to determine whether news is fake or not. [PERSON] et al. prepared a review on this problem while focusing on social media [24]. They mentioned propaganda as a way of conveying false stories whose aim was to change the way of thinking and behavior of people, often to advocate for a particular point of view. The authors claimed that the automatic detection of misinformation and disinformation acts is challenging, as the content itself often looks real but needs further checking by humans to confirm the veracity. Another survey centered around fake news, propaganda, misinformation and disinformation in various online content, including text, images and videos [25]. Authors reviewed available state-of-the-art methods of multimodal (combining various input data) disinformation detection methods and stated that the lack of datasets are a big hindrance for future research in this area. Table 1 presents some of the most prominent organizations that deal with the media bias problem and fact-checking task, focusing mostly on the English news providers [26]. As we can see, none of them dealt with propaganda as their core activity, but they all revolved around this subject. Propaganda in online news has become a visible problem in the digital age, and is connected with the easy reach and big influence of online media platforms [27]. Online news outlets can have a strong impact on shaping public opinion and beliefs. It may appear in various forms, such as biased reporting, selective facts, sensationalism and presenting misleading information. [PERSON] et al. focused on generating fake news that is more human-like [28]. They openly claimed that neural models are not ready in their current form to effectively detect human-written disinformation. Proppy is one of the earliest systems to automatically assess the intensity of propagandist content (score) based on the style of writing and presence of particular keywords [29]. Authors also created QProp, which is a propaganda corpus prepared with distant supervision. Binary labels for propaganda content, as well as media bias level (left, center or right), were extracted from the Media Bias/Fact Check website. The key conclusion was that at the article level, the writing style representations and complexity of text are more effective than n-grams, and at the topic-level, the consideration of stylistic features provides better results for propaganda detection. Another work proposed a fine-grained analysis of information on the detection of propaganda techniques and spans in which they were used [30; 31]. Based on previous works [32; 33; 34; 35; 36; 37; 38; 39; 40], the authors prepared a list of 18 propaganda techniques, which where described and provided examples of use from news excerpts. Additionally, they prepared an annotated corpus of news that have propaganda examples marked. The results of BERT-based models and designed multi-granularity neural networks were given as a baseline for two tasks:  
* Sentence-level classification (SLC): prediction of at least one propaganda technique at the sentence level; best F1 score--multi-granularity with ReLU (60.98%). * Fragment-level classification (FLC): identification of a span and the type of propaganda technique; best F1 score--multi-granularity with sigmoid (38.98% for the span task and 22.58% for the full task). In the NLP4 IF-2019 Shared Task, different teams participated in the aforementioned tasks and managed to obtain scores better than the baseline [41].