UTB - NLP at S em E val-2023 Task 3: Weirdness, Lexical Features for Detecting Categorical Framings, and Persuasion in Online News (2023)

Abstract: Nowadays, persuasive messages are more and more frequent in social networks, which generates great concern in several communities, given that persuasion seeks to guide others towards the adoption of ideas, attitudes or actions that they consider to be beneficial to themselves. The efficient detection of news genre categories, detection of framing and detection of persuasion techniques requires several scientific disciplines, such as computational linguistics and sociology. Here we illustrate how we use lexical features given a news article, determine whether it is an opinion piece, aims to report factual news, or is satire. This paper presents a novel strategy for news based on Lexical Weirdness. The results are part of our participation in subtasks 1 and 2 in SemEval 2023 Task 3.

## introduction frame identification is a critical task in natural language processing (nlp) that recognizes the frames or semantic structures used in an article or text. in the semeval 2023 challenge context, we propose the task of frame detection in news articles. in this case, we selected the english language. this task is essential because it allows a better understanding of the perspective and tone of news articles and other media. in addition, it can help detect bias and improve accuracy in news and opinion article classification, which is relevant for applications such as news monitoring, sentiment analysis, and text classification. in particular, the frame detection challenge in which this paper seeks to promote the development of new nlp techniques and models for frame identification in complex and realistic texts and builds on the dataset and evaluation presented in the challenge overview article. the approach presented in this paper is lexical weirdness. this approach seeks a more appropriate lexicon for the domain as we generated these lexicons focusing on having a more appropriate vocabulary. the first thing that we did was to create a corpus associated with each of the framing categories. then, we scrapped wikipedia-related articles. subsequently, we extracted the nominal unigrams of these articles. after acquiring the unigrams for each category, we calculated the relevance of words within the context by comparing the lexicon of each context with the lexicon generated by google. finally, we selected the words with the highest occurrence, which had the most significant relevance in the context, to become the specific lexicon for each domain we implemented in a bag of words model. finally, the frequency of words per category was obtained and combined with the frequencies of unigrams using the weirdness technique to generate a lexicon. from this lexicon constructed in the previous phases, we tested different classification techniques to obtain the prediction of frames in news articles. the rest of the paper is structured as follows. first, we introduce the related work in section 2. then, section 3 presents the details of the proposed strategy. in section 4 and section 5, we discuss the setup and analysis of the experiment of results. we conclude in section 6 with remarks and future work. the system receives news articles as input in the task setup, already filtered from any html or special characters. then, the articles undergo feature extraction based on linguistics, and the extraction of noun phrases, vectorized using term frequency -inverse document frequency (tf-idf) @xcite . additionally, we oversample the data using smote @xcite . for the news genre categorization, we only used the english dataset, which consisted of 433 labeled elements for training and 83 unlabeled aspects for development. in addition, two articles focus on the classification of satire news vs. reportage. the other focuses on the distinction between opinion and reportage, which were reliable signals to identify satire news and reportage. @xcite uses various feature extractions such as the absurdity feature (abs), humor (hum), text processing and feature weighting, grammar (gram), negative effect (neg), and punctuation (pun) to identify satire within the text. the system's basis proposes text vectorization with then tf-idf. @xcite proposes a system that uses both paragraph-level and document-level linguistic features to identify satire. the attributes for the document level include psycholinguistic, writing stylistic, readability, and structural characteristics that distinguish how a journalist writes conservatively versus a satirical text that intentionally uses a humorous and aggressive message to entertain. @xcite proposes the classification between a report and an opinion and suggests feature extractions such as linguistic, document-level contextualized embeddings, and argumentation. all these articles have in common the extraction of linguistic features, which focuses on the document level. therefore, this study focuses on this feature and uses the vectorization of the noun phrases with tf-idf as a basis for the components. compared to these articles, we propose implementing linguistic feature extraction for a multiclass classification system (single label) for reportage, option, and satire classes. in addition, we present the extraction of noun phrases. ## system overview this section outlines the predictive models utilized in our proposed solutions for task 3-sub-tasks 1 and 2 of semeval 2023. this task involves detecting the genre, framing, and persuasion techniques in online news across multiple languages. using a four-step method, we employed the models proposed for solving sub-tasks 1 and 2 to identify elements contributing to text persuasiveness, including genre and framing of opinion, report, or satire. the first step involves pre-processing the data @xcite , the second step uses feature extraction(puertas and martinez-santos, 2021) that we regularized in the third step @xcite . then the regularized data is submitted through a classifying voting system evaluated using f1-score macro and f1-score micro for sub-tasks 1 and 2, respectively. figure 1 shows the proposed pipeline for both sub-task. this pipeline comprises the following stages: pre-processing, feature extractions, regularization, classifiers, and evaluation. this section outlines the predictive models utilized in our proposed solutions for task 3, sub-tasks 1 and 2 of semeval 2023, which involves detecting the genre, framing, and persuasion techniques in online news across multiple languages. to identify elements contributing to text persuasiveness, including genre and framing of opinion, report, or satire, we employed the models proposed for solving sub-tasks 1 and 2 using a four-step method. the first step involves pre-processing the data @xcite , followed by feature extraction @xcite , which is then regularized in the third step @xcite . finally, we submitted the regularized data through a classifying voting system evaluated using f1-score macro and f1-score micro for sub-tasks 1 and 2, respectively. figure 1 illustrates the proposed pipeline for both sub-tasks, which includes pre-processing, feature extraction, regularization, classifiers, and evaluation. ## conclusion in conclusion, the research conducted in this paper focused on the prediction of topic frames in news articles. the proposed methodology involved creating a vocabulary of topic frames and using different classification techniques, which showed the potential to improve the system's performance. however, as the results have shown, the unbalanced and limited training data and the use of a linguistic classification model limited the system's performance. to overcome these limitations and enhance the system's accuracy and robustness, we require more diverse and balanced data and explore other natural language processing models and techniques, such as deep learning. furthermore, additional linguistic features, such as noun phrases, can be used to improve model performance. finally, the results indicated that the system tended to extract more information from opinion articles, resulting in more predictions for this class. we could address this by exploring more balanced data and feature extraction techniques. overall, this work provides a solid foundation for future advances and developments in topic frame prediction in news articles. furthermore, it emphasizes the importance of acquiring more diverse and balanced data and exploring different natural language processing models and techniques to improve system performance.