F raming F reaks at S em E val-2023 Task 3: Detecting the Category and the Framing of Texts as Subword Units with Traditional Machine Learning (2023)

Abstract: This paper describes our participation as team FramingFreaks in the SemEval-2023 task 3 “Category and Framing Predictions in online news in a multi-lingual setup.” We participated in subtasks 1 and 2. Our approach was to classify texts by splitting them into subwords to reduce the feature set size and then using these tokens as input in Support Vector Machine (SVM) or logistic regression classifiers. Our results are similar to the baseline results.

## introduction fake news and false arguments are everywhere. in twitter posts or news articles, people use persuasion techniques to influence the opinion of other people. for this reason, it is important to detect the framing and persuasion techniques in texts automatically. this was the aim of the third shared task of semeval 2023 @xcite , in which we took part as team framingfreaks. this task focuses on the detection of non-neutral texts (first subtask), evaluating the frame of the argumentation on the article level (second subtask), and finding out which techniques were used on the sentence level @xcite . the data consists of online news and twitter posts in english, french, german, italian, polish, and russian. for the test set the organizers of the task proposed a few-shot learning task where our systems had to generalize to sources of other languages, namely spanish, greek, and georgian. we participated in the first and second subtasks in all languages. we did not tackle the third subtask. the focus of our contribution is on computationally simple and efficient models. we use 'traditional' machine learning models, like svm. however, unlike word or character-based features that are commonly used with these models, we use subword units to further reduce the feature set size. besides their computational efficiency, the traditional classifiers based on bag-of-words (or ngrams) features have several other interesting prop-erties. for example, unlike recent deep-learning models, they can process sequences of arbitrary length, the computational cost comes mainly comes from the size of the feature set. convex optimization methods used in (some of) these models make training and tuning these models more straightforward. furthermore, in some cases, they may work as well or even better than deep learning models, especially when information from pre-training is not available (see, for example, @xcite . however, the bag-of-words features may typically result in sparse features. this may particularly be bad for morphologically complex languages as many words will be observed only a few times, and the relation between the morphologically-related words cannot be used by the classifier. a common alternative is to use character n-grams (e.g., @xcite @xcite , often in combination with word n-grams. the character n-gram features, on the other hand, may result in very large feature sets. inspired by recent deep learning approaches that use subwords as input units, we use subword ngrams as features used by traditional machine learning models. subwords may allow models to learn from smaller, meaningful units, by reducing the sparsity and the number of features at the same time. this should reduce the vocabulary size because parts of words should be more frequent than the word itself. especially in languages with rich morphology, since the stem often does not change. we release our code on github. foot_0 2 background ## post-analysis of subtask 1 we run all the following experiments on the test set. ## conclusion the first subtask was difficult because the datasets were not balanced and the input length of the texts differed between small paragraphs and longer news articles. the first fact results in the problem that the majority class of the training set is predicted over proportional often. the different lengths of the texts could lead to a classification that is just based on length. this may be due to the fact, that news articles are longer, in general, and therefore the class 'reporting' is just predicted by the size. the difficulty in subtask 2 was to not just find one or two of the correct framings but all of them. the majority class in the english dev set was opinion. this lead our system to ignore the minority classes and predict the opinion class in each case of the test set, although we balanced the class weights to prevent this problem. our system just modeled the high-frequency classes. we assumed that a bag-of-word model should be worse, especially for morphologically complex languages, but the baseline used these bag-of-word models and was better in most of the languages. a monolingual setup also did not improve our results. our analysis of the data shows that our system does not seem to classify meaningful features but takes the frequency of the training data and low features like the article length as hints for categorization. maybe the use of a transformer model or other language model that takes the relationship of the words into account would have led to a better result. methods for data argumentation like oversampling could have been helpful as well.