3218 IR at S em E val-2020 Task 11: C onv1 D and Word Embedding in Propaganda Span Identification at News Articles (2020)

Abstract: In this paper, we present the result of our experiment with a variant of 1 Dimensional Convolutional Neural Network (Conv1D) hyper-parameters value. We describe the system entered by the team of Information Retrieval Lab. Universitas Indonesia (3218IR) in the SemEval 2020 Task 11 Sub Task 1 about propaganda span identification in news articles. The best model obtained an F1 score of 0.24 in the development set and 0.23 in the test set. We show that there is a potential for performance improvement through the use of models with appropriate hyper-parameters. Our system uses a combination of Conv1D and GloVe as Word Embedding to detect propaganda in the fragment text level.

## introduction news articles can have many forms of presentation. a news article often tends to influence the reader to have some particular view following the author's agendas. the method usually used is to add some bias to the article being written @xcite . on a different level, news management sometimes has lacking neutrality shapes information by emphasizing positive or negative aspects purposefully @xcite . identifying bias is not easy. often the writer is not realized that he has included his bias in his writing. in another case, the writer deliberately introduces bias into his writings, intending to influence his readers. the last condition is known as propaganda. propaganda is a shape of opinion or action by individuals or groups deliberately designed to change the perspective or actions of other individuals or groups regarding predetermined ends @xcite . da san martino et al. @xcite organized the semeval 2020 task 11: detection of propaganda techniques in news articles. this propaganda refers to whenever information is shaping purposefully to foster a predetermined agenda. this research focuses on subtask 1 about span identification that indicated the location of the text fragment of propaganda in news articles. propaganda span identification is a problem of detecting text fragment in a sentence or paragraph containing propaganda techniques. the purpose of this task is to detect the presence of words or phrases that cause bias in the information on purpose or called propaganda. this study proposes a deep learning approach, specifically 1 dimensional convolutional neural network (conv1d) combined with word embedding (glove) in detecting text fragment related to propaganda. conv1d, as part of convolutional neural network (cnn), is one of the deep learning approaches that has been widely used. convolutional neural network is widely used in computer vision, especially in image classifications where data is processed in the form of 2 dimensions @xcite . whereas in text processing, 1-dimensional convolutional neural network (conv1d) is used in relation to its suitability for the characteristics of text that only has 1 dimension in the form of word sequence. conv1d was chosen because it has been known to have excellent performance in the natural language processing (nlp) task such as sentiment analysis and classification of text @xcite especially in the categorization of text fragments @xcite and classification in word and character level @xcite . word embedding is a language modelling and feature learning technique in (nlp) that maps words into a low dimensional continuous space @xcite . word embedding as input representation has been known to increase the performance of tasks in nlp through mapping words in the form of a low-dimensional vector that allows a more expressive and efficient representation by maintaining the contextual similarity between words @xcite . glove or global vector is one type of word embedding that is known to have an excellent performance. glove combines global matrix factorization and local context window methods. rather than training on the entire sparse matrix or individual context windows in the corpus, glove model efficiently captures statistical information by training only on the nonzero elements in a word-word co-occurrence matrix. which then becomes a solution to the problem of computational costs and context constraints @xcite . in our work, we explain the motivation of using conv1d and word embedding (glove) in the related work section. in the system overview section, we describe the system that we used in our submission. we then describe the variant of the model with conv1d that we use to reach the best performance of our classification model in the experimental setup section. we also describe our results on the development and test data, as well as a short analysis in the experiment result and analysis section. ## related work some research has been done in detecting propaganda in news articles. previous research on the language of news media in the context of political fact-checking and fake news detection trying to compare the language of real news with that of satire, hoaxes, and propaganda by finding linguistic characteristics of untrustworthy text @xcite . other research proposes a model to automatically assess the level of propagandistic content in an article based on different representations, from writing style and readability level to the presence of specific keywords @xcite . another research is trying to utilize bert in classifying propaganda techniques in news articles @xcite . however, the detection of propaganda related to this task tends to be challenging because they have to be done to the level of a text fragment, and this kind of task at least was first competed on semeval. in other words, this task still leaves opportunities in terms of performance improvement or analysis related to the possibility of using other approaches than previously done. conv1d in several previous studies has been shown to have an excellent performance related to the classification of texts in word and character level @xcite and also in text fragment level @xcite . this can be achieved by utilizing the process of convolution and pooling in the word, character and text fragment level by pick out stand out features like tokens or token sequences in a way that is not fixed in their position in the input sequence @xcite . the use of word embedding as word vector representation has also been shown to have better performance than the use of one-hot vector directly @xcite . these facts motivate us to explore the potential of using conv1d combined with word embedding in text classification, especially at text fragment level by utilizing automatic feature extraction in news articles related to propaganda. ## experiment setup and analysis this section described the setup and analysis for the approaches used in this research, such as data, standard parameters settings, hyper-parameters tuning, and analysis of the experiment results. ## conclusions based on the results obtained from this research, it was concluded that conv1d combined with word embedding could be used as a model in propaganda span identification problem. the best results are generated from scenarios that use several convolution layers with the same number of filters but have different kernel sizes and pooling sizes for each layer. this scenario also shows an increasing pattern in the kernel size and decreasing pattern in the pooling size from the initial layer to the next layer. this pattern can be used as consideration related to hyper-parameters tuning at the convolution layer. furthermore, the limited combination of hyper-parameters values used in this research leaves the possibility of achieving better performance by using combinations that have not yet been tested. other parameters whose values are made static are also likely hiding the potential for improved performance.