Exploratory Analysis of News Sentiment Using Subgroup Discovery (2021)

Abstract: In this study, we present an exploratory analysis of a Slovenian news corpus, in which we investigate the association between named entities and sentiment in the news. We propose a methodology that combines Named Entity Recognition and Subgroup Discovery - a descriptive rule learning technique for identifying groups of examples that share the same class label (sentiment) and pattern (features - Named Entities). The approach is used to induce the positive and negative sentiment class rules that reveal interesting patterns related to different Slovenian and international politicians, organizations, and locations.

## introduction traditionally, sentiment analysis refers to the use of natural language processing to systematically identify, extract, quantify, and study affective states and subjective information. most frequently, it is used as a predictive technique used to model social media @xcite , more specifically to predict or summarize opinions, attitudes and emotions in tweets, comments, online reviews etc., where the main focus is on predicting attitudes expressed towards a specific entity @xcite . another line of research applies sentiment analysis on news text, where the focus has shifted from analyzing sentiment towards a specific target to analyzing the intrinsic mood of the text itself @xcite . authors aimed to model feelings (positive, negative, or neutral) that readers feel while reading a certain piece of news @xcite @xcite , also in relation to news covering covid-19 @xcite , modelled news sentiment in relation to stock market and economic conditions @xcite @xcite . sentiment analysis has been also used in fake news identification @xcite and in media bias analysis @xcite . in the current trend of natural language processing research @xcite , the main focus is on improving the predictive performance over state-of-the art especially using deep learning-based methods. the drawback of these models is in their very limited interpretability. in contrast, several data and text mining techniques have been developed to improve domain understanding and support exploratory analysis of data, with focus on explainable models, which is crucial e.g. in medical applications, but also interesting for interdisciplinary research in the field of digital humanities and digital social sciences. our research falls under this line of research. the aim of our study is to gain better understanding into news sentiment by analysis of named entities in a manually annotated corpus of slovenian news articles @xcite . more specifically, our aim is to identify groups of topics with negative or positive sentiment in slovenian news, where topics are identified by named entities and their interaction forms the context of the reported stories. we propose the employment of subgroup discovery -a descriptive rule learning technique for identification of groups of examples sharing the same class label (sentiment) and same pattern (features). the task of subgroup discovery is the combination of predictive and descriptive rule induction. the result of subgroup discovery is to provide understandable descriptions of subgroups of individuals which share a common target property of interest. subgroup discovery methods have traditionally be successfully applied to in different medical applications (e.g. detecting of groups of patients at risk for atherosclerotic cardiovascular disease @xcite , supporting factors for brain ischemia @xcite , and psychiatric emergency @xcite , but only rarely applied to model textual data. the closest to our study is the work by @xcite using the subgroup discovery system hedwig for analyzing news articles about portugal focusing on interesting vocabulary patterns that reflect credit default swap. the authors focused on financial entities, geographical entities and a specialized vocabulary of the european sovereign debt crisis. the main contributions of this paper are two fold. first, we propose a novel approach using named entity recognition and linking in a subgroup discovery setting. next, we apply the method on the slovenian news dataset, getting new insights into slovenian news reporting in terms of news sentiment and showcase the potential of our approach for digital social science research. the paper is structured as follows: in section 2, we present the data used in the experimental work. section 3 presents a short outline of the employed methodology. in section 4 and section 5, we present our results and offer our conclusions and ideas for further work. ## data in our experiments, we used the manually sentiment annotated slovenian news corpus sentinews 1.0 @xcite foot_0 . the corpus consists of slovene web-crawled news containing more than 250,000 documents with political, business, economic and financial content from five slovene media resources on the web. the data covers the period between 1 september 2007 to 31 december 2013. data used in the experiments is a manually sentiment annotated stratified random sample of 10,427 documents from news portals 24ur, dnevnik, finance, rtvslo, and žurnal24. data was independently annotated by 2-6 annotators, using the five-level lickert scale (1 -very negative, 2 -negative, 3 -neutral, 4 -positive, and 5 -very positive) on three levels of granularity, i.e. on document, paragraph, and sentence level. the sentiment of an instance is defined as the average of the sentiment scores given by the different annotators, where an instance labeled as negative has received an average score less than or equal to 2.4 and an instance labeled as positive has received an average score to 3.6. instances with an average score in-between were labeled as neutral. the analysis of the agreement between annotators is available in @xcite . the value of annotators agreement on document level as measured by the cronbach's alpha is 0.903. in this paper we are interested only in documents with either positive or negative sentiment, which corresponds to 1665 positive and 3337 negative articles, respectively. note that the dataset is thus imbalanced towards the negative class (which is also matching the observations of media researchers that attention to negative news is disproportionate (e.g. @xcite ). ## methodology the methodology for named entity-based sentiment subgroup discovery consists of three steps. ## results for example, the interpretation of the first rule, loc nova @xmath1: articles that talk about the person igor bavčar and do not talk about the locations nova gorica and grad brdo pri kranju (corresponding to brdo pri kranju, the location of slovenian government's main venue for diplomatic meetings) are articles with negative sentiment. this rule covers 28 documents, out of which 26 are actually labeled with negative sentiment, while 2 are annotated with positive sentiment. rules describing articles with negative sentiment concern articles written about igor bavčar, a slovenian politician and manager. as politician, he played an important role during the slovenian independence period (he acted as minister of internal affairs in the beginning of the 90s and later as slovenian minister for european affairs) @xcite . in 2002, he withdraw from politics to engage in the private sector and was connected to financial affairs, which explains the negative sentiment association in our corpus. bavčar became the chairman of the istrabenz holding company in 2002. on 31 march 2009 bavčar resigned as the president of istrabenz due to poor business performance as the company was forced into financial restructuring. bavčar was indicted for disputed trading in istrabenz shares in 2017, when he attempted to take over the company. he was arrested in financial fraud investigation and released after 10 hours. 6 the discovered subgroups of articles with positive sentiment contain articles talking either about slovenia's former president, danilo türk, or the city of new york, new yorku. danilo türk is a slovenian diplomat, professor of international law, human rights expert, and political figure who served as president of slovenia from 2007 to 2012. note that the rules of discovered subgroups have a low precision (as the distribution of true positive and false positive is 53% vs. 47%). however one should take into account that the dataset is very imbalanced -with only 33% of articles annotated with a positive sentiment class label. these rules also show the main objective of subgroup discovery -discovering and describing groups of examples with significantly different class distribution to the the population class distribution rather than prediction. it is interesting that the positive sentiment for new york is connected to the absence of other named entities, closely related to new york including the former president of the united states, barack obama. another entity whose omission in news articles strengthens the positive senti-6 https://en.wikipedia.org/wiki/igor_ bav%c4%8dar , last accessed: 8 february 2021. ment associated with new york is the investment bank lehman brothers. on september 15, 2008, lehman brothers holdings, inc. sought chapter 11 protection, initiating the largest bankruptcy proceeding in united states (u.s.) history. it declared @xmath0613 billion in debts. at the time, lehman was the fourth-largest u.s. investment bank. despite being thought "too big to fail", the federal government did not employ extraordinary measures to save lehman, such as the enabling financing it had facilitated for j.p. morgan chase's purchase of a failing bear stearns just six months earlier. lehman's demise was a seminal event in the financial crisis that began in the u.s. subprime mortgage industry in 2007, spread to the credit markets, and then burned through the world's financial markets. the crisis resulted in significant and wide losses to the economy. @xcite other rules of positive sentiment with named entity new york, also include the absence of standard & poor's, an american credit rating agency and a division of s&p global (org standard & pools) that publishes financial research and analysis on stocks, bonds, and commodities, and the bank of america (org bank of america). the positive sentiment of danilo türk is related to the omission of the above mentioned politician involved in financial affairs, igor bavčar. the quality of the rules presented in ## conclusions in this paper, we presented a sentiment subgroup discovery approach through the lenses of named entities. experiments were performed on a corpora of slovene news data, manually labeled with positive and negative sentiment. our approach utilizes the fact that the sentiment in news articles is tied-up to their discussed named entities. we identified groups of news articles with negative and positive sentiment. the identified groups of articles were described by rules containing only one positive condition, confirming the presence of a named entity in an article. many of the identified subgroups for a chosen sentiment focus on the same entity. one of the key person entities associated with negative sentiment is igor bavčar -a polarizing slovenian politician and manager, whose presence in the news from 2009 is accompanied by negative sentiment due to r.nr. rule sentiment tp fp n1 loc nova @xmath2 in future work we will adapt the used algorithm for subgroup discovery to identify subgroups that are not overlapping, thus potentially involving multiple entities. stefan wrobel. 1997. an algorithm for multi-relational discovery of subgroups. in proceedings of the first european symposium on principles of data mining and knowledge discovery, pkdd 1997, pages 78-87.