EMBEDDIA hackathon report: Automatic sentiment and viewpoint analysis of S lovenian news corpus on the topic of LGBTIQ + (2021)

Abstract: We conduct automatic sentiment and viewpoint analysis of the newly created Slovenian news corpus containing articles related to the topic of LGBTIQ+ by employing the state-of-the-art news sentiment classifier and a system for semantic change detection. The focus is on the differences in reporting between quality news media with long tradition and news media with financial and political connections to SDS, a Slovene right-wing political party. The results suggest that political affiliation of the media can affect the sentiment distribution of articles and the framing of specific LGBTIQ+ specific topics, such as same-sex marriage.

## introduction quantitative content analysis of news related to lgbtiq+ in general, and specifically, to marriage equality debates show that distinctions can be drawn between those media articles that express positive, neutral or negative stance towards samesex marriage. those media articles that express positive stance are grounded in human rights/civil equality discourses and access to benefits @xcite @xcite , and frame marriage equality as an inevitable path towards equality, as a civil right issue that would reduce existing prejudices and discrimination, and protect threatened lgbtiq+ minority @xcite . for media articles that express negative stance towards marriage equality, distinctive discursive elements are present, such as "equal, but separate" (marriage equality should be implemented, but differentiating labels should be kept in the name of protecting the institute of marriage) @xcite @xcite , and reference procreation/welfare of children @xcite , public objection @xcite and church -state opposition @xcite . the related work also shows that the differences between "liberal" and "conservative" arguments are not emphasised, mostly because both sides refer to each other's arguments, if only to negate them; yet, political orientation can be identified through the tone of the article @xcite . when it comes to methods employed for automatic analysis of the lgbtiq+ topic, most recent approaches rely on embeddings. @xcite employed embeddings to research how words (among them also word gay) change meaning through time. they built static embedding models for each time slice of the corpus and then make these representations comparable by employing vector space alignment by optimising a geometric transformation. this research was recently expanded by @xcite , who employed embeddings to explore semantic shifts of six descriptive lgbtiq+ words from the 1860s to the 2000s: homosexual, lesbian, gay, bisexual, transgender, and queer. there are also several general news analysis techniques that can be employed for the task at hand. @xcite developed a system for semantic shift detection for viewpoint analysis of political and media discourse. a recent study by @xcite tried to identify biased terms in news articles by comparing news media outlet specific word embeddings. on the other hand, pelicon et al. (2020) developed a system for analysing the sentiment of news media articles. while the above described analyses in a large majority of cases covered news in english speaking countries, in this research, we expand the quantitative analysis to slovenian news, in order to determine whether attitudes towards lgbtiq+ differs in different cultural environments. we created a corpus of lgbtiq+ related news and conducted an automatic analysis of its content covering several aspects: the research was performed in the scope of the embeddia hackashop (hackaton track) at eacl 2021 and employs several of the proposed resources and tools @xcite . ## methodology for sentiment analysis we used a multilingual news sentiment analysis tool. the tool was trained using a two-step approach, described in @xcite . for training, a corpus of sentimentlabeled news articles in slovenian was used @xcite with news covering predominantly the financial and political domains. this model was subsequently applied to the lgbtiq+ corpus where each news article was labeled with one of the sentiment labels, namely negative, neutral or positive. this allowed us to generate a sentiment distribution of articles for each media source in the corpus. for word usage viewpoints analysis, we applied a system originally employed for diachronic shift detection @xcite . our word usage detection pipeline follows the procedure proposed in the previous work @xcite : the created lgbtiq+ corpus is split into two slices containing news from different news source according to procedure described in section 3. next, the corpus is lemmatized, using the stanza library @xcite , and lowercased. for each lemma that appears more than 100 times in each slice and is not considered a stopword, we generate a slice specific set of contextual embeddings using bert @xcite pretrained on the slovenian, croatian and english texts (ulčar and robnik-šikonja, 2020). these representations are clustered using k-means and the derived cluster distributions are compared across slices by employing wasserstein distance @xcite . it is assumed that the ranking resembles a relative degree of usage change, therefore words are ranked according to the distance. once the most changed words are identified, the next step is to understand how their usage differs in the distinct corpus slices. the hypothesis is that specific clusters of bert embeddings resemble specific word usages of a specific word. the problem is that these clusters may consist of several hundreds or even thousands of word usages, i.e. sentences, therefore manual inspection of these usages would be time-consuming. for this reason, we extract the most discriminating unigrams, bigrams, trigrams and fourgrams for each cluster using the following procedure: we compute the term frequency -inverse document frequency (tf-idf) score of each n-gram and the n-grams appearing in more than 80% of the clusters are excluded to ensure that the selected keywords are the most discriminant. this gives us a ranked list of keywords for each cluster and the top-ranked keywords (according to tf-idf) are used for the interpretation of the cluster. ## conclusions we conducted a content analysis of the slovenian news corpus containing articles related to the topic of lgbtiq+. the sentiment analysis study shows that there are some differences in the sentiment of reporting about lgbtiq+ between two distinct groups of media and that the three media houses connected to political right tend to cover the subject in a more negative manner. this supports the thesis by @xcite , who suggested that political orientation can be identified through the tone of the article. nevertheless, the obtained results should be interpreted with the grain of caution, since the sentiment classifier we employed cannot distinguish whether it is the stance expressed towards the lgbtiq+ community, or is it rather the event on which the article is reporting, that is positive or negative (e.g., an attack on the lgbtiq+ activist). the distinction between these two "types" of sentiment will be analysed in the future work. the viewpoint analysis suggests that the usage of some specific words has been adapted in order to express specific ideological point of view of the media. for example, the analysis of the word relationship suggests that the more conservative media more likely frame lgbtiq+ relationships as a partnership of two homosexual (or even polygamous) partners. on the other hand, they rarely consider lgbtiq+ relationships as family or talk about marriage. in the future we plan to conduct topic analysis of the corpus in order to identify the most common lgbtiq+ related topics covered by the news media. we will also employ embeddings to research relations between lgbtiq+ specific words.