Changes in E uropean Solidarity Before and During COVID -19: Evidence from a Large Crowd- and Expert-Annotated T witter Dataset (2021)

Abstract: We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58% macro-F1 to almost 85%. We use this high-quality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.

## introduction social solidarity statements and other forms of collective pro-social behavior expressed in online media have been argued to affect public opinion and political mobilization @xcite @xcite . the ubiquity of social media enables individuals to feel and relate to real-world problems through solidarity statements expressed online and to act accordingly @xcite . social solidarity is a key feature that keeps modern societies integrated, functioning and cohesive. it constitutes a moral and normative bond between individuals and society, affecting people's willingness to help others and share own resources beyond immediate rational individually-, group-or class-based interests @xcite . national and international crises intensify the need for social solidarity, as crises diminish the resources available, raise demand for new and additional resources, and/or require readjustment of established collective redistributive patterns, e.g. inclusion of new groups. because principles of inclusion and redistribution are contested in modern societies and related opinions fragmented @xcite , collective expressions of social solidarity online are likely contested. such statements, which we refer to as anti-solidarity, question calls for social solidarity and its framing, i.e. towards whom individuals should show solidarity, and in what ways @xcite . for a long time, social solidarity was considered to be confined to local, national or cultural groups. the concept of a european society and european solidarity @xcite , a form of solidarity that goes beyond the nation state, is rather new. european solidarity gained relevance with the rise and expansion of the european union (eu) and its legislative and administrative power vis-à-vis the eu member states since the 1950s @xcite @xcite . after decades of increasing european integration and institutionalization, the eu entered into a continued succession of deep crises, beginning with the european financial crisis in 2010 @xcite . experiences of recurring european crises raise concerns regarding the future of european society and its foundation, european solidarity. eurosceptics and right-wing populists claim that social solidarity is, and should be, confined within the nation state, whereas supporters of the european project see european solidarity as a means to overcome the great challenges imposed on eu countries and its citizens today @xcite . to date, it is an open empirical question how strong and contested social solidarity really is in europe, and how it has changed since the onset of the covid-19 pandemic. against this background, we ask whether we can detect changes in the debates on european solidarity before and after the outbreak of covid-19. our contributions are: our data and code are available from https://github.com/lalashiwoya/ socialsolidaritycovid19 . in (computational) social science, several studies investigated the european migration crisis and/or the financial crisis as displayed in media discourses. these studies focused on differences in perspectives and narratives between mainstream media and twitter, using topic models @xcite , and the coverage and kinds of solidarity addressed in leftist and conservative newspaper media @xcite , as well as relevant actors in discourses on solidarity, using discourse network measures @xcite . while these studies offer insight into solidarity discourses during crises, they all share a strong focus on mainstream media, which is unlikely to publicly reject solidarity claims @xcite . social media, in contrast, allows its users to perpetuate, challenge and open new perspectives on mainstream narratives @xcite . a first attempt to study solidarity expressed by social media users during crises has been presented by @xcite . they assessed how emojis are used in tweets expressing solidarity relating to two crises through hashtag-based manual annotationignoring actual content of the tweets-and utilizing a lstm network for automatic classification. their approach, while insightful, provides a rather simple operationalization of solidarity, which neglects its contested, consequential and obligatory aspects vis-à-vis other social groups. the current state of social science research on european social solidarity poses a puzzle. on the one hand, most survey research paints a rather optimistic view regarding social solidarity in the eu, despite marked cross-national variation @xcite @xcite . on the other hand, the rise of political polarization and eurosceptic political parties @xcite suggests that the opinions, orientations and fears of a potentially growing political minority is underrepresented in this research. people holding extreme opinions have been found to be reluctant to participate in surveys and adopt their survey-responses to social norms (social desirability bias) @xcite @xcite . research indicates that such minorities may grow in times of crises, with both short-term and long-term effects for public opinion and political trust @xcite . our paper addresses these problems by drawing on large volumes of longitudinal social media data that reflect potential fragmentation of political opinion @xcite and its change over time. our approach will thus uncover how contested european solidarity is and how it developed since the onset of covid-19. ## data and annotations we use the unforeseen onset of the covid-19 crisis, beginning with the first european lockdown, enacted late february to early march 2020, to analyze and compare social solidarity data before and during the covid-19 crisis as if it were a natural experiment @xcite . in order to utilize this strategy and keep the baseline solidarity debate comparable before and after the onset of the covid-19 crisis, we confined our sample to tweets with hashtags predominantly relating to two previous european crises whose effects continue to concern europe, its member states and citizens: (i) migration and the distribution of refugees among european member states, and (ii) financial solidarity, i.e. financial support for indebted eu countries. the former solidarity debate predominantly refers to the refugee crisis since 2015 and the living situation of migrants, the latter mostly relates to the financial crisis, followed by the euro crisis, and concerns the excessive indebtedness of some eu countries since 2010. 1 data. we crawled 271,930 tweets between 01.09.2019 and 31.12.2020, written in english or german and geographically restricted to europe, to obtain setups comparable to the survey-based social science literature on european solidarity. we only crawled tweets that contained specific hashtags, to filter for our two topics, i.e. refugee and financial solidarity. we started with an initial list of hashtags (e.g., "#refugeecrisis", "#eurobonds"), which we then expanded via co-occurrence statistics. we manually evaluated 456 co-occurring hashtags with at least 100 occurrences to see if they represented the topics we are interested in. ultimately, we selected 45 hashtags (see appendix) to capture a wide range of the discourse on migration and financial solidarity. importantly, we keep the hashtag list associated with our 270k tweets constant over time. 2 definition of social solidarity. in line with social scientific concepts of social solidarity, we define social solidarity as expressed and/or called for in online media as "the preparedness to share one's own resources with others, be that directly by donating money or time in support of others or indirectly by supporting the state to reallocate and redistribute some of the funds gathered through taxes or contributions" @xcite . we define anti-solidarity as expressions that contest this type of social solidarity and/or deny solidarity towards vulnerable social groups and other european states, e.g. by promoting nationalism or the closure of national borders @xcite @xcite . ## methods we use multilingual bert @xcite / xlm-r @xcite to classify our tweets in a 3-way classification problem (solidarity, anti-solidarity, other), not differentiating between the classes ambivalent and non-applicable since our main focus is on the analysis of changes in (anti-)solidarity. we use the baseline mbert model: bert-base-multilingual-cased and the base xlm-r model: xlm-roberta-base. we implemented several data augmentation/transfer learning techniques to improve model performance: we also experimented with re-mapping multilin-gual bert and xlm-r @xcite as they have not seen parallel data during training, but found only minor effects in initial experiments. ## experiments in §5.1, we describe our experimental setup. in §5.2, we show the classification results of our baseline models on the annotated data and the effects of our various data augmentation and transfer learning strategies. in §5.3, we analyze performances of our best-performing models. in §5.4, we automatically label our whole dataset of 270k tweets and analyze changes in solidarity over time. ## conclusion in this paper, we contributed the first large-scale human and automatically annotated dataset labeled for solidarity and its contestation, anti-solidarity. the dataset uses the textual material in social media posts to determine whether a post shows (anti-)solidarity with respect to relevant target groups. our annotations, conducted by both trained experts and student crowd-workers, show overall good agreement levels for a challenging novel nlp task. we further trained augmented bert models whose performance is close to the agreement levels of the experts and which we used for large-scale trend analysis of over 270k media posts before and after the onset of the covid-19 pandemic. our findings show that (anti-)solidarity statements climaxed momentarily with the first lockdown, but the predominance of solidarity expressions was quickly restored at higher levels than before. solidarity and anti-solidarity statements were balanced by the end of the year 2020, when infection rates were rising. the covid-19 pandemic constitutes a worldwide crisis, with profound economic and social consequences for contemporary societies. it manifests yet another challenge for european solidarity, by putting a severe strain on available resources, i.e. national economies, health systems, and individual freedom. while the eu, its member countries and residents continued to struggle with the consequences of the financial crisis and its aftermath, as well as migration, the covid-19 pandemic has accelerated the problems related to these former crises. our data suggests that the covid-19 pandemic has not severely negatively impacted the willingness of european twitter users to take responsibility for refugees, while financial solidarity with other eu countries remained low on the agenda. over time, however, this form of expressed solidarity became more controversial. on one hand, these findings are in line with survey-based, quantitative research and its rather optimistic overall picture regarding social solidarity in the eu during earlier crises @xcite @xcite ; on the other hand, results from our correlation analysis suggests that severe strains during crises coincide with increased levels of antisolidarity statements. we conclude that a convergence of opinion @xcite among the european twitter-using public regarding the target audiences of solidarity, and the limits of european solidarity vs. national interests, is not in sight. instead, our widened analytic focus has allowed us to examine pro-social online behavior during crises and its opposition, revealing that european twitter users remain divided on issues of european solidarity.