Using Word Embeddings to Quantify Ethnic Stereotypes in 12 years of S panish News (2021)

Abstract: The current study provides a diachronic analysis of the stereotypical portrayals concerning seven of the most prominent foreign nationalities living in Spain in a Spanish news outlet. We use 12 years (2007-2018) of news articles to train word embedding models to quantify the association of such outgroups with drug use, prostitution, crimes, and poverty concepts. Then, we investigate the effects of sociopolitical variables on the computed bias series, such as the outgroup size in the host country and the rate of the population receiving unemployment benefits. Our findings indicate that the texts exhibit bias against foreign-born people, especially in the case of outgroups for which the country of origin has a lower Gross Domestic Product per capita (PPP) than Spain.

## introduction languages are complex and systematic instruments of communication that reflect the culture of a given population. by studying language, it is possible to observe stereotypes, a type of social bias that is present when discourse about a given group overlooks the diversity of its members and focuses only on a small set of features @xcite . as such, language analysis is a good way to depict, understand, and demonstrate stereotypes @xcite @xcite . nonetheless, like society, languages are not static. variations in lexical systems can be observed over time due to a myriad of intraand extra-linguistic factors. by analyzing extralinguistic aspects, it is possible to gain insights into the dynamics of social, cultural, and political phenomena reflected in texts @xcite . efficient methods for performing diachronic analysis are crucial, as manually evaluating several years of text collections is unfeasible due to the large amount of data involved. as such, computational methods for diachronic linguistic analysis are of utmost importance, and ongoing research shows that word embeddings models are helpful tools to this end @xcite @xcite @xcite . word embeddings are powerful representations of language, that allow for the quantification of relationships between words through efficient numerical operations inside the vector space. in this context, previous works demonstrated that such models contain machine-learned biases in their geometry that closely depict societal stereotypes @xcite @xcite , which is not surprising since stereotypes are massively present in texts used to train computational models @xcite . although such language models should be carefully tested for biases and not blindly applied to widely computational applications due to ethically concerning outcomes @xcite @xcite , they can be a valuable tool for enabling sociolinguistic analysis on large volumes of textual data. this topic establishes a collaboration between computer science, social sciences, and linguistics, as hypotheses about social phenomena can be tested on language using computational methods. in this study, we analyze the dynamics of stereotypical associations with seven nationalities, in the period of 2007 to 2018. we train our word embedding models using 1,757,331 news articles published in the spanish newspaper 20 minutos, for the aforementioned time span. we adopt a culturally diverse perspective by taking into account some of the most representative foreign nationalities that lived in spain in the aforementioned period according to the instituto nacional de estadística (ine) 1 . namely, british, colombian, ecuadorian, german, italian, moroccan, and romanian are included in this study. we conduct a fine-grained analysis, studying the association of such nationalities with drug use, prostitution, crimes, and poverty concepts. then, we compare our findings with sociopolitical variables, such as survey items from the european social survey (ess), number of residents by nationality living in spain, the rate of the population receiving unemployment benefits from the spanish government, and the number of offenses committed in spain by outgroup background. additionally, we investigate the effect of the outgroups' countries of origin having a lower gross domestic product per capita (ppp) than the host country (spain) 2 . to account for both group effects and error correlation, we use multilevel random effects (re) models in our analysis. this paper is organized as follows. in section 2 we discuss related works. subsequently, in section 3 we state our research questions, present metrics, data, model training, and evaluation. section 4 comprises the findings and discussion about results derived from this study. finally, in section 5 we present our conclusions, limitations, and future work. ## related work word embeddings showed as a valuable tool, by means of enabling efficient methods for analyzing and quantifying linguistic and social phenomena in natural language. in the context of model stereotypical bias analysis, which is the focus of this paper, the first disseminated studies concern gender bias @xcite @xcite @xcite . nonetheless, biases can exist in many shapes and forms, which can lead to unfairness in subsequent downstream tasks @xcite . garg et al., used both pre-trained models and models trained with the new york times annotated corpus to quantify gender and ethnic stereotypes in 100 years of data for the english language. the reported bias series showed strong correlations with census data and demographic changes in the united states for gender and ethnic stereotypes. similarly, kozlowski et al. analyzed english em- most works concerning the study of machine learned biases have english as target language, since there is more availability of linguistic resources that favors such analysis. here we cite four relevant works conducted on non-english target languages. wevers quantified gender biases in 40 years of dutch newspapers categorized ideologically as liberal, social-democratic, neutral/conservative, protestant, and catholic. the results depict differences in gender bias and changes within and between newspapers over time. @xcite the antisemitism in public discourse in france, by using diachronic word embeddings trained on a large corpus of french books and periodicals containing keywords related to jews. using the changes over time and embedding projections, they tracked the dynamics of antisemitic bias in the religious, economic, sociopolitical, racial, ethnic and conspiratorial domains. sánchez-junquera et al. detected stereotypes towards immigrants in political discourse by focusing in the narrative scenarios, i.e. the frames, used by political actors. they propose a taxonomy to capture immigrant stereotype dimensions and produced an annotated dataset with sentences that spanish politicians have stated in the congress of deputies. such dataset was used to train classifiers that detect and distinguish between stereotype categories. more similar to ours, is the work of kroon et al. in their study, the authors quantify the dynamics of stereotypical associations with different outgroups concerning low-status and high-threat concepts in 11 years of dutch news data. the authors investigate both time invariant and time variant hypotheses, focusing on the difference of associations regarding the group membership (ingroup vs outgroups). our study distinguishes itself from the aforementioned studies by (i) the interdisciplinarity with social survey research, as the selected survey questions measure attitudes of spanish people (the ingroup) towards immigrants (the outgroups) and can be interpreted as a proxy for cultural/economic threat perception; (ii) our choice of multilevel modeling (re model), to combine types of phenomena (linguistic and social) and account for group effects; and (iii) the use of fine-grained lists representing crimes, drugs, poverty and prostitution concepts to investigate stereotypical portrayals. additionally, we contribute to the scarce literature on stereotypical bias analysis with non-english data sources by using spanish from spain as a target language. ## method in this work, we aim to study the dynamics of the stereotypical portrayals of british, colombian, ecuadorian, german, italian, moroccan and romanian nationalities with drugs, prostitution, crimes, and poverty concepts, which are some of the stereotypical frames associated to immigrants in the literature @xcite @xcite @xcite . we investigate the effect that the gross domestic product per capita (ppp) of the outgroup's country of origin has in the strength of stereotypical association. namely, our hypothesis is that outgroups coming from countries with lower ppp than the host country (spain), are more strongly associated with such concepts, due to posing a greater economic threat to the ingroup @xcite 3 . then, we evaluate to what extent our findings can be explained by (i) the number of residents per nationality in spain (i.e, the size of outgroup); (ii) rates of population receiving unemployment benefits; (iii) the number of offenses committed in the spanish territory by outgroup background and; (iii) public opinion. in order to investigate such hypothesis, we adopt the following metrics, procedures and data. ## results and discussion in this section we discuss the findings and limitations of the present research. we analyse the dynamics of stereotypical associations comprised 9 series named "gdp per capita, ppp (current international $)" available in the world development indicators series. in 12 years @xcite @xcite @xcite @xcite @xcite @xcite of spanish local news published in the newspaper 20 minutos, comprising 1,757,331 news items, by training and analyzing yearly word embedding language models. our objective is to quantify stereotypes in such items towards the aforementioned outgroups, taking into account a cultural dimension by studying seven of the most prominent foreign outgroups living in spain considering the aforementioned period of analysis. we explore the hypothesis that outgroups coming from countries which have a lower ppp than the host country (spain), have stronger stereotypical associations with concepts related to crimes, drugs, poverty and prostitution, as a consequence of representing a greater social threat to the ingroup. the yearly average bias scores concerning concepts related to crimes and drugs are depicted in figures 3 and 4 . the trends in figure 3 show that, most of the outgroups are more strongly associated with the crimes concepts than the spanish ingroup. the colombian and the romanian are the outgroups more strongly associated with crimes concepts, while the german and the british are the two outgroups less associated. in fact, for most years, the bias score values are negative for the german and the british outgroups. in contrast, for the colombian, ecuadorian, morrocan, and romanian outgroups, bias score values are always positive. a similar pattern can be observed in figure 4 , in the case of stereotypes concerning drugs. the results of the random effects model for the aforementioned series are presented in to further inspect the effects of the lower ppp variable, we add interaction terms in model 2. for both series, there is a strongly significant relationship between lower ppp and unemployment benefits, such that when the rate of population receiving unemployment benefits increases, the stereotype association for colombian, ecuadorian, moroccan and romanian (lowerp p @xmath0, but decreases for german, british and italian outgroups. similarly, the interaction with the number of committed offenses in the drugs series reveals that an increase in the offenses lead to stronger stereotypical associations for the first outgroups, but not for the latter. for the series concerning crimes concepts, it is also possible to observe that the public opinion threat perception decreases as stereotypical associations increases. the yearly average bias scores for concepts related to poverty and prostitution are depicted in figures 5 and 6. for poverty related concepts, german, italian, and british bias score values are negative for most years, meaning that poverty concepts are actually more associated with the spanish ingroup when compared to such outgroups. the same is not true for colombian, ecuadorian, moroccan, and romanian outgroups. again, in figure 6 it is possible to observe that same division between outgroups. the descriptive analysis show that, overall, outgroups in the lower ppp classification exhibit stronger association with concepts related to prostitution and poverty. the concerning time effects, only the years 2009 and 2011 affect significantly the poverty series, while the year trend is not significant for the prostitution stereotypical associations. comparably to the findings described for the crimes and drugs concepts, the unemployment benefits predictor has a significant involvement with the dependent series, indicating discrepancy between lower and higher ppp groups. aside from the interaction with the unemployment benefits predictor, which has the same pattern described above for the crimes and drugs series, no other predictor interacts significantly with the lower ppp group. the strong effect of the lower ppp predictor on our analysis that news discourse emphasises the ethnicity of certain outgroups more than others. furthermore, the interpretation of main effects and interactions with sociopolitical variables indicates that stereotypical portrayals seem to be dissociated from real demographic trends. discourse is one of the everyday social practices that may be used for discriminatory purposes, for instance in intra-group discourse about resident minorities or immigrants frame these "others" negatively, thus leading to the reproduction of ethnic prejudices or ideologies @xcite . our findings go in line with frames described in other studies made with european newspapers, which indicate the semantic link between foreigners, prostitution, criminality and degeneracy @xcite @xcite @xcite , especially for eastern european and latin american backgrounds. we join previous studies pointing that media coverage can be stereotypical, associating ethnic outgroups with stigmatized attributes, and therefore having serious negative effects both on individuals and society, as news are powerful sources of the discursive demoralization of marginalised groups @xcite @xcite @xcite @xcite @xcite . we cite the following limitations of our findings. the present analysis considers only one data source, therefore our conclusions cannot be generalized to other spanish media outlets. although the unavailability of other diachronic corpora for spanish from spain limits our conclusion to a single news outlet, we argue that this study is a valuable contribution to stereotype analysis in media discourse using a non-english target language. further, we acknowledge that by excluding gender inflected words, stereotypes about women that could be informative were left out. we do wish to explore gender inflected words in future work with a more suitable dataset. lastly, we would like to point that all these nationalities have intricate and deep political relationships with spain which certainly go beyond having a higher or lower gpd per capita. ## conclusion in this work we analyzed the dynamics of stereotypical associations concerning seven of the most prominent ethnic outgroups living in spain using language models trained with 12 years of news items from the spanish newspaper 20 minutos. we investigated biases concerning concepts related to crimes, drugs poverty and prostitution, exploring the relation between the stereotypical associations and the gpd per capita (ppp) of the outgroups' countries of origin, public opinion, outgroup size, unemployment subsidy, and number of committed offenses in the spanish territory. our results show that the texts exhibit stereotypical associations, especially for the colombian, ecuadorian, moroccan and romanian outgroups. we conclude that the examined news articles emphasize the nationality of certain ethnicities, which hinder the integration process of already marginalized outgroups. moreover, these associations can be further propagated and amplified through computational algorithms if available data indiscriminately @xcite , leading to concerning outcomes. as future work, we aim to move to a multilingual perspective and compare outgroup stereotypes across different languages. furthermore, we wish to examine stereotypes in political discourse, to inspect if patterns similar to the ones found in this work can be observed.