dataesr/scientific-paragraphs-categorization huggingface dataset:
The low interest from the online community users that we targeted to partake in the annotation task meant that we had to rely on more traditional social networks, mainly professional and personal. Overall, the participation of non-experts and the number of data annotated through this process was lower than expected, but this does not diminish the validity of the methodology. A workaround solution would involve non-experts with existing interest in research, such as students. Furthermore, this research proposes a sophisticated definition of online Mexican antiimmigrant speech based on the empirical observation of the phenomenon. This definition is functional in the Mexican context but is also general enough to be used in other contexts. The greatest challenge that we had to overcome during the definition work was the rapid evolution of migration and migration policies, which transformed the terms of the online debate throughout the period of interest (2018-2022).
To address this limitation comprehensively, we have devised a strategy to leverage white box learners that enhance interpretability and transparency. We will implement a continuous monitoring mechanism to assess the frequency of incorrect predictions made by our classification models, enabling us to identify whether such mistakes are due to algorithmic factors or the introduction of new data in the corpus. With the same method, we can scrutinize the words associated with the highest correlated class and determine their semantic relevance to Mexican anti-immigrant HS, thus enabling our team to conduct thorough verification.
Conclusions and Future Work In conclusion, we describe in this paper the methodology used to create an ad-hoc antiimmigrant dataset to train automatic detectors of online anti-immigrant speech published in the Mexican context. We elaborate in detail on the methodology, which introduces elements not yet considered in the literature, and discuss the quality of the obtained data. The experiments conducted in this study provide valuable insights and answer the research questions. We have verified that the kappa coefficient value can be an efficient mechanism for quality annotations from non-experts. It also enhances the temporal efficiency by involving and utilizing contributions from non-expert annotators who meet the expert criteria, rather than relying solely on expert annotators (RQ1). We have confirmed that, given a sophisticated but concise guideline, non-expert annotators can detect almost all categories of anti-immigrant speech (RQ2), although some deeply rooted stereotypes remain undetected (RQ2A).
We have verified that the data obtained from experts and non-experts represent a valid input for any learning algorithm whenever there is a quality control mechanism in the annotation process (RQ3). An analysis based on a white box model proved that the annotation process induces semantic aspects directly associated with our study object, which allows us to assert that our annotation approach is somewhat context-sensitive (RQ4). Overall, we have demonstrated in this paper that it is possible to build a good-quality, context-sensitive training dataset for machine learning purposes by combining experts' and non-experts' annotations supervised by a control mechanism. As we move towards the creation and analysis of the Mexican anti-immigrant speech corpus, we must conduct a formal external validation of our classifier to ensure its effectiveness when dealing with real new data.
Acknowledgments: The authors would like to express their gratitude to  Oscar Gerardo Sánchez  and  Daniela Alejandra Moctezuma Ochoa  for providing the raw data. Special thanks to  Beatriz Zepeda Rivera  and  Loraine Morales Pino  for their expert annotations,  Isabel Sofía de la Cruz Abrín  for English revision, and all the non-expert annotators, who made a significant contribution to the development of this research work.
We study the impact of lockdown measures on beliefs about gender roles. We collect data from a representative sample of 1,000 individuals in France during the first COVID-19 lockdown in 2020. To measure beliefs about gender roles, we use questions from the 2018 wave of the European Values Study, and match respondents from the two surveys to compare beliefs before and during lockdown. We find evidence that the lockdown period was associated with a shift towards more traditional beliefs about gender roles. The effects are concentrated among men from the most time-constrained households and from households where bargaining with a partner over sharing responsibility for household production was likely to be an issue. Finally, we find evidence that beliefs about gender equality may be a luxury good: beliefs in equal gender roles increase with household income.
We study changes in beliefs about gender roles using data from a survey we designed during the first lockdown period in France, and that a survey institute conducted on a representative sample of 1,000 individuals from the French working population. We measure beliefs about gender roles by asking respondents about their opinions on six statements from the EVS. 2  We examine changes in beliefs about gender roles by combining responses to our survey with the responses of individuals from the latest wave of the EVS for France. 3 We perform a Nearest-Neighbor Match on respondents' observable characteristics from both surveys to estimate how the respondents to our survey would have likely responded before lockdown to the EVS questions about gender roles. We then use the predicted values to estimate changes in beliefs during lockdown.
, 2017). In previous research,  Pedulla and Thébaud (2015)  also found that women's support for equal gender roles tends to be more elastic than men's to the removal of household production constraints. Our results suggest that men's beliefs in traditional gender roles are more elastic than women's when household production constraints increase. This interpretation of our results suggests that beliefs in gender equality may be a luxury good. As household income increases, households can outsource household production more easily, which could lead to increased beliefs in equal gender roles. We examine this luxury good hypothesis by using cross-country European data from the latest wave of the EVS (pre-pandemic) to estimate the relationship between household income and beliefs about gender roles.
Our results are consistent with egalitarian beliefs about gender roles being a luxury good: throughout Europe, both men and women are more likely to believe in equal gender roles as they move up the income distribution. We also find that women tend to believe in more equal gender roles than men across the entire income distribution. The paper is structured as follows. Section 2 describes the data, including information on how we match respondents of the Lockdown and the EVS datasets. Section 3 provides descriptive evidence on beliefs about gender roles and household production constraints. Section 4 presents the results of our estimates of changes in beliefs about gender roles during lockdown, which can be related to household production constraints. Section 5 discusses our results by studying the relationship between beliefs about gender roles and income.
Fitted values Source: The data for beliefs about gender roles are from the fifth wave (2017) of the EVS (EVS, 2020). The data for enrolment rates are from the OECD Family Database, and are for 2017 or the latest year available. The OECD defines these enrolment rates as the "percent of children enrolled in early childhood education and care services (ISCED 0 and other registered ECEC services), 0-to 2-year-old". The EVS data are available at https: //europeanvaluesstudy. eu/methodology-data-documentation/survey-2017/. The OECD data are available at https://www. oecd. org/els/family/database. htm, Table  PF3. 2. Notes: This figure presents the correlation between the percentage of individuals who agree or strongly agree with the statement "a man's job is to earn money; a woman's job is to look after home and family" and the enrolment rates in early childhood education and care services in European countries.
The value of the Pearson correlation is 0. 86 and is significant at the 1% level. Source: The data for beliefs about gender roles are from the fifth wave of the EVS  (EVS, 2020). The data for maternal employment rates are from the OECD Family Database, and are for 2019 or the latest year available. The OECD defines maternal employment rates as employment rates for women (15-64 year olds) with at least one child aged 0-14, who are working full-time or part-time. The EVS data are available at https://europeanvaluesstudy. eu/ methodology-data-documentation/survey-2017/. The OECD data are available at https://www. oecd. org/els/ family/database. htm, Table  LMF1. 2. Notes: This figure presents the correlation between the percentage of individuals who agree or strongly agree with the statement "a man's job is to earn money; a woman's job is to look after home and family" and maternal employment rates in European countries.
Data In this section, we describe the datasets that we combine to perform our analysis. The first dataset is the "Lockdown survey", which includes information from the survey that we designed and that IPSOS, a survey agency, conducted on a representative sample of 1,000 working individuals in France during the first lockdown period, between May 4th and May 8th, 2020. 5  The second dataset is the "EVS survey", which includes data from the fifth wave of the EVS for France, from 2018. We then describe how we match respondents of both surveys to build the dataset for our examination of changes in beliefs about gender roles during lockdown. When the first lockdown in France occurred on March 17th, 2020, all Lockdown survey respondents were at least 18 years old and were either employed or independent workers.
The survey agency applied a quota sampling method to ensure that the respondents were representative of the French population, based on gender, age, professional activity, as well as the region and the type of environment (rural or urban) where the respondent lived at the time of the survey. 6  Since time constraints were an issue during lockdown, we opted for a short, ten-minute, online survey. The Lockdown survey includes questions from the EVS (2020) to measure respondents' beliefs about gender roles. The EVS dataset for France includes information collected between March 3rd and August 16th, 2018, two years before the first COVID-19 lockdown. We kept the same format as the EVS questions, asking respondents whether they strongly agree, agree, disagree, or strongly disagree with six statements about gender roles, which we describe in Section 3.
1. For each statement measuring beliefs about gender roles, we construct a binary variable equal to one if the respondent answered either agree or strongly agree, and zero if the respondent answered either disagree or strongly disagree. These six measures of beliefs are the main dependent variables of our analysis. After collecting the responses for each statement during lockdown, we match respondents from our survey with respondents from the EVS. Because we do not have panel data to measure within respondent variations in beliefs regarding gender roles before and during lockdown, we conduct a Nearest-Neighbor Match to estimate the Lockdown survey respondents' beliefs before lockdown. To have a matching set of respondents, we selected the 871 individuals in France from the EVS who were at least 18 years old and employed when they answered the survey in 2018.
The characteristics of respondents in the EVS survey are generally similar to the ones of the Lockdown survey. The main difference between the two datasets relates to education: respondents from the Lockdown survey have higher education on average than respondents from the EVS survey. The matching exercise takes into account such differences. We predict the Lockdown survey respondents' beliefs before lockdown by conducting a Nearest-Neighbor Match with Mahalanobis distances on the following characteristics: age, education category, marital status, the region the respondent lives in, the number of children living in the household, and the employment status of the partner before lockdown. We conduct an exact match on whether the respondent is female and whether the respondent has a child who is 12 years old or under living in the household.
More specifically, if beliefs in equal gender roles increase with income, then these egalitarian beliefs are a luxury good. We examine the relationship between beliefs about gender roles and income by using crosscountry data from the fifth wave of the EVS. We conduct a regression analysis where an individual's agreement with each statement is the dependent variable and household income decile is the main independent variable, controlling for gender, age, marital status, number of children, and country fixed effects. On the whole men make better business executives than women do.
Male Female Source: EVS (2018). Notes: We use data from 30 different countries to examine the relationship between beliefs in equal gender roles and income. The figure presents the estimates of an OLS regression, controlling for gender, age, number of children, marital status, and country fixed effects. We include all working individuals from the full dataset of the fifth wave of the EVS. Figure  8  depicts the predicted percentages of men and women who agree with each statement by household income decile. The results show that beliefs in gender equality are a luxury good: as income levels increase, both men and women are more likely to believe in equal gender roles. This relationship is strongest for the statements ascribing a role to women. We also find that women are more likely than men to agree with equal gender roles.
Estimation and matching alternatives Our estimations rely on the predictions from the Nearest-Neighbor Match. In this section, we present the results of several analyses we conducted to test the robustness of our estimates to different empirical strategies and alternatives to our baseline matching exercise. First, instead of matching respondents from the EVS survey and our Lockdown survey, we estimate our benchmark model using an OLS cross section analysis. Respondents from both surveys are representative of the French population, and descriptive statistics comparing respondents from both surveys (Table  1 ) suggest that both samples are comparable (the main difference between the two datasets concerns the education variable). Using this unmatched dataset, we find similar results to the analysis on the matched dataset.
Results in Table  A4  in the Appendix suggest that the lockdown period is associated with a statistically significant increase in beliefs in unequal gender roles across the same four out of six statements. Statement (2) is also significant in the unmatched dataset. Furthermore, the results from the unmatched dataset confirm that the main effects are different for men and women, and that men's beliefs are associated with a change during lockdown. Second, we run our baseline model directly on the matched data. Compared to the main analysis (Table  4 ), we use the control variables from the Nearest-Neighbor Match in this exercise. Table  A5  in the Appendix presents the results, which are similar in size and significance compared to the ones we present in Table  4. Finally, Table  A6  in the Appendix compares average treatment effects estimated using different respondent characteristics to conduct the match, and using either Mahalanobis distances (columns (1) to (  6 )) or Euclidean distances (columns (7) to (  12 )).
Social-desirability bias We check whether social-desirability or type-of-interview bias can explain our results. Indeed, respondents to our survey may have been more willing to express beliefs in unequal gender roles because our survey was conducted online, compared to the EVS survey which was conducted in person. We measure desirability bias by using data from the fifth wave of the EVS for six countries (Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland), where some respondents were interviewed in person and other respondents were surveyed online. In these countries, we compare respondents who answered each statement in the online version of the survey with the in-person interview version of the survey. In Panel A of Table  A7  in the Appendix, we show results of regressions where the coefficient on the variable Online survey measures the difference in the share of individuals who agree with each statement in the online version of the survey compared to the in-person interviews.
Acknowledgment This paper resulted from discussions in the network of the UNESCO Chair Fleuves et Patrimoine (River Culture), the development of the book River Culture -Life as a Dance to the Rhythm of the Waters, two research clusters at the universities of Tours (CITERES) and Strasbourg (LIVE), respectively, and the Research Initiative in Sustainability and the Environment (RISE). This contribution was peer-reviewed. Karl M. Wantzen's expertise covers ecology, biodiversity and human-nature interactions, specifically in continental hydrosystems. He obtained a PhD at the Max-Planck-Institute of Limnology, Plön, Germany. During an eight-year delegation at the Federal University of Mato Grosso, Brazil, he coordinated a research cluster on the ecology of the Pantanal wetland, contributing to the establishment of an MSc/PhD program and two national research institutes.
In fact, in the model, the time-dependence of a subject's actions only results from the explicit time-dependence of the cell colors and of their 3 best discovered cells. Again, we did not consider, say, time-dependent visit parameters (ε and α parameters), for the sake of simplicity of the model, but also due to the fact that identifying the possible time-dependence of these parameters with reasonable statistical accuracy would require a much larger dataset. Yet, despite the model's imperfection in reproducing certain observables, the worst agreement between experimental and model results typically remains within 2 experimental standard errors (for instance, see Fig. 2I  for Rule 1). Considering the number and diversity of observables that we have considered (see figures in the main text and the SI-Appendix), this level of agreement can be regarded as very satisfactory, suggesting that the model grasps the main ingredients of the actual visit and rating dynamics.
ACKNOWLEDGMENTS. This work was supported by grants from the  CNRS Mission for Interdisciplinarity  (project  SmartCrowd ,  AMI S2C3 ) and the  CNRS  Project  80   | Prime ALTHEA. T. B. was supported by a doctoral fellowship from the  CNRS. R. E. was supported by  Marie Curie Core Grant Funding  (grant no. 655235-Smart-Mass ).
By examining a selection of Western European countries and contrasting them with the United States, we align with the recent recommendation that the scholarship should move beyond 'Eurocentrism' (Castelli  Gattinara 2020). The article is organized as follows. First, we define the core and intertwined concepts of economic populism and sovereignism and discuss their connection with globalization in the context of the populist radical right. Next, we outline the objectives of our analysis, clarify the rationale behind our case selection and describe our methodology. We analyse data from an original cross-national survey conducted in September 2020, which covered France, Germany, Italy, Switzerland and the United States. The survey used a set of items specifically designed to capture economic populist and sovereignist attitudes, as well as attitudes towards economic globalization among the general public.
Data and methods The analysis conducted in this article is essentially exploratory. We analyse data from an original survey administered in September 2020 in five countries: France, Germany, Italy, Switzerland and the United States. The survey was conducted online by YouGov in France, Germany, Italy and the United States, while the Link Institute conducted it in Switzerland. The data consist of nationally representative samples of voting-age citizens, with about 2,000 respondents from each country. Participants were selected from their national panels by the two polling companies recruited via the Internet, employing quota sampling based on gender, age, occupation and municipality size, along with regional stratification. To maintain data quality, respondents who had completed the questionnaire in less than half the median time (i.
Therefore, we exclude it from subsequent analysis. It is worth noting that a similar EFA conducted for each individual country confirms the factor structure observed in the pooled dataset (see Appendix A4 in the Supplementary Material for more details). Next, we turn to CFA to assess the validity of the three-factor model derived from the exploratory factor analysis (see details in Appendix A5 in the Supplementary Material). We use multiple group confirmatory factor analysis (MGCFA) with a diagonally weighted least squares (DWLS) model designed for ordinal data, taking 'country' as the grouping variable. We allow the covariance between all three latent variables to be free in order to take into account the correlations observed between the factors from the preliminary EFA. The CFA fit statistics confirm that the data are consistent with the hypothesized three-factor structure.
RRP electoral support The final set of questions in this article concern how economic populist and sovereignist attitudes, in conjunction with views of globalization, may shape individual electoral support for RRP parties and leaders. Using the CFA factors, we test three distinct Mokken scales: economic populism (EcoPop), economic sovereignism (EcoSov) and positive views of globalization (Global). These scales serve as our main independent variables in the pooled dataset (see Appendix A6 in the Supplementary Material for more details). In the pooled data, the coefficients of homogeneity for all three scales exceed the 0. 4 threshold, which indicates 'moderate' Mokken scales and a sufficient level of homogeneity. Furthermore, all three scales satisfy the classic IRT assumptions of unidimensionality, local independence of items and monotonicity (refer to the summary table of scale properties in Appendix A6).
We anticipate that these effects will be visible alongside well-established correlates of the RRP vote, including the perception of a cultural threat posed by immigration, as well as attitudes to law and order. Importantly, these effects should also be independent of voter economic preferences within the domestic realm, which we assess by measuring attitudes towards economic redistribution (see Table  2  for a description of the attitudinal correlates used in this analysis). To start, we run a binary logistic regression analysis that compares RRP voters with all other voters in the pooled European dataset (see Model 1 in Appendix A8 in the Supplementary Material). We specify country-fixed effects to account for unobserved country variance. RRP voters are selected based on their responses to a voting intention question: RN in France (n = 234 voters against 696 for other parties), AfD in Germany (n = 135/970), Lega and FdI in Italy (n = 427/633) and the SVP in Switzerland (n = 258/655).
Acknowledgements. Previous versions of this article have been presented at the international Workshop 'Sovereignism as a Global Phenomenon?A  Comparative Perspective between Europe and the Americas', TU Dresden, March 2022 , and at the  ECPR Joint Sessions, April 2022. We are grateful to  Nicolò Conti ,  Marianne Kneuer  and the anonymous reviewers for their useful comments and suggestions. Oscar Mazzoleni  and  Gilles Ivaldi  https://doi. org/10. 1017/gov. 2023. 28Published online by  Cambridge University Press.