dataesr/scientific-paragraphs-categorization huggingface dataset:
, by searching databases and bibliographies in published research, including the Oxford Bibliography of Young Carers  (Becker and Leu 2014, 1). Additionally, the authors have collected and collated grey literature, policy documents and information from their contacts and networks in countries where young carers' services have emerged or developed; and they have drawn on their site visits and knowledge of country-level responses gained from over twenty years of engagements with policy makers, governments and service providers internationally. The classification of the countries was developed based on the following five questions: 1. Does country X. recognise or do anything specifically about YCs? 2. Does country X. define them as YCs (or anything else i. e. , 'children as next of kin')? 3. What does country X.
To our knowledge, most social media studies regarding COVID-19 vaccine hesitancy in France are qualitative, and do not benefit from advanced and efficient machine learning. Our approach relies on FastText  [27] , which can provide better performance when the vocabulary contains many syntactic or orthographic variations of the same word. We privileged transfer learning that allows to learn with a limited amount of data or low computational capacity. There are now many pre-trained NLP models available, depending on the languages or texts used during training. Models in English were created to facilitate research, such as COVID-Twitter-BERT  [28]  which is trained on English tweets mentioning the COVID-19 pandemic. As such resources are not available in French, it is necessary to fine tune existing models for the classification of opinions on the announced vaccines before we may apply them to Twitter's data.
Overview This study was conducted in two steps that were designed to be accessible and easily adapted to any other subject at a specific time. The first step consisted in constructing a dataset of French tweets related to COVID-19. From this dataset, a subset of keywords relative to the word "vaccin" (vaccine in French) was selected. By restricting the dataset to the tweets containing at least one of these keywords, a dataset related to the topic of vaccine mistrust in French during the COVID-19 pandemic was reached. The second step focused on sentiment analysis. As this part requires machine learning, a small part of the restricted dataset needs to be hand labeled to fine-tune a pretrained model. After training and evaluating the model, label prediction was performed on the whole restricted dataset and the predictions were explored in terms of vocabulary and timeline.
Data Collection Data were collected before vaccines were available on the market. Between February and August 2020, about 700 million unique tweets were identified by the PanaceaLab team at Georgia State University  [29]  using a selection of keywords that are mainly COVID-19 designation variants, and the corresponding Ids were made available on GitHub. A new version is made available every week and collects approximately 3 million tweets per day. Once restricted to tweets in French, the database has yet to be restricted to the vaccination topic. On this purpose, word embeddings were trained on the whole French tweet dataset. Word embeddings are a way of representing a word of the vocabulary in a mathematical space. Words are transformed into vectors of a fixed number of dimensions to embed information about their meaning in the corpus.
For this reason, we trained our own word embeddings model in order to be specific to the content present in the tweets at the time of the study, and take into account associations between concepts relative to the pandemics. Since the data is composed of short, noisy messages with uncertain spelling, the word embeddings generated by FastText offer significant advantages. Notably, this method is considered to be fast and enables words with a similar spelling to be brought together by using parameter sharing. Most word embedding models learn a vectorial representation from the word's context, but as FastText also learns additional embeddings at the character level of the word, the decomposition of an unknown word enables to learn more relevant embeddings for rare words. The aforementioned technique named parameter sharing is an advantage since most models do not address the diversity of morphologically rich languages, such as French.
Accurate word representations are difficult to learn since many word forms occur too rarely in the training corpus. Parameter sharing also enables to handle the uncertain spelling observed in tweets. As word embedding models come with hyperparameters, tuning is necessary to fit the corpus at best. The metric used for that task is a criterion named the Discounted Cumulative Gain (DCG), which considers a user-defined collection of word pairs that are known to be close in meaning or context and computes the score of the cumulative closeness of the embeddings associated with the pairs. The higher this metric is, the more the model is expected to fit the corpus. To complete the restriction of the French tweets to a vaccine centered dataset, a semantic field of the word "vaccin" is built: by iterating from the word "vaccin", words close in the embedding space were added to the semantic field by computing their distance.
Data Preprocessing Data preprocessing was designed to be as minimal as possible and focused on two main tasks. The first one was to delete unnecessary information and the second was to lower the noise in texts. Firstly, the following steps were applied to the restricted dataset: URLs, punctuation, and special characters were removed. Tweets were then split into lists of lowercased words, and words that do not add any real value to the meaning (Stop Words) are also removed. In addition, some words such as "hydroxychloroquine" have many spelling variations. In order to suppress a part of the noise to obtain better performances, a spelling correction step was applied. We observed that correcting the 3 000 most common mistakes could correct around 70% of the misspelled words in the corpus. Thus, a dictionary of 3,482 words was manually created to correct this noise.
The negative ones evoke the potential vaccine in a mistrustful, and possibly conspiratorial way, and they seek to warn of its possible risks or manipulation or to relay information spreading doubt about its effectiveness. The neutral ones are unrelated to vaccines or do not contain any judgment about them. For training, a total of 4,548 tweets (1. 3% of the restricted dataset) were labelled, of which 26. 9% were negative, 21. 2% positive and 51,9% neutral. The classification task was therefore imbalanced, and this had to be addressed by using specific methods. Firstly, a stratified training method was required so that the training and test sets respect the proportion of each class. In order to compare methods, metrics must ensure that each class is best predicted not only as a whole, but also class by class.
Collected Data The initial dataset restricted to French tweets consisted of 894,315 unique words for 4,020,525 tweets. After training word embeddings of 300 dimensions, relations between words were explored, and 69 keywords were identified as the semantic concept of the word "vaccin". Using Principal Component Analysis (PCA), we displayed a projection of the vector representations of each keyword on a shared two-dimensional plan (Figure  2 ). This projection seems to group keywords into 3 clusters surrounding the word "vaccin", which could be summarized as: potential treatments (top left), conspiracy (bottom left), and pharmaceutical companies (right). The projection of the terms "vaccination" and "vacciner" (to vaccinate, in French) sets these words among the ones related to conspiracy theories terms like "complot" (conspiracy), "puce" (chip), and mentions of Bill "Gates" and George "Soros".
Although "vaccination" and "vacciner" have the same root as "vaccin", the embeddings captured nuances in their contexts. This information already unveils some polarity that is associated with the vocabulary. Tweets filtering based on word embeddings provided a specific dataset by restricting the collection to 344k tweets in French (around 9% of the initial French dataset). Filtering based on the "vaccin" word alone would have generated a subset of 75k tweets, thus ignoring a large number of tweets potentially related to aspects of vaccine mistrust. The results presented in the rest of this paper concern the 344k French tweets related to a potential vaccine against COVID-19. Figure  2. Two-dimensional projection of the 69 identified keywords, with the starting word "vaccin" in red, and "vaccination" and "vacciner" in blue for readability reason.
Conclusions Our study showed that Twitter could be a useful tool to investigate the arguments of vaccine mistrust. Our results unveil that political aspects of vaccination overshadow its usual criticisms about adverse drug reactions. As the opposition rhetoric is generally more homogenous and more widely spread than the positive rhetoric, we believe that this study provides effective tools to help health authorities better understand vaccine mistrust. ADZ implemented the embeddings and the classifier, and performed the analysis. BA collected data from the Panacelab github repository and extracted tweets from github, administered the server, and installed the necessary libraries and tools. ADZ and CB drafted the manuscript. AGB provided support on designing the study and reviewing results. All co-authors revised the article critically for important intellectual content and provided final approval of the version to be submitted.
This work was funded by the grant N° ANR-16-CE23-0011-01 from the ANR, the French Agence nationale de la Recherche through the PEGASE (Pharmacovigilance enrichie par des Groupements Améliorant la detection des Signaux Emergents) project. Exaion, a new subsidiary of the EDF group, a cloud provider of blockchain and highperformance computing solutions, lent one of its servers with two graphical processing units free of charge for the duration of the study. We thank the representatives of Exaion who helped us: Fatih Balyeli, Laurent Bernou-Mazars, Nicolas Meaux, and Vivien Sayve. We acknowledge the work of the Panacea Lab in collecting tweets related to the COVID-19 pandemics. The views mentioned in this article are those of the authors only.
In this project, we adopted a many-analysts approach to investigate two research questions using a new large cross-cultural dataset featuring N = 10,535 participants from 24 countries. Specifically, we recruited 120 teams to conduct analyzes in order to answer the following two research questions: (1) "Do religious people self-report greater well-being?," and (2) "Does the relation between religiosity and self-reported well-being depend on perceived cultural norms regarding religion?. " In the subsequent sections, we will first introduce our theoretical framework, dataset, and the many-analysts approach, before describing the key results with respect to the stated research questions and the varying approaches taken by the many-analysts teams. A general discussion of the project and the results is included in the closing article  (Hoogeveen et al.
A many-analysts approach In the current project, we aim to shed light on the association between religion and well-being and the extent to which different theoretically-or methodologically-motivated analytic choices affect the results. To this end, we initiated a many-analysts project, in which several independent analysis teams analyze the same dataset in order to answer a specific research question (e. g. ,  Bastiaansen et al. , 2020; Boehm et al. , 2018; Botvinik-Nezer et al. , 2020; Silberzahn & Uhlmann, 2015; van Dongen et al. , 2019). A many-analysts approach has been proposed as a way to mitigate the influence of individual-researcher biases (e. g. , confirmation bias by the proponent of a theory or disconfirmation bias by the skeptic), especially since the analysis teams are not typically invested in the outcome.
1 Inviting independent analysts from various backgrounds including but not restricted to religious studies attenuates this potential concern. Moreover, in addition to quantifying variability, with a sufficiently large number of analysis teams one can also investigate factors that might explain observed variability, such as those related to theoretical or methodological expertize and prior beliefs  (Aczel et al. , 2021). 2  In addition to the theoretical rationale for using a many-analysts approach to answer the research questions at hand, we also consider the current dataset particularly appropriate for such an approach. That is, the complexity of the data allows for many justifiable choices for the operationalization of the variables and the statistical approach to be employed. While the questions posed to the participants in the cross-cultural study could no longer be changed, the specific method of derivation for the religiosity and well-being scores was at the discretion of the many analysts.
At the same time, the research questions and data structure (cross-sectional correlational data) were sufficiently intuitive and manageable to inspire many researchers in the fields of (social) psychology, religious studies, health science, and general methodology to propose an analysis. Finally, we believe that our project involves a combination of elements that extend existing many-analysts work. First, we collected new data for this project with the aim to provide new evidence for the research questions of interest, as opposed to using an existing dataset that has been analyzed before. Second, we targeted both researchers interested in methodology and open science, as well as researchers from the field of the scientific study of religion and health to encourage both methodologically sound and theoretically relevant decisions (see the section "Analysis teams").
The dataset The dataset provided to the analysts featured data from 10,535 participants from 24 countries collected in 2019. The data were collected as part of the cross-cultural religious replication project (see also  Hoogeveen et al. , 2021; Hoogeveen & van Elk, 2018). The dataset contained measures of religiosity, well-being, perceived cultural norms of religion, as well as some demographic items. The full dataset, the data documentation file, and original questionnaire can be found on the OSF project page (osf. io/qbdce/).
Participants Participants were recruited from university student samples, from personal networks, and from (demographically representative) samples accessed by panel agencies and online platforms (MTurk, Kieskompas, Sojump, TurkPrime, Lancers, Qualtrics panels, Crowdpanel, and Prolific). Participants were compensated for participation by financial remuneration, the possibility for a reward through a raffle, course credits, or received no compensation. Everyone aged 18 years or above could participate. 3 Participants were required to answer all multiple choice questions, and hence there were no missing data (except for 36 people who did not provide a numeric age and 995 people who chose not to answer the item on sexual satisfaction, as this was the only item for which participants were not required to provide an answer.
) The countries were convenience-sampled (i. e. , through personal networks), but were selected to cover six continents and include different ethnic and religious majorities. The final sample included individuals who identified as Christian (31. 2%), Muslim (6. 1%), Hindu (2. 9%), Buddhist (2. 0%), Jewish (1. 0%), or were part of another religious group (2. 9%). Finally, 53. 9% of participants did not identify with any religion. See Tables  A1  and A2  in Appendix 1 for the full descriptive statistics of the dataset.
, 2022). Note that we did not include our own estimated effect sizes in the results as shown below. Our results, however, do corroborate the overall pattern of results from the analysis teams. Interested readers can access our preregistered analysis of the research questions on the OSF (osf. io/vy8z7/). All documents provided to the analysis teams (dataset, documentation, questionnaire), as well as the administered surveys, the anonymized raw and processed data (including relevant documentation), and the R code to conduct all analyzes (including all figures), can be found on the project page on the OSF (osf. io/vy8z7/). Identifying information (such as names, email-addresses, universities) was removed from all free-text answers. See also Table  2  for an overview of all resources.
e. , stage 2). Specifically, per research question, teams were asked about their statistical approach, the operationalization of the independent variable(s) and dependent variable(s), included covariates, analytic sample size, (unit of) effect size, p-value or Bayes factor, and additional steps they took for the analysis. Furthermore, for both research questions, the teams gave a subjective conclusion about the evidence for the effect (i. e. , "good evidence for a relation," "ambiguous evidence," or "good evidence against a relation"), about the practical meaningfulness/relevance of the effect (based on the data; "yes" or "no"), and indicated again the likelihood of the effects of interest (on a 7-pt Likert scale). Additionally, teams indicated the appropriateness of their statistical approach (7-pt Likert scale), the suitability of the dataset for answering each research question (7-pt Likert scale), and whether or not they deviated from their planned analysis.
Procedure After signing up, participating teams received a document outlining the aim of the project, the timeline, a short theoretical background with respect to the research questions, and a description of the dataset. Then, after completing the pre-survey, teams could access the full data documentation, the questionnaire as presented to the participants of the cross-cultural study, and either a blinded version of the data or a preregistration template, depending on which condition they had been assigned to. Teams could then design their analysis and upload their documents on their own team page on the OSF (deadline: December 22nd, 2020). The project leaders "froze" the stage 1 documents and sent the link to the mid-survey. Upon completion of this survey, teams automatically received access to the real data.
Specifically, for (multilevel) linear regression models we used the effectsize package or the jtools package to extract standardized coefficients in R. For analyzes in SPSS and non-standard models in R, we standardized the data manually prior to executing the analyzes. Finally, many teams reported multiple effect sizes, as they either separately considered multiple predictors (e. g. , religious beliefs and religious behaviors) and/or multiple dependent variables (e. g. , psychological well-being and physical well-being). In that case, we asked the teams to provide us with one primary effect size they considered most relevant to answer the research question or to select one randomly. In the Online Appendix, we additionally list (1) effect sizes for the different subscales of the well-being measure as reported by the teams and (2) effect sizes from teams that could not provide a beta coefficient (e.
Perceived suitability of dataset At the end of the project, all teams reported how suitable they found the current dataset for answering the research questions. As shown at the top of Figure  6 , most teams considered the data (very) suitable for answering the research questions: for research question 1, 86% found the data suitable, 8% neutral, and 6% unsuitable; for research question 2, 70% found the data suitable, 19% neutral, and 11% unsuitable.
Variable inclusion For each team we coded which of the items provided in the dataset were included as (1) dependent variable, (2) independent variable, and (3) covariates in the analysis for each research question. 7 4. 2. 3. 1 Dependent variable. The subjective well-being measure consisted of three subscales (psychological, physical, social), as well as two general items. In the dataset, we provided responses for all 18 individual items as well as an overall mean and one mean for each of the three subscales. Teams could decide to either use any of the provided averages or combine specific items themselves (e. g. , take the mean, median, sum). In addition, some teams conducted a factor analysis and used one or multiple extracted factors as the dependent variable. In this case, we coded which items were used as input for the factor analysis.
Independent variable. The religiosity measure consisted of 9 primary items on response scales ranging from dichotomous to 8-points and the cultural norms of religiosity measure consisted of two items on a 5-point scale. Averages were not provided in the dataset, but could be created by the teams themselves. Figure  8  shows the included items as independent variable aggregated Figure  6. Responses to the survey questions about the suitability of the dataset for answering the research questions (top) and the teams' confidence in their analytic approach (bottom). For question 1, the top bar represents the teams' answers with respect to research question 1 and the bottom bar represents the teams' answers for research question 2. For each item, the number to the left of the data bar (in brown/orange) indicates the percentage of teams that considered the data (very) unsuitable/were not (at all) confident in their approach.
Summary In the current project, 120 analysis teams were given a large cross-cultural dataset (N = 10, 535, 24 countries) in order to investigate two research questions: (1) "Do religious people self-report higher well-being?" and (2) "Does the relation between religiosity and self-reported well-being depend on perceived cultural norms of religion?. " In a two-stage procedure, the teams first proposed an analysis and then executed their planned analysis on the data. Perhaps surprisingly in light of previous many-analysts projects, results were fairly consistent across teams. For research question 1 on the relation between religiosity and self-reported wellbeing, all but three teams reported a positive effect size and confidence/credible intervals that exclude zero. For research question 2, the results were somewhat more variable: 95% of the teams reported a positive effect size for the moderating influence of cultural norms of religion on the association between religiosity and self-reported well-being, with 65% of the confidence/ credible intervals excluding zero.
All participants were treated in accordance with the Declaration of Helsinki. Descriptive statistics of the cross-cultural data. Here we show the variable inclusion for each team; TableA3displays the included items for the dependent variable for research question 1 and TableA4displays the included items for the dependent variable for research question 2. TableA5displays the included items for the independent variable for research question 1 and TableA6displays the included items for the independent variable for research question 2. Table A7 displays the included items as covariates for research question 1 and TableA8displays the included items as covariates for research question 2. Appendices Appendix 1. Descriptives of the dataset A. 1. Ethical approval cross-cultural data collection Country N Age (SD) Women Australia 482 47.
Research general background. Information regarding soil knowledge and soil education was gathered from 43 countries worldwide, which countries together are home to 62 % of the world's population (Fig. 1 ). Data collection required the involvement of soil scientists in contact with teachers, as well as the analysis of school textbooks (ISCED 3rd level). Due to the Covid-19 pandemic, there were a number of challenges, including school and library closures as well as limited contacts with relevant contributors. Similarly, it was not possible to include a number of countries where political, ideological or language factors presented insurmountable logistical problems. Potential participants in more than 50 countries were invited to cooperate, and 43 of these were able to obtain and contribute information.