dataesr/scientific-paragraphs-categorization huggingface dataset:
From CIVED 1999, the scientific monograph which was published in 2003 in the Slovene language was not open sourced. This means we could expect less "impact" because other reports, scientific monographs and papers from ICCS 2009 were open sourced and free. Starting with ICCS 2016 reports/monographs are printed in addition to being open sourced. Reports/monographs are sent to all schools who participated in the ICCS 2016. Therefore, we can expect an even bigger "impact" on educational practices. In the database published for the period 2011-2018 (based on ICCS 2009 and 2016 results, with criteria mentioned above) several results were found: 3 scientific monographs, 1 special issue of a scientific journal, 4 research reports, 8 articles, 1 bachelor's thesis, and 1 master's thesis  (Cobiss 2018 ), all of them in Slovene language.
Background: Early Career Development and Introduction to IEA Working with large-scale datasets had been part of my studies in sociology at Johns Hopkins University, where I analyzed data from the universe of public secondary schools in the United States (rather than samples of students). However, interest in the interplay between the contexts of schools and the lives of the students who attended them led me to pursue graduate study in human development at the University of Maryland (College Park), specializing in educational psychology. There, I joined the CIVED research team led by Judith Torney-Purta and Jo-Ann Amadeo, who had been with this project from its inception nearly 10 years earlier. The CIVED project's book of national case studies  (Torney-Purta et al. 1999 ) and report on international comparisons of 14-year-olds in CIVED  (Torney-Purta et al.
T. Grant Foundation to support this work had been funded. Independently, William Galston and Peter Levine had recently established the Center for Information and Research on Civic Learning and Engagement (CIRCLE), in the School of Public Affairs at the University of Maryland (and later moved it to Tufts University in Massachusetts, where it remains influential). CIRCLE's leadership was interested in disseminating brief reports of empirical studies to scholarly, practitioner, and policymaker audiences interested in youth civic engagement. This provided a way to disseminate short user-friendly presentations of findings from CIVED secondary analyses on topics such as media usage  (Amadeo et al. 2004)  or patterns of civic knowledge  (Torney-Purta and Barber 2004 ). The team from the University of Maryland also obtained funding from CIRCLE to establish the Civic Education Data and Researcher Services Center (CEDARS), whose goal was to encourage the use of data from the IEA civics studies by providing technical support, short reports for interested researchers, and enhanced datasets including scores on scales not included in the international data files.
A collaborator in this work on the expanded datasets was Vera Husfeldt, who visited Maryland in 2005 after her decade of work in the international coordinating center for CIVED at Humboldt University  (Husfeldt et al. 2005). This report, available on the University of Maryland website, details eight additional attitudinal scales derived from the IEA CIVED data, including attitudes toward ethnic minority groups and protective attitudes toward one's nation to supplement the 12 scales available on the IEA data sets. 1 During doctoral studies I co-authored a number of articles, technical reports, and research briefs communicating CIVED research results to policymakers and educators while also working as a consultant on statistical issues and on substantive issues pertaining to classroom and school climate.
In 2007, I accepted a faculty position at an urban research university in the Midwestern United States, with the expectation of sustaining a coherent line of scholarship. Analysis of the civics studies (first CIVED, then later the IEA International Civic and Citizenship Education Study [ICCS] of 2009) served as the core of this scholarly agenda. Taking a faculty position also meant expanding roles as a mentor. This resulted in a research team to encourage students' use of several large-scale datasets to answer questions of interest to counseling and educational psychologists (e. g. ,  Barber and Ross 2018, 2020; Barber et al. 2015). The communities of practice idea served as a model when forming this team. Arriving at the University of Maryland just as potential for secondary analyses of the CIVED data was being recognized created a unique experience for professional development.
Contextualized Approaches to the Study of Educational Psychology Like other branches of the discipline, educational psychology has historically relied on studies from non-representative and local data sources (a fact that Duncan, 1991, documented and lamented in developmental psychology several decades ago). However, results from such studies have limited contextual variability and often fail to generalize meaningfully to broader groups. Enhanced external validity is a benefit of conducting secondary analyses on nationallyrepresentative datasets. One key way in which work with the CIVED/ICCS studies can have an impact upon educational psychologists is by making them sensitive to issues of context and generalizability. Mueller and Hart (2008)  noted that a benefit of large-scale surveys, above and beyond their generalizability, is the ability to understand which aspects of context are most relevant to psychological and educational processes and outcomes.
g. ,  Barber et al. 2013; Sandoval-Hernandez et al. 2018; Torney-Purta et al. 2008). With large sample sizes it is also possible to conduct robust analyses comparing predictors of student outcomes by gender, ethnic identity, or immigrant status  (Barber and Torney-Purta 2009; Barber et al. 2015; Torney-Purta et al. 2007). For example, within the broader community of psychologists,  Diemer and Rapa (2016) , and  Godfrey and Grayman (2013)  have used CIVED data from the United States to examine the experiences of members of minoritized racial groups using moderation or multi-group techniques. Many benefits of working with large-scale databases for educational researchers apply regardless of content area; in other words, similar benefits would be gained from looking at large-scale studies of math, science, or literacy.
2018 ) and secondary analyses incorporating these variables were featured prominently in an examination of learning environments as supports for civic reasoning and discourse sponsored by the United States-based National Academy of Education  (Barber et al. 2021). The focus on participatory intensions and on attitudes alongside civic knowledge as outcomes has expanded interest in studying how context may shape key aspects of development. An early product of the team at Maryland was a "28-country database" containing national indicators for countries participating in the CIVED study. While many of these variables can be studied in relation to knowledge (a classic outcome for educational psychologists regardless of subject area), indicators added to this data source have also been used to predict citizenship norms (including those pertaining to human rights), inclusive attitudes toward participation, and political self-efficacy  (Barber et al.
2005 ). This remains one of the few attempts to link specific teachers with their classes. To place the contributions of such work in perspective, it is important to note a conclusion in a handbook chapter describing the potential contribution of secondary data sources (such as CIVED and ICCS) to research in social studies education. Heafner et al. (2016)  concluded that the lack of training to analyze data from complex surveys has been a barrier to many researchers who have substantive interests in research on social studies and civic engagement. While it is not necessary to receive a degree in statistics to work with large-scale data, developing methodological expertise is vital. While some challenges (such as appropriately utilizing sample weights) are characteristic of work with large-scale data across subject areas, working with civics datasets can be especially challenging due to the focus on attitudinal/behavioral data measured on rating scales rather than the right/wrong items found in assessments of knowledge.
As outlined both in the technical reports for CIVED  (Schulz and Sibberns 2004)  and ICCS  (Schulz et al. 2011) , as well as several secondary data analyses (e. g. ,  Husfeldt et al. 2005; Munck et al. 2018) , polytomous items add another level of difficulty to the already complex measurement characteristic of large scale assessments. In many cases, researchers working with CIVED/ICCS data expand their community of practice by networking with others interested in effectively using large-scale datasets, in order to remain aware of up-and-coming methodologies, analytic tools, and data sources. Special interest groups (SIGs) in the American Educational Research Association (AERA) and the Comparative International Education Society (CIES) have been instrumental in developing and sustaining this network, and researchers working with the civics data have served in leadership roles within these groups.
Cross-disciplinary Collaborations In these groups organized around methodology, membership is drawn from across disciplinary backgrounds. In fact, some of the most valuable discussion takes place when subject-area educators, educational policy scholars, and methodologists have the opportunity to communicate. This speaks to another feature of work on CIVED and ICCS: work with these datasets brings together researchers from multiple disciplines. Scholars such as  Duncan (1991)  and  Friedman (2007)  have long highlighted the potential for secondary analyses to support opportunities for interdisciplinary collaboration among scientists. Such opportunities are particularly prominent in the CIVED and ICCS studies, as the measures they included have been drawn from a wide range of areas and have received widespread interest from a range of social and behavioral scientists: political scientists, educational researchers (especially those interested in social studies), psychologists, sociologists, comparative education researchers, social policy specialists, and political psychologists (a feature noted in the literature review from  Knowles et al.
Established and Recent Themes in the European Research Landscape Making Use of IEA Civic and Citizenship Education Studies The educational policy initiatives and priority areas were accompanied by prolific empirical research on civic and citizenship education making use of IEA data. Indeed, the body of comparative research including European countries and based on data from the IEA civic and citizenship education studies has been growing considerably, sometimes with implicit connections to European issues. For example, a recent summary of research  (Knowles et al. 2018 ) identified over 100 published articles reporting secondary analysis of CIVED 1999 and ICCS 2009 data. A large proportion of these publications are international comparative analyses that included several European countries, but some also report analyses focused on individual European countries.
Most use data from the international instruments, and a few use data from the European regional module. A partially overlapping list of publications with a few more recent entries can be found at the website of the CivicLEADS Project (Regents of the University of Michigan 2020). Within and across countries groups of collaborating researchers are coalescing around common interests in the dataset as a whole. The contents of these publications to a large extent address policymakers' needs for comparative data on outcomes and processes of citizenship education: understanding what appears to be successful in European school systems for the promotion of civic and citizenship competences (i. e. , enhancing the knowledge of democracy, trust in democratic institutions, citizenship efficacy, civic engagement, acceptance of those who come from diverse backgrounds).
Indeed, existing research, recently enriched by empirical articles reporting secondary analyses of ICCS 2016 data, falls largely into two categories: a) comparative studies focused on citizenship competences in youth, and b) comparative and country-specific studies focused on structural and learning approaches to citizenship education and their potential impact on youth citizenship competences. In what follows, I give a brief overview of topics approached within these two research themes that have been addressed by secondary data analysis of IEA studies. Then, I point out some recent conceptual and methodological developments, which I illustrate by a few (by no means exhaustive) examples of empirical studies.
Studies Focused on Citizenship Competences in Youth To give an overview, several studies using data from the IEA civic and citizenship education studies were dedicated to defining and then comparing citizenship competences in European youth. Reflecting both societal concerns and policy initiatives, some projects created composite or summary measures of citizenship competences  (Hoskins et al. 2011). Some other studies focused on comparing attitudinal measures such as young people's institutional trust, voting intentions, and their attitudes toward support for immigrants' rights and gender equality  (Barber and Ross 2018; Isac et al. 2019; Malak-Minkiewicz 2007; Miranda and Castillo 2018; Mirazchiyski et al. 2014; Torney-Purta and Barber 2011). Moreover, other research addressed comparisons of citizenship norms or perceptions of "good citizenship" behaviors among youth  (Hooghe and Oser 2015; Hooghe et al.
2019; Miranda and Castillo 2018; Munck et al. 2017). These studies demonstrated that it is important to assess cross-cultural comparability (or measurement invariance) when comparing attitudinal measures across countries and over time. The IEA studies focus considerable attention on the validity and comparability of measurement instruments, but cultural, linguistic, and geographic differences do exist and should be the explicit focus of secondary analysis of ICCS data  (He and Van de Vijver 2013; Rutkowski and Svetina 2017). Here it is important to note that their findings indicated that European young people tend to be, on average, supportive of immigrants' rights. However, these attitudes show high variability within countries. For example, in ICCS 2016 Swedish students have both the highest average scores in Europe and the widest gap between the average scores of the students in the 5th and 95th percentile (see  Isac et al.
2016; Reichert 2016; Reichert et al. 2018; Torney-Purta and Barber 2011). This approach applies in particular to the topic of young people's citizenship norms. For example, guided by Dalton's theoretical work and studies of adults on norms of civic duty (2008), researchers have used IEA data and employed latent class analysis to identify groups of students with different perceptions of "good citizenship" behaviors  (Hooghe and Oser 2015; Hooghe et al. 2016). Consistent with Dalton's distinction, their results indicate that a large proportion of young people in a range of democracies (38 countries in ICCS 2009) express either "engaged citizenship" norms (highly endorsing the protection of human rights and community involvement, while downplaying the importance of traditional duty-based political participation) or "dutybased" citizenship norms (highly endorsing traditional political participation).
Consistent with the findings of  Hooghe et al. (2006) , ongoing work with ICCS 2016 data  (Treviño et al. 2021, in press)  further confirms these findings by identifying five types or profiles of citizenship norms. These are comprehensive, socially engaged, duty-based, monitorial, and anomic. They find that across the European countries participating in ICCS 2016 large proportions of young people can be regarded as involved citizens who tend to hold holistic visions of good citizenship that are not limited to the fulfillment of duties but incorporate aspects of respect for the environment and human rights into their behaviors.
In addition, these substantive and methodological approaches have been used to examine interactions between factors situated at different levels (e. g. , interactions between student background characteristics such as gender or socioeconomic background and aspects of class or school climate). Furthermore researchers have focused on several explanatory variables that can vary in their impact depending on school or country contexts. Sandoval-Hernández et al. (2018) , for example, used a framework rooted in ecological systems theory and educational effectiveness research to examine factors and conditions that have the potential to promote positive attitudes toward the rights of immigrants, ethnic groups, and women. This book-length report of analysis commissioned by IEA used data from 38 countries participating in ICCS 2009 (of which 26 were European).
e. , acquiring information about social, political, and economic issues in other European countries; opportunities to interact with European peers) and young people's feelings of belonging to Europe and to the EU (i. e. , European identity). They found a particularly strong association between cognitive learning opportunities (i. e. , opportunities to learn about social, political, and economic issues in other European countries) and European identity. This finding was replicated and further enriched by the study of  Keating (2016)  that made use of the same dataset (for 20 EU countries) and established a link not only between specific learning opportunities and young people's European identity but also between these opportunities and their endorsement of equal rights for other European citizens.
Secondary Analyses of Teacher Questionnaire Data In the last 20 years, a number of secondary analyses were carried out using CIVED and ICCS teacher questionnaire data. Two recent reviews of the research conducted across different disciplines published in English have highlighted the multifaceted nature of the studies that used datasets from the IEA large-scale international surveys on civic and citizenship education  (Knowles et al. 2018; Knowles and Di Stefano 2015). The 2015 piece was preliminary to the extensive search for secondary analysis published in 2018, which found and summarized 100 articles using CIVED and ICCS 2009 data. In relation to the studies using the teacher questionnaire data, we have identified two main strands of academic research in these reviews. The first strand focused on teachers' beliefs and teachers' confidence in teaching CCE-related topics, and the second aimed at understanding the associations between specific teachers' characteristics or practices and students' civic knowledge.
2010, p. 182). Reducing xenophobia was also rarely endorsed as an aim of civic education. An average of only 10% of teachers across countries answered that this was one of the three most important objectives. On the other hand, promoting students' knowledge of citizens' rights and responsibilities and students' critical and independent thinking were frequently chosen by teachers as of among the three most important aims (with an average across countries of 60 and 52% respectively). In the second strand, we included studies based on a CIVED dataset such as the article by  Chin and Barber (2010) , which examined the relation between teachers' beliefs and their organization of classes and student learning in Australia, England, and the United States, and also the contribution of  Torney-Purta et al.
It can be said that some of the main achievements of ICCS in Latin America have been the development of technical expertise for the evaluation of civic and citizenship education, impact on the policy and practice of the countries that have participated in the IEA studies, and the establishment of a community of researchers (many at universities) who are interested in the development of in-depth studies based on secondary analyses of ICCS data. In this chapter, we focus on the academic products of this community of researchers, a characterization of the topics that have particular relevance for the region and a mapping of the specific overlaps between these topics and the contents of the ICCS datasets. In the remainder of this chapter, we outline a landscape of the research outputs of ICCS in Latin America, as well as some specific suggestions to raise the profile of ICCS as a key resource for enhancing future discussions on civic education in the region.
Lack of technical documentation and a limited number of training events available in Spanish The communications team at IEA does a great job at disseminating their studies and results, and the Research and Analysis Unit does an excellent job at offering options to build capacity in secondary analysis techniques of their datasets. Good examples of this are the offering of a very comprehensive Training Portfolio, the video tutorials available on the web, and the IERI Academies in collaboration with the Educational Testing Service (ETS). 3 All these efforts, however, use English as the main language of instruction. This is a major barrier for many researchers in Latin America since English is not widely spoken in the region. According to the English Proficiency Index (Education First 2018), Latin America is the weakest of all regions in the world, with an average English proficiency score barely surpassing the low proficiency cut-off.
Develop open-access software to analyze large-scale assessment data Similar to the technical documentation and training events, the IEA has been doing a very good job at making tools for analyzing large-scale assessments data freely available. The IEA's IDB (International Database) Analyzer, Data Visualizer, and the NCES (National Center for Education Statistics) Data Explorer are good examples of this. The Data Visualizer, however, can be used to produce only very basic visualizations of descriptive statistics; the NCES Data Explorer can only be used with a limited number of studies (i. e. , PISA, TIMSS, PIRLS, PIAAC, and TALIS). The IDB Analyzer offers greater flexibility and the possibility of running slightly more complex analyses (e. g. , linear and logistic regressions). The main limitation of the IDB Analyzer is, however, that it uses either SPSS (e.
One invaluable benefit of IEA studies is their provision of databases for secondary analyses. For example, the datasets from IEA studies of civic and citizenship education are also available at CivicLEADS. org and have been widely used. A recent review found about 100 articles reporting secondary analysis of CIVED and ICCS 2009 data  (Knowles et al. 2018). Across the first ICCS cycles some measures were based on exactly the same question stem, item wording, and response category format. However, comparisons between CIVED 1999 and later administrations should be undertaken with caution (except for the content knowledge subscale for which equated scales score were provided in the ICCS 2009 database). The development of the IEA studies on civic and citizenship education since 1999 shows that there is a need to adapt study content to new developments, as well as occasionally to update design features and methodological aspects.