dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements. This research was supported by the  Institute of Education Sciences, U. S. Department of Education , through Grant  R305A150155  to the  University of Pittsburgh. The opinions expressed are those of the authors and do not necessarily represent the views of the Institute or the  U. S. Department of Education. We thank  Sarah Birmingham ,  Dennis Lusetich , and  Scott Silliman  for their contributions. Acknowledgements. The authors wish to thank  Patrice Lombardo , head of the  IFSI, Centre Hospitalier Annecy -Genevois , who helped make this research possible, and  Jean-Luc Rinaudo ,  University of Rouen , for his valuable input all along the phases of this research. This research was partially supported by the  FP7 2008-212578   LTfLL  project, by the  644187 EC H2020   Realising  an  Applied Gaming Eco-system  (RAGE) project, as well as by  University Politehnica of Bucharest  through the " Excellence Research Grants" Programs   UPB-GEX 12/26.
Acknowledgements. We thank the team of bachelor students of  Radboud University  for collecting data, as well as master student  Liset Onnink  and post-doc  Karly van Gorp. Acknowledgements. We would like to thank  Sorbonne Universités  for funding this research project and teachers, but also students, who accepted to participate to this experiment. Acknowledgements. This research has been partially funded by the  European Union  in the context of  CEITER  and the  Next-Lab (Horizon 2020 Research and Innovation Programme , grant agreements no. 669074  and  731685 ). Acknowledgements. This research has been partially supported by the  Junta de Castilla y León, Spain  ( VA082U16 ) and  Ministerio de Economía y Competitividad, Spain  ( TIN2014-53199-C3-2-R ). The authors thank the rest of the  GSIC/EMIC research team , as well as  Roberto Castellanos  and the  Canvas  team for their valuable ideas and support.
The authors also thank the  Spanish network of excellence SNOLA  ( TIN2015-71669-REDT ). Acknowledgements. We would like to thank the  SURF Foundation & NRO  for supporting the efforts of  Marcel Schmitz ,  Evelien van Limbeek  and  Hendrik Drachsler  under the  REFLECTOR  project grant. Acknowledgements. This work is partially supported by the  eMadrid  project (funded by the  Regional Government of Madrid)  under grant no  S2013/ICE-2715 , the  Commin  project (funded by the  Spanish Ministry of Economy and Competitiveness)  under grant no  IPT-2012-0883-430000  and the  RESET project (Ministry of Economy and Competiveness)  under grant  RESET TIN2014-53199-C3-1-R. The research has been partially financed by the  SURF Foundation of the Netherlands  and the  KU Leuven Research Council  (grant agreement no.
C24/16/017 ). Acknowledgments. The authors would like to thank the  MOOC Maker Project  (http://www. moocmaker. org/),  Universidad Galileo  and  Curtin University  for providing the datasets for the analysis and the  Graz University of Technology and Curtin University  for supporting the research visits of  Massimo Vitiello  and  Christian Guetl. Acknowledgments. We thank all the teachers, schools and students who took part in our study,  Gail Kusbit ,  Kenneth Holstein , the coders and graders for the project. This work is supported by  NSF  Award #  1530726.
Acknowledgements. This work has been co-funded by the  Erasmus +  projects  MOOC-Maker  ( 561533-EPP-1-2015-1-ES-EPPKA2-CBHE-JP ),  SHEILA  ( 562080-EPP-1-2015-BE-EPPKA3-PI-FORWARD ), and  COMPETEN-SEA  ( 574212-EPP-1-2016-1-NL-EPPKA2-CBHE-JP ), by the  eMadrid Network  ( S2013/ICE-2715 ), and by project  RESET  ( TIN2014-53199-C3-1-R ). Acknowledgments. This study is partly funded by the  I SECURE -Empowering education systems in information security project  (n. 2015-1-IT02-KA201-015005 ) under the  Erasmus+ programme  of the  European Commission. We would like to thank the participants that voluntarily took part in this study. Acknowledgements. This research was supported by the  Irish Research Council  in association with the  National Forum for the Enhancement of Teaching and Learning in Ireland  under project number  GOIPG/2015/3497 , and by  Science Foundation Ireland  under grant number  12/RC/2289.
The authors are indebted to  Dr. Stephen Blott , Lecturer on the module which is the subject of this work, for his help. Acknowledgments. We thank the teachers, learners and school principals who participated in the study and we are grateful to the  Swedish Research Council  for funding this research. Acknowledgements. We would like to thank the  EIAH Chair of Sorbonne-Universités  for financing this work. Fig. 1. Conceptual model of MLPGO. Acknowledgements. This work was partially funded by the  European Union  in the context of the  Go-Lab  project (Grant Agreement no. 317601 ) under the  Information and Communication Technologies (ICT)  theme of the  7th Framework Programme for R&D  (FP7). This document does not represent the opinion of the  European Union , and the  European Union  is not responsible for any use that might be made of its content.
Acknowledgments. This work was supported by the  Research Council of Norway  under the project  FUTURE LEARNING  ( 255129/H20 ). Acknowledgements. This research has been partially supported by the  Junta de Castilla y León  ( VA082U16 ) and  Ministerio de Economía y Competitividad  ( TIN2014-53199-C3-2-R ). The authors thank the  GSIC-EMIC research team  for their ideas and support and the " Movilidad Doctorandos UVa 2017" grant program.
Acknowledgements. This project has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement No. 669074. Acknowledgments. This work was supported by  FONDECYT  ( 11150231 ),  University of Costa Rica (UCR) ,  MOOC-Maker  ( 561533-EPP-1-2015-1-ESEPPKA2-CBHE-JP ),  LALA  ( 586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP ),  CONICYT Doctorado Nacional   2017/21170467 ,  CONICYT Doctorado Nacional   2016/21160081. Acknowledgments. The  Hubble  project (e. Learning Traces Observatory ) has been funded by the  French National Agency for Research (ANR). Acknowledgement. This paper is based on research conducted with the support of  Science Foundation Ireland  under grant  SFI/12/RC/2289. Acknowledgements. This work is supported by the  Dutch National Initiative for Education Research/The Netherlands Organisation for Scientific Research  and the  Dutch Ministry of Education, Culture and Science  under grant  405-15-705  (www.
sooner. nu). Acknowledgments. This research was partially supported by the  FP7 2008-212578   LTfLL  project, by the  644187 EC H2020   RAGE  project, by the  ANR-10-blan-1907-01   DEVCOMP  project, as well as by  University Politehnica of Bucharest  through the " Excellence Research Grants" Program   UPB-GEX 12/26. 09. 2016. Acknowledgments. This research is partly funded by  RecerCaixa CoT  project and the  Spanish Ministry of Economy and Competitiveness  under  RESET  ( TIN2014-53199-C3-3-R ) and the  Maria de Maeztu Units of Excellence Programme  ( MDM-2015-0502 ). DHL is a  Serra Hunter Fellow. The authors want to sincerely thank  Pablo Abenia  for the support to the development of this application. Acknowledgments. This project is partially funded by  SmartZoos ,  CB64 ,  Central Bal-tic Programme 2014 -2020 ; and the  European Union 's  Horizon 2020 Research and Innovation Programme  under grant agreement No.
The work is financed by the  Netherlands Organization for Scientific Research (NWO) , project number  405-16-5010. Thanks to the  PlatOOlab schools , and Eduventure for their input to the project. Acknowledgment. We thank  Ian Anthony Flitman  and  Jean-Cédric Chappelier  from  EPFL  for their support and for providing us the EPFL grader data. Acknowledgements. Funding from  PIA-CONICYT Basal Funds for Centers of Excellence  Project  FB0003  is gratefully acknowledged and to the  AKA-EDU -01 grant from  CONICYT. The authors are also thankful for the funding provided to project No. 294218  by the  Academy of Finland  Acknowledgements. This research was supported by  Science Foundation Ireland  under grant number  SFI/12/RC/2289 , and by  Dublin City University. Acknowledgement. These programs are available at tinyurl.
com/ai-snap-demos. This research was supported by the  eCraft2Learn  project [6] funded by the  European Union 's  Horizon 2020 Coordination & Research and Innovation Action  under Grant Agreement No  731345. Child-Friendly Programming Interfaces to AI Cloud Services  Acknowledgements. This research was partially supported by the  FP7 2008-212578   LTfLL  project, and by  European Union 's  Seventh Framework Programme  ( FP7/2007-2013 ), project  EarlyNutrition  under grant agreement n°[ 289346 ]. This work has received an unrestricted educational grant by  Wyeth Nutrition. Acknowledgements. This work was supported by a grant of the Romanian National Authority for Scientific Research and Innovation, CNCS -UEFISCDI, project number PN-II-RU-TE-2014-4-2604. Acknowledgements. We thank the authors of the Cyberlearning Community Report: June Ahn, Jodi Asbell-Clarke, Matthew Berland, Catherine Chase, Judith Fusco, Erica Halverson, Kemi Jona, Chad Lane, Emma Mercier, Tom Moher, Amy Ogan, Nichole Pinkard, Joseph Polman, Katie Headrick Taylor, Michelle Wilkerson, and Marcelo Worsley.
This material is based in part upon work supported by the National Science Foundation under Grant No. IIS-1233722. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Acknowledgments. The underlying project to this article is funded by the Federal Office of Bundeswehr Equipment, Information Technology and In-Service Support under promotional references. The authors are responsible for the content of this article. Acknowledgment. This work was partially funded by the FP7 2008-212578 LTfLL project, by the 644187 EC H2020 Realising an Applied Gaming Eco-system (RAGE) project and by University Politehnica of Bucharest through the Excellence Research Grants Program UPB-GEX 12/26.
M. user modelling), information model (F. folksonomies, R. M. relational model or ontologies, S. M. statistical model, S. N. social network), technical context, learning metaphor (K. A. knowledge acquisition, K. C. knowledge creation, P. participation), evaluation type (A. D. validation using an artificial dataset, C. S. case study, E. experimental study, I. E. informal evaluation, O. S. observational study, P. C. proof of concept), evaluation methodology (M. mixed, Q. quantitative) and evaluation data (D. demographics, L. logs, O. observations, P. predictions, R. recordings, U. F. user feedback). C. P. F. S. N. -P.
In our process, we have started with this kind of testing when the requirement specifications or user stories (in Scrum projects) are not provided by the client. In such a case, the client must provide at least the test acceptance criteria. • LL11: Sharing the same database between tester team and development team could lead to rework, especially in the test documentation because the test data defined before could not be more available (e. g. , if a developer deletes a database while implementing some new feature). Such problem also impacts on several test scripts that have to be updated. Furthermore, the previous versions of the application could be used as test oracle in the regression testing and also to verify intermittent bugs; and • LL12: An emergency release called "Hot fix" requires a fast test feedback.
g. , TestRail 6  ). With regard to the failures, they are reported in both JIRA 7  software, which is shared with the development team, and spreadsheets. Note that, only the bug repository is shared with both teams. Our experience shows that we have to clearly separate the development and test environments (see LL11). In such a case, the configuration manager is responsible to handle the changes in both environments. Moreover, the repository's permissions are strictly managed, i. e. , none of the team have permission to delete bugs. For example, once a bug is registered, it is analysed by the test analyst, if the bug is invalid, only its state changes in that repository. Another important point to be discussed is the team qualification (LL02). In the first cycle, the test team works together with the development team as a "Pair Testing" approach (LL04).
Fewer than 40% of taxa with multiple 16S rRNA genes have identical 16S rRNA sequences in each operon, although sequence divergence between copies is typically < 1%  (3, 190). 16S rRNA gene reference databases generally do not take this into account, and instead contain composite or consensus sequences obtained from simultaneous PCR amplification and pooled sequencing of all 16S gene copies on a genome  (90). The combined effects of multiple rrn operons per genome and different 16S rRNA gene sequences per operon result in systematic skewing of relative taxonomic abundance and overestimation of microbial diversity in mixed microbial communities  (190). The Ribosomal RNA Operon Copy Number Database (rrnDB) maintains updated, annotated lists of rRNA operon copy numbers  (180). Other widely used methods for defining species boundaries include multilocus sequence analysis and multilocus sequence typing, which are similar to the method described above but rely on a panel of usually seven to ten core genes rather than focusing on a single gene  (60, 146) , as well as genome-wide average nucleotide identity, which compares the sequences of all orthologous genes in the complete genomes of species pairs  (87, 160).
For example, genome size among 17 strains of the periopathogen P. gingivalis ranges from 2. 2 to 2. 4 Mb, a difference of 8%, but the number of genes differs by 22%, ranging from 1,870 genes in strain F0569 to 2,405 genes in strain JCVI SC001. This is true even though these strains exhibit >99. 4% sequence identity in the 16S rRNA gene and >98. 8% sequence identity across a panel of 11 housekeeping genes (coa, dnaK, ef-tu, ftsQ, gdpxJ, hagB, mcmA, nah, pga, recA, and pepO)  (93)  [analysis performed on all complete or nearly complete (scaffolded) genomes available in GenBank as of November 2016; for details, see Supplemental  Appendix 1]. By contrast, all members of a eukaryotic species carry a nearly identical gene set, and 75% of human genes have homologs in the genome of the puffer fish Takifugu rubripes, which diverged from mammals more than 450 Mya  (8, 148).
However, the apparent parasitic lifestyle of such taxa precludes attempts to classify them using conventional systematics criteria. Similar challenges face other microbial groups that are resistant to isolation in pure culture, making them difficult to discuss and study  (21, 50). Additionally, such a standard could never be applied to ancient microbes, effectively shutting the door to the possibility of discovering and naming extinct species. As a consequence of the high bar set by the ICSP for obtaining a valid species name, comparatively few microbial species have been officially named and validated-15,974 as of 2014 (  141 ), compared with the >645,000 for which there is currently 16S rRNA gene sequence evidence (OTUs clustered at 99% in the SILVA SSU Ref NR 99 database, release 128; https://www.
arbsilva. de) (152, 204). As a result, most microbial taxa are currently nameless but not necessarily unknown. The challenge of how to devise a functioning nomenclature scheme for such a situation is clearly illustrated by the taxon table maintained by the Human Oral Microbiome Database (HOMD), a public scientific resource that curates an up-to-date list of human oral microbes  (27). As of November 2016, the HOMD included 687 species-level oral taxa, of which 335 had both a valid species name and at least one sequenced genome, 36 had a valid species name and no sequenced genome, 88 had no valid species name but at least one sequenced genome, and 228 had no valid species name and no sequenced genome. The HOMD addressed this problem by developing a provisional naming scheme based on binning 16S rRNA gene sequences into unique phylotypes that are then assigned a Human Oral Taxon number.
This number then allows phenotypic, phylogenetic, genomic, clinical, and other data types to be linked within the HOMD portal. This scheme, however, is limited to microbes of the human oral cavity and is not generalizable to other microbiomes, such as those of the human gut, soil, or ocean. Alternatively, as of March 2017, GenBank (11, 131) contains 14,022 sequenced microbial genomes and maintains a taxonomy common tree of 23,653 named and candidatus microbial species (993 archaea and 22,660 bacteria) that "does not follow a single taxonomic treatise but rather attempts to incorporate phylogenetic and taxonomic knowledge from a variety of sources"  (127). By taking this pragmatic approach, they are able to utilize a diverse range of existing phenotypic, genetic, and genomic microbial data in a common phylogenetic framework  (165).
The ICSP has procedures for correcting such problems  (98) , and many taxa have been reclassified under this scheme. For example, Bacteroides forsythus, a gram-negative anaerobic rod first isolated from periodontal patients  (182) , was reclassified into its own genus as Tannerella forsythensis after major differences with other Bacteroides taxa were discovered  (163)  and was later renamed Tannerella forsythia to adhere to ICSP guidelines regarding gender consistency in Latin genus-species name combinations  (111). Given that microbial classifications frequently change as new information becomes available, it is important to be aware of nomenclature history, especially when conducting literature searches and comparing results generated using different databases.
METATAXONOMIC TOOLS AND DATABASES Numerous tools and databases are available that can facilitate metataxonomic analyses of both modern and ancient samples. Here, we provide an overview of different approaches and strategies commonly used in microbial archaeology studies, as well as a selection of frequently used software (for a detailed description and systematic comparison, see  106, 142). Supplemental Appendix 2 provides a list of tools and databases mentioned in this article, along with website links and references.
The reference strategy- The way a reference database is constructed and queried by a software tool has a strong influence on the sensitivity and specificity of the analysis, as well as on the potential biases that can occur. Commonly applied reference strategies can be divided into three classes: single-locus approaches, which focus on rRNA genes or other genes that show orthologs in all species represented in the database; multilocus approaches, which represent an extension of the previous approach applied to multiple genes; and wholegenome approaches, which use nearly all available genomic information.
Single-locus approaches: Tools that focus on single loci, such as the 16S rRNA gene, have the advantage that single-locus databases contain more taxonomic entries than those covering multiple loci or complete genomes. For investigations of microbial diversity, no other gene has been studied as extensively as the 16S rRNA gene. Several large-scale, publicly available 16S rRNA gene databases are available, each of which contains more than 1 million aligned reference sequences: the Ribosomal Database Project (RDP) (3,356,809 sequences)  (30) , Greengenes (2013 release, 1,262,986 sequences)  (37) , and SILVA (SSU Ref, 1,922,213 sequences)  (152) , the last of which contains entries for more than 645,000 distinct bacterial taxa (as defined by a 99% identity clustering cutoff for OTU assignment; the number of database entries is reported as of December 1, 2016).
This presents a limitation for the detection of low-abundance taxa. Finally, when working with short-read metagenomic data, we have found that minor changes in closed reference OTU picking parameters (such as UCLUST --max_accepts and --max_rejects values) can dramatically alter diversity estimates; consequently, default settings should be adjusted in microbial archaeology studies to optimize performance for very short sequences, which are typical of aDNA. Two widely used software pipelines that use 16S rRNA reference databases are QIIME  (24)  and mothur  (168). Metagenome Analyzer (MEGAN)  (80)  can also be used in connection with rRNA gene reference data.
Multilocus approaches: An alternative technique for metataxonomic assessment focuses on multiple loci, usually a small set of (single-copy) housekeeping genes. Here, the advantage is that more multilocus sequence analysis and multilocus sequence typing data sets are available in comparison with full genomes, and the sequence divergence level can be higher in comparison with rRNA genes, potentially allowing for the differentiation of bacterial strains. Another advantage is that genes can be selected that occur only as single copies in each reference genome sequence. One widely used software pipeline that uses a variant of a multilocus sequence analysisbased approach is the Metagenomic Phylogenetic Analysis (MetaPhlAn) pipeline  (172). The current MetaPhlAn database (version 2) includes ~1 million marker genes identified from ~17,000 reference genome sequences obtained from >7,000 species-level taxa; the marker genes were selected to define specific microbial clades (mainly bacterial and archaeal, but also including viruses and eukaryotes) and are spread across all functional annotation classes  (188).
The latter is essential because the microorganisms present in biological or environmental samples may differ in, for example, their metabolic pathways and can be detected only if they are represented in the reference database. In contrast to whole-genome alignment, read alignment against clade-specific markers is fast, comparing favorably to the Basic Local Alignment Search Tool (BLAST)  (172) , although not quite as fast as Kraken  (201). Once normalized by the sequence length of each individual marker, the total number of high-quality hits offers a simple measure for calculating taxon abundances. Because gene elements that are shared among clades do not fulfill the definition of markers, MetaPhlAn assignment is robust to horizontal gene transfer. Quasi markers, which are informative only in the absence of particular taxa sharing closely related sequences, can also improve sensitivity and enable strain detection.
Whole-genome approaches: The third reference strategy is to use complete genomes as a target database. Using whole genomes has the advantage that known taxa present in low abundances can potentially be identified because any sequenced DNA fragment that originates from a species contained in the target database can be (theoretically) assigned. Therefore, this strategy can be beneficial in ancient pathogen detection studies, where only traces of DNA from a pathogenic organism are expected to be present. Additionally, because variation across the entire genome is considered, this method maximizes the power to make fine-scale distinctions among species and strains. Challenges, however, can arise from horizontal gene transfer, bacterial recombination, or mobile elements, which can influence the precision of assignments.
Additionally, genome sizes of free-living taxa vary by nearly an order of magnitude, ranging from the tiny 0. 16-Mb genome of Candidatus Carsonella ruddii to the massive 10-Mb genome of Solibacter usitatus  (118) ; as a result, taxa with larger genomes contribute proportionally more fragment reads per cell to a metagenome, potentially making them appear more abundant if genome size differences are not taken into account. Whole-genome databases are also far less complete in comparison with rRNA databases, and thus, for a given sample, far fewer taxa may have a close representative in the reference database. This may lead to a larger number of false negative (unidentified) or false positive (identified as the closest representative in the database) assignments, a phenomenon often referred to as database bias.