dataesr/scientific-paragraphs-categorization huggingface dataset:
2009). For instance, the proportion of species listed as threatened in the IUCN Red List is much higher in amphibians than in birds (International Union for Conservation of Nature 2013). Geographic breadth is needed because human impacts show strong spatial variation: most of Western Europe has long been dominated by human land use, for example, whereas much of the Amazon basin is still close to a natural state  (Ellis et al. 2010). Thus, in the absence of broad coverage, any pattern seen in a dataset is prone to reflect the choice of taxa and region as much as true global patterns and trends. The most direct way to capture the effects of human activities on biodiversity is by analysis of time-series data from ecological communities, assemblages or populations, relating changes in biodiversity to changes in human activity (Va ck a r 2012).
g. ,  UNEP-WCMC 2009; Feld et al. 2010; Heink and Kowarik 2010)  but the datasets underpinning previous policy-relevant analyses have not always been made publicly available. We present a new database that collates published, in-press and other quality-assured spatial comparisons of community composition and site-level biodiversity from terrestrial sites around the world. The underlying data are made up of abundance, presence/absence and speciesrichness measures of a wide range of taxa that face many different anthropogenic pressures. As of March 2014, the dataset contains more than 1. 6 million samples from 78 countries representing over 28,000 species. The dataset, which is still being added to, is being assembled as part of the PREDICTS project (Projecting Responses of Ecological Diversity In Changing Terrestrial Systemshttp://www.
predicts. org. uk), the primary purpose of which is to model and project how biodiversity in terrestrial communities responds to human activity. The dataset is already considerably larger and more representative than those used in existing quantitative models of biodiversity trends such as the Living Planet Index (WWF International 2012) and GLOBIO3  (Alkemade et al. 2009). In this paper we introduce the database, describe in detail how it was collated, validated and curated, and assess its taxonomic, geographic and temporal coverage. We make available a summary dataset that contains, for each sampling location, the predominant land use, landuse intensity, type of habitat fragmentation, geographic coordinates, sampling dates, country, biogeographic realm, ecoregion, biome, biodiversity hotspot, taxonomic group studied and the number of measurements taken.
The full dataset constitutes a large evidence base for the analysis of: • The responses of biodiversity to human impacts for different countries, biomes and major taxonomic groups; • The differing responses within and outside protected areas; • How traits such as body size, range size and ecological specialism mediate responses and • How human impacts alter community composition. The summary dataset permits analysis of geographic and taxonomic variation in study size and design. The complete database, which will be made freely available at the end of the current phase of the project in 2015, will be of use to all researchers interested in producing models of how biodiversity responds to human pressures.
Data collection To maximize consistency in how incoming data were treated, we developed customized metadata and data capture toolsa PDF form and a structured Excel filetogether with detailed definitions and instructions on their usage. The PDF form was used to capture bibliographic information, corresponding author contact details and meta-data such as the country or countries in which data were collected, the number of taxa sampled, the number of sampling locations and the approximate geographical center(s) of the study area(s). The Excel file was used to capture details of each sampling site and the diversity measurements themselves. The PDF form and Excel file are available in Supplementary Information. We wrote software that comprehensively validates pairs of PDF and Excel files for consistency; details are in the "Database" section.
We also recorded authors' descriptions of the habitat at each Site and of any transects walked. For each Site we recorded the dates during which sampling took place. Not all authors presented precise sampling datessome gave them to the nearest month or year. We therefore recorded the earliest possible start date, the latest possible end date and the resolution of the dates that were given to us. Where dates were given to the nearest month or year, we recorded the start and end dates as the earliest and latest possible day, respectively. For example, if the authors reported that sampling took place between June and August of 2007, we recorded the date resolution as "month," the start of sampling as June 1, 2007 and end of sampling as August 31, 2007. This scheme meant that we could store sampling dates using regular database structures (which require that the year, month, and day are all present), while retaining information about the precision of sampling dates that were given to us.
Database Completed PDF and Excel files were uploaded to a Post-greSQL 9. 1 database (PostgreSQL Global Development Group, http://www. postgresql. org/) with the PostGIS 2. 0. 1 spatial extension (Refractions Research Inc, http://www. postgis. net/). The database schema is shown in Figure  S3. We wrote software in the Python programming language (http://www. python. org/) to perform comprehensive data validation; files were fully validated before their data were added to the database. Examples of lower level invalid data included missing values for mandatory fields, a negative time since conversion, a latitude given as 1°6 1', a date given as 32nd January, duplicated Site names and duplicated taxon names. Commonly encountered higher level problems included mistakes in coordinates, such as latitude and longitude swapped, decimal latitude and longitude incorrectly assembled from DD/MM/SS components, and direction (north/south, east/west) swapped round.
These mistakes typically resulted in coordinates that plotted in countries not matching those given in the metadata and/or out to sea. The former was detected automatically by validation software, which required that the GIS-matched country for each Site (see "Biogeographical coverage" below) matched the country name entered in the PDF file for the Study; where a Study spanned several countries, we set the country name to "Multiple countries. " We visually inspected all Site locations on a map and compared them to maps presented in the source article or given to us by the authors, catching coordinates that were mistakenly out to sea and providing a check of accuracy. Our database linked each Data Source to the relevant record in our Insightly contact management database. This allows us to trace each datum back to the email that granted permission for us to include it in our database.
Biogeographical coverage In order to assess the data's geographical and biogeographical coverage, we matched each Site's coordinates to GIS datasets that were loaded into our database: Global GIS layers appear coarse at local scales and we anticipated that Sites on coasts or on islands could fall slightly outside the relevant polygons. Our software therefore matched Sites to the nearest ecoregion and nearest country polygons, and recorded the distance in meters to that polygon, with a value of zero for Sites that fell within a polygon; we reviewed Sites with non-zero distances. The software precisely matched Sites to hotspot polygons. The relative coarseness of GIS polygons might result in small errors in our assessments of coverage (i. e. , at borders between biomes, ecoregions and countries, and at the edges of hotspots)we expect that these errors should be small in number and unbiased.
• We also estimated the yearly value of total net primary production (TNPP) for biomes and five-degree latitudinal belts, using 2010 spatial (0. 1-degree resolution) monthly datasets "NPP -Net Primary Productivity 1 month-Terra/MODIS" compiled and distributed by NASA Earth Observations (http://neo. sci. gsfc. nasa. gov/view. php? datasetId=MOD17A2_M_PSN&year=2010). We used the NPP values (average for each month assimilation measured in grams of carbon per square meter per day) to estimate monthly and annual NPP. We then derived TNPP values by multiplying NPP values by the total terrestrial area for that ecoregion/latitudinal belt. We assessed the representativeness of land use and land-use intensity combinations by comparing the proportion of Sites in each combination to a corresponding estimate of the proportion of total terrestrial area for 2005, computed using land-use data from the HYDE historical reconstruction  (Hurtt et al.
Taxonomic names and classification We wanted to identify taxa in our database as precisely as possible and to place them in higher level groups, which required relating the taxonomic names presented in our datasets to a stable and authoritative resource for nomenclature. We used the Catalogue of Life (http://www. cata logueoflife. org/) for three main reasons. First, it provides broad taxonomic coverage. Second, Catalogue of Life publishes Annual Checklists. Third, Catalogue of Life provides a single accepted taxonomic classification for each species that is represented. Not all databases provide this guarantee; for example, Encyclopedia of Life (http:// www. eol. org/) provides zero, one or more taxonomic classifications for each represented species. We therefore matched taxonomic names to the Catalogue of Life 2013 Annual Checklist  (Roskov et al.
2013, henceforth COL). There was large variation in the form of the taxonomic names presented in the source datasets, for example: • A Latin binomial, with and without authority, year and other information; • A generic name, possibly with a number to distinguish morphospecies from congenerics in the same Study (e. g. , "Bracon sp. 1"); • The name of a higher taxonomic rank such as family, order, class; • A common name (usually for birds), sometimes not in English; • A textual description, code, letter or number with no further information except an indication of some aspect of higher taxonomy. Most names were Latin binomials, generic names or morphospecies names. Few binomials were associated with an authorityeven when they were, time constraints mean that it would not have been practical to make use of this information.
Many names contained typographical errors. We represented each taxon by three different names: "Name entered," "Parsed name," and "COL query name. " "Name entered" was the name assigned to the taxon in the dataset provided to us by the investigators who collected the data. We used the Global Names Architecture's biodiversity package (https://github. com/GlobalNames Architecture/biodiversity) to parse "Name entered" and extract a putative Latin binomial, which we assigned to both "Parsed name" and "COL query name. " For example, the result of parsing the name "Ancistrocerus trifasciatus M€ ull. " was "Ancistrocerus trifasciatus. " The parser treated all names as if they were scientific taxonomic names, so the result of parsing common names was not sensible: e. g. "Black and White Casqued Hornbill" was parsed as "Black and.
When a COL search returned more than one result, and the results included zero or two or more accepted names, we used the lowest level of classification common to all results. For example, COL lists Notiophilus as an accepted genus in two beetle families -Carabidae and Erirhinidae. This is a violation of the rules of nomenclature, but taxonomic databases are imperfect and such violations are to be expected. In this case, the lowest rank common to both families is the order Coleoptera.
Counting the number of species It was not possible to precisely count the number of species represented in our database because of ambiguity inherent in the taxon names provided with the data. We estimated the number of species as follows. Names with a COL result at either species or infraspecies level were counted once per name. Names with a COL result resolved to higher taxonomic ranks were counted once per Study. To illustrate this scheme, consider the bat genus Eonycteris, which contains three species. Suppose that Study A sampled all three species and that the investigators could distinguish individuals as belonging to three separate species but could not assign them to named species, reporting them as Eonycteris sp. 1, Eonycteris sp. 2 and Eonycteris sp. 3. Study B also sampled all three species of Eonycteris and again reported Eonycteris sp.
1, Eonycteris sp. 2 and Eonycteris sp. 3. We would erroneously consider these taxa to be six different species. We did not attempt to determine how often, if at all, such inflation occurred. In order to assess the taxonomic coverage of our data, we computed a higher taxonomic grouping for each taxon as: (1) order where class was Insecta or Entognatha; (2) class where phylum was Arthropoda (excluding Insecta), Chordata or Tracheophyta; otherwise 3) phylum. So the higher taxonomic group of a bee is order Hymenoptera (following rule 1), the higher taxonomic group of a wolf is class Mammalia (rule 2), and the higher taxonomic group of a snail is phylum Gastropoda (rule 3). For each higher taxonomic group, we compared the numbers of species in our database to the estimated number of described species presented by  Chapman (2009).
Some of the higher taxonomic groups that we computed did not directly relate to the groups presented by  Chapman (2009)  so, in order to compare counts, we computed Magnoliophyta as the sum of Magnoliopsida and Liliopsida; Gymnosperms as the sum of Pinopsida and Gnetopsida; Ferns and allies as Biome 1 9 9 2 1 9 9 3 1 9 9 4 1 9 9 5 1 9 9 6 1 9 9 7 1 9 9 8 1 9 9 9 2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2  the sum of Polypodiopsida, Lycopodiopsida, Psilotopsida, Equisetopsida and Marattiopsida; and Crustacea as Malacostraca. P N M L K J H G F E D C B A For some of our analyses, we related taxonomic names to databases of species' traits. To do this, we synthesized, for each taxon, a "Best guess binomial": • The COL taxon name if the COL rank was Species; • The first two words of the COL taxon if the rank was Infraspecies; • The first two words of "Parsed name" if the rank was neither Species nor Infraspecies and "Parsed name" contained two or more words; • Empty in other cases.
Results Between   1 ). Hotspots together account for just 16% of the world's terrestrial surface, yet 47. 67% of our measurements were taken in hotspots. The vast majority of measurements in hotspots were taken in the Sundaland hotspot (Southeast Asia) and the latitudinal band with the most samples is 0°to 5°N (Fig. 2 ); many of these data come from two studies of higher plants from Indonesia that between them contribute just 284 sites but over 320,000 samples  (Sheil et al. 2002). The best-represented biomes are "Temperate Broadleaf and Mixed Forests" and "Tropical and Subtropical Moist Broadleaf Forests" (Figs  3, 4 ). "Flooded Grasslands and Savannas" is the only biome that is unrepresented in our database  (Figs 3, 4) ; although this biome is responsible for only 0. 7% of global terrestrial net primary productivity, it is nevertheless ecologically important and will be a priority for future collation efforts.
Two biomes -"Tundra" and "Deserts and Xeric Shrublands"are underrepresented relative to their areas. Of the world's 17 megadiverse countries identified by  Mittermeier et al. (1997) , only Democratic Republic of Congo is not represented (Figure  S4 ). The vast majority of sampling took place after the year 2000 (Fig. 3 ), reflecting our desire to collate diversity data that can be related to MODIS data,    (Hurtt et al. 2011)  and Global Land Systems datasets (van Asselen and Verburg 2012); no difference is shown for "Urban"/"Light use" because these datasets did not allow us to compute an estimate for this combination. The 12. 15% of Sites that could not be assigned a classification for predominant land use and/or landuse intensity are not shown. (Jones et al. 2009) , TRY  (Kattge et al. 2011).
Best guess binomials: the number of unique "Best guess binomials" in the PREDICTS database within that taxonomic group. Attribute database names: the number of unique binomials and trinomials for that attribute in attribute database. Species matches: the number of "Best guess binomials" that exactly match a record in the attribute database. Genus matches: the number of generic names in the PREDICTS database with a matching record in the attribute database (only for binomials for which there was not a species match). Total matches: sum of species matches and genus matches. We did not match generic names for GBIF range size, IUCN category or CITES appendix because we did not expect these traits to be highly conserved within genera. which are available from early 2000 onwards. The database's coverage of realms, biomes, countries, regions and subregions is shown in Supplementary Tables  S5-S11.
The distribution of Site-level predominant land use and use intensity is different from the distribution of the estimated total terrestrial area in each land use/landuse intensity combination for 2005 (v2 = 28,243. 21, df = 16, P < 2. 2 9 10 À16 ; we excluded "Urban"/"Light use" from this test because the HYDE and Global Land Systems datasets did not allow us to compute an estimate for this combination). The main discrepancies are that the database has far fewer than expected Sites that are classified as "Primary habitat"/"Minimal use", "Secondary vegetation"/"Light use" and "Pasture"/"Light use" (Fig. 5 ). We were unable to assign a classification of predominant land use to 3. 34% of Sites and of use intensity to 12. 09% of Sites. The most common fragmentation layout was "Representative part of a fragmented landscape" (27.
49% of Sites values are approximately log-normally distributed (median 40,000 square meters; Figure  S7 ). We reviewed all cases of Sites falling outside the GIS polygons for countries (0. 82% of Sites; Figure  S8 ) and ecoregions (0. 52% of Sites; Figure  S9 ). These Sites were either on coasts and/or on islands too small to be included in the GIS dataset in question. The database contains measurements of approximately 28,735 species (see "Counting the number of species" in Methods) -17,733 animals, 10,201 plants, 800 fungi and 1 protozoan. We were unable to place 97 taxa in a higher taxonomic group because they were not sufficiently well resolved. The database contains more than 1% as many species as have been described within 20 higher taxonomic groups (Fig. 6 ). Birds are particularly well represented, reflecting the sampling bias in favor of this charismatic group.
Our database contains measurements of 2,479 species of birds -24. 81% of those described  (Chapman 2009 )and 2,368 of these are resolved to either species or infraspecies levels. A total of 228,644 samplesmore than 14% of the entire database are of birds. In contrast, just 397 species of mammals are represented, but even this constitutes 7. 24% of described species. Chiroptera (bats) are the best-represented mammalian order with 188 species. Of the 115,000 estimated described species of Hymenoptera, 3,556 (3. 09%) are represented in the database, the best representation of an invertebrate group. The hymenopteran family with the most species in the database is Formicidae with 2,060 species. The database contains data for 4,056 species of Coleoptera -1. 07% of described beetles. Carabidae is the best-represented beetle family with 2,060 species.
Some higher taxonomic groups have well below 1% representation and, as might be expected, the database has poor coverage of groups for which the majority of species are marinenematodes, crustaceans and molluscs. Of the 28,735 species, 43. 26% are matched to a COL record with a rank of species or infraspecies, 37. 47% to a COL record with a rank of genus and 19. 27% to a COL record with a higher taxonomic rank (Fig. 7 ). The species with the largest number of measurements -1,305is Bombus pascuorum (the common carder bee), and bees constitute 35 of the top 100 most frequently sampled species: this results from a PREDICTS subproject that is examining pollinators. Birds make up most of the remaining top 100, with 36 species. Of the 407 Studies, 126 sampled within a single order (Fig. 8 ); just 12 Studies    6.
2010; 281 Hanson et al. 2008; 282 Ferreira and Alves 2005; 283 Luskin 2010; 284 Grogan et al. 2008. databases is often much higher than would be expected by chance (Table  2 ), greatly facilitating analyses that integrate PREDICTS data with species attributes  (Newbold et al. 2013 (Newbold et al. , 2014a,b),b). Of the 284 Data Sources, 271 were taken from articles published in scientific peer-reviewed journals; the rest came from unpublished data (5), internet databases (3), PhD theses (2), agency reports (1) and other sources (2). The vast majority -273 (96. 13%)of Data Sources are taken from English articles; the remainder are in Mandarin (0. 35%), Portuguese (1. 06%) or Spanish (2. 46%). 29. 15% of Data Sources come from just four journals (Fig. 9 ): Biological Conservation (11. 07%), Biodiversity and Conservation (8.
Discussion The coverage of the PREDICTS dataset illustrates the large number of published articles that are based on   8,18,36 196,209 206-208 46,204-205 164,199-203 12,196-198 185-195 18,31,45,176-184 36,166-175 29,34,155-165 34,64,143-  local-scale empirical data of the responses of diversity either to a difference in land-use type or along a gradient of land-use intensity or other human pressure. Such data can be used to model spatial responses of local communities to anthropogenic pressures and thus changes over time. This is essential for understanding the impact of biodiversity loss on ecosystem function and ecosystem services, which operate at the local level  (Fontaine et al. 2006; Isbell et al. 2011; Cardinale et al. 2012; Hooper et al. 2012). Regardless of scale, no single Study is or could ever be representative, but the sheer number and diversity of Studies means that a collation of these data can provide relatively representative coverage of biodiversity.
The majority of Data Sources (271 of 284) come from peerreviewed publications and all data have used peer-reviewed sampling procedures. There are doubtless very many more published data than we have so far acquired and been given permission to use. For the majority of Data Sources (225), it was necessary to contact the author(s) in order to get more information such as the Site coordinates or the names of the taxa studied: even now that supplementary data are commonplace and often extensive, we usually had to request more detail than had been published. The database currently lacks Sites in ten biodiversity hotspots and one megadiverse country (Democratic Republic of the Congo). It also has no data from many large tropical or partially tropical countries such as Angola, Tanzania and Zambia.
Many countries are underrepresented given their area and/or the distinctiveness of their biota e. g. , Australia, China, Madagascar, New Zealand, Russia and South Africa. We have few data from islands and just 57 Sites from the biogeographic realm of Oceania (Fig. 3  and Table  S8 ): we have not yet directly targeted Oceania or island biota more generally. The database contains no studies of microbial diversity and few of parasitesmajor shortcomings that also apply to other large biodiversity databases such as the Living Planet Index (WWF International 2012), the IUCN Red List (International Union for Conservation of Nature 2013) and BIOFRAG  (Pfeifer et al. 2014). Fewer than 50% of the taxa in our database are matched to a Catalogue of Life record with a rank of species or infraspecies (Fig.