dataesr/scientific-paragraphs-categorization huggingface dataset:
Correction for the following covariates was undertaken: age at recruitment (UK Biobank data-field 21022), sex (data-field 31), height (data-field 50), weight (data-field 21002) and genetic principal components 1 to 20 (data-field 22009). Variants were considered significantly associated if they met the genome-wide significance level (p-value < 5 × 10 -8 ). 1. 1 (https://rgcgithub. github. io/regenie/) (Mbatchou 2021). The following quality-control filters were applied on the imputed genotype data (UK Biobank data-field 22828) during the creation of the whole-genome regression model (REGENIE step 1): a minor allele frequency !"#$%& '& (); Hardy-Weinberg equilibrium test not exceeding P > 1 × 10 -15 ; a genotyping rate above 99%; not present in a low-complexity region, a region of long-range linkage disequilibrium or a sex chromosome (Mbatchou 2021).
The IntCal family of radiocarbon ( 14 C) calibration curves is based on research spanning more than three decades. The IntCal group have collated the 14 C and calendar age data (mostly derived from primary publications with other types of data and meta-data) and, since 2010, made them available for other sorts of analysis through an openaccess database. This has ensured transparency in terms of the data used in the construction of the ratified calibration curves. As the IntCal database expands, work is underway to facilitate best practice for new data submissions, make more of the associated metadata available in a structured form, and help those wishing to process the data with programming languages such as R, Python, and MATLAB. The data and metadata are complex because of the range of different types of archives.
INTRODUCTION The generation of the IntCal, Marine, and SHCal radiocarbon ( 14 C) age calibration curves  (Heaton et al. 2020a; Hogg et al. 2020; Reimer et al. 2020 ) is only possible because of the research and care which goes into generating and checking the underlying 14 C datasets used in their construction. In particular, the assessment of data quality relative to set criteria and the collection of associated metadata  (Reimer et al. 2002 (Reimer et al. , 2013) )  are key elements in the ongoing compilation of these datasets. In addition to their value for 14 C calibration, these 14 C datasets also have an importance in their own right for a wide spectrum of different areas of research  (Heaton et al. 2021)  and have routinely been made available to the international research community through the IntCal database (previously available from the Queen's University Belfast webserver).
The developments reported here build upon that initiative with the intention of facilitating access to, and enhancing transparency of, the broad range of information used in the construction of the calibration curves. At present, the data collected and discussed here relate only to the construction of the pre-1950 IntCal, Marine, and SHCal curves that span from 55,000-0 cal yr BP. However, the intention is to broaden this to include the data for calibration in the post-1950 period  (Hua et al. 2022).
AIMS OF THE DEVELOPMENT WORK The main aims of the new database structures are to facilitate the sharing of not only the primary, raw, 14 C data used for calibration purposes but also the related data (such as tree ring measurements), metadata (including method statements and images), and to do so in a way that enables the data to be manipulated programmatically using tools such as R (R Core Team 2022), Python (e. g. ,  Van  Rossum and Drake 2009), and MATLAB (e. g. ,  Higham and Higham 2016). The specific objectives are to: 1. Make all the primary data (the 14 C measurements and accompanying calendar age information with their associated uncertainties) that are used in construction of the IntCal family of calibration curves available in a digitally readable form. 2. Organize this data, as far as possible, by primary record rather than measurement initiative, in order to facilitate comparison of results from multiple laboratories and to put the data into a geographic context.
Associate (or link ) the data and metadata with the relevant publications, providing complete DOI information and URL links to the publications. This includes links to complimentary data archives, particularly those relating to the supporting dendrochronological data. 6. Provide methods for visualization of the 14 C and dendrochronological data. 7. Include tools for checking and assessing data. 8. Allow for the easy import and export of both data and references to a range of other software. These aims are intended to help different groups of researchers working with the data: those wishing to use or independently evaluate the data, those working on the preparation of new datasets, and the members of the IntCal group working on curve construction. It is important to remember that research on the underlying records is an ongoing process and that, for example, the timescales for the speleothems and their derived chronologies will change in future iterations  (Cheng et al.
OVERALL DATA MODEL In order to achieve these aims, use has been made of the pre-existing IntChron framework  (Bronk Ramsey et al. 2019). This is specifically designed for sharing linked data and includes elements relevant to chronological data, particularly the ability to handle different timescale units and tools for data visualization. The data schema and associated tools have been updated to include elements required for the IntCal datasets, but the overall data model has been found to cover all the core IntCal requirements.
Data Organization The data structure is essentially hierarchical but with the ability to link and associate information by searches. At the top level there are three main classes of information, with all other information organized within this structure: 1. Records: These contain all information relating to individual sites or records; each record has a unique short name, or site code, which is used as a key for accessing information. Within the record, there are three types of information: a. Information on the location (latitude, longitude, elevation) and type (e. g. , Marine, Terrestrial, Speleothem, : : : ) of the record. b. Data series lists which are used to hold data for the record. For this application the main data series types are: i. IntCal data: the primary 14 C calibration data (containing 14 C measurements and quoted uncertainties, as well as the accompanying calendar age information).
ii. Dendrochronological sample data: the ring-width or oxygen isotope data associated with the measurement series (if appropriate). iii. Metadata: other descriptive information about the record. iv. Attachments: files (typically images) cited in the metadata. v. Other data, such as ages and correlation or covariance files for records with a level of dependence between the calendar age estimates can be added with specific series types. c. Reference links for the record as a whole and for the series within it. 2. Project data: which refers to data series that do not have a specific link to single records or sites. In the case of the IntCal database, information held at this level includes the calibration curves themselves and an index of data series organized by the IntCal dataset number. This also includes information on the time-variant relationship between the calibration curve and other (e.
Data Storage The underlying storage format within IntChron is JSON because this facilitates easy interfacing with programming tools (most easily using tools such as R, Python, and MATLAB, but also in principle with languages such as C and C#), static archiving/storing of data without the need for software installation  (Bronk Ramsey et al. 2019) , and is commonly used in web-based applications. The data will be stored and used in three different ways: i. There is an active database accessible to members of the IntCal working group which is intended to help develop new calibration curves. This allows for the addition of new records and the updating of existing ones. ii. There is a static archive of the IntCal20 datasets at https://intchron. org/archive/IntCal/ IntCal20/index. json, which holds all the information accumulated for IntCal20 as it was when the curve was constructed.
This is open access. iii. It is possible for users to make their own copies of this archive (or parts of it) for their research and to prepare new data for inclusion in IntCal. These can be stored on the IntChron server in users' own areas, or on users' own computers. In the case of the IntCal database itself (point "i" above), a hybrid approach has been adopted. The user interface, archives and file transfer all use JSON, but for the primary IntCal data, an underlying MySQL data table has been retained to minimise the risks of unintended changes. In addition, the JSON data records are stored in a database rather than as files. This has some advantages for a multi-user system and enables the database maintainers to use an automated archiving system on the computer server which holds the database.
However, from the perspective of a user of the database or of the derived archives, the data organization is effectively in the form of JSON objects. Reflecting the data organization, there are only three types of JSON files that are used for data exchange and presentation. These are: 1. Project data: these files hold links to all of the relevant records and project-level data series. In addition, the project data file contains all of the relevant publication information and details of parameter characteristics. 2. Record data: these contain all the information relevant to a specific record, including the data series contained within it. 3. Series data: these are for project-level data series which do not relate to specific records (such as calibration curves). The overall database model is based on linked data, so it can also include references to files held outside the JSON data structure via attachments (typically images).
In general, the aim has however been to avoid putting key information in such attachments because it makes for more difficult data distribution. As a variant of the above model, it is possible to have all record and series data embedded into a single project data file. We have used this option for the data archive because it means that all of the data (other than attachments) can be retrieved from a single file rather than having to make multiple requests for each record and series. This approach would not be efficient for very large projects but poses no problems for the present IntCal datasets in terms of data handling (the entire IntCal20 archive is < 10 MB). Attachment files are organized in a hierarchical file structure based on record or series name. A full archive of the IntCal20 data files is included as supplementary online information in this publication, allowing the complete archive to be reproduced without access to the current site.
Software Overview In principle, the archive can be worked with entirely using software tools such as R, Python, and MATLAB. However, the IntChron integration tool (Bronk Ramsey et al. 2019) has been further developed to facilitate work with the IntCal database and provide a user interface for some pre-defined data manipulations. This tool can be accessed and preloaded with the open access static IntCal20 data using the url: https://intchron. org/tools/integrate/integrate. html?filename=https://intchron. org/archive/ IntCal/IntCal20/index. json The integration tool has been specifically designed to work with data organized in the format described above, and the user is presented with a list of records and data series. The data can be explored by following links from this level. There are also search facilities built into the system which enable lists of relevant data series or primary data to be extracted.
Record Level Information At the record level, the key information held is about the location. For dendrochronological samples, the genus and species of wood sampled (e. g. , Quercus robur, Agathis australis) is also included (see Table  1  and Figure  2  for details). There is an optional comment field at this level which can consist of information not included in the standard parameters. The references at this record level should only be the key papers relevant to the sample set within IntCal. More specific references, for example to the dendrochronology, can be included with the dendrochronological series or metadata. The record name is ideally a formal site code (such as SG06) or failing that a short form of the site name (such as Maraa). In the case of records containing compilations of data (mostly older datasets) the lab code, country of origin of the samples, and taxon are used together (as in QL_DE_Oak).
IntCal Data Series For IntCal, these are the most important element of the data held in the database. The definitions of the parameters included are given in Table  1  and also in the online schema. For more recent data, each data series (with a set and division number) contains measurements from a single laboratory on a single sample set (environmental record or single tree sample). For some older datasets, multiple samples from a combined chronology are summarised within a single set/division. Ideally, for dendrochronological samples, the sample parameter in the series header will be the same as the series name for the associated dendrochronological data, allowing the two to be cross-checked especially where there are multiple-year block samples used. Each sample has its midpoint age expressed in two ways.
0. For samples dated by means other than dendrochronology, the calage is simply the Key 2 Example of plot drawn from the IntCal database showing the main datasets in IntCal20 over the time period 6000-5000 calBP. For the calibration curves, the series name is given; for all other data the record name is given with the series name in brackets. See section on record level information and Table  1  for details of record and series names. 10 C Bronk Ramsey et al. prior age estimate before AD1950 with an associated uncertainty. For those samples/archives which have uncertain calendar ages when entering IntCal, the calage values provide the prior calendar age estimate before curve construction. Consequently, in the construction of the calibration curve the calage parameter is used as the inputs for curve construction, while the t parameter provides the posterior estimate of the age after curve construction has been completed and is the measure used for plotting purposes.
Dendrochronological Sample Data Series These can be imported from various dendrochronological formats directly, or from the NOAA International Tree Ring Data Bank (ITRDB: NOAA NECI 2022) exported as Tucson. rwl files. The internal format of these files has some unique features needed for this global dataset. Ring numbering is typically from old to young but can be reversed where this is the case for the primary data. The date (t) parameter is a floating-point astronomical year and should reflect the growing season for the wood. As previously described, NH wood grown in 1950 will be stored as 1950. 5, whereas SH wood that starts to grow in 1950 will be stored as 1951. 0. In the interface you can choose whether to use the Schulman convention for display purposes. If this is selected, 1951. 0 will be shown as AD1950⊣ showing that it is the end of this year, whereas if the convention is not applied it will show as AD1951⊢ indicating the start of the year.
Metadata In order to properly understand the background to the dendrochronology underpinning the datasets used in IntCal, the database includes metadata. Such metadata are of greatest importance when the associated data are not already published elsewhere. The data included have been selected to include key information needed as outlined in  Reimer et al. (2013). The metadata are usually structured to address key questions (see Table  2 ) and are in the form of a plain text file. Where tables are required, these can be incorporated within the main notes field using tab characters. A "code" field can also be added if needed (for example, COFECHA output extracts). The metadata should be kept succinct and not include unnecessary detail published or deposited elsewhere. Some of the information in the metadata is also held in a more structured way in the main database structure.
Other Data The calendar age correlation or covariance matrices, for those records with timescales that have been obtained using approaches that introduce a level of dependence between the calendar age estimates, are the main other data required for the IntCal calibration curve Development of the IntCal Database 11 generation. These are stored in a particular IntCal_Correlation dataset which holds the matrix as a simple tab-delimited text matrix that can be read by standard software packages. See for example the SG06 record. This Lake Suigetsu record has an adjusted varve-counted chronology  (Bronk Ramsey et al. 2020 ) whereby calendar age uncertainties at any individual depth within the core are propagated to the other depths (due to both the necessary depth ordering and limitations on changing sedimentation rates).
TOOLS FOR DATA PREPARATION AND ASSESSMENT One of the main reasons for the choice of the underlying data model for the IntCal database was to make the task of preparing, adding, and assessing data easier for both submitters of data and those involved in the compilation. Clearly, the central database itself cannot be open for modification, but by enabling users to make their own copies of the database, this allows additions and changes to be tested with full access to the associated tools. It is also hoped that by enabling this, data providers (who normally understand their data best) will be able to get the data ready for submission themselves. There is a help facility within the software, available through the [Help] menu, and this contains specific information for IntCal data which will be kept up-to-date with any developments in the interface.
CONCLUSIONS It is hoped that the provision of this comprehensive database for the IntCal project will facilitate the work of the 14 C research community wishing to use the IntCal data and those wishing to contribute to it. The intention is to make the tools and all of the data used within the group open for all researchers. We consider it particularly important that all of the extensive work which underlies the datasets is fully referenced and that these references are easy to use for those accessing the data. Work is underway to prepare for the next update of the IntCal calibration curves, and loading data into this updatable format is intended to be the first stage in that process.
ACKNOWLEDGMENTS The authors of this paper would like to acknowledge all of the work which has gone into generating the datasets included within this dataset, and all of those who have helped in their compilation for IntCal20 and earlier iterations of the calibration curves. In addition, we would like to acknowledge a number of young researchers who have tested and worked with the IntChron interface through its development over the last few years, particularly  Rebecca Kearney ,  Lorena Becerra-Valdivia  and  Jakov Mlinaravic. Finally, we would like to acknowledge the support of the  NERC NEIF facility  ( PR19008-NEIF- Oxford) for the underlying computing infrastructure and support for the  IntChron  interface development.
DATA ELEMENTS The overall data schema for IntChron (including the parameters used for the IntCal dataset) is given at https://intchron. org/schema with the current version supplied in the supplementary information for this paper. Here we will focus on how the data are used specifically for IntCal, concentrating specifically on elements which might be less intuitive or where this might not otherwise be obvious. Each parameter has a formal name used in the JSON objects and for searches, and a display-orientated name (see Table 1). Parameter, series names and record Development of the IntCal Database 5 https://doi. org/10. 1017/RDC. 2023. 53 Published online by Cambridge University Press.
Division number within the set. The sets are subdivided into divisions where this is thought to be useful. For dendrochronological samples this should normally be the tree sample code (such as ABCD012). Depending on the context, shows the linear mean position of a sample. It should be the mid-value and so might be non-integer, for example for a decadal block. This can either be displayed separately or as a range of values. For tree rings this is zero for a single ring. (Continued) Development of the IntCal Database 7 parameters are used for this record. Typically here, "rings" for dendrochronological samples, "depth" for sediments, or "height" for speleothems. z_units z units Most relevant for depths and heights where the data is stored in m but can be displayed in "m", "cm", or "mm". Set to "rings" for dendrochronology.
Another hurdle in the development of AI-based histopathology classifiers is the huge work burden for annotation of large datasets and which support the use of weakly supervised deep learning pipelines  [8, 14]. This applies also for the prediction of defined genetic mutations in histopathology specimens, e. g. , colorectal, gastric and bladder cancer  [14]. A different AI-based approach was able to replicate histochemical and immunohistochemical stainings from unstained slides without actually performing the staining procedures  [12, 29]. In contrast, AI-based classifiers are increasingly developed in neuroradiology in order to reduce the burden of reviewing large stacks of MR images at various protocols and to detect small lesions in drug-resistant focal epilepsy, i. e. , FCD  [15]. However, the full potential of cost-effective and AI-based digital pathology classifiers needs to be further explored.
Dataset splitting In order to monitor and control the training process, we divided the total set of available training images into a training group and a validation group. To do this, we determined the total number of training images for each of our eight main entities to be examined (see Supplement Figure  1 ) and assigned 20% from each group for the validation process. The splitting process was carried out using a random number generator at the patient level and subsequently using an iteration selection, also at the patient level, until the required number of validation images had been separated, i. e. , all images of a specific patient were available in either the training group or in the validation group. The aim of this approach was to prevent the system from assignments based on specific individual characteristics of the individual patient samples in order to positively support the generalization of the model and avoid overfitting in the context of hidden variables within the patient data, e.
5 (Table  1 ). Values below this threshold were considered to not meet the prediction (negative). These predictions were converted into a one-hot-encoded format and compared with the basic ground truth, i. e. , the manually labeled values for the corresponding image. The mean of all element-wise compared values then provides the accuracy_multi score metric (see below). We applied weighted averaging for PrecisionMulti, RecallMulti and F1ScoreMulti across all labels in this work to take the presence of unbalanced categories into account (see dataset splitting)  [26]. This means that the calculated metrics for the respective individual classes are weighted with the proportion of training images in this class in relation to the size of the total population of training image data. Other averaging regimes were not used herein.