dataesr/scientific-paragraphs-categorization huggingface dataset:
All other PG Chla and TChla were well predicted from our regression models based on the EOF scores derived from the three hyperspectral AOP data sets combined with the matchup HPLC PG Chla (Table  3 ). Full fit results are best (indicated as bold in Table  3 ) for Ǩd (λ), closely followed by T(z, λ) and then K d (z 1 → z 2 , λ) based models (e. g. , R 2 ≥ 0. 71, ≥0. 65 and ≥0. 39, and MPD ≤ 46%, ≤47% and ≤56%, respectively). In addition, the results for the cross validation statistics based on 500 permutations using different sub-samples are similar for the three AOP data set based models. Full fit R 2 , RMSD, and MPD values show better results for all three AOP based PG data sets as compared to R 2 cv, RMSDcv, and MPDcv values, respectively. However, for the T(z, λ) based PG data sets, the difference between the statistics of the full fit and that of the mean of the cross validation is lowest (here R 2 cv ≥0.
Although the vertical speed of the system was around 1 m/s (in addition to the 4 m/s horizontal speed) reliable spectral transmission data mostly until below the Z eu could be obtained. The depth limit until where information can be obtained is clearly dependent on incident light which was very high during our undulating platform operations in the tropics and subtropics. In the future obtaining PG Chla and TChla data from hyperspectral IOP sensors would be favorable and should be explored because radiometers only provide valid data under daylight (leading to the absence of AOP information in our transects during night-time), while IOP instruments use their own light source. However, hyperspectral IOP sensors need even higher energy supply than the radiometers. The later have the advantage to be rather self-calibrating since AOPs are derived.
ACKNOWLEDGMENTS We thank  H. Becker ,  S. Spahic ,  J. Hagemann , and  S. El Naggar  for their invaluable help in collecting the Triaxus data presented here. The CTD at the hydrographic stations was operated by  H. Leach  and  J. Gerken. We thank  B. Bracher  for supporting HPLC sample analysis and the captain, the crew and other scientists on board PS113 for their valuable support on board. We thank  ESA ,  EUMETSAT , and  CMEMS GlobColour program  for providing OLCI Chla level-3 products.
Aim: To enable a world-leading research dataset of routinely collected clinical images linked to other routinely collected data from the whole Scottish national population. This includes more than 30 million different radiological examinations from a population of 5. 4 million and >2 PB of data collected since 2010. Methods: Scotland has a central archive of radiological data used to directly provide clinical care to patients. We have developed an architecture and platform to securely extract a copy of those data, link it to other clinical or social datasets, remove personal data to protect privacy, and.
Use of routinely collected images, in contrast, opens up the potential for very large-scale studies, which not only efficiently and effectively complement smaller disease-based cohorts of patients but are also extremely flexible when linked to extensive electronic medical records, allowing for a wide range of disease areas to be examined. However, whereas research images are typically collected using specific image acquisition protocols under ideal conditions, routinely collected clinical images are more heterogeneous. Using clinical images for research and linking them to other routinely collected clinical data is challenging because: 1) Existing software used to query/search for images from the Picture Archive Communication System (PACS) is designed for clinical care rather than research.
The software makes it easy to find all images for a particular patient but is not designed to facilitate searching for all images with particular characteristics such as body part; slice thickness, scanning protocol, contrast agent, or patient medication; or linking to other Electronic Health Record (EHR) datasets (e. g. , outcome data, prescription data). 2) Re-use of clinical images for research required deidentification. However, identifiable data can be present in many areas of the associated DICOM (Digital Imaging and Communications in Medicine, RRID:SCR 018878)  [16]  file metadata and/or may be present within the pixel data themselves, "burned on" to the actual image. 3) Anonymisation of images can reduce the ability to perform linkage to other datasets, e. g. , demography, prescribing, hospital admissions.
Provision of routinely collected text-based clinical data for research Scotland has a relatively stable population with long-established use of a unique healthcare identifier (the Community Health Index [CHI] number) that is also increasingly seeded in data in other sectors such as social care. A National Health Service (NHS) Scotland service, called the electronic Data Research and Innovation Service (eDRIS)  [17] , provides a National Safe Haven environment (hosted by the University of Edinburgh) to support research access to anonymised extracts of linked data from different data controllers to answer specific approved research questions. The linkable phenotypic data include a range of national datasets including, e. g. , prescribing, death data, and hospital admissions. Subject to robust pseudonymisation safeguards and approval by the Public Benefit and Privacy Panel for Health and Social Care, individual patient consent is not required in Scotland.
This project assembles a library of imaging data and then generates thoroughly redacted subsets for research projects. Multiple safeguards are applied. The subsets are themselves only released to approved research projects within the controlled environment of the National Safe Haven computers; any extraction of data beyond that is subject to further controls to protect the privacy of patients. The technical safeguards described in this article and implemented in this project are not perfect and are not the sole protection: rare medical conditions or identifying features may still be present within the research extracts generated. Contractual and administrative precautions manage these risks: researchers are both contractually prohibited from attempting to re-identify patients or link against unapproved datasets and prevented from exporting the raw data beyond the confines of the Safe Haven because any such export could enable such an attempt to be made.
Incorporating clinical imaging data into the wealth of available datasets for research A research copy of the data held within the Scottish National Clinical PACS system has been created to enable the clinical imaging data to be linked with the other routinely collected datasets and be made accessible for research (given appropriate data governance approvals). The research copy of the Clinical PACS system is called the Scottish Medical Imaging (SMI) Database. The data are held in the non-proprietary DICOM format. The management of imaging data for research presents a substantial set of challenges beyond those encountered in the management of purely text-based records. Some of these are variations on familiar challenges, such as deidentification, whilst others are novel and intrinsic to this type of dataset, such as size and compute requirements for big data processing.
Requirement gathering An initial requirement gathering exercise was undertaken at the project inception eliciting requirements from the research community who will use the data extracts provided by the platform, the National Health Service Data Governance representatives as the data controllers of the data, and the National Safe Haven staff who will use the platform to build cohorts and provision relevant data extracts to researchers for analysis. We also investigated other open source and freely available platforms for hosting and/or anonymising imaging data to see if any of these could be used entirely or in part within our solution. We researched both functional and non-functional requirements of the solution.
Testing and case studies on sample data The software was then tested on a 180-TB subset of the full dataset including ∼3 million studies (which were loaded into the Document Store-see below). These were images generated across Scotland during the same 2-week period in February for each year in a 7-year period. Linkage, extraction, and anonymisation was performed for a range of case studies along with performance and functional testing. (Case studies are listed in Online Appendix D and included linking against the Scottish Cancer Registry's SMR06 diagnosis data and Radiology Information System records, as well as the DICOM metadata. ) The full set of historical data was still in the process of being decrypted from the proprietary PACS vendor format and could therefore not be used for complete testing at this stage.
The CHI number encodes potentially identifiable information such as date of birth, so we use an encrypted version known as EUPI (Encrypted Universal Patient Identifier) within the deidentified zone, and this is typically used for patient-linking tasks. Two processes (historic and ongoing) extract DICOM files from the proprietary PACS format received from the NHS. These DICOM files contain identifiable metadata (DICOM Tags) that are read and stored in full in the Document Store (a MongoDB database) by the Loader. The DICOM files also contain the pixel image information. The "Identifiable DICOM Tag Data" are analysed for potentially identifiable data. Relative file paths and a subset of DI-COM Metadata that can be de-identified and is useful for cohort building are sent to the "inventory tables" (a relational database) and stored using EUPIs within the Identifiable Zone.
A mirror of the Inventory Tables is provided to data analysts in the deidentified zone. This mirror supports condensing tag data, e. g. , to study/series level to improve performance of cohort generation/linkage. The inventory tables can be queried like any other database by research co-coordinators to construct cohorts using their existing working procedures. Once a research extract has been created, the corresponding image UIDs (study/series or SOPIn-stanceUID) are supplied to the extraction process. The extraction process uses the inventory tables to locate and anonymise the DICOM files held in the data store. These anonymised files are then provided to the researcher in the Safe Haven environment. A summary of the expected functionality of the data stores and process within the architecture is provided in Tables  1  and 2.
Architectural support for feedback and enhancement from other sources The architecture uses a microservices architecture. Individual components (microservices) can be turned on or off as needed and support multi-process execution for linear scaling. The architecture has been designed to support iterative enhancement based on feedback from research outputs generated in the researcher Safe Haven environment or directly from external sources (e. g. , clinical experts). Such enhancements could be: r New datasets, such as clinical mark-up (capturing ground truth data that have been generated by a radiologist marking up data for a set of images) r New processes to improve cohort generation or dataset preparation (e. g. , software that runs over pixel data and returns the size of the airways shown in CT scans), i.
e. , derived datasets. r Algorithms that could run over source images or textual data (e. g. , software that uses natural language processing on imaging metadata to find images that show signs of dementia). There are several key benefits: r This is an opportunity to incrementally improve the quality and value of datasets from SMI. r Research projects could add expertise at a scale that will never be available within a single development team. r It can improve collaboration and sharing across projects. r It supports active engagement by the user community and increases support for the service.
Current status of the data A copy of the historical imaging data (from 2006 to September 2018) in PACS proprietary format has been transferred onto the hardware environment and converted into the non-proprietary DICOM format. A feed from the National PACS to retrieve the data from October 2018 onwards is in the process of being commissioned. The first system test was conducted using ∼3 million studies (∼10 million series and ∼300 million images). These data (all the scans taken during the same 2-week period in February for 7 consecutive years) have been used as test data for software development. The implementation has enabled extraction of images based on cohorts built from data captured in DICOM tags and linking data from other sources-as illustrated in the "Use Cases" section. At this stage, operation still involves some manual intervention that we intend to automate as development progresses, and only the initial subset of DICOM tags (as recommended by a domain expert) is promoted, but the system is designed to facilitate enhancement and extension in future  [18].
Downloaded from https://academic. oup. com/gigascience/article/9/10/giaa095/5912826 by guest on 07 October 2020 DICOM files are stored unaltered in a file archive. There are many reasons why we wish to keep the identifiable data and store the original DICOM images: • Any program developed to strip all identifiable data from the DICOM files and tags risks rendering the whole dataset unusable if this is done incorrectly. Linkage to other datasets would subsequently be either incorrect or impossible. • It is conceivable that future data de-identification strategies will wish to make use of some identifiable data and removing that data would therefore limit future options. • The NHS may wish to use the data as a secondary offline disaster recovery system or use the data to populate a clinical system from an alternative provider.
In this case it needs to be technically feasible to generate the data in identifiable form in a format that is non-proprietary and as close as possible to the DICOM files as they were originally captured. Identifiable DICOM tag data All tag metadata from the DICOM files are extracted to a MongoDB database in a searchable format. These tag metadata are stored in an identifiable format because de-identification analysts need to know what the identifiable data are so that they can remove them, e. g. , • If the patient name is Mrs Jones, then a de-identification analyst searching for identifiable data in the clinical report will need to know to look for the text "Jones" in order to remove it. • To check whether an image is identifiable, the de-identification analyst might need to know the CHI number in order to check that it is not burnt into the pixel data.
Inventory tables A subset of data from the Identifiable set above is copied here. This is a relational database that contains suitably cleansed and de-identified image metadata (and file paths), i. e. , has been confirmed to be well populated, of high quality, and does not contain identifiable data. This is used by the research co-ordinators for cohort creation and extraction to the Safe Haven. The data are indexed using EUPIs. For example, DICOM age strings can express the age in years, months, or days (e. g. , 075Y, 006M, or 002D). The cleaned and homogenized metadata will store these in a consistent and easily queried numeric format. Other metadata fields may be a single value summarising data stored in multiple different DICOM tags. For example, by analysing the acquisition position of the images it is possible to identify examinations in which the same volume has been acquired repeatedly in a single series; when used in conjunction with tags that indicate whether contrast was used during the examination, this can be used to disambiguate contrast bolus imaging from other acquisitions that may also use contrast.
Cohort and associated anonymous research extracts Any research project will start by defining a relevant cohort and obtaining the necessary ethical/administrative approval in consultation between the researchers and the research coordinators-this is an out-of-band process outwith the iRDMP system, so not shown here. The data analysts then assemble a dataset (the anonymous research extract) for that research project by querying the inventory table, possibly linked against other data sources via the EUPI (pseudonymised patient ID, explained below) and trigger the Extraction Microservices to export the appropriate subset of columns made available to the research users. For example, a project might request all available brain MRI scans from patients who have been prescribed gabapentin and want the dosage information and patient age; they would be given a set of image data (the scans themselves, passed through the DICOM file anonymiser described later) and a table of associated metadata including the dosage information for each de-identified patient.
Some images, particularly ultrasounds, may have identifiable information such as patient name or CHI watermarked on the image. Which images can be de-identified is stored in the metadata catalogue, but the rules regarding how images are de-identified are stored in CTP  [17]  anonymisation scripts that apply to all images. These scripts contain rules such as "if Modality is US and Manufacturer is X and model is Y then blank out pixels in the rectangle (0 0 1000 200). ".
Mapping (CHI-EUPI) This process is called when metadata are promoted to the de-identifiable zone to replace identifiable CHIs with EUPI. It is an automated process so that no individuals can see this mapping. Cohort creation process A set of software tools (or manual SQL queries if the user prefers) that query the DICOM metadata within the inventory tables to select images relevant for a particular cohort (by applying filters that describe researcher requirements). The resulting cohort forms the basis for both the initial and subsequent releases of data to the Safe Haven for the relevant study, and as such it is critically important that the cohort be identified and managed correctly.
Researcher VM with tools to view and manipulate images There are 2 main use cases: small-scale studies in which a researcher team may wish to open and mark up each image by eye and large-scale studies in which software and algorithms will be developed by the users of the system to analyse the images for their specific project. The different tools available within the Safe Haven meet both sets of requirements. The researcher VM image includes a standard set of tools, which will be increased over time as the requirements increase. Example tools are MicroDICOM (simple DICOM viewer), ClearCanvas (open-source PACS client, cf. Carestream), and XNAT. The VM should have the capability for users to securely add their own tools. The VM provides access to the associated data from study-specific image metadata and pixel data but does not allow row-level or pixel data to be extracted.
Non-structured database solution for identifiable metadata The data in the DICOM tags are largely unstructured and deeply hierarchical. This is challenging to represent in a relational store. Moreover, their structure may change over time (e. g. , new tags). Consequently, the use of a document-oriented, flexible, and dynamic data storage system was deemed necessary. MongoDB  [24]  was selected as the NoSQL database technology because the Research coordinators understand where the data are stored and how to link across datasets and will run software, write scripts, query databases, etc. , to produce the final cohort datasets.
Software developers Produce any new software required within any zones of the environment. The software is developed and tested outwith the production environment. Deployment of software updates will be carried out by system administrators. hierarchy of DICOM tags can be mapped directly to a JSON document, then indexed and queried efficiently. This facilitated the transfer of images across database collections and use of queries against DICOM tags to select and control the promotion of data to later stages in the process. Within the architecture the MongoDB database provided a middle ground within the ETL data flow. It allows mappings to the relational database schemas to be quickly modified and tested, while being able to quickly reload and reprocess data from MongoDB rather than the slow process of going back to DI-COM files, reducing the petabytes of raw DICOM files to terabytes of queryable data.
Structured database solution for de-identifiable metadata Many DICOM servers and APIs have a way of representing DI-COM in a relational schema, e. g. , dcm4chee  [25]. We have used our own (dynamic) cut-down schema for several reasons: r To present something to data analysts that has a simplicity (without requiring DICOM expertise) on the same level as the other linkable datasets hosted on the National Safe Haven. r To optimise for linkage, i. e. , the ability to limit the number of table joins needed and create efficient query-oriented indexes, e. g. , PatientId+ImageType+StudyDescription. r To be able to adjust this schema and regenerate the data as future development requires. r To be able to add additional curated fields from external sources or transformed columns such as results from expert mark-up as ground truth data.
Anonymisation tools There are many different software programs that can be used to de-identify imaging data. We tested the feasibility of 3 different widely used programs (DICOM Confidential  [26] , XNAT  [27, 28] , CTP  [29] ) in deciding which to adopt as part of the pipeline. A summary of each is provided in Online Appendix B. For a meaningful comparison of the tools, a set of criteria were devised, and each de-identification program was examined in turn against these criteria using a rating of 1-5 (where 5 is the best). We grouped the results into 3 different categories: core functionality, user friendliness, and support. Table  4  provides a summary of the scores for each category, with the detailed analysis provided in Online Appendix A. In summary, DICOM Confidential was ruled out owing to the quality of the documentation and the lack of first-party or community support.
Comparisons to other existing systems Given the different imaging platforms in active development to support research projects, we investigated alternative platforms so that we did not re-invent the wheel. In general terms, other solutions have concentrated on consented cohorts from researcher-collected research images rather than much larger unconsented data from routinely collected "real-world" images. The architectural solution developed by others is generally a large anonymised database (sometimes distributed) containing all the images with permissions to see, extract, and run pipelines on the imaging data configured for each research group. The metadata provided are limited and relatively clean in comparison with routinely collected data. The architectural challenges and solutions are therefore very different.
Exemplar-driven use cases A number of use case scenarios were defined with input from researchers (listing in Online Appendix D). These scenarios were further elaborated by a team from eDRIS, in effect, performing a dry run as though these were real research projects. The test cases assessed included the following scenarios: r Cohorts can be generated using the metadata repository in- formation r Images can be returned where the cohort has been generated from another dataset r Cohorts and images can be identified using a combination of the metadata repository and other data sources.
Potential impact of enabling this resource The SMI data, linked to other datasets, along with the secure iRDMP platform we have developed have the potential to reduce the costs and widen access to large quantities of routinely collected de-identified images at scale. They also have the potential to reduce the effort of obtaining governance approval because a data controller-approved method for de-identification and access has already been agreed. Increasing the availability of large-scale routinely collected imaging datasets linked to other forms of health data for both industry and academic use will hopefully lead to a greater likelihood of achieving results translatable into diagnoses and treatments.