dataesr/scientific-paragraphs-categorization huggingface dataset:
42 In 2014 the OMOP research team launched the Observational Health Data Sciences and Informatics (OHDSI) (pronounced "Odyssey") program  43  which is currently continuing the activity of OMOP. The MATRICE project was funded by the Italian Ministry of Health and ran from 2011 to 2014 under the coordination of the Italian National Agency for Regional Health Services to measure quality of health care for chronic diseases. MATRICE developed a distributed network infrastructure specific to local and regional Italian administrative databases and is rapidly growing to include participants beyond the project. Currently, it covers a population of about 9 million subjects living in some of the Local Health Authorities in 9 of the 21 regional health care systems in the country. Studies completed so far using data from this network were aimed at evaluating the quality and equity of primary care, the impact of policies in this field [Visca2013,  44, 45, 46, 47  and methodological challenges of such studies [Gini2014].
DATA SETS AND DATA TRANSFORMATIONS: D1 (original databases: DBs) is a collection of data sources controlled by a single organization that has procedures in place to link them with each other at the individual level, thus creating a single data pool on the same subjects. The term "DB" refers to an organization that has access to the data. T1 (data reorganization) is a data modeling step: transformation from the locally defined data repository into a global (common) schema with standardized variable and attribute names, without loss of information. Simple one-on-one recoding is performed as well, such as making data formats and coding of attributes (e. g. , gender) identical. T1 is specific per DB but independent of the specific study. D2 (global schema, GS) is a general database schema that contains all the attributes thatare necessary to answer a realm of study questions ("use cases") that are of general interest to the network, such as incidence of disease, drug utilization, or association studies.
D2 has a defined set of table names, attribute names, and formats. D2 plays the same role as a GS of a data integration system. 50 herefore, a set of correspondences are defined between this schema and the D1. Note that (1) these correspondences may not be complete for all databases: for instance, if a D1 does not have information about primary care diagnoses, these attributes will remain empty in the D2; and (2) some attributes (typically, diagnoses or drugs) might have different coding for different DBs in the network.
DATA SETS AND DATA TRANSFORMATIONS (CONT'D): T2 (data derivation) is the step where novel meaning is obtained from D2 by means of an explicit manipulation and combination of D2data. These manipulations are necessary when a study variable is not among those collected by one of the DBs in the network, and must therefore be represented, by proxy, as a combination of whatever pertinent information is available. When the study variable is a disease, this process is referred to in the literature as disease phenotyping. 51 T2 is often specific per DB, as it depends on the information that was originally collected, and is often specific per study, although conceivably past data derivations could be reused in new studies. As an example of T2, if the presence of diabetes in study subjects needs to be assessed, DBs collecting data from primary care can identify the information from a general practitioner's (GP's) diagnosis, whereas claims databases without clinical data from primary care may use dispensing of antidiabetic drugs as proxy, and combinations may also be possible.
T3 (study design application) is data transformation for a specific analytic: based on the protocol of a study with specific design (application of inclusion and exclusion criteria, selection of exposure windows, propensity and disease score estimation, control selection, matching). T3 produces the data sets for statistical analysis. Within this transformation data may be de-identified and aggregated to various levels. T3 is specific to the study, butis the same across participant DBs. D4 (data sets for analysis) is the result of T3. D4s from all the partners in the network are similar. Based on the level of sharing that is allowed, D4 may stay local at the database custodian or be pooled in a central repository. In both situations, statistical analysis on D4 follows and produces estimates to be interpreted.
Box 2. An Example of Data Management in the MATRICE Network The Italian National Agency for Regional Health Services promoted a study to assess whether regional Italian administrative databases can be used to measure whether patients with Chronic Obstructive Pulmonary Disease (COPD) are treated with recommended therapies. The study objective was to establish whether different cohorts, defined with different case-identification strategies, resulted in consistent estimates of therapy adherence. The MATRICE network was used for this study. Five regions were involved in the study. In each Italian region several tables of administrative data are collected with content regulated by national law, in particular the following: the list of residents (citizens and regular migrants) entitled to receive health care; hospital discharge records, with six diagnosis codes; exemptions from copayments for health care; and drug prescriptions.
In each region participating in the study, a copy of the four tables (D1) was stored, with different data models and format. The MATRICE network has established a specific data model for the above mentioned four tables(list of residents; hospital discharge records; exemptions from copayments for health care; and drug prescriptions), and the format is flat comma-separated files (D2). Two of the regions had already participated in a previous study of the MATRICE network, so T1 had already been performed. In the other three regions, the format D2 was explained to a local expert by means of structured documents and a teleconference, a common software named TheMatrix was installed (see "T1 (reorganization)" inthe "FINDINGS" Sectionbelow), and T1 was performed by the local expert and was checked with standard procedures embedded in the software.
The study protocol had defined several variables to be extracted or derived: gender, presence in the region at index date, age at index date, presence of a COPD diagnosis in the 1-5 years before index date, presence of some patterns of utilization of respiratory drugs in the 1-3 years before index date, and adherence to recommended therapies during follow-up. D3 was composed ofa group of data sets, one per derived variable, each with a single observation per subject. Since in MATRICE all the participating data partners share the same data content (see "D1: original DBs" in the "FINDINGS" Section below), the transformation T2 was uniform across data partners. T2 was therefore embedded by the principal investigator in a single ad hoc procedure of the software TheMatrix, shared with the local partners and executed locally.
Differences and Similarities Besides local storage, in OMOP some databasesalso allow creating a central and cloud-based copy of the transformed data. In MATRICE and Mini-Sentinel, all original databases used the same coding systems, while in OMOP participating databases used different coding systems and even unstructured free text in different languages, in EU-ADR. If Irina, Louise, and Mario were part of the database population of the four networks, the image of the story would be different. For databases lacking diagnosis from primary or secondary care, like M1 or EU1-EU4, Irina's beginning to take insulin could be misinterpreted as an occurrence of diabetes, even though a complex algorithm using hospital admittance for delivery or the ending of insulin prescriptions could effectively avoid misclassification.
Louise's vaccine would be detected by MS1, MS2, and MS3. When Mario moved to Irina's region and entered the database population, only databases collecting clinical history from primary care-like EU6, EU7, MS1, and O4-could have detected that he was the father of Irina and was a smoker. While the history of coronary heart disease could also be deduced from the same databases or clinical notes of a cardiologist in MS1, the presence of the disease may be inferred from drug utilization data in all the databases, and angina precisely in 2010 in databases with diagnoses from inpatient care (MA1, EU1-EU4, all MS, and O1-O3 In Table  2 , D2 is compared across case studies.
Differences and Similarities The main difference we observed in the evaluation of the data models was the way two main characteristics of an encounter were captured: the setting where the health care was administered (e. g. , general practice, inpatient care, laboratory) and the medical content of the encounter (e. g. , diagnosis, procedure, laboratory test). One possibility was that information was grouped in tables according to the setting (e. g. , a table for hospital admissions, another for laboratory tests) and facts were recorded as attributes. The alternative was that encounters were grouped in tables defined by medical content (e. g. , a table for diagnoses, a table for procedures) and the care setting was recorded as an attribute. EU-ADR and MATRICE adopted the first approach, OMOP adopted the second, and Mini-Sentinel adopted a combination of the two approaches-death and pharmacy dispensations were organized in the first way and other information was organized in the second.
Differences and Similarities In EU-ADR each data custodian executed its algorithm with its own usual extraction tool to derive simple input files for a specific study, while execution was performed with common software on the GS in the other networks. OMOP and Mini-Sentinel adopted shared SQL and SAS code, respectively. In MATRICE an ad hoc scripting language was designed and a compiler (a computer program that transforms source code written in a programming language into another) from this language toward the Java virtual machine was developed; extraction in a shared code was then executed locally. Since OMOP focused on methods development, it often used multiple algorithms for data derivation, to study the impact of the differences. In MATRICE, ongoing validation studies test several algorithms, but the plan is to use a single best definition per study in the end.
In EU-ADR, to overcome the heterogeneity across terminologies, a shared semantic foundation was built by using Unified Medical Language System (UMLS) concepts to define events  5. Then, the definitive choice of algorithms was obtained through an iterative negotiation between databases: DBs with similar structures were invited to query the same tables and fields. 54 In Mini-Sentinel, algorithms are developed (or reused) for specific analyses and applied at the time of analysis; the result of those algorithms is not stored in the database, but analytic files for each assessment are retained locally. As for validation of the event resulting from data derivation, all the networks compared incidenceor prevalence rates among databases as a tool to assess consistency. OMOP did not routinely compare with external standards nor with the literature.
Differences and Similarities During steps T1 and T2, local partners in some of the networks were asked to implement the processes that had been agreed upon in their own local procedures; moreover the procedures were not shared. In step T3 (study design application), data transformation into analytical data sets was performed in all four networks using shared and common software. In Mini-Sentinel and OMOP, statistical analysis was needed in T3 to estimate propensity and disease scores, while in the studies implemented in the other networks only simpler tasks were needed: linkage between different tables, time splitting, random selection, matching, de-identification, and aggregation. The software Jerboa was developed and used by EU-ADR to execute T3. The software TheMatrix developed by MATRICE executes both T2 and T3: a Domain Specific Language (DSL) was designed and developed for this purpose.
DSLs are computer programming languages whose features and expressiveness are restricted and designed ad hoc to fit a given field of application. They target a narrower set of programs than general-purpose languages like Java, but in exchange they provide a higher level of abstraction and can be programmed directly by domain experts rather than computer programmers. 55 In MATRICE, a DSL generating tool called Neverlang was used to develop the language,  16, 56  and scripts in the language were generated by domain experts. Mini-Sentinel and OMOP both used existing software (SQL, SAS, C, Java and R). In Table  3 , D4 is compared across case studies.
Differences and Similarities In OMOP only final estimates were shared, while in the other networks integrated data sets were shared to be pooled before statistical analysis. EU-ADR and OMOP both adopted a similar validation strategy for their methodological studies in drug safety, which implicitly validated the whole sequence of data transformations at once: a set of positive controls (known adverse drug reactions) and negative controls (drug-outcome pairs that are believed to have no causal relationship) was created. The quality of each method of analysis was assessed by measuring its discriminating power, i. e. , the ability of telling positive from negative controls.
Differences in Original Databases (of DBs) Have Huge Consequences The differences observed in the four networks when comparing the original databases (D1) are huge. Understanding such differences is a challenge in itself, as terminology describing health data sources is not shared across countries. 57 The three national networks (MATRICE, OMOP, and Mini-Sentinel) were much more homogeneous than EU-ADR. Since we expect that networks will continue to grow and new DBs will be different from existing DBs, the.
Substantial Differences in the GS (D2) between the networks exist but are not substantial, as each GS can be mapped into another, except for those data items that are specifically collected in a single network (for instance, exemptions from copayment, which are documented only in Italian DBs). It would be very valuable, however, to explicitly create such a mapping, as this would make it possible to run existing software procedures embodying T2 and T3 independently of the network: this happened, for instance, in a study replicating-in the EU-ADR network-results from the OMOP network. 58 One area of research should be the impact of different formats of GSs on study outcomes.
Different Approaches to Terminology Mapping In two networks (OMOP and EU-ADR), different disease and drug coding systems needed to be managed. In OMOP the differences were addressed by mapping to homogeneous coding systems during T1, although the original codes were not discarded but were also included in D2. In EU-ADR, mapping was not conducted in T1, therefore all mapping was performed during T2 and only for study-specific conditions. Due to the large differences in the granularity and type of coding schemes, in European databases mapping was very time-consuming-yet this was necessary to obtain consensus across data custodians and investigators  6  -and is progressively growing a shared library. The impact of different mapping strategies, and whether mapping should be done at all versus addressed in the analytic phase, should be investigated.
Software Tools, Professional Skills, and Information Technology Software tools used during the transformation process differed across case studies. This had implications for the type of professional skills needed to perform studies in the network as well as the readability of the programs for other investigators. In principle, all data transformations must be documented to allow investigators to correctly interpret study results and to understand study limitations and strengths. OMOP and Mini-Sentinel have complete websites where information is stored and can be openly accessed, while EU-ADR and MATRICE rely mainly on scientific papers and reports, a less efficient way of storing information. How to develop transparent programs and how to store and share the corresponding complex body of information to make it easily available to investigators is also a relevant research topic.
Validation of derived data is an imperative condition to produce good epidemiological estimates,  62  and this is even truer when heterogeneous databases participate ina network. Indeed, regularizing the process of creating research data sets from secondary data sets, although necessary, is not enough to ensure high data quality; and validation can quantify how much derived data fail in correctly identifying the study variables-failure that can differ across data partners. In MATRICE-as data from primary care is lacking and information from secondary care is sparsederiving chronic conditions, the primary focus of the network, is cumbersome. This is why MATRICE is leading a population-based validation study using diagnosis from a sample of GPs as a gold standard. In Mini-Sentinel a model for a typical validation study was developed  13  and implemented for some events, in particular acute myocardial infarction.
14 EU-ADR adopted a similar study design in some validation studies [, Valkhoff2014]. 7, 8 Only positive predictive value could be estimated from the study design adopted in the two networks. A similar study was performed on an occasional basis in OMOP. 63 In order to estimate sensitivity, access to a populationbased data source would be required, which is more complex than accessing clinical charts of selected candidate events. However, in the specific case of Volume 4 (2016) Issue Number 1 acute myocardial infarction, death registries are estimated to add from 15 percent to 25 percent of cases to inpatient data where both data sources are available  6. Therefore misclassification of non-cases, in principle, could have a relevant impact on study results, especially in older subpopulations.
Limitations The conceptual framework was useful to interpret similarities and differences among the four networks, which are heterogeneous for geographical coverage and purpose. However the choice of the sample of four was nonsystematic, therefore the framework may prove insufficient to include other networks in the comparison. Data processing in networks of databases may suffer from subtle challenges: privacy laws may enable patients to opt out of sharing information based on some encounters only (for instance, for mental health issues); some databases may collect information from smaller health care providers, whose information is not effectively shared in digital form; regional or national differences in privacy regulations may affect differentially the partners of a network. We did not investigate how the four networks faced such challenges.
Conclusion We proposed a conceptual framework to analyze the data management process involved in observational studies taking place in distributed networks of databases. The framework was applied to four case studies to identify similarities and differences. Several research questions were highlighted by this comparison, including interoperability among the available GSs, optimization of data harmonization, use of validity indices in study design and statistical analysis, development of an information infrastructure to support investigators in accessing details of data transformation, and optimal level of programming skills needed to manage the process. Medical informatics is called on to support transparency, and quick and sound application of the experimental method to the production of empirical knowledge.
Table1summarizes the DBs participating in the four networks. For each network a column represents a combination of data sources that are linked in at least one database. We classified data sources according to their provenance, and we indicated the data items available in the DB from that data source. If more than one DB in a network share the same combination, only one column is shown: the number of columns fora network in Table1is therefore a measure of heterogeneity of the DBs participating in the network. MATRICE has a single.
Volume 4 (2016) Issue Number 1 Notes: If more than one database in a network has access to the same combination of data, they are represented by a single column. Data items-Dx: diagnostic codes; Proc: procedure codes; Rx: prescriptions or dispensings of drugs; Spec: specialty of secondary care encounters; Refsec: Referrals from secondary care; Description of the D1 (Original DBs) Databases in Terms of Provenance and Data Items Collected from Box 3. A Short Illustration of the Differences in Original Data In 2005, Irina, age 36, developed gestational diabetes during her second pregnancy, which was diagnosed by her gynecologist and treated with insulin prescribed by her GP. Irina gave birth to Louise in a hospital, and had her vaccinated against tetanus and diphtheria when the baby was six months old.
Acknowledgements This study was funded by the  Integrazione dei Contenuti Informativi per la Gestione sul Territorio di Pazienti con Patologie Complesse o con Patologie Croniche(MATRICE)  project, which is funded by the  Italian Ministry of Health  in the framework of the  MATTONI Program. Drs. Ryan  and Schuemie are employees of  Janssen Research and Development. The other authors declare no competing interests.
Table1summarizes the DBs participating in the four networks. For each network a column represents a combination of data sources that are linked in at least one database. We classified data sources according to their provenance, and we indicated the data items available in the DB from that data source. If more than one DB in a network share the same combination, only one column is shown: the number of columns fora network in Table1is therefore a measure of heterogeneity of the DBs participating in the network. MATRICE has a single codes (M1 andEU1-EU2). Fourth, access to laboratory test results is rare among databases in all networks. Fifth, in all but one United States database, enrollment of subjects in the data collection is due to the eligibility criteria for social insurance or an insurance company, while in Europe criteria include geographical residence or being listed with a GP.
Sixth, only in EU-ADR and Mini-Sentinel are death and immunization registries available. Finally, only Mini-Sentinel involves partners collecting information from both clinical and administrative data sources. This is achieved by integrated delivery systems that operate medical facilities from which they collect electronic health care records data. Differences and Similarities First, in the two United States-based networks (OMOP and Mini-Sentinel) almost all databases (O1-O3 and MS1-MS3) obtain administrative information from primary, secondary, and inpatient care, while in both European networks (EU-ADR and MATRICE) each database lacks at least one setting. Second, EU-ADR pools data from the most heterogeneous databases: the eight databases showed seven different combinations. Third, in Italy, although administrative information from secondary care (such as specialty of the physician visiting the patient) is available, it does not contain diagnostic combination (M1), EU-ADR has seven (EU1-EU7), Mini-Sentinel has three (MS1-MS3), and OMOP has four (O1-O4).