dataesr/scientific-paragraphs-categorization huggingface dataset:
About 26% of index cases carrying such a mutation (i. e. the first individual tested in the family) are included in GEMO, and 21% in GENEPSO  [18]. Therefore, it is essential to identify the overlap between participants of the both studies by linking the two data sources, which will allow setting up studies evaluating simultaneously genetic and nongenetic factors modifying cancer risk of carriers of a BRCA1 or BRCA2 mutation. Studies conducted in subjects enrolled in both studies will also allow, for instance, assessment of whether it is possible to predict response to treatment according to BRCA1/2 mutation status and other genetic variant profile.
Data In September 2016, 4688 participants had been enrolled in GEMO and 3339 in GENEPSO. This initial dataset (dataset 1) was used for building ML algorithms and determining an optimal linkage method. This optimal linkage method was then applied on the updated version (dataset 2) of the two studies as of December 2019. The updated version counted 6375 participants for GEMO (i. e. 1687 new participants), and 4925 participants for GENEPSO (i. e. 1586 new participants). Name and address of individuals were not available here due to privacy and confidentiality policies. Ten matching variables shared between GEMO and GENE PSO were used for comparison (Table  1 ): recruiting center number (CTR), family number (NUMFAM), individual number in the family (SUJID), gender (GENDER), year of birth (Yob), month of birth (Mob), day of birth (Dob), BRCA1 mutational status (BRCA1), BRCA2 mutational status (BRCA2) and mutation description using the HGVS nomenclature (MUT_HGVS).
Recruiting centers may be coded differently in GEMO and GENEPSO, however we were able to standardize the CTR variable during data pre-processing (see next paragraph). NUM-FAM and SUJID are assigned by the recruiting center at the time of recruitment. In principle, same family number and individual number should be provided by the clinician to GEMO and GENEPSO investigators. However, the family number may be recorded under different formats in the two databases. Moreover, an individual may be assigned a different individual number on the pedigree (SUJID) in GEMO and in GENEPSO, if included at a different time in the two studies. These inconsistencies may be corrected later by manual review if other variables such as recruiting center, sex, BRCA mutation (HGVS nomenclature) and date of birth are identical in the two databases.
BRCA1/BRCA2 mutation descriptions should theoretically use the international HGVS nomenclature in both database  [20]. However, the two studies were independently set up more than 20 years ago and standards for mutation descriptions have evolved during these years. Therefore, the same mutation identified at different times by two laboratories may have been recorded with a different nomenclature. We thus had to standardize this annotation during data pre-processing (see next paragraph).
Record pair comparison Let X and Y be two databases and x ∈ X and y ∈ Y two arbitrary records in form of a d-dimensional vector, i. e. x Because of the evolution in standards for mutation descriptions during the last twenty years, a same BRCA mutation may have been annotated differently at different time by different laboratories from database to database. Even though we standardized MUT_HGVS variable as much as possible during data preprocessing, the inconsistencies in reporting the same mutation in GEMO and GENEPSO still existed. This is why we allowed inexact matching for the HGVS_MUT variable.
Supervised machine learning (ML) linkage We first used blocking to reduce the number of possible record pair comparisons. Missing data in blocking variables (BRCA1, BRCA2, GENDER and Yob) were tolerated here. After blocking, the imputation of missing data could be then performed. The missing data in similarity for MUT_HGVS (numeric) were imputed by Bayesian linear regression and those for other categorical matching variables were imputed by logistic regression. The labeled record pairs were randomly partitioned into two sets: the training dataset (60%) on which we trained ML models, and the test dataset (40%) on which we evaluated the predictive performance of the trained models. ML models were built by using the similarity vector of the six variables (CTR, NUMFAM, SUJID, MUT_HGVS, Mob, Dob).
Hybrid record linkage process Step 1 (Fig. 1a ): after data pre-processing, we built record pair comparisons using dataset 1. We then computed a PRL score S from Eq. 1 for each pair. After examining the empirical distribution of PRL scores, we chose a threshold t to separate all record pairs into potential matches (S(x, y) > t) and non-matches (S(x, y) ≤ t). We manually reviewed not only these potential matches  [28] , but also the potential FNs. We launched several rounds of searches to find out the record pairs with PRL scores below the chosen threshold that may be FN. This exhaustive manual review was essential to minimize the number of FN, so that we could obtain a set of true matches representing the gold standard used to generate the confusion matrix of the dataset 1. Step 2 (Fig. 1b, c, d ): based on the gold standard of true matches from step 1, we selected the optimal linkage method by using a nested validation procedure.
More specifically, we used a 5-fold cross-validation to compare the average performances of PRL, ML and PRL + ML. In the PRL + ML method, we first implemented PRL and ML in parallel, then combined together the candidate matches classified by both approaches. Within the cross-validation, we used a 60-40% traintest split procedure to pick the ML algorithm having the best average performance. Within this procedure, we used a 5-fold inner cross-validation to determine the optimal parameters for each of the ML algorithms we investigated (See Supplementary Data for details). We evaluated the performances of ML algorithms and the three linkage methods (PRL, ML, PRL + ML) based on the gold standard of true matches from step 1. Once the best performing ML algorithm and the optimal linkage method was chosen, it is necessary to train a final ML model for the ML or PRL + ML linkage method to predict the new true matches on the updated dataset.
To achieve this, after the blocking step and the imputation of missing data on the whole record pairs (from step 1), we trained a ML model on a larger subset than any "dataset A" used in the cross-validations step (Fig. 1b  and c ), and verified whether this model still had a good performance on the remaining subset. Step 3 (Fig. 1e ): we applied the selected linkage method followed by manual review to identify new true matches on the updated dataset.
Performance measures The predictive performance of all algorithms was assessed. In record linkage, the data is imbalanced, meaning that the two classes are not represented equally. Indeed, there are far more non-matches than matches. In such cases, standard accuracy is not a good measure of performance. Instead, precision and recall are commonly used for evaluating the linkage quality. We calculated these performance metrics based on the confusion matrix described in Table  S1. The precision is the proportion of classified matches that are true matches. precision ¼ TP TP þ FP The recall is the proportion of true matches that have been classified correctly. A good linkage algorithm will typically have values of precision and recall greater than 0. 95  [29].
Results Matching status assignment by PRL followed by manual review Up to September 2016, 4688 individuals had been enrolled in GEMO and 3339 in GENEPSO,. After data preprocessing, 15,653,232 record pairs were built as the Cartesian product of the two databases in dataset 1. The PRL score of each record pair was computed from Eq. 1 (see Methods). The empirical score distribution is given in Fig. 2  and Table  S2. We observed a large peak of low scores, corresponding to a large number of nonmatches, and a small peak of high scores, corresponding to a small number of matches. This bimodal distribution suggests that PRL worked as expected. After examining the distribution of PRL scores, we chose the threshold separating matches from non-matches at 0. 6 and performed a step of manual reviews. 2,664 record pairs had a PRL score above 0.
Selection of a supervised machine learning algorithm We applied 5-fold cross-validation on the 15,653,195 record pairs that were labeled in the previous step. Within this cross-validation, we call A the training dataset (containing 12,522,556 record pairs) and B the test dataset (containing the remaining 3,130,639 record pairs). After blocking, each dataset A was randomly partitioned into a set Atrain containing 60% of the record pairs and a set Atest containing the remaining 40% of the record pairs (Table  S3 ). The average predictive precision and recall of the Bernoulli model and of the six ML models, trained on Atrain and evaluated on Atest (Fig. 1b ), are presented in Table  2. The six ML models outperformed the naive Bernoulli model, and their performance values, whether precision or recall, were all higher than 0.
Evaluation of three linkage methods In the 5-fold cross-validation step, the averaged performance of three linkage methods (PRL, RF, PRL + RF) was assessed on dataset B (Figs. 1c  and 3 ). PRL and PRL + RF had thresholds varying from 0. 6 to 0. 8. As expected, increasing the threshold resulted in fewer FP, which led to an increase in precision, but more FN, which led to a decrease in recall. Across all thresholds, the PRL + RF method showed a higher recall than the RF (Fig. 3a ). The recall of PRL decreased significantly with the threshold. The recall of RF was similar to that of PRL at threshold 0. 65. RF achieved a higher precision than that of PRL and PRL + RF across all thresholds, while PRL and PRL + RF achieved similar precisions, which increased significantly with the threshold (Fig.
Linking records in updated GEMO and GENEPSO databases As of December 2019, 1687 new BRCA1/2 mutation carriers had been enrolled in GEMO and 1586 in GENE PSO. These updated GEMO and GENEPSO samples constitute dataset 2. The combined linkage method PRL + RF was applied on this dataset to identify the new true matches (Fig. 1e ). Linkage was first performed by PRL and by RF separately. The RF model here was trained on a subset of dataset 1 (i. e. using GEMO and GENEPSO participants enrolled before September 2016) (Fig. 1d ). RF and PRL predicted 819 and 1268 candidate matches, respectively (Fig. 4a ). Besides the 772 matches that were common to these two linkage methods, RF had an additional 47 candidate matches and PRL had 496 additional candidates. Those 1315 (772 + 496 + 47) candidate matches were then manually reviewed.
The PRL approach was correct for 57. 3% (727 out of 1268) records; while the RF was correct for 87. 3% (715 out of 819) records (Fig. 4b ). Finally, 738 of the 1315 candidate pairs suggested by the combined linkage method were true new matches. This is consistent with the precision achieved with a threshold of 0. 6 on dataset 1 (Fig. 3b ). The PRL + RF method identified 11 more true matches than PRL alone, corresponding to a gain of 1. 5% (11/738), at a cost of manually examining 47 more candidate pairs. It also identified 23 more true matches than RF alone, corresponding to a gain of 3. 1% (23/738), at a cost of manually examining 496 more candidate pairs (Fig. 4  and Table  S4 ). This confirmed also that the PRL + RF method had a higher recall than PRL and RF alone. To summarize, in December 2019, GEMO included 6375 participants and GENEPSO included 4925 participants, and our hybrid record linkage identified 1995 BRCA1/2 mutation carriers from 1693 families that had been enrolled in both studies.
Discussion PRL has a lower computational cost but the linkage quality is impacted by the choice of the threshold on the likelihood score. Lower thresholds lead to more FP whereas higher thresholds lead to more FN. The ML approach reaches higher precision, requesting fewer manual reviews. However, the blocking step can lead to FN if the data contain errors in blocking variables. We found that the PRL + ML combined method, having the highest recall compared to either of the two methods alone, improves linkage by identifying more true matches, but at the cost of additional manual reviews. In a context where manual review cost is to be capped and missing true matches is tolerated, the ML approach, which has a much higher precision to the expense of a lower recall, is an interesting option. Another possibility, which we expect from our results on dataset 1 (Fig.
3 ) to reach higher recall and higher precision, would be to use PRL + ML but with a higher threshold for PRL (such as 0. 68 in our study). Here, our goal was to identify as many common participants as possible between the two studies, so as to facilitate research projects requiring both genetic and follow-up data. We therefore chose the PRL + ML approach with a relatively low threshold of 0. 6 for PRL, so as to maximize recall. We expect linkage performance to be related to the number of matching variables. Had more matching Fig. 4  Comparison of candidate matches predicted by the RF and PRL models for the updated databases. a RF and PRL identified 819 and 1268 new candidate matches, respectively; 772 candidate matches were common to both approaches. b After manual review, PRL + RF led to the identification of 738 true matches, among which 727 were identified by PRL alone and 715 by RF alone.
704 true matches were identified by both approaches. 23 true matches were identified only by PRL, and 11 true matches were identified only by the RF model variables been shared between GEMO and GENEPSO, the most discriminating matching variables could have been identified using feature selection algorithms, resulting in a lower computational cost. Here, with 10 matching variables, such a strategy was not necessary. On the other hand, if too few matching variables had been available, one could expect ML models to have lower performance, giving the advantage to PRL. Previously, Elfeky et al. [30]  described a hybrid technique for record linkage, combining both supervised and unsupervised machine learning methods. Record pairs were assigned a matching or nonmatching status through unsupervised clustering, and the resulting labeled data was then used as a training dataset for a supervised model  [2].
However, this technique was not suitable here since two unsupervised machine learning methods (K-means and bagged k-means) showed independently poor performance (Supplementary Data, Table  S5 ), probably due to our imbalanced data. In this study, the two databases are limited in size. However, larger databases may be challenging for record linkage. In this case, the traditional blocking technique that we employed here is a first step towards reducing computational complexity. In addition, partitioning the data into a larger number of smaller blocks and processing them in parallel using our hybrid record linkage process could be used to maintain a reasonable computational time. Besides, In order to decrease the burden of manual review, we could aim to achieve a high precision instead of having high recall by choosing a higher PRL score threshold.
Conclusions In this paper, we propose a hybrid record linkage process which involves both PRL and ML approaches (Fig. 5 ). The hybrid process was developed using datasets from GEMO and GENEPSO which are two independent ongoing nation-wide epidemiological studies involving BRCA1/2 mutation carriers. PRL and ML were combined to classify the record pairs into matches and nonmatches, and the ML model was built on a training set labeled by using PRL followed by manual review. GEMO and GENEPSO are ongoing studies and their respective databases are continuously updated. About 730 new subjects are included each year in GEMO, and 590 in GENEPSO. Hence, we will apply our hybrid approach on the updated versions of the two databases on a regularly basis, so as to identify new matches and increase the statistical power of research projects involving linked participants.
Acknowledgements We thank the patients and the participants in the contributing studies. The Genetic Modifiers of Cancer Risk in BRCA1/2 Mutation Carriers (GEMO) study is a study from the  National Cancer Genetics Network «UNICANCER Genetic Group», France. We wish to pay a tribute to  Olga M. Sinilnikova , who with  Dominique Stoppa-Lyonnet  initiated and coordinated GEMO until she.
This article presents and discusses different kinds of data documents, including data sets, data studies, data papers and data journals. It provides descriptive and bibliometric data on different kinds of data documents and discusses the theoretical and philosophical problems by classifying documents according to the DIKW model (data documents, information documents, knowledge documents and wisdom documents). Data documents are, on the one hand, an established category today, even with its own data citation index (DCI). On the other hand, data documents have blurred boundaries in relation to other kinds of documents and seem sometimes to be understood from the problematic philosophical assumption that a datum can be understood as "a single, fixed truth, valid for everyone, everywhere, at all times" 1.
Introduction: data and data documents The relation between data documents and knowledge organization primarily concerns metadata: how data documents should be described, indexed, and classified. Secondarily, the concept of data document represents a kind of knowledge organization represented by different document types. As  Furner (2016, 288)  expressed, the relation between data and documents seems often to be confused in the literature: There is little consensus on the precise nature of the conceptual relationship between "data" and "document. " The default position appears to be the view that all documents are in some sense made up of data  […]. The position I wish to develop in this paper, however, is that it is not in fact the case that documents are made up of data. On the contrary, it is the other way round: datasets are made up of documents.
Furner's view is supported by data indexing practices, such as Clarivate Analytics' Data Citation Index (DCI), which were introduced in 2012. 2 The "data" indexed in DCI are of four kinds (Clarivate Analytics n. d. ): -data repositories, which consist of data studies and data sets and provide access to the data. 411 repositories were indexed 'DT=repository' by DCI on April 16, 2020. -data studies which are descriptions of studies or experiments with the associated data used in the study. Includes serial or longitudinal studies over time. 1, 221, 993  items data studies were indexed 'DT=data study' by DCI on April 16, 2020. -datasets consisting of a single or coherent set of data, or a data file, provided by the repository as part of a collection, data study, or experiment. 9,349,330 items were indexed 'DT= data set' by DCI on April 16, 2020.
We have so far presented four kinds of data documents all of which have been connected to DCI and thus to data indexing as opposed to the more traditional indexing of literatures, which has been covered by traditional bibliographic databases for a very long time, and by the Science Ci-tation Index since 1964, which today is available with other citation indexes in Clarivate Analytics' platform Web of Science (WoS). 4  WoS has two document types related to data: data paper 5 and database review. 6   1. "Data Paper: A scholarly publication describing a particular dataset or collection of datasets and usually published in the form of a peer-reviewed article 7 in a scholarly journal. The main purpose of a data paper is to provide facts about the data (metadata, such as data collection, access, features etc.
) rather than analysis and research in support of the data, as found in a conventional research article". 6621 documents in WoS were assigned the document type (DT) category "data paper" on March 21, 2020. (All 6621 were also assigned DT=Article 8 ). 2. "Database Review: a critical appraisal of a database, often reflecting a reviewer's personal opinion or recommendation. Refers to a structured collection of records or data that is stored in a computer system. " (WoS has no document type named "database", but other bibliographic databases, such as MEDLINE have. 9 ) Another kind of data document is the data handbook, which exists in many forms and sizes, and are often published under other titles such as "statistical handbook" or "statistical yearbook". 10  A prominent example of a data handbook is CRC Handbook of Chemistry and Physics with the nickname: The Rubber Bible  (Rumble 2019).
This is a one volume book updated annually (now in its 100th edition and today available in print as well as online). By contrast  Gmelin (1924 Gmelin ( -1997) )   11  is the most comprehensive handbook of inorganic chemistry ever published (more than 400 volumes), and today only published as the Gmelin Database. Such electronic databases are probably the dominant medium of present-day data handbooks. A further kind of data document is the data journal (presented in Section 4 below). This brings the number of different kinds identified up to the following nine: -data repository -data study -data set -software -data paper -database -database review -data handbook -data journal What about data themselves? Furner (2016, 288)  claimed that data could not exist without documents: In this case, "document" is the primary concept: if documents did not exist, data could not; even though documents do exist, data need not.
e. , their documents, exclusively to themselves. " Therefore, overall, the conclusions of Furner seem right: documents, not data, is the primary concept for data-and information sciences. However, in many cases, data exist only in highly dynamic databases, where it may not be possible to reconstruct the exact data at a given former time, and, in some contexts, the very meaning of "data" may be very different from one time to another. 13  Two kinds of data documents: data journal and data paper are more fully described below in Sections 4 and 5 of the present article (and kinds of emergent document types are mentioned in Section 5. 6). Database reviews will not be discussed, while 'Database' is planned to be an independent article.
Documents classified according to the datainformation-knowledge hierarchy In information science the so-called knowledge pyramid (or DIKW model) is a suggestion of three or four layers of knowledge: -Data -Information -Knowledge -Wisdom (only in certain versions of the model) It is further assumed that these layers form a pyramid (there is much data, less information, lesser knowledge, and little wisdom): It seems obvious to discuss this model because, if one of its categories, data, is represented with its own document types (data documents), this seems to indicate that each category in the knowledge pyramid corresponds to specific document types: if data are published in data documents, where, then, are the other categories published? The DIKW model has been discussed by a comprehensive literature, including  Ackoff (1989) ,  Frické (2007) ,  Rowley (2007)  and  Zins (2007).
The characteristic of data papers as documents which are expected to provide simple descriptions of facts (data) as opposed to research articles which are expected to provide insight, understanding, interpretations, hypotheses etc. are partly blurred and some data papers do more than carrying information about data, in particular when they include sections with data analysis results and discussions. So at least partly, data papers also convey knowledge, even if this is not seen as part of their core function. Our conclusion of this section is that classification of documents according to the DIKW model seems not feasible, raising difficult questions about the identity and attributes of data documents.