dataesr/scientific-paragraphs-categorization huggingface dataset:
Machine learning employs a variety of statistical, probabilistic, fuzzy and optimization techniques that allow computers to "learn" from examples and to detect hard-to-discern patterns from large, noisy or complex datasets. This capability is well-suited to medical applications, and machine learning techniques have been frequently used in cancer diagnosis and prognosis. In general, machine learning techniques usually work in two phases: training and testing. Some parameters, with regards to the underlying machine learning technique, must be tuned in the training phase in order to best "learn" from the dataset. On the other hand, belief merging operators integrate inconsistent information, which may come from different sources, into a unique consistent belief set (base). Implementations of merging operators do not require tuning any parameters apart from the number of sources and the number of topics to be merged.
Both Tongue and Salivary Gland cancers are higher among men as compared to women. However, mouth cancer is higher among women (56. 5%) than men (43. 5%) with women of Indian ethnicity forming 71% of the total incidence among Malaysian women. As in other parts of the world, the incidence of oral cancer in Malaysia increases with age, with more oral squamous cell carcinoma occurring in individuals over 40 years of age  [8]. A coordinated and standardized data collection of oral cancer cases from multiple centres involving eight hospitals in Malaysia was initiated by the Malaysian Oral Cancer Research and Coordinating Centre (OCRCC), University of Malaya, Malaysia  [9]. The data collected includes parameters on sociodemographic, clinical, pathological, quality of life measures, details of treatment methods, vital status and dietary intake.
The establishment of this database, named as, the Malaysian Oral Cancer Database and Tumour Bank System, aims in encouraging and supporting researches on oral cancer. Machine learning, a sub-branch of artificial intelligence, has been in existence for more than 50 years with a wide variety of learning tasks and successful applications  [10]. Machine learning techniques have been applied to medicine for diagnosis, prognosis, bio-medical analysis, image analysis and drug development  [11, 12]. Machine learning techniques invariably involve parameter tuning with regards to the underlying technique, such as, the number of neurons, layers or epochs in a neural network technique; membership function selection in fuzzy logic; population size, selection strategy, mutation rate, crossover rate in genetic algorithms as well as in the hybrid techniques that use fuzzy logic or neural network or both.
Thus, a good strategy for testing the merging operator is to evaluate their results when they are used to implement algorithms for some real world problems, such as, the diagnosis of a particular medical condition using real world medical datasets. In this paper, we discuss an application of belief merging on a set of data for which a prior research involving oral cancer diagnosis has been carried out using machine learning techniques  [22]. We implement a new method for such diagnosis based on the PS-Merge operator implementation. Our aim is to give a real world application to belief merging, for which the technique is sadly lacking. Any reference made to machine learning is merely to put the research in perspective and not to act as a comparison on the classification superiority of either technique.
Previous work Commonly used machine learning techniques are artificial neural networks, fuzzy logic, support vector machines and genetic algorithms. In the works of medical diagnosis, machine learning has been used in the diagnosis of breast cancer, sexually transmitted diseases, sepsis, oral cancer, leukaemia, etc. [23, 24, 22, 25]. In medical prognosis, machine learning has been used in naso-pharyngeal carcinoma, colo-rectal cancer, cardiac diseases, gestational trophoblastic tumours, septicaemia, etc. [12, 26, 27]. A retrospective cohort study was conducted using a sample of 171 data obtained from the Malaysian Oral Cancer Database and Tumour Bank System (MOCDTBS) courtesy of the OCRCC. The full dataset were split randomly into a modelling dataset (67% of the total) and testing dataset (the remaining 33%).
Materials Table 2 lists all the binary variables used as input for the prediction models together with a brief description of the variables. In order to investigate the possibility of applying belief merging in the diagnosis of oral cancer, we use the data set discussed in the last section. The data was earlier obtained from the Oral Cancer Database and Tumour Bank System (MOCDTBS) provided by the Oral Cancer Research and Coordinating Center (OCRCC), University of Malaya, Malaysia. Demographic profiles (age, gender) and oral cancer risk habits (cigarette smoking, alcohol drinking, tobacco and betel-quid chewing) of the cancer patients and control group were used as input variables as listed in Table  2. The dichotomous output or outcome refers to the health state of either "cancer" (1) or "healthy" (0).
The method considers every case in the medical database as a source of information; thus, every case (belief base) K is of the form l 1 ∧ • • • ∧ l 8 → l 9 with the first variables or factors represented as follows: l 1 represents age, l 2 represents gender, l 3 represents ethnicity, l 4 represents cigarette smoking, l 5 represents alcohol drinking, l 6 represents betel quid chewing, l 7 represents GSTM1 and l 8 represents GSTT1, while, the last variable, l 9 , represents the current diagnosis. For example, suppose that the medical database contains case 13 which is a diagnosis of oral cancer for a smoker, non-aboriginal male over forty years of age, who does not drink and does not chew betel quid and is positive for chromosome GSTM1 but negative for chromosome GSTT1. Then, we have K 13 = ¬ p 1 ∧ p 2 ∧ ¬ p 3 ∧ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ p 7 ∧ ¬ p 8 → p 9 ; given that the input and output data to the proposed algorithm is in a binary format, we use an equivalent notation with 1's and 0's, thus writing K 13 = (0, 1, 0, 1, 0, 0, 1, 0, 1).
Based on the following equivalence: p → q ≡ ¬ p ∨ q, a DNF or CNF equivalent to K can be easily found negating every factor and connecting all the elements with disjunctions, ¬l 1 ∨ • • • ∨ ¬ l 8 ∨ l 9. For the previous example, we have K 13 = p 1 ∨ ¬ p 2 ∨ p 3 ∨ ¬ p 4 ∨ p 5 ∨ p 6 ∨ ¬ p 7 ∨ p 8 ∨ p 9. After the transformation of the 171 cases into their normal forms we obtain the medical dataset in normal form E, then we apply Algorithm 1 to E, where the profile holds two third (  2 3 ) of the 171 belief bases in normal forms. Then the algorithm obtains the diagnosis D for one third (  1 3 ) of the cases. The diagnosis has the format: (target, outcome) where target is the current diagnosis stored in the medical dataset and outcome is the diagnosis obtained by the algorithm.
42. We call this method the Proportional p method; the results are shown in Table  7 , again rounding to the nearest whole number as in Table  6. The accuracy scores are fairly similar in Tables 6, 8 and 7. The p = 1 method (Table  8 ) has very high values of sensitivity (true positive rate) but the p = 0. 5 and Proportional p methods have better values of specificity (true negative rate). On this dataset, the slightly more sophisticated Proportional p method does not do better than the simpler p = 0. 5 method.
With this work being an initial exploration of using belief merging in a medical diagnosis scenario, we have not used all permutations of the input variables nor reduced the variables based on the variable selection methods carried out by  [22]. We have merely repeated a subset of the set of all permutations of the input factors that produced the best results in the works by  [29] [30] [31] 22]. Notice that only with 8 factors (see Table  8 ), we obtained a value of FN different from zero, i. e. , FN = 1. A shallow analysis would lead to the conclusion that the lack of preprocessing introduced noise to the merge (training) phase. When the input of 8 factors was used, the medical dataset included two instances of the form The two instances holding non-cancer diagnosis were in the training base and the instance holding cancer diagnosis was in the testing base; the system thus learned from the training data that when the input factors were p 1 ∧ ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 , the output should be 0 (because of the two instances in the training base), and so, in the testing phase, the system responded with 0 for these factors.
However, the implementation may be limited by the available resources (hardware and software). The belief merging method for oral cancer diagnosis presented was easily implemented with a response time of 65 s, using a PC operating a Windows 7 and Matlab 9. In future, we would like to try to create an open source implementation that could be used by researchers for "training" and "testing" their data sources. In order to attain this goal, we would need to determine the maximum size of the input binary data set allowed. That is, we would need to analyze how many factors and how many cases would be allowed using similar resources to mentioned earlier. We would also like to adapt this implementation of oral cancer, in order to carry out the diagnosis for other types of cancer, such as breast cancer, depending on the availability of datasets.
¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 → ¬ p 9 and one instance of the form p 1 ∧ ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 → p 9 , i. e. , the dataset held a contradiction when the factors were of the form p 1 ∧ ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 , having two instances with the diagnosis of non-cancer and one instance with a cancer diagnosis.
Acknowledgements We would like to thank  Weiru Liu  and  Queen's University Belfast  for hosting this work. We would also like to extend our appreciation to  Rosma Mohd Dom  for the extensive use of her previous work in this research. This research was supported by the  University of Malaya Sabbatical Leave  in the case of the first author and the  Programme Postdoctoral and Sabbatical Stays Abroad for the Consolidation of Research Groups -CONACyT  and  UJAT  in the case of the second author.
p 1 ∧ 1 ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 → ¬ p 9 and one instance of the form p 1 ∧ ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 → p 9 , i. e. , the dataset held a contradiction when the factors were of the form p 1 ∧ ¬ p 2 ∧ ¬ p 3 ∧ ¬ p 4 ∧ ¬ p 5 ∧ ¬ p 6 ∧ ¬ p 7 ∧ ¬ p 8 , having two instances with the diagnosis of non-cancer and one instance with a cancer diagnosis.
Statistic data is an important sub-category of open data; it is interesting for many applications, including but not limited to data journalism, as such data is typically of high quality, and reflects (under an aggregated form) important aspects of a society's life such as births, immigration, economic output etc. However, such open data is often not published as Linked Open Data (LOD) limiting its usability. We provide a conceptual model for the open data comprised in statistic files published by INSEE, the leading French economic and societal statistics institute. Then, we describe a novel method for extracting RDF LOD populating an instance of this conceptual model. Our method was used to produce RDF data out of 20k+ Excel spreadsheets, and our validation indicates a 91% rate of successful extraction.
In the best-case scenario, reference data sources are available to fact-checkers in a format and organized according to a model that they understand well. For instance, in [WAL + 14], unemployment data is assumed available in a temporal database relation. This assumption is very reasonable for statisticians and other scientists working within the statistics institutes. However, unfortunately, most fact-checking actors are outside these institutes, and only have access to the data through publication by the institute. This raises the question of the format and organization of the data. While the W3C's best practices argue for the usage of RDF in publishing open data  [The14] , in practice open data is often published in other formats, such as Excel, HTML or PDF; in the latter formats, statistic data is typically shown as tables or charts.
In the particular case of the INSEE 2 , the leading French social and economic statistics institute, while searching for some hot topics in the current French political debate (youth unemployment, immigrant population etc. ) we found many very useful datasets, available as Excel and/or HTML and sometimes accompanied by a PDF report explaining and commenting the data, but we did not find the data in a structured, machine-readable format. Motivated by this limitation, we propose an approach for extracting Linked Open Data from Below, we describe the data sources we work with and the model for the extracted data (Section 2) and our extraction method (Section 3), outline the format of the extracted RDF data (Section 4), and present an evaluation of our extraction process (Section 5), before discussing related work and concluding.
Reference statistic data In Section 2. 1, we present the input INSEE statistics, while Section 2. 2 describes our conceptual model for this data. could be quite easily adapted to HTML tables in the future. Also, while we targeted INSEE in this work, we found similar-structure files published in other Open Data government servers in the UK 3  and the US 4. Thus, we believe our tool can be used to handle more than the data sources we considered so far.
In a majority of cases (about two thirds in our estimation), each data sheet D comprises one statistics table, which is typically a two-dimensional aggregate table. Data is aggregated since statistics institutes such as IN-SEE are concerned with building such global measures of society or economy phenomena, and the aggregate tends to be bidimensional since they are meant to be laid out in a format easy for humans to read. For instance, Figure  1  illustrates a data sheet for the birth statistic dataset mentioned above. In this example, the two dimensions are the mother's age interval, e. g. , "16-20", "21-25", "26-30" etc. and her geographic area, specified as a department, e. g. , "Essonne", "Val-de-Marne" etc. For a given age interval and region, the table includes the number of women which were first-time mothers in France in 2015, in that age interval and department.
Extraction We developed a tool for extracting data from spreadsheet files so as to populate an instance of our conceptual data model; it uses the Python library xlrd 7  for accessing spreadsheet content. Below, we present the extraction procedure. Unlike related work (see Section 6), it is not based on a sequence classification with a supervised learning approach, but on a more global rule-based system aiming at framing the data cell range and extracting the headers around these data cell. For each sheet, the first task is to identify the data cells. In our corpus, these cells contain numeric information, that is: integers or real numbers, but also interval specifications such as "20+" (standing for "at least 20"), special null tokens such as "n. s. " for "not specified". Our approach is to identify the rectangular area containing the main table (of data cells), then build data cells out of the (row, column) locations found in this area.
Linked Data vocabulary To represent the extracted data in RDF, we created an RDF class for each entity, and assigned an URI to each entity instance; the properties and their typing follow directly from the entity attributes and relationships shown in Figure  2. Some sample triples resulting from the extraction are shown in Figure  3. We use the namespace inseeXtr. excel: for classes and properties we introduce in our extraction from INSEE Excel tables, and the namespace rdf: to denote the standard namespace associated to the W3C's type property. The snippet in Figure  3  corresponds to one Excel file, one sheet in the file, a row header cell, a column header cell, a data cell associated to them, and the location of the data cell.
Evaluation From 20,743 Excel files collected (see 2. 1), we selected at random 100 unseen files to evaluate the reliability of the extraction process; these files contained a total of 2432 tables. To avoid very similar files in our evaluation, e. g. , tables showing the same metric for distinct years, we required the first three letters in the name of each newly selected file, to be different from the first three letters in the names of all the files previously selected to be included in the test batch. We did the evaluation manually by looking at Excel files and identifying their header cells, data cells and header's hierarchy; then comparing them with the results from our extractor. We only assign the label "extracted correctly" for one table when there is an exact match of all cells and header's hierarchy.
9% Figure  3 : Sample extracted RDF triples. We have inspected the tables which were not handled correctly, and we saw that most of them had sheets with several tables (not just one). Our system is currently not able to correctly recognize these; we plan to address this limitation in the near future. Overall, the extraction produced 2. 68 • 10 6 row header cells, 122 • 10 6 column header cells (table height is often higher than the table width), and 2. 24•10 9 data cells (three orders of magnitude more data cells than header cells). It took 5 hours to run our extraction algorithm.
Related work Closest to our work, Chen et al  [CC13]  built a system that could automatically extract relational data from spreadsheets. Their system used conditional random field (CRF)  [LMP01]  to identify data frames: such a frame is a value region which contains numeric cells, top attributes and left attributes corresponding to header cells and their hierarchies. From a collection of 410,554 Excel files collected from the Web, they selected randomly 100 files, labeled 27,531 non-empty rows (as title, header, data or footnote) manually by human experts, then used this dataset for training and evaluation. They obtained significant better precision and recall metrics when taking into account both textual and layout features for CRF. SVM was applied on header cells to learn about their hierachies.
The model achieved a precision of 0. 921 for top headers, and 0. 852 for left headers. However, these numbers should not be compared with our results, because the datasets are different (tables encountered on the Web vs. government statistics), as well as the evaluation metrics (cell-by-cell assessment in  [CC13]  vs. binary assessment over the entire sheet extraction in our case). Overall, it can be said that their method is more involved whereas our method has the advantage of not requiring manual labeling at all. As we continue in our work (see below), we may experiment with learning-based approaches also. Apart from spreadsheet data extraction, the high number of public available spreadsheets on the Web also attracted interest for other data management tasks. For example, Ahsan, Neamt ¸u et al.
[ANR16]  built an extractor to automatically identify in spreadsheets entities like location and time. When one considers HTML and not Excel data, data extraction from tables encoded as HTML lists has been addressed e. g. , in lists  [EMH11]. Structured fact extraction from text Web pages is an extremely hot topic, and has lead to the construction of high-quality knowledge bases such as Yago [MBS15] and Google's Knowledge Graph [DGH + 14] built out of Freebase and enriched through Web extraction. Our work has a related but different focus as we focus on high-value, highconfidence, fine-granularity, somehow-structured data such as government-issue statistics; this calls different technical methods. In prior work [GKK + 13], we devised a tool for exploiting background information (in particular, we used DBPedia RDF data) to add context to a claim to be fact-checked; in this work, we tackle the complementary problem of producing high-value, high-confidence Linked Open Data out of databases of statistics.
Conclusion and perspectives We have described an ongoing effort to extract Linked Open Data from data tables produced by the INSEE statistics institute, and shared by them under the form of Excel tables. We do this as part of our work in the ContentCheck R&D project, which aims at investigating content management techniques (from databases, knowledge management, natural language processing, information extraction and data mining) to provide tools toward automating and supporting fact-checker journalists' efforts. The goal is to use the RDF data thus extracted as reference information, when trying to determine the degree of thruthfulness of a claim. We foresee several directions for continuing this work: Improving the extractor As previously stated, we currently do not correctly detect several data tables in a single sheet; we believe this can be achieved with some weighted extraction rules (the current extraction method does not use weights).
INSEE publishes a variety of datasets in many different formats. In particular, we performed a crawl of the INSEE web site under the categories Data/Key figures, Data/Detailed figures, Data/Databases, and Publications/Wideaudience publications, which has lead to about 11,000 web pages including 20,743 Excel files (with total size of 36GB) and 20,116 HTML tables. In this work, we focused on the set of Excel files; we believe our techniques 2 INSEE: French National Institute of Statistics and Economic Studies, http://www. insee. fr.
rdf:type <http://inseeXtr. excel:Dataset>; inseeXtr:source "http://insee. fr/. /Age Mère. xls"; inseeXtr:name "Age Mère. xls"; inseeXtr:description "The data reflects children. ". <http://inseeXtr. excel:Sheet0> rdf:type <http://inseeXtr. excel:Sheet>; inseeXtr:title "sheet title"; inseeXtr:belongsTo <http://inseeXtr. excel:File0>. <http://inseeXtr. excel:HeaderCellY0> inseeXtr:value "x1"; rdf:type <http://inseeXtr. excel:HeaderCellY>; inseeXtr:YHierarchy <http://inseeXtr. excel:Sheet0>. <http://inseeXtr. excel:HeaderCellX2> inseeXtr:value "a1"; rdf:type <http://inseeXtr. excel:HeaderCellX>; inseeXtr:aggregatedBy <http://inseeXtr. excel:HeaderCellX3>; inseeXtr:XHierarchy <http://inseeXtr. excel:Sheet0>. <http://inseeXtr. excel:DataCell0> inseeXtr:value "1"; rdf:type <http://inseeXtr. excel:DataCell>; inseeXtr:location <http://inseeXtr.
Variable Meaning Sex 1 Male 2 Female AGEPYR10 Age group 00 Less than 3 years old 03 3 to 5 years old 06 6 to 10 years old 11 11 to 17 years old 18 18 to 24 years old 25 25 to 39 years old. 80 80 years old 2. 2 Conceptual data model From the above description of the INSEE statistic datasets, we extract a conceptual data model shown in Figure2. Entities are shown as boxes while attributes apper in oval shapes. We simplified the notation to depict relationships as directed arrows, labeled with the relationship name. Data cells and header cells each have an attribute indicating their value (content); by definition, any data or header cell has a non-empty content. Moreover, a data cell has one associated location, i. e. , (row number, column number), while a header cell may occupy one or several (adjacent) locations.
ACKNOWLEDGMENTS We thank  Vitaly Kresin ,  Klaus Mølmer ,  Janne Ruostekoski ,  Markus Greiner ,  Zoran Hadzibabic ,  Wilhelm Zwerger  for fruitful discussions. This work is supported by  DIM NanoK ,  ERC  (Synergy UQUAM). L. C. acknowledges the support from  DGA. This project has re-ceived funding from the  European Union 's  Horizon 2020 research and innovation programme  under the  Marie Sk lodowska-Curie grant  agreement  N • 703926.