dataesr/scientific-paragraphs-categorization huggingface dataset:
Similarly to the existing literature, SCI-CERO exploits a transformer-based model which is fine-tuned on a set of reliable triples to identify triples that might describe erroneous facts about the Computer Science domain. Finally, existing approaches lack well-defined semantics and semantic web best practices to make sense of data and enable users to explore and reason on the generated graph. For example, the triples generated by DyGIEpp are not described by means of ontologies thus limiting the possibility to reason and make sense out of them. In comparison, SCICERO addresses the above-mentioned challenges by (i) integrating entities and relationships that come from different papers and extracted from different tools, (ii) validating the triples by means of ontology-and machine learning-based approaches, and (iii) following best practices in the design of ontologies to express the domain semantics.
3. 2). Each statement is encoded as a rdf:Statement instance and contains the following information: • the triple describing the relationship between two entities in the form of <subject, predicate, object> (using the object properties: rdf:subject, rdf:predicate, and rdf:object); • the number of articles from which the claim was extracted (cskg-ont:has-Support) • the set of IDs of the articles from which the statement was extracted (provo:wasDerivedFrom); • provenance and versioning information of the combination of techniques used to detect this specific statement (provo:wasGeneratedBy). The architecture of SCICERO is depicted in Fig. 1. It includes three main components. First, the extraction modules (Section 3. 1) apply two methods for identifying entities (CSO Classifier and DyGIEpp) and four techniques for extracting relationships between these entities (DyGIEpp, OpenIE, PoST, and DepT).
Extraction Modules This section describes the tools used to extract the entities and the relationships from natural language text. Specifically, we extract entities using two modules: 1) DyGIEpp Module  [58]  (Section 3. 1. 1, also used for identifying relationships) and 2) the CSO Classifier Module  [59]  (Section 3. 1. 2). We detect relationships between these entities according to four approaches: 1) DyGIEpp Module  [58]  (Section 3. 1. 1), 2) the Stanford Core NLP -OpenIE Module  [60]  (Section 3. 1. 3), 3) PoS Tag-based Relationships Extractor Module  [61]  (Section 3. 1. 4), and 4) Path-based Relationships Extractor Module (Section 3. 1. 5), a novel approach that we introduce in this paper.
Stanford Core NLP -OpenIE Module This module is built on top of the Stanford Core NLP suite  [60]. Ope-nIE  [62]  is an annotator that extracts domain-independent triples from a text. To start with, it builds clauses which are groups of terms that include at least a subject-noun and a verb by traversing the parsing tree of the input text. Clauses are shortened thus creating a set of short sentence fragments, that are subsequently transformed into triples. In the context of the proposed approach, the resulting triples are filtered by checking the overlapping of subjects or objects with the set E CSO ∪E DyGIEpp. Triples whose subject or object are not part of E CSO ∪ E DyGIEpp are discarded. Moreover, only triples that have a relationship whose tokens are verbs according to the associated PoS tags are used to build the KG.
For example, if nlp is found within E, then the entity nlp is replaced with the entity natural language processing in the triples of sets T DyGIEpp , T OpenIE , T P oS , and T Dep. The linkage to Wikidata is done by using a SPARQL query which performs string matching with entities and their alternative labels (i. e. , linked by the property rdfs:altLabel similarly to what was performed with the CSO). To limit potential false matches with entities of other domains, the SPARQL query is designed to find entities which are: i) instances of the entity Computer Science  24  , ii) subclasses of Computer Science, iii) instances of a subclass of Computer Science, iv) part-of or facet-of Computer Science, and v) not ambiguous within Wikidata (i. e. , they do not have "Wikimedia disambiguation page" as value of the property schema:description  25  which is present when a certain entity appears multiple times in Wikidata with different meanings).
Triple Validation Modules This section describes the two methodologies applied by SCICERO to discard triples that i) are erroneous artifacts of the information extraction process or the integration procedure, or ii) do not align with the knowledge contained in the generated KG or the domain ontology. Figure  3  shows the schema of the two modules (i. e. , the Transformer-based Validation Module and the Ontology-based Validation Module) that perform this validation. The first method exploits transformers to identify low-quality triples. The second one checks that the type of entities and the relationships align with the ontological schema.
Transformer-based Validation Module Intuitively, a triple that is extracted from multiple documents by multiple approaches has a higher probability to be correct than a triple that is extracted from one or a few documents using one or a few approaches. Section 5. 3 discusses the quantitative data that confirm this intuition. Therefore, within SCICERO, we use two scores to define a triple as reliable or uncertain, thus selecting triples which can be directly included in the resulting KG and triples which need to be validated. The first score, s 1 , is given by the number of papers from which a triple was extracted from. The second score, s 2 , is given by the number of extractor tools that were able to detect the triple. For example, if a triple t is extracted from 10 papers and the tools that were able to detect it were DyGIEpp Module and the Path-based Relationships Extractor Module, the two scores will be s 1 = 10 and s 2 = 2.
In other words, the transformer model is tuned to verify whether a triple from the set T uncertain can be part of the KG whose triples are in the set T reliable  [57]. This module uses the scibert scivocab uncased  [30]  transformer model which is pre-trained on the full texts of 1. 14M papers from Semantic Scholar  29. The transformer model is fine-tuned according to the set T reliable which includes all triples with support s 1 ≥ th s1 or s 2 ≥ th s2. The reader notices that SCICERO is aimed to be applied on large collections of research papers which will naturally include redundant statements appearing in multiple triples. Hence, it can always rely on a set of triples with high support for the training phase. For this reason our approach does not typically suffer from the cold start problem.
Ontology-based Validation Module This section describes the ontology used to represent scientific facts and how SCICERO exploits it to filter out semantically erroneous triples. The ontology uses the namespace http://scholkg. kmi. open. ac. uk/cskg/ontology# (prefix cskg-ont) and builds on top of SKOS 30  and PROVO  31. It is designed to represent the domain semantics on top of the types of entities identified by the DyGIEpp tool (i. e. , Method, Task, Material, Metric, Oth-erEntity) and the 39 representative verbs (described in Section 3. 2. 3). It describes five classes of research entities: cskg-ont:Method, cskg-ont:Metric, cskg-ont:Material, cskg-ont:Task, cskg-ont:OtherEntity. The types Other-Scientific-Term and Generic from DyGIEpp, which are associated to generic entities with no specific categorization, were both mapped to the class cskgont:OtherEntity.
At the same time, it is incorrect that a Material uses a Method, therefore, in the definition of the object property usesMethod, the class Material was not included in the domain of the property. The reader can find the ontology at http://scholkg. kmi. open. ac. uk/cskg/ontology#. This module analyzes all triples from the set T reliable ∪ T consistent and discards them if the subject or the object do not comply with the range and domain of the relation as defined in the ontology. For example, the triple <dbpedia, usesMethod, deep learning>, where dbpedia is a Material and deep learning is a Method, is discarded, whereas the triple <sentiment analysis, usesMaterial, lexical resource>, with sentiment analysis being a Task and lexical resource being a Material, is included in the KG. Finally, the triples and their relevant metadata (e.
Implementation Settings This section describes the setting parameters as well as the statistics about the SCICERO KG generation process. The settings used to generate the scientific KG are reported in Table  1. Thresholds and scores were empirically set to find a balance between the quality of the overall triples and the coverage of the KG. The Extraction Modules produced 53M triples from DyGIEpp, 34M from OpenIE Module, 58M from the PoS Tag-based Relationships Extractor, and 14M from the Path-based Relationships Extractor. Entities and Relationships were cleaned by the Entities and Relationships handling modules yielding a total of 68M candidate triples. Among them, 2. 6M had a high support according to the thresholds of s 1 and s 2 reported in Table  1  and composed the set T reliable. 500K of these highly supported triples were used in a train and test split setting to fine-tune the transformer model within the Transformerbased Validation Module.
Finally, the Ontology-based Validation Module was applied on the set T reliable ∪ T consistent and filtered out about 7M triples that were inconsistent with the ontology. The remaining 41M triples were reified to generate the KG. In Table  3 , we report some examples of i) triples that were considered valid by SCICERO and included in the KG, ii) triples that were discarded by the Transformer-based Validation Module, and iii) triples that were discarded by the Ontology-based Validation Module. The CS-KG generated by SCICERO includes 41M statements extracted from the 6. 7M input abstracts. Statements are formalized according to the domain ontology described in Section 3. 3. 2.
The Computer Science Knowledge Graph Research entities are described within the http://scholkg. kmi. open. ac. uk/cskg/resource/ namespace (prefix cskg). Each statement is associated with the research papers it was generated from (using the property provo:wasDerivedFrom), the number of source papers (cskg-ont:hasSupport), and the modules that were involved in its extraction (provo:wasGeneratedBy). An example of statement generated by SCICERO is depicted in Fig. 4. The triple <face recognition, providesTask, authentication> is reified and linked to research papers where it was extracted from (e. g. , 2940931619, 2966617151, and so on), the extraction modules that generated it (i. e. , Ope-nIE and PoSTagger ), and its support (i. e. , 5).
Extractors Performance Table  4  reports the performances of the 15 approaches in terms of precision, recall, and F-measure. The four basic methods for extracting triples (DyGIEpp, OpenIE, PoST, and DepT) obtained a precision score in the range 0. 57 -0. 68. In terms of recall, the extractors obtained values lower or equal than 0. 4, which indicates that a single tool or module is not sufficient to extract an amount of information that can be used to adequately describe the Computer Science domain. The F-measure of the extractors ranges from 0. 24 to 0. 49. However, when the extractors are combined, the recall increases without paying too much in terms of precision. The version which integrates all of them yields an F-measure score of 0. 69. The F-measure increases with the number of tools that are combined: the combination of two tools leads to a maximum F-measure of 0.
75) and F-measure (0. 77). Table  6  reports the performance of the full version of SCICERO on the three sub-fields: Natural Language Processing, Information Retrieval, and Machine Learning. The performance are similar across the different research areas, suggesting that SCICERO is quite consistent in the Computer Science domain. In conclusion, the experiments yielded two main insights. First, tools only working at level of single documents may be not sufficient to produce a comprehensive representation of the domain. Therefore, we may need to develop solutions that are able to integrate and validate information from multiple tools and multiple documents. Second, the approaches for filtering triples based on their consistency with highly supported ones (the Transformerbased Validation Module) and compliance with the domain ontology (the Ontology-based Validation Module) can drastically improve the quality of the produced triples.
Conclusion In this paper we presented SCICERO, a novel approach for automatically generating scientific KGs from the titles and abstracts of research papers. SCICERO relies on a modular architecture that combines five extraction modules (the DyGIEpp Module, the CSO Classifier Module, the Stanford Core NLP -OpenIE Module, the PoS Tag-based Relationships Extractor Module, and the Path-based Relationships Extractor Module), three modules for integrating entities and relationships, and two validation modules. Within SCICERO, entities are extracted from the input text and linked by exploiting both pre-defined and verb-based relations. They are then merged by mapping them to external resources such as CSO, Wikidata, and DBpedia. Moreover, deep learning transformers are employed for two different purposes: i) merging entities that are not linked to external resources by using the framework SentenceTransformers and encoding the research entities with the paraphrase-distilroberta-base-v2 transformer model; ii) predict whether a certain triple is correct or not using the scibert scivocab uncased transformer model pretrained on 1.
14M papers from Semantic Scholar. SCI-CERO also validates the resulting triples against an ontology that describes 179 possible relations between scientific entities. SCICERO has been applied to a collection of about 6. 7M research paper titles and abstracts from Computer Science ranging from 2010 to 2021. The evaluation that has been carried out concerns the ability of SCICERO to extract and recognize correct triples. It has been performed on three different domains: Machine Learning, Natural Language Processing, and Information Retrieval. The experiments shows that SCICERO yields a precision of 0. 75, a recall of 0. 79, and a F-measure of 0. 77 on a gold standard of 3, 600 triples annotated by three humans experts. Last but not least, we publicly release the source code of SCICERO to make our results reproducible and to support the scientific community, as it can be easily adapted to other domains.
33 The relevant topics on CSO are Machine Learning (https://cso. kmi. open. ac. uk/ topics/machine_learning), Natural Language Processing ( https://cso. kmi. open. ac. uk/topics/natural_language_processing), and Information Retrieval (https:// cso. kmi. open. ac. uk/topics/information_retrieval). Set 5 (Discarded by the Ontology-based Validation), which includes triples have been discarded by the Ontology-based Validation Module during the creation of CS-KG. • 34 CSO Topics -https://github. com/danilo-dessi/SKG-pipeline/blob/main/ resources/cso_topcis. txt.
Natural minerals and synthetic glass were used for standardization, and the ZAF matrix procedure was applied for data reduction. X-ray maps were processed using the XmapTools software  (Lanari et al. , 2014 (Lanari et al. , , 2018)). Zr and other trace elements (Nb, Fe, Cr) in rutile were measured in thin section on four samples , with the JEOL JXA-8230 electron microprobe at ISTerre Grenoble. The instrument was operated with an accelerating voltage of 20 kV, a beam current of 600 nA, and a total counting time of 600 s, with spot size set to 1-3 μm. Standardization was done on synthetic and natural standards of TiO 2 ZrO 2 , Cr 2 O 3 , Fe 2 O 3 and Nb. The detection limit is around 27 ppm for Zr and below 35 ppm for other trace elements. Accuracy check was done on standard rutile R632  (Axelsson et al.
, 2018) , which returned Zr concentrations between 2 and 5% of the reference value. This uncertainty is consistent with counting error statistics returned by the JEOL microprobe software for rutile analysis, which gives a 2σ uncertainty between 40 and 50 ppm. In addition, rutile grains separated from sample JB-19-56 were analyzed for trace elements by LA-ICPMS at the University of Bern, and were subsequently dated (see below). Zircon and rutile were separated using conventional rock crushing and heavy liquids. Crystals were mounted in epoxy resin and polished to expose the grain center for analysis. Charge contrast (CC) images of zircon grains were obtained with a ZEISS EV050 scanning electron microscope at the Institute of Geological Sciences, University of Bern, at low vacuum conditions (18 Pa), 12 kV, a beam current of 100 mA and a working distance of 9.
, 2011). Accuracy was monitored using the reference material GSD-1G  (Jochum et al. , 2005)  and zircon 91,500  (Wiedenbeck et al. , 2004) , and was assessed to be better than 10% for all elements. Data reduction was performed using the software Iolite  (Paton et al. , 2011). U-Pb dating of rutile was performed with the same LA-ICP-MS system at the University of Bern with conditions of 5 Hz and 3. 5 J/cm 2. The count rate ratio for Th/U was 98-97 and the ThO/Th < 0. 2. Measurements consisted of 30s background and 30s signal acquisition sweeping through the masses 206 Pb, 207 Pb, 208 Pb, 232 Th and 238 U. Spot size was μm with a pre-cleaning with a 60 μm beam. U-Pb ratios were standardized to rutile R10 (1091. 6 ± 3. 5 Ma,  Luvizotto et al. , 2009) , whereas secondary reference material was rutile R632 (496 ± 2 Ma,  Axelsson et al.
, 2018) , which returned an average 206 Pb/ 238 U of 496 ± 11 Ma. Data reduction was performed using the software Iolite  (Paton et al. , 2011)  running the DRS VizualAge_UcomPbine  (Chew et al. , 2014) , although no common Pb correction was applied to the data of either standards or unknown. The single spot 2σ standard error external on the primary standard was 7. 5% and propagated in full to the unknowns. U-Pb geochronology of zircon was carried out with the SwissSIMS Cameca IMS 1280-HR at Université de Lausanne, which is equipped with an high-brightness Hyperion H201 RF plasma oxygen ion source. Basic instrument set up parameters were 6-8 nA, 20 μm O 2 -primary beam, mass resolution M/ΔM ~ 5000, energy window = 40 eV. Data were acquired in mono-collection, peak-hopping mode. U-Pb-Th data were standardized to TEM2 zircon (417 Ma,  Black et al.
, 2003)  and accuracy was checked with zircon standard Plešovice (337. 17 ± 0. 37 Ma,  Sláma et al. , 2008) , which were cast in the same epoxy mount as the unknowns and returned an age at 337. 7 ± 3. 5 Ma. The uncertainty on standard 206 Pb/ 238 U -UO 2 /U calibration was 1. 0 and was propagated to the data. Common Pb correction was based on the measured 204 Pb signal (when significant relative to background) assuming the present-day model terrestrial Pb composition of  Stacey and Kramers (1975). Radiogenic ratios and single ages were calculated using the CIPS program compiled by Martin Whitehouse. Age calculations use the decay constant recommendations of  Steiger and Jäger (1977). The IsoplotR package (Vermeesch, 2018) was employed to plot the diagrams and calculate concordia and weighted average ages.
Thermobarometry Zr-in-rutile temperatures for five samples were calculated using the calibration of  Kohn (2020) , constrained with combined experimental and natural datasets. Two samples of garnet-bearing mafic granulites  were selected for comprehensive P-T estimations using a combination of forward phase equilibrium modeling with PerpleX 6. 9. 0  (Connolly, 2009)  and inverse thermobarometry on plagioclase-amphibole pairs using the edenite-richterite exchange thermometer of  Holland and Blundy (1994). Phase equilibrium models were computed in the compositional space MnNCKFMASHTO, using the TC-DS622 thermodynamic database of Thermocalc  (Holland and Powell, 2011) , with a set of solution models from  Fuhrman and Lindsley (1988) ,  Green et al. (2007 Green et al. ( , 2016)) ,  Holland and Powell (1998, 2011)  and  White et al.
Evolution at the regional scale Comparison of the zircon rims ages from this study with data from the surrounding metamorphic rocks provides insight into the metamorphic evolution of the Oisans-Pelvoux massif. A summary of geochronological data from  Fréville (2016)  and this work is presented in Fig. 13. The data were split by domains between the widely migmatized inner Oisans-Pelvoux zone and the lower grade cortical zone in the western part of the massif (Fig. 1 ). The distribution of zircon and monazite U-Pb dates in the high grade inner zone marks three distinct peaks corresponding to the Visean (ca. 340-325 Ma), the Late Carboniferous -Early Permian (ca. 310-295 Ma) and the mid-Permian (ca. 285-270 Ma). Only the Visean peak is recorded in the lower grade cortical zone, which does not display any trace of younger events.
Acknowledgements This work was supported by the  BRGM (French Geological Survey)  through the " Référentiel Géologique de la France" program  (RGF). J-B Jacob  was funded by a doctoral grant from the  ENS de Lyon. We wish to thank all the technical staff at  ISTerre in Grenoble, UNIL in Lausanne and at the University of Bern , who greatly helped for the preparation of samples and during the multiple analytical sessions. This manuscript also benefited from interesting and constructive reviews by  Julien Berger  and an anonymous reviewer, and from editorial handling and advice from  Nadia Malaspina. Most  of the field investigations were carried out in the central zone of the Ecrins National park, in which collecting rock samples is usually strictly forbidden. We therefore wish to thank the administration staff of the  Ecrins National park , who gave us authorization to collect the samples and followed with great interest the advancement of this work.
Additionally, radiotherapy is related to complications such as avascular necrosis, osteoarthritis and even radiation-induced malignancies, an important issue for a locally aggressive yet benign disease  [14,16e18]. More recently, novel drugs targeting CSF1R are being developed and the safety and efficacy are evaluated for patients with relapsing or inoperable D-TGCT  [19e24]. CSF1R-inhibiting drugs, consisting of CSF1R antibodies and tyrosine kinase inhibitors (TKI), have shown substantial clinical activity  [25]. A phase II clinical trial evaluating the effect of nilotinib in patients with locally advanced D-TGCT, was started in 2010 (ClinicalTrials. gov: NCT01261429)  [23]. Nilotinib is a phenylaminopyrimidine, inhibiting several tyrosine kinases, including ABL, KIT, platelet-derived growth factor receptors and CSF1R.
Methods This study describes the long-term effect of nilotinib in patients with locally advanced D-TGCT. This is a longterm report of a multi-centre, open-label, single-arm, phase II trial, registered with ClinicalTrials. gov, number NCT01261429  [23, 27]. Patients were enrolled at 11 cancer centres or hospitals in four countries (France, the Netherlands, Italy and Australia) between December 2010 and September 2012. A summary of this study and a comprehensive overview of the in-and exclusion criteria can be found in the appendix (supplementary Table  1 ). The study protocol of the phase II trial was approved by the local ethics committee at each site and is available online  [23, 27]. This study was performed in accordance with the Good Clinical Practice guidelines and the Declaration of Helsinki.
Statistical analysis Continuous data were described using means and standard deviations (SD) or medians and interquartile ranges (IQR). KaplaneMeier method was used to analyse PFS. The statistical analyses were performed in IBM Statistical Package for Social Statistics (SPSS) 25 (Chicago, IL, USA) was used for analysis. A swimmer plot was created using RStudio version 4. 1. 0 (RStudio Team, Boston, United States).