dataesr/scientific-paragraphs-categorization huggingface dataset:
This confidence is used to perform an automatic selection process and reduce the number of rules for each class; we only retain rules that have the best confidence among redundant rules (two rules are redundant if they apply to the same candidates in their class). Training a classification model for candidate verification. Once rules are generated for a candidate class, we still need to decide whether a candidate is correct or incorrect. We thus create a description matrix (Table  1 ) for each class, where candidates are rows and rules are columns. A candidate that satisfies a rule then gets a value 1 and otherwise it gets a value 0. We use this description matrix to train a Support Vector Machine (SVM) model with a Radial basis function kernel  [22] [23] [24] [25]. We train the SVM model on candidates from a labelled dataset described in the Evaluation protocol for information extraction section.
Evaluation protocol for information extraction We evaluated the accuracy, precision, recall and F-score of the IE method to correctly classify candidates for diseases, dates, locations, hosts and number of cases in a dataset of labelled relevant news. Dataset. Two IE stages require labelled data: i) automatic rule discovery for each candidate class, and ii) training of an SVM model for candidate verification. We therefore created a labelled dataset for each candidate type (diseases, locations, dates, hosts and number of cases), which we then used for both training and evaluation (S1 Dataset). The dataset consisted of English language news reports collected from the Google News aggregator that mentioned outbreaks notified to OIE from January 2014 to December 2015. We only selected news reports that were published before an OIE notification date.
PADI-web is thus trained to find relevant information early, before official notification. Among the 532 news pieces collected, 352 were evaluated and classified as relevant by a veterinary epidemiologist (EA), i. e. disease-related news. These 352 news reports constituted the training and evaluation dataset. In each of these 352 news reports, the candidates were automatically identified using the method described in the Candidate identification step. A veterinary epidemiologist (EA) and a computer scientist (JR) subsequently labelled these candidates as correct or incorrect regarding their relevance to an outbreak. Metrics. Accuracy (Acc) was the proportion of correctly classified candidates by PADIweb over the total number of candidates. Precision (Pr) was the proportion of correct candidates over the total number of candidates classified as correct by PADI-web.
PADI-web evaluation protocol We assessed PADI-web regarding the relevance, sensitivity and timeliness in detecting ASF, FMD, BTV and AI signals outside France from 1 January to 30 June 2016. Using the same diseases, evaluation period, geographical coverage and language, we compared PADI-web to two major biosurveillance systems operational today, i. e. ProMED and HealthMap. All signals detected in France were excluded from the analysis since the goal is to monitor disease emergence outside France. Dataset. PADI-web is an automatic biosurveillance system that has been operational since January 2016. PADI-web currently collects English language news using the Google News aggregator. The system design and its detailed description are given in the Methods section. We downloaded the freely available PADI-web dataset (S2 Dataset) from the dedicated webpage  [29].
Each row in the PADI-web dataset corresponds to a location candidate extracted from a news report and classified as "correct" (see Information extraction section). Each location candidate is associated with all other correct candidates (diseases, dates, hosts and numbers of cases) extracted from the same news report, and represents a signal (see Definitions section). First, we cleaned this set of signals by removing duplicates and locations having a feature code in GeoNames starting with "PCL" (i. e. political entities, that usually correspond to countries). Such locations were removed because their geographical information scale was considered too broad to reveal any useful knowledge. Second, for each studied disease, we selected signals containing: i) at least one reference to the disease, and ii) one or several hosts specific to the disease.
ProMED is a human moderated biosurveillance system that has been operating since 1994. It is the most extensive open-access biosurveillance system and is available in five languages  [30, 31]. ProMED's main source for signals of animal diseases is the OIE, followed by sanitary reports sent from subscribers worldwide. ProMED moderators verify the signals before sharing them with its network of subscribers. Any signals that cannot be verified are tagged with a 'request for information (RFI)'  [31]. Major signals of international concern are shared in English through ProMED-mail, ProMED's main network. Specific signals of regional concern are shared with ProMED's regional networks, e. g. English language posts in ProMED-MBDS (Mekong Basin region of Southeast Asia), ProMED-EAFR (Anglophone Africa), ProMED-MENA (Middle East) and ProMED-SoAs (South Asia).
HealthMap is a semi-automatic biosurveillance system that was launched in 2006  [13]. The system automatically collects, processes and displays signals from online sources available in seven languages. HealthMap sources for signals of animal diseases include news aggregators (e. g. Google News, Baidu, Soso, VeriSign), the OIE, as well as national (animal) health authorities and other biosurveillance systems (currently ProMED). Before publishing, HealthMap moderators rate signals on a 1 to 5 scale according to relevance. Signals of minor international concern are rated as 1, while signals of major international concern are rated as 5  [32]. For accurate comparison with PADI-web, we only included HealthMap data obtained from its news aggregators, hereafter referred to as HealthMap (Agg. ). Data from ProMED and HealthMap (Agg.
) were freely downloaded from the HealthMap webpage  [33]. For each studied disease, one veterinary epidemiologist (EA) filtered the signals reported outside France from January to June 2016. The epidemiologist evaluated each signal by reading the news report content, the location of the event, date and, where relevant, the affected hosts. One news report may consist of one or several signals. A ProMED report sometimes summarizes information on several outbreaks within the same geographical zone or outbreaks subject to the same immediate notification to OIE. Each summarized ProMED report was considered as one signal. One veterinary epidemiologist (EA) cross-checked the ProMED signals in the HealthMap database with reports published on the ProMED webpage  [34] , and supplemented the dataset if signals were missing.
We assessed ProMED using signals from all networks publishing English language reports, i. e. ProMED-mail, ProMED-MBDS, ProME-D-EAFR, ProMED-MENA and ProMED-SoAs. Confirmed outbreaks included all data collected from the Emergency prevention system (Empres-i) of the UN Food and Agriculture Organization (FAO) with the same geographical coverage and diseases as included in data collected from PADI-web, ProMED and HealthMap (Agg. ). Empres-i centralises data for outbreaks of major diseases of animal and public health concern, collected from OIE (immediate notifications for ongoing and resolved primary outbreaks and follow-up reports for ongoing secondary outbreaks), WHO, national (animal) health authorities, FAO field officers, reference laboratories and research. We downloaded open-access Empres-i data from the Empres-i webpage  [35].
The Empres-i dataset does not provide the reasons for outbreak notifications. Therefore, for OIE-sourced Empres-i data, one veterinary epidemiologist (EA) manually matched outbreaks that corresponded to an immediate OIE notification (primary outbreak) or follow-up report (secondary outbreak). All immediate notifications and follow-up reports sent to OIE from January to June 2016 were available in the OIE World Animal Health Disease Information System (WAHID) immediate notification and follow-up report archive  [36]. Metrics. Relevance (Rel) was the percentage of signals detected by PADI-web, ProMED and HealthMap (Agg. ) that corresponded to: i) diseases notifiable to OIE where an outbreak was recorded in the Empres-i database during the evaluation period, from January to June 2016 (notifiable, current epizootic category), or before the evaluation period, from January 2014 to December 2015 (notifiable, recent epizootic category), or 6 months after the evaluation period, from July to December 2016 (notifiable, new epizootic category), ii) diseases notifiable to OIE where no outbreaks were recorded in the Empres-i database during the evaluation period, from January to June 2016, and according to the OIE 6-monthly country reports  [37]  or disease timeline reports, the disease was noted as continuously present or suspected in the country or in one or more of its zones,  [38] , since at least January 2013 (notifiable, enzootic category), iii) outbreak of another disease, notifiable or not to OIE (other diseases category), iv) disease outbreak awareness, prevention and surveillance (disease vigilance category).
If a signal could not be confirmed through the OIE 6-monthly reports or the Empres-i database, and consisted of relevant epidemiological information on the spread or (re-)emergence of a disease, it was categorized as 'alert'. Signals were considered irrelevant if they did not fit in these categories. Two veterinary epidemiologists (EA and SV) evaluated the relevance of signals based on the content (disease, location, host and date of the events) detected by PADIweb, ProMED and HealthMap (Agg. ). Sensitivity (Se) was the percentage of ongoing primary outbreaks from immediate notifications to OIE (hereafter referred to as primary outbreaks) that were recorded in the Empres-i database and which were detected by PADI-web, ProMED and HealthMap (Agg. ) (True Positive, TP). False Negatives (FN) were all primary outbreaks from immediate notifications to OIE which were not detected by the evaluated biosurveillance systems.
Timeliness was the lag in days from the date of immediate notification to OIE (day 0), as recorded in the Empres-i database, to the date of detection by PADI-web, ProMED and HealthMap (Agg. ) during the evaluation period. A negative lag meant that a biosurveillance system was timely in detecting a primary outbreak, i. e. before the date of immediate notification. A positive lag indicated that a biosurveillance system was untimely in detecting a primary outbreak, i. e. after the immediate notification date. Libraries and tools. All statistical computing and evaluations were performed with the R statistical language  [39]  using existing libraries. Detected signals and notified outbreaks were plotted using the world map of the Natural Earth project (1:50 m resolution) implemented in the R "maps" library  [40].
Information extraction evaluation The results of the assessment of our combined IE candidate identification/verification based method, ranged from 80% to 96% (Fig  2 ). Our results were obtained via tenfold cross-validation on a manually labelled dataset used both for training and evaluation (see Evaluation protocol for information extraction section). Our combined IE method achieved the highest performance to correctly classify diseases and hosts (F-score of 95%) and lower performance to classify the number of cases, spatial and temporal information types (F-score of 85%, 80%, and 83%, respectively). The IE accuracy when based only on candidate identification ranged from 45% to 97%. Our combined method for IE was, on average, 30% more accurate than the baseline method which considered all candidates as correct.
The majority of signals detected by ProMED and HealthMap were related to current epizootic ASF outbreaks (94% and 79%, respectively), compared to 13% of the PADI-web detected signals that were related to current epizootic ASF outbreaks (Table  3 ). Both PADI-web (n = 1) and HealthMap (n = 9) detected signals for ASF outbreaks within an Eastern European zone that we could not confirm via the WAHID 6-monthly country reports and Empres-i databases. In addition, HealthMap (Agg. ) reported four signals of vigilance for potential ASF outbreaks within unaffected zones in Eastern Europe, three of which reported primary ASF outbreaks, after our study period, from July to September 2016 (data shown in category new outbreaks) (Table  3.
, Fig 3). Foot-and-mouth disease (FMD). From January to June 2016, 41% of the signals that PADI-web detected for FMD were relevant to the disease or its current outbreaks, compared to all signals detected by ProMED and 97% of the signals detected by HealthMap (Agg. ) that were relevant to FMD. Incorrect location of an event was the reason for 61% of irrelevant signals detected by PADI-web and the two irrelevant signals from HealthMap (Agg. ) (Table  3 , Fig  4 ). The majority of signals from ProMED and HealthMap (Agg. ) were related to current epizootic FMD outbreaks (24% and 13%, respectively). In contrast, PADI-web was more exhaustive, detecting signals for current epizootic (4%) and enzootic FMD outbreaks (10%), along with FMD vigilance (18%) (Table  3 ). Both PADI-web (n = 2) and ProMED (n = 1) detected signals for FMD outbreaks in zones in Asia that we could not confirm via the WAHID 6-monthly country reports and Empres-i databases.
Meanwhile, HealthMap (Agg. ) alerted signals (n = 5) on outbreaks in Southeast Asian zones that we were not able to confirm via the WAHID 6-monthly country reports and the Empres-i databases (Table  3 , Fig  6 ). Both PADI-web and HealthMap (Agg. ) reported signals on disease vigilance (n = 83, 18% and n = 47, 22%) for awareness and preparedness for a possible spread of AI due to the current AI outbreaks in North America and Asia. These two biosurveillance systems also reported signals on human cases of AI (n = 41, 10% and n = 11, 5%, respectively)(Table  3 ,  Fig 6). Sensitivity and timeliness of signals. African swine fever (ASF). From January to June 2016, OIE received immediate notifications for 11 primary ASF outbreaks in Ukraine (n = 5), South Africa (n = 2), Burundi (n = 2), Kenya (n = 1) and Mali (n = 1).
Key information extraction features The IE method that we implement in PADI-web offers several key features. We obtain relevant results regarding accuracy and F-score using a unique IE approach applicable to different information types (diseases, locations, dates, hosts and the number of cases). While some methods are specially designed and fine-tuned for a given information type, our IE approach is flexible and can cover multiple information types. Moreover, our IE approach has prospects for further development, e. g. integration of new information types and multiple languages. We also show that the IE efficiency depends on the information type that we want to extract from the news. For example, the extraction of diseases and hosts is equally efficient with the candidate identification based method alone and the combined candidate identification/verification based method (F-score of 95%).
Current limitations of PADI-web PADI-web currently detects a substantial number of false positive signals. Based on our definition, each single location extracted from the news and labelled as correct creates a PADI-web signal. This approach allows detection of all potentially relevant locations. However, it leads to numerous false positive signals due to: i) multi-references to the same location in the same article, but not identified as duplicates (e. g. references to the same place at different levels of granularity), ii) reference to an irrelevant location regarding a current outbreak (e. g. reference to a former outbreak in another geographical zone). As this was the reason underlying most of the irrelevant signals, we aim to improve the quality of our signal dataset by identifying the nested locations contained in each report, as well as by removing the irrelevant locations.
We are thus currently evaluating an SVM model as part of the Data processing step. We intend to tap the labelled dataset used for candidate verification to train the SVM model to filter news more efficiently. Additional factors that may have influenced the performance of PADI-web were the language and scope of geographical coverage of the news providers  [44]. PADI-web currently collects English language news from Google News. In comparison, HealthMap has access to several news aggregators in seven languages, and ProMED gets first-hand information from subscribers from around the world and its regional networks. Other automated biosurveillance systems, such as BioCaster have achieved high performance for given regions, sources and languages, such as the Asia-Pacific region  [45]. The MedISys system favors broader coverage of sources in more than 40 languages but its efficiency has yet to be evaluated  [46].
The work was funded by the  French General Directorate for Food (DGAL) , the  French  foot-and-mouth disease in Asia, i. e. they were detected 8 and 3 days before immediate notification to OIE, respectively. Agricultural  Research Centre for International Development (CIRAD) ,  #DigitAg , the  SONGES  Project ( FEDER, Occitanie, and Languedoc-Roussillon ), and the  LabEx NUMEV  project (number  ANR-10-LABX-20 ) funded by the " Investissements d'Avenir" French Government program , managed by the  French National Research Agency (ANR). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
S1 file is available from the CIRAD Dataverse database (accession number(s) doi:10. 18167/DVN1/KMTIFG, https:// dataverse. cirad. fr/dataset. xhtml?persistentId= doi:10. 18167/DVN1/KMTIFG). S2 file is available from the CIRAD Dataverse database (accession number(s) doi:10. 18167/DVN1/JZM34U, https:// dataverse. cirad. fr/dataset. xhtml?persistentId= doi:10. 18167/DVN1/JZM34U).
Strong-gravitational lens systems with quadruply-imaged quasars (quads) are unique probes to address several fundamental problems in cosmology and astrophysics. Although they are intrinsically very rare, ongoing and planned wide-field deep-sky surveys are set to discover thousands of such systems in the next decade. It is thus paramount to devise a general framework to model strong-lens systems to cope with this large influx without being limited by expert investigator time. We propose such a general modelling framework (implemented with the publicly available software Lenstronomy) and apply it to uniformly model three-band Hubble Space Telescope Wide Field Camera 3 images of 13 quads. This is the largest uniformly modelled sample of quads to date and paves the way for a variety of studies.
To illustrate the scientific content of the sample, we investigate the alignment between the mass and light distribution in the deflectors. The position angles of these distributions are wellaligned, except when there is strong external shear. However, we find no correlation between the ellipticity of the light and mass distributions. We also show that the observed flux-ratios between the images depart significantly from the predictions of simple smooth models. The departures are strongest in the bluest band, consistent with microlensing being the dominant cause in addition to millilensing. Future papers will exploit this rich dataset in combination with ground based spectroscopy and time delays to determine quantities such as the Hubble constant, the free streaming length of dark matter, and the normalization of the initial stellar mass function.
Nightingale et al. (2018)  devised an automated lens modelling procedure using Bayesian model comparison. Hezaveh et al. (2017)  and Perreault  Levasseur et al. (2017)  applied machine learning techniques to automatically model strong gravitational lenses and constrain the model parameters. In this paper, we devise a general framework or decision-tree that can be applied to modelfitting of quads both in a single band and simultaneously in multiple bands. We implement this uniform modelling approach using the publicly available lens-modelling software Lenstronomy  (Birrer & Amara 2018 , based on Birrer et al. 2015)  to a sample of 13 quads from the Hubble Space Telescope (HST ) data in three bands. Lenstronomy comes with sufficient modelling tools and the architecture allows a build-up in complexity as presented in this work.
LENS MODELLING To devise a uniform approach that will suit a wide range of quads that vary in size, configuration, light profiles, etc. , we need to choose from the most general models for the lens mass profile and the light distributions. It is often required to fine-tune the choice of models by adding complexities to the lens model in a case-by-case basis to suit the purpose of the specific science driver of an investigator. However, such detailed lens-modelling is outside of the scope of this paper. We only require our models to satisfactorily (χ 2  red ∼ 1) fit the data while being general enough to be applicable to a wide variety of lens systems. We use the publicly available software package Lenstronomy 2  (Birrer & Amara 2018 , based on Birrer et al. 2015)  to model the quads in our sample.
The PSO and MCMC routines in Lenstronomy utilize the cos-moHammer package  (Akeret et al. 2013). cosmoHammer itself embeds emcee (Foreman-Mackey et al. 2013), which is an affine-invariant ensemble sampler for MCMC  (Goodman & Weare 2010)  written in Python. In this section, we first describe the profiles used to parameterize the mass and light distributions. Then, we explain the decision-tree of the modelling procedure.
Based in part on observations at Cerro Tololo Inter-American Observatory, National Optical Astronomy Observatory, which is operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation. The DES data management system is supported by the National Science Foundation under Grant Numbers AST-1138766 and AST-1536171. The DES participants from Spanish institutions are partially supported by MINECO under grants  AYA2015-71825, ESP2015-66861, FPA2015-68048, SEV-2016 -0588, SEV-2016 -0597, and MDM-2015-0509, some.
of which include ERDF funds from the European Union. IFAE is partially funded by the CERCA program of the Generalitat de Catalunya. Research leading to these results has received funding from the European Research Council under the European Union's Seventh Framework Program (FP7/2007-2013) including ERC grant agreements 240672, 291329, and 306478. We acknowledge support from the Australian Research Council Centre of Excellence for All-sky Astrophysics (CAASTRO), through project number CE110001020, and the Brazilian Instituto Nacional de Ciência e Tecnologia (INCT) e-Universe (CNPq grant 465376/2014-2). This manuscript has been authored by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U. S. Department of Energy, Office of Science, Office of High Energy Physics. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes.