dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements This work was funded by  EPSRC   Early Career Fellowship   EP/M008517/1 , the  Winton Program for the Physics of Sustainability  and the  EU CELINA COST action. DSH acknowledges a  Girton College Pfeiffer scholarship  and support from the  EPSRC CDT in Nanoscience and Nanotechnology. AHR and SMV acknowledge funding from the  EU   Horizon 2020 program  through  Marie Skłodowska-Curie Action   H2020-MSCA-IF-2016-74695. CD acknowledges funding from  Leverhulme Trust  ( ECF-2018-016 ),  Isaac Newton Trust  ( 18-08 ) and a  L'Oréal-UNESCO UK and Ireland Fellowship for Women in Science 2019. Funding by the  Spanish Ministry of Science  is acknowledged, grants  MAT2017-82970-C2-1-R ,  MAT2017-82970-C2-2-R  and  MAT2018-102627-T , and by  Aragon Government (Construyendo Europa desde Aragón) , grant  E13_17R  including  European Social Fund.
Materials and Methods Crystallization and Data Collection. Crystals of the deglycosylated SGN3 in complex with the CIF2 peptide developed at room temperature in hanging drops composed of 1 μL protein solution (1 mg mL -1 ) containing 0. 5 mM CIF2 and 1 μL of crystallization buffer (17% [wt/vol] PEG 6000, 0. 1 M Tris pH 7. 5, 0. 2 M LiCl), suspended above 1. 0 mL of the latter as reservoir solution and using microseeding protocols. Crystals of SGN3 in complex with the CIF2  Hyp69,71 ) and cryocooled in liquid nitrogen. Sulfur single-wavelength anomalous diffraction (SAD) data to 4. 0-Å resolution was collected at beam-line PXIII at the Swiss Light Source (Villigen, Switzerland), CH with λ = 2. 066 Å. A native dataset to 2. 95-Å resolution was collected on a crystal from the same drop cryoprotected in the same way with λ = 1.
0 Å. Data processing and scaling was done in XDS  (40). Structure Solution and Refinement. The structure was solved using the molecular replacement method as implemented in the program PHASER  (41) , and using the isolated ectodomain of the LRR-RK PEPR1 as search model (PDB ID 5gr8). The solution comprised a dimer in the asymmetric unit and the structure was completed in alternative cycles of manual model building in COOT  (42)  and restrained TLS refinement in phenix. refine (43). A phased anomalous difference electron density map calculated with the program ANODE (44) was used to assign the position of disulfide bonds and free cysteines/methionines in the structure. Analysis with phenix. molprobity (45)  reveal good stereochemistry of the final model. Structural diagrams were prepared using Pymol (https://sourceforge.
Titration Calorimetry. All ITC experiments were perfomed on a MicroCal PEAQ-ITC (Malvern Panalytical) with a 200-μL sample cell and a 40-μL injection syringe at 25 °C. Proteins were dialyzed into ITC buffer (20 mM sodium citrate pH 5. 0, 250 mM NaCl, exceptionally containing 5% [vol/vol] DMSO for CIF4 experiments) prior to all titrations. A typical experiment consisted of injecting 200 μM CIF peptide in 2-μL intervals into the cell containing 20 μM GSO1/SGN3 receptor. The MicroCal PEAQ-ITC analysis software (version 1. 21) was used for data analysis. Data Availability. Atomic coordinates and structure factors have been deposited in the Protein Data Bank (PDB) under ID code 6S6Q.
ACKNOWLEDGMENTS. We thank the staff of  Beam Line PXIII of the Swiss Light Source (Villigen, Switzerland)  for their technical assistance during data collection. This work was supported by grants  31003A_176237  and  31CP30_180213  (M. H. ),  31003A_156261  and  310030E_176090  (N. G. ) from the  Swiss National Science Foundation , a  European Research Council   Consolidator Grant  ( 616228-ENDOFUN ) (N. G. ),  Human Frontier Science Program Organization postdoctoral  fellowship  LT000567/2016-L (S. O. ) , a  Japan Society for the Promotion of Science (JSPS)  fellowship (S. F. ), and by an  International Research Scholar Award  from the  Howard Hughes Medical Institute  (M. H. ).
Isothermal Titration Calorimetry. All ITC experiments were perfomed on a MicroCal PEAQ-ITC (Malvern Panalytical) with a 200-μL sample cell and a 40-μL injection syringe at 25 °C. Proteins were dialyzed into ITC buffer (20 mM sodium citrate pH 5. 0, 250 mM NaCl, exceptionally containing 5% [vol/vol] DMSO for CIF4 experiments) prior to all titrations. A typical experiment consisted of injecting 200 μM CIF peptide in 2-μL intervals into the cell containing 20 μM GSO1/SGN3 receptor. The MicroCal PEAQ-ITC analysis software (version 1. 21) was used for data analysis. Data Availability. Atomic coordinates and structure factors have been deposited in the Protein Data Bank (PDB) under ID code 6S6Q.
Summary of Analyses We generated RADseq data for 63 individuals from 6 lineages, of which 48 were from the two focal clades and passed quality control. First, we used maximum likelihood approaches to infer relationships among lineages and to provide a framework for subsequent species delimitation analyses (Fig. 2A  and C ). To delimit species, we performed clustering (Fig. 2B ) and PCA analyses (Fig. 3A-C ), as well as formal MSC species delimitation analyses using SNAPP and BPP. We also used the recently developed genealogical divergence index gdi based on BPP parameter estimates (Fig. 3D ) and performed an isolation-by-distance analysis (Fig. 4 ). To determine to what extent ongoing and ancestral gene flow may have contributed to current patterns of divergence, we used G-PhoCS and D-statistics (Fig.
Sequencing Data, Genotyping and Genome Assembly We generated RADseq libraries using the SbfI restriction enzyme, following three protocols (Supplementary Methods, Table  S1 ). Sequences were aligned to the M. sp. #3 nuclear genome generated by this study, and to the published M. murinus mitochondrial genome  (LeCompte et al. 2016). We used two genotyping approaches to ensure robustness of our results. First, we estimated genotype likelihoods (GL) with ANGSD v0. 92  (Nielsen et al. 2012; Korneliussen et al. 2014) , which retains information about uncertainty in base calls, thereby alleviating some issues commonly associated with RADseq data such as unevenness in sequencing depth and allele dropout  (Lozier 2014; Pedersen et al. 2018; Warmuth and Ellegren 2019). Second, we called genotypes with GATK v4.
0. 7. 0  (DePristo et al. 2011) , and filtered GATK genotypes following the "FS6" filter of O'  Leary et al. (2018; their Table 2). We furthermore used three mtDNA fragments [Cytochrome Oxidase II (COII), Cytochrome B (cytB), and d-loop] that were amplified and Sanger sequenced. The genome of the M. sp. #3 individual sampled in Mananara-Nord NP (Table  S3 ) was sequenced with a single 500bp insert library on a single lane of an Illumina HiSeq 3000 with paired-end 150bp reads. We used MaSuRCA v3. 2. 2  (Zimin et al. 2013)  for contig assembly and SSPACE  (Boetzer et al. 2011)  for scaffolding. Scaffolds potentially containing mitochondrial or X-chromosome sequence data were removed in downstream analyses.
Phylogenetic Analyses We used three phylogenetic approaches to infer relationships among lineages: (1) maximum likelihood using RAxML v8. 2. 11  (Stamatakis 2014) , (2) SVDquartets, an MSC method that uses phylogenetic invariants, implemented in PAUP v4a163  (Chifman and Kubatko 2014) , and (3) SNAPP, a full-likelihood MSC method for biallelic data that does not require joint gene tree estimation (v1. 3. 0; Bryant et al. 2012). Analyses with RAxML and SVDquartets used all available individuals, whereas SNAPP analyses were performed with subsets of 12 and 22 individuals for computational feasibility (see Supplementary Methods).
2, (b) is unambiguously two separate species for gdi > 0. 7, and (c) falls in an ambiguous zone for 0. 7 > gdi < 0. 2. Isolation-by-distance. -We tested for isolation-by-distance using the VCF file produced by GATK with the gl. ibd() function in the R package dartR 1. 1. 11 (Gruber et al. 2018).
Inference of gene flow and divergence times G-PhoCS v1. 3  (Gronau et al. 2011 ), a Bayesian MSC approach that allows for the estimation of periods of gene flow (i. e. "migration bands"), was used to jointly infer divergence times, population sizes, and rates of gene flow between specific lineages. Based on the results of exploratory models that each contained a single "migration band" between two lineages, we ran a final model with a migration bands allowing gene flow from mittermeieri to lehilahytsara and from macarthurii to M. sp. #3. Given the observed mitonuclear discordance between M. sp. #3 and M. macarthurii (see Results), we investigated gene flow between them in more detail by running G-PhoCS using a dataset with only M. sp. #3, M. macarthurii, and M. lehilahytsara individuals, wherein M.
Mutation Rate and Generation Time We used empirical estimates of mutation rate and generation time to convert coalescent units from BPP, G-PhoCS and MSMC analyses into absolute times and population sizes. We incorporated uncertainty by drawing from mutation rate and generation time distributions for each sampled generation of the MCMC chains in BPP and G-PhoCS (MSMC parameter estimates were converted using the point estimates). For the mutation rate, we used a gamma distribution based on the mean (1. 236 x 10 -8 ) and variance (0. 107 x 10 -8 ) of seven pedigree-based mutation rate estimates for primates (see  Campbell et al. 2019 , Table  S1 ). For the generation time, we used a lognormal distribution with a mean of ln(3. 5) and standard deviation of ln(1. 16) based on estimates of 4. 5 years calculated from survival data  (Zohdy et al.
SNAPP and BPP. -SNAPP Bayes factors strongly favored splitting Clade I into two species (2lnBF = 17,304 and 34,326 for two different datasets, Table  S6 ), as well as splitting Clade II, although with a smaller difference in marginal likelihood scores (2lnBF = 1,828 and CRYPTIC SPECIATION IN MOUSE LEMURS 20 993). All putative species assignments were recovered by the guided delimitation analysis with BPP (Fig. S12 ).
, in revision). Only six other cases of sympatry among mouse lemur species are known, five of which include M. murinus as one of the co-occurring species  (Radespiel 2016; Sgarlata et al. 2019). Given that the sympatrically occurring species were estimated to have had a common ancestor as recently as ~700-800 kya (i. e. , the divergence time between Clade I and Clade II, see Fig. 5 ), this suggests rapid evolution of reproductive isolation and a short time to sympatry among mouse lemurs. By comparison,  Pigot & Tobias (2015)  estimated that after 5 Ma of divergence, only 21-23% of primate species have attained sympatry. In fact, the one sympatric pair within their dataset of 74 sister species pairs younger than 2. 5 million years consisted of Galago gallarum and G. senegalensis  (Pigot and Tobias 2015) , which are also Strepsirrhini.
Biology at The University of Western Australia to AM, the Fundação para a Ciência e a Tecnologia, Portugal (PTDC/BIA-BEC/100176/2008, PTDC/BIA-BIC/4476/2012, and SFRH/BD/64875/2009), the Groupement de Recherche International (GDRI) Biodiversité et développement durable -Madagascar, the Laboratoire d'Excellence (LABEX) TULIP (ANR-10-LABX-41) and CEBA (ANR-10-LABX-25-01, the Instituto Gulbenkian de Ciência, Portugal to LC and JS, the ERA-NET BiodivERsA project: INFRAGECO (Inference, Fragmentation, Genomics, and Conservation, ANR-16-EBI3-0014 & FCT-Biodiversa/0003/2015) the LIA BEEG-B (Laboratoire International Associé -Bioinformatics, Ecology, Evolution, Genomics and Behaviour, CNRS) to LC and JS. Further financial support came from the Institute of Zoology, University of Veterinary Medicine Hannover and UR acknowledges the long-term support of the late Elke Zimmermann for her research activities on Madagascar.
The genomic data were generated with funds from NSF DEB-1354610 to ADY and DWW and from the EDB Lab to JS. ADY also gratefully acknowledges support from the John Simon Guggenheim Memorial Foundation, the Alexander von Humboldt Foundation, and the Duke Tropical Conservation Initiative. EELJ would like to acknowledge support from the Ahmanson Foundation for the data generation. This work was performed in collaboration with the GeT core facility, Toulouse, France (http://get. genotoul. fr), and was supported by France Génomique National infrastructure, funded as part of "Investissement d'avenir" program managed by Agence Nationale pour la Recherche (contract ANR-10-INBS-09). JS, UR & LC also gratefully CRYPTIC SPECIATION IN MOUSE LEMURS 36 acknowledge support from the Get-Plage sequencing and Genotoul bioinformatics 614 (BioinfoGenotoul) platforms Toulouse Midi-Pyrenees.
ACKNOWLEDGMENTS We thank the editors and three anonymous reviewers for their constructive input which substantially improved our manuscript. The study was conducted under the research permit No. 197/17/MEEF/SG/DGF/DSAP/SCB. Re  (DS),  072/15/MEEMF/SG/DGF/DCB. SAP/SCB (MBB), 137/13/MEF/SG/DGF/DCB. SAP/SCB  (DWR),  175/14/MEF/SG/DGF/DCB. SAP/SCB  ( A. Miller ), kindly issued by the directeur du système des aires protégées, Antananarivo and the regional authorities ( Direction Régional de l'Environnement, de l'Ecologie et de Forêts ). We are endebted to  J. H. Ratsimbazafy ,  N. V. Andriaholinirina ,  C. Misandeau ,  B. Le Pors  and S. Rasoloharijaona, for their help with administrative tasks, to  A. Miller  for sharing samples, and to  G. Besnard  for facilitating this study. We thank our field assistants ( T.
The quality of OCR has a direct impact on information access, and an indirect impact on the performance of natural language processing applications, making fine-grained (e. g. , semantic) information access even harder. This work proposes a novel post-OCR approach based on a contextual language model and neural machine translation, aiming to improve the quality of OCRed text by detecting and rectifying erroneous tokens. This new technique obtains results comparable to the best-performing approaches on English datasets of the competition on post-OCR text correction in ICDAR 2017/2019.
A shared task is a good chance to compare techniques. Therefore, we use the evaluation metrics and English datasets of the two occurrences of the competition on post-OCR text correction in 2017  [3]  and 2019  [14]  to evaluate the performance of our proposed methods. Experimental results show that our approach performs slightly better than the winners of the competition on error detection and obtains comparable improvements on error correction. Our three contributions are mentioned as follows. The first one is to apply static word embeddings in fine-tuned BERT models, which increases performance of error detection. Our character embeddings created by training NMT on aligned OCRed text and its ground truth (GT) achieve some positive results in rectifying errors. The last contribution is to utilize a length difference for removing irrelevant candidates, which improves correction output.
ERROR CORRECTION As mentioned in Section 2, character-level MT is the state of the art for the error correction task, where it enables to tackle the problem of data sparsity. Regarding MT techniques, SMT consists of many small sub-components that are tuned separately. In contrast, NMT aims at building a single neural network which maximizes the translation performance. Its performance is comparable to the existing state-of-the-art phrase-based model  [18]. Consequently, we employ NMT at character level to translate OCRed text into its corrected version (in the same language). Our models are built on an open-source toolkit for neural machine translation (OpenNMT)  [8]. We use most of the default values of OpenNMT, except for embedding, hidden layer size, sequence length. Input and output texts are written in the same language, therefore we configure to share embeddings between the source and target side with embedding size of 160 (tested against 100).
Hidden layer size is increased from 500 to 1000 in order to learn more information. We set the maximum sequence length to 70 (instead of the default one, 50) to cover longer sequences of training data. It is the fact that most of OCRed tokens are correct. If the MT system is trained on a dataset with a large proportion of valid tokens, then it might not rectify errors. In order to reduce the negative impact of imbalanced data and deal with real-word errors, we consider erroneous OCRed tokens and some nearby tokens (which can be correct or incorrect) as input; the corresponding GT texts are provided as output of NMT models. Particularly, given one error and its four neighbors, we generate five word 5-grams which are represented at character level and used as input sequences. By doing this, we augment data for training NMT models.
An example of an input sequence of Monograph dataset with factored representation is shown in Table  2. Factored NMT model is the first version of our approach (denoted as Correction 1). MT techniques apply pre-trained word embeddings to improve translation performance. Several word embeddings are available and free to access while it is not easy to find a character embedding. McCann et al. [10]  reported that a pre-trained encoder of a MT model increases the performance of other NLP tasks. Their contextualized word vectors are known as Context Vectors (CoVe). Broadening this idea, we extract embeddings from character-level NMT model trained with an aligned data. Particularly, we align OCRed text with its corresponding GT text, then we generate input sequences from each aligned error with its contextual tokens.
Datasets English OCRed texts of both rounds of the competition are exploited as evaluation data. The dataset of Comp2017 consists of 813 English written files that were either published in periodicals or monographs. Therefore, they were divided by the competition organizers into two datasets: Monograph and Periodical. The dataset of Comp2019 contains 200 files in English, which are from IMPACT project. The corresponding GT is created by different projects such as Europeana Newspapers, IMPACT, Project Gutenberg, Perseus and Wikisource. These datasets are distributed as a training set of 80% and an evaluation set of 20%. The detailed characteristics of the used evaluation datasets are shown in Table  3.
Results Tables  4  and 5  illustrate the performance of the proposed detection and correction approaches on the competition datasets in ICDAR 2017, ICDAR 2019, respectively. Error Detection. In overall, our approach surpasses other approaches on Periodical (with 4% higher F-score) and Comp2019 (with 1% higher F-score) but not on Monograph. These results are partly explained by the rate of real-word and non-word errors in each dataset and the strength of our neural network based approach. In fact, there are more real-word errors in two datasets (Periodical and Comp2019) than in Monograph. BERT is a contextual language model, so it is reasonable that the BERT-based model can detect more real-word errors. The rate of correctly detected real-word errors supports our assumption. Our approach is able to identify 64% of context-sensitive errors on Monograph, 63% on Periodical, 48% on Comp2019, which is better than the results reported in the prior work  [12]  (43% on Monograph, 49% on Periodical, no report on Comp2019).
This approach also gets higher results of correctly detected non-word errors with 95% on Periodical, 93% on Comp2019, but not on Monograph (82%). Similarly, the percentage of correctly recognized OOV words is comparable to the ones reported in the related work  [12]  with about 61% on average on three datasets. Error Correction. There is an important difference between the 2017 and 2019 competitions. While the organisers provide the predefined list of error positions, the correction task of the second round is more challenging as it requires not only to identify error positions but also to rectify such detected errors. In general, the best model of our approach outperforms some of our counterparts. In terms of the 2017 competition, our single model performs better than most of participants, except for the Table  4 : Results on the English datasets of the competition in ICDAR2017 (F-score: detection, Impr.
: correction); '-' marks no improvement, 'x' denotes no reported result by the competition or the prior approaches that only work on detection  [12]  or correction task  [11]. state-of-the-art approach (Char-SMT/NMT) which combines five different models of statistical MT and neural MT. The authors of Char-SMT/NMT claimed that their system is complicated and difficult to apply to new datasets. Therefore, they suggested the most promising single system  [1]  which works across all data sets. However, its performance is significantly lower than the ensemble model as well as our proposals. In contrast, our model is easy to implement with available data. Moreover, it should be emphasized that our improvement is much higher than the neural MT based approach (CLAM) or statistical MT based one (MMDT)  [16].
Consequently, we think that our model can be considered as a reliable solution to reduce OCR errors across various data sets. In terms of the 2019 competition (i. e. , without the provided error list) we have to use our list of errors obtained from the detection task for generating data sequences. Our best model still underperforms some other methods, including RAE2, RAE1 and CCC. In our opinion, the reason is that our models are built on the limited resources of the 2019 competition which is small and contains several real-word errors involving wrong line recognition. The RAE1, RAE2 competitors and the CCC team benefit from using external materials like the Google Books Ngram Corpus. Nonetheless, there is no clear conclusion between the performance of our best model and that of RAE1&2 (called WFST-PostOCR) as the former performed better in the first round.
Data were collected retrospectively in November 2018. A medical chart review was conducted to extract clinical and demographic data including age, gender, body mass index (BMI), CF center of enrollment, place of residence, comorbidities (pancreatic insufficiency, diabetes, chronic Pseudomonas aeruginosa infection), percent predicted forced expiratory volume in 1s (ppFEV1), health status (death, transplantation and number of days of hospitalization). At the time of the study, in France, lumacaftor-ivacaftor therapy was only available in hospital pharmacies. To obtain patients pharmacy refills, hospital pharmacies providing lumacaftor-ivacaftor therapy to patients were contacted. Adherence to lumacaftor-ivacaftor was calculated using proportion of days covered (PDC), which is the total number of medication-covered days divided by the number of days in a certain period.
Statistical analyses Continuous variables were described as means and standard deviations (SD). Categorical variables were described as frequencies. Patients were categorized into 2 groups, adherent (PDC ≥ 80%) and non-adherent (PDC < 80%) at 6 and 12 months and their clinical characteristics were compared using χ2 test or the exact Fisher test. Multivariate logistic regression analyze was conducted to examine the association between lumacaftor-ivacaftor adherence and different factors: travel time from the place of residence to pharmacy, ppFEV1, gender and age. A p-value < 0. 05 was considered statistically significant. All analyses were performed using SPSS version 21. 0.
Patients' characteristics Out of the 963 CF patients followed up in Auvergne Rhône-Alpes CF centers, 192 patients were treated with lumacaftor-ivacaftor. After applying the inclusion and exclusion criteria, 96 patients were included in the final cohort for analysis. All of them were homozygous for the F508del-CFTR mutation. Table  1  shows the demographic and clinical data of the 96 patients included. The mean age of the cohort was 22 years (± 9). The overall cohort had predominantly adults (55% vs 45% under 18), male (54%). Mean ppFEV1 was 77% (± 25), with well-preserved lung function in 57% (ppFEV1 > 70%). All patients had an altered exocrine pancreatic function; cystic fibrosis-related diabetes was diagnosed in 13%, and chronic Pseudomonas aeruginosa colonization in 37%. The mean number of hospitalization days per patient was 3.