dataesr/scientific-paragraphs-categorization huggingface dataset:
The K-FSI coupling tool The present work has been implemented in K-FSI, a commercial tool coupling the structural code K-STRUCT and the fluid solver ISIS-CFD, the latter being part of the FINE™/MARINE suite. K-FSI resolves the Neumann-Dirichlet boundary conditions at the wet surface. The coupling between the two solvers is carried out using a block-LU decomposition of the monolithic system  [6]. This approach, along with the approximation of the first Jacobian of the interface, allows K-FSI to predict the behavior of applications ranging from weakly to strongly coupled. Writing the lower decomposition of the system given by Eq. (1) the system of equations reads { (𝑆 -𝐶 ⃗ 𝑓𝑠 𝐹 -1 𝐶 ⃗ 𝑠𝑓 ) 𝑥 𝑠 = 𝑆 𝑠 -𝐶 ⃗ 𝑓𝑠 𝐹 -1 𝑆 𝑓 (a) 𝐹 𝑥 𝑓 = 𝑆 𝑓 -𝐶 ⃗ 𝑠𝑓 𝑥 𝑠 (b) (2) where the operators are defined in a similar way as for the Eq.
An advantage of dictionary-based approaches over rule-based ones is that they are equally well suited for recognising all types of taxon names. On top of that, entity normalisation is straightforward since dictionaries are generally derived from taxonomic databases such as the NCBI Taxonomy  (Federhen, 2002). Although these databases contain a huge number of taxonomic names, they cannot be considered exhaustive as new taxa are continuously described. Therefore, these systems are often characterised by a low recall, as they cannot handle new or abandoned taxon names, misspellings or other unexpected naming variants. In addition, dictionary-based approaches cannot resolve the ambiguity due to homonymy between taxon names and common words as matching is context-agnostic. Machine learning-based systems replace human-curated rules or fixed lists of names by a statistical model that has been trained to recognise taxon mentions from a feature vector representation of input data  (Campos et al.
, 2020). In particular, remarkable progress has been made in biomedical information extraction through the widespread application of deep learning techniques  (Liu et al. , 2016; Perera et al. , 2020). Deep learning refers to a class of machine learning techniques that use multiple processing layers (typically artificial neural networks) to learn latent representations of data with multiple levels of abstraction  (Goodfellow et al. , 2016). The ability of deep neural networks (DNNs) to auto-detect hidden features in complex, highly dimensional data removes the burden of task-specific, knowledge-centred feature engineering. In return, their performance largely depends on the availability of large amounts of high quality, manually annotated data in the form of gold standard corpora (GSCs). Indeed, DNNs usually have a large number of parameters, which make them overfit on small training datasets, with the consequence that the resulting models perform poorly on unseen data.
As new use cases emerge, including the need to extract factual statements from the ecological literature to augment biodiversity databases with up-to-date information, ecological information extraction sees a resurgence of interest from the community  (Tamaddoni-Nezhad et al. , 2013; Thessen and Parr, 2014; Compson et al. , 2018; Nguyen et al. , 2019; Chaix et al. , 2019; Muñoz et al. , 2019). However, ecologists still lack the tools to build biodiversity information extraction pipelines with state-of-the-art performance. This paper addresses the task of taxonomic NER as a critical component of such pipelines, and a first step towards the development of a toolkit of state-of-the-art algorithms that would help the ecology and evolution community make the most of the ever-growing corpus of texts on biodiversity.
In addition,  Lee et al. (2020)  obtained state-of-the-art results in a number of biomedical NLP tasks, including named entity recognition, relation extraction, and question answering, using BioBERT word vector representations. A recent survey also showed that of 6 open-source language models, BioBERT performed best on biomedical tasks  Lewis et al. (2020). Large pretrained Transformer models are tremendously effective for many NLP tasks. However, they have two main limitations. First, they usually require a large training corpus and easily overfit on small or modestly-sized datasets. Although large-scale unlabelled corpora are far easier to obtain than large-scale GSCs, building a corpus large enough (several billion words) to learn such a model requires significant engineering efforts. Second, inference in large Transformer models is prohibitively slow and expensive due to the use of self-attention in multiple layers.
| Training TaxoNERD's models using transfer learning Performance of DNN-based approaches to NER largely depends on the availability of large amounts of high quality, manually annotated data in the form of gold standard corpora. The deeper the network architecture, the better the expected performance, but a much larger dataset is needed to fully train model parameters and prevent overfitting. In domains where GSCs tend to be small, as is the case in ecology and evolution, training such large neural networks from scratch, starting with randomly initialised weights, would overfit the training set badly, which would cause the resulting models to perform poorly on unseen data. One approach to get around this problem is to first pretrain a DNN on a source task for which a large dataset is available.
Then, the pretrained weights of this network are used to initialise the weights of a second network, which we continue to train on our typically smaller dataset for the target task (Fig. 4 ). This process, called transfer learning, has been shown to improve generalisation of the model, reduce training time on the target dataset, and reduce the amount of labelled data needed to obtain high performance  (Giorgi and Bader, 2018). There are basically two common ways to transfer knowledge learned from one task or domain to another: feature extraction and fine-tuning. Feature extraction. The pretrained model is used as an off-the-shelf feature extractor. The pretrained weights of the feature extraction layer are frozen, and a new classification layer is trained on the target dataset. Fine-tuning.
All the pretrained network's weights are unfrozen and updated (fine-tuned) for the target task. The choice between feature extraction and fine-tuning may be guided by some criteria such as the size of the target dataset, and the similarity between the source and target datasets. In practice, fine-tuning is usually more general and convenient for many different downstream tasks, while requiring minimal architecture modifications. Following the pretrain-and-finetune approach that has become the dominant paradigm for NLP applications in the last few years, we trained TaxoNERD models by reusing embedding layers that have been pretrained on large biomedical corpora, and fine-tuning the pretrained models on a GSC which was built by combining the COPIOUS and Bacteria Biotope task corpora. More precisely, the en_ner_eco_md model is trained by fine-tuning the en_sci_core_md model provided by the scispaCy library.
While there exists a number of pretrained language models for biomedical NLP, there exists none for ecological applications. Although we demonstrated with TaxoNERD that biomedical language models can be finetuned on ecological datasets with satisfactory performance, a domain-specific language representation pretrained on a large-scale ecological corpus would surely boost the performance of ecological information extraction tools. Post-processing can also improve the quality and accuracy of predictions  (Perera et al. , 2020). For example, if a certain named entity is tagged once or several times in a document, and the same entity exist elsewhere in the text, untagged, then post-processing could make sure these missed NEs are also tagged with their predicted class, thus increasing recall. Another important subtask at this point is to resolve coreferences, i.
g. interspecific interactions), and other NLP tools that will hopefully facilitate access to the considerable amount of knowledge held by the current (and future) body of published literature in ecology and evolution. F I G U R E 4 State-of-the-art NER systems are based on deep neural networks that learn latent features from large amounts of data. When only small datasets are available for the target task, a common approach is to use transfer learning. In this example, transfer learning is used to adapt a DNN trained on a large biomedical corpus to the ecological domain. The feature extraction subnetwork (pink nodes) is frozen, while the NER layers (green, then blue nodes) are retrained on the ecological corpus. Alternatively, the pretrained model parameters can be unfrozen and the whole network be fine-tuned on the target task corpus.
data availability All the scripts used for the evaluation and the links to the four gold standard corpora are available on the GitHub repository accompanying this paper (https://github. com/nleguillarme/snr_tools_and_methods). TaxoNERD is available on PyPI under a MIT license; the sources, including the configuration files used to fine-tune the models, can be downloaded from the project's GitHub repository (https://github. com/nleguillarme/taxonerd). references Akella, L. M. ,  Norton, C. N. and Miller, H. (2012)  NetiNeti: discovery of scientific names from text using machine learning methods. BMC bioinformatics, 13, 211. Author One et al. TA B L E 3 Precision, recall and F-score for the eight taxonomic NER systems evaluated on the four gold standard corpora, using exact match and approximate match as evaluation criteria.
Signs of the external field effect have been detected too  [80] [81] [82] [83] [84] [85]. In our simulation, we employed the QUMOND modified gravity theory of MOND  [57]. In it, the gravitational potential φ is related to the distribution of the density of matter ρ through the equations: ∆φ N = 4πGρ, ∆φ = ∇ • ν ∇φ N /a 0 ∇φ N (2) with the boundary conditions φ N ( r) → 0 and ∇φ( r) → 0 for | r| → ∞. Here ν is the inverse interpolating function defined by the equations x = yµ(y), y = xν(x). We decided for QUMOND for our simulation, because of publicly available software that is suitable for simulations of galaxy interactions and can work in QUMOND. It is the Phantom of RAMSES code, or PoR 1   [87, 88] , that is based on the popular adaptive-mesh-refinement code RAMSES  [89]. The simulation was done with the spatial resolution of 120 pc.
The fluorescence high frequency features are extracted from the variance of the raw images, as in fluctuation microscopy, through a demodulation process using the speckle autocorrelation as a carrier wave, as in Structured Illumination Microscopy. AlgoRIM does not invoke the sparsity of the excitation or the binarity of the fluorescence for achieving super-resolution and, even though it is based on the variance of the speckle images, it yields a linear response to brightness (for more details, see Supplemental information). In Figure  1C , we compare the reconstructions of the same sample of tagged F-actin network in podosomes obtained by second-order statistics dSTORM (NanoJ software, Super Resolution Radial Fluctuation)  (Gustafsson et al. , 2016)  and RIM. Podosomes are actin-rich, cell adhesion structures applying protrusive forces on the extra cellular environment, recently observed by 3D dSTORM  (Bouissou et al.
RIM allows high fidelity super-resolved imaging in three-dimensions RIM three-dimensional (3D) imaging is obtained by translating the sample through the focal plane (Movie S1) and by recording several speckle images at each position. The speckle illumination and the variancebased data processing ensure an efficient optical sectioning  (Ventalon and Mertz, 2005 ; Ventalon et al, 2007). Note that, when necessary, the spherical aberration induced by the index mismatch between the objective immersion oil and the mounting medium has been accounted for when reconstructing the 3D image  (Sibarita, 2005). To test the fidelity of the super-resolved images obtained with 3D RIM, we focused on dense filamentous structures. We imaged the vimentin network from fixed HUVEC cells and reconstructed the whole network from 200 speckles per slice, 12 slices and an axial step of 100 nm (see in Figure  2A  the color-coded axial position of the filaments, and Movie S2).
RIM provides super-resolved movies of live specimens at high temporal resolution and low toxicity. To optimize the temporal resolution of RIM and limit phototoxicity, one should know the minimal number of raw speckle images that is necessary for a faithful reconstruction of a given specimen. The latter depends on the nature of sample (dense or sparse) and its dynamics. Tests on fixed samples showed that super-resolved images can be obtained with only 50 speckles but at the cost of a residual illumination inhomogeneity (Figure  S3A ). Increasing the number of speckles improves the illumination homogeneity and the signal to noise ratio but movements of the live specimen during the recording may blur the reconstruction (Figure  S3B ). The speckle images being recorded regularly during the observation time, different trade-offs can be tested with the same data set.
Declaration of interests AlgoRIM described herein is covered by a provisional invention statement filed by S. L. , T. M. , J. I. , M. A. , and A. S. and covered by the CNRS. (C) Super-resolved images of the macrophages F-actin network, obtained by processing 10,000 dSTORM raw images as in (B) using (Nanoj software SRRF;  Gustafsson et al. , 2016)  or 800 raw RIM images using algoRIM, compared to a reference Scanning Electron Microscopy image. Note that RIM reconstruction is closer to the SEM image and does not suffer from the non-linear response to brightness of fluctuation microscopy. (E) Comparison between RIM and SIM resolution using the commercial ELYRA microscope for two-color imaging. The sample is a calibrated DNA nanoruler (SIM 140 YBY), where two red fluorophores (Alexa 561) attached at the DNA ends are separated by 140 nm, and are equidistant (70 nm) to a green fluorophore (Alexa 488).
Acknowledgements We thank  François Payre ,  Paul Mangeat  and  Malek Djabali  for proofreading the manuscript. We thank the  Imaging Core TRI  and  Drosophila  facilities of the  CBI , and  Isabelle Fourqueaux  ( CMEAB ) for SEM imaging. We thank  INSERM Plan Cancer   2014-2019  and  Toulouse Cancer Foundation  for partial financial support. We thank  David Villa  for video editing (https://www. scienceimage. fr/).
Landcover estimates Site-specific landcover estimates were calculated in QGIS v3. 2. 3 by first using the Semi-Automatic Classification Plugin v6. 2. 9 to transform LANDSAT 8 data (https://earthexplorer. usgs. gov/) into reflectance values  (Congedo 2016 , QGIS Development Team (2019). These values were used to calculate the normalized difference vegetation index (NDVI) for each site using the QGIS Raster Calculator tool and the equation: NDVI = (Band 5 -Band 4) / (Band 5 + Band 4), where Band 4 = red and Band 5 = near-infrared. Pixels with NDVI values of -1. 000 -0. 150 were considered water, 0. 151 -0. 700 were considered urban (grey), and 0. 701 -1. 000 were considered green. For downstream analyses, we calculated: (i) NDVI_mean as an estimate of the average proportion of green landcover at each site; (ii) NDVI_stdev as an estimate of landcover variability across a site; (iii) distance_to_urban, the distance from the center of each site to the nearest 'urban' pixel (an indicator of access to resources associated with urban landcover); and (iv) distance_to_water, the distance from the center of each site to the nearest 'water' pixel.
Finally, we explored using datasets from the Global Road Inventory Project (GRIP); https://datacatalog. worldbank. org/dataset/grip-globalroads-inventory-project-2018), the Global Urban Landcover dataset (ftp://115. 239. 252. 28/GlobalUrbanLand/), and the WorldPop demographic dataset (https://www. worldpop. org/) to calculate additional estimates of urbanization; however, none of these were of appropriate resolution for the study area.
2002). Next, we calculated the modularity of each network to identify subsets (modules) of the network that are more densely connected than other subsets. Higher values of modularity indicate that subsets of rodents and parasites interact more strongly amongst themselves (i. e, within a module), than with other subsets (Thébaud 2013). Finally, we calculated the nestedness temperature of each network, which provides an estimate of the orderliness of the network and reflects both species composition and incidence  (Ulrich et al. 2009). Nestedness temperature values range from 0 to 100, with higher values observed when more abundant parasites interact more frequently with a wider range of hosts. Finally, we transformed each bipartite network into a unipartite network by connecting individual rodents based on shared ticks/microbes using the 'igraph' package in R  (Csardi and Nepusz 2006).
Nonparametric resampling was used to estimate the edge density with confidence intervals of each unipartite network using the R package 'snowboot'  (Ramirez-Ramirez et al. 2019). Eigenvector centrality was calculated from each projected unipartite network to identify individuals, collection sites, or rodent species occupying central (i. e. , wellconnected) positions in each network, and which may be of importance for the transmission of ticks or microbes at each location.
Acknowledgments The authors thank  Andrew Alek Tuen  ( Universiti Malaysia Sarawak ),  Andrew Joris Noyen  ( Padawan Municipal Council ),  Basheer Ahmed  ( Kuching North Municipal Council ),  Danielle Levesque  ( Universiti Malaysia Sarawak ),  Dilop Jina  ( Padawan Municipal Council ),  Jean-Bernard Duchemin  (CSIRO),  Lee Trinidad (CSIRO) ,  Nurshilawati Abdul Latip  ( Universiti Malaysia Sarawak ),  Patrick Lai Ganum (Kuching South Municipal Council) ,  Rachel Amos-Ritchie (CSIRO) , and  Samuel Wong  ( Universiti Malaysia Sarawak ).
Background/Aims: Accurate stool consistency classification of non-toilettrained children remains challenging. This study evaluated the feasibility of automated classification of stool consistencies from diaper photos using machine learning (ML). Methods: In total, 2687 usable smartphone photos of diapers with stool from 96 children younger than 24 months were obtained after independent ethical study approval. Stool consistency was assessed from each photo according to the original 7 types of the Brussels Infant and Toddler Stool Scale independently by study participants and 2 researchers. A health care professional assigned a final score in case of scoring disagreement between the researchers. A proof-of-concept ML model was built upon this collected photo database, using transfer learning to re-train the classification layer of a pretrained deep convolutional neural network model.
METHODS This observational study was conducted between May and December 2018 in the United States of America. An independent institutional review board (IntegReview IRB, Austin, TX) reviewed and approved the study protocol and documents used for informed consent before study initiation. The study was conducted according to the principles and rules laid down in the Declaration of Helsinki, the International Conference for Harmonization of Technical Requirements for Pharmaceuticals for Human Use guidelines, and the Council for International Organizations of Medical Sciences. All participants provided informed electronic consent before any study-related activities were initiated. This study was registered at ClinicalTrials. gov (identifier NCT03402555). All coauthors had access to the study data and reviewed and approved the final manuscript.
Machine Learning The above described collection of labeled photos was used as input for the ML model. In cases of scoring disagreements between the 2 independent researchers' scores, a final stool consistency score was assigned by a health care professional. Significant progress in the field of image classification has been made possible by the availability of large annotated datasets and advances in deep convolutional neural networks. Due to the limited size of the stool photo dataset (>100-fold fewer images than datasets typically used to train deep neural network models for image recognition), it was not feasible to consider complete training of such a model for stool consistency prediction. Instead, our proofof-concept project assessed the feasibility of adapting an existing model for classifying images (MobileNet  (17) , an efficient convolutional neural network for mobile vision applications) for the stool photo classification task.
This transfer learning approach assumes that the selected source domain (images on which the existing model was trained) is sufficiently related to the target domain (stool photos). MobileNet's streamlined architecture uses depthwise separable convolutions to build light-weight deep neural networks  (17) , and consists of a set of stacked layers, ending with the final classification layer. We took advantage of knowledge learned by the MobileNet model, which was pretrained using the ImageNet Only the final classification layer of our model was retrained; all other layers of the network were kept ''frozen,'' retaining the optimal performing values and configurations defined during the initial training phase on the ImageNet database. Each stool photo had to be read several times in each fold of the crossvalidation, and the values of the layers before the classification layer had to be generated multiple times with the same output.
Statistical Analyses No formal sample size calculation was performed. The estimated number of photos needed per class was based on tests with a previous ML photo recognition task. The primary objective of the study was to create a database with at least 200 photos per BITSS type (1-7) and 200 photos of empty diapers. The recruitment target of 100 participants was estimated based on the number of photos required, assuming a withdrawal rate of 15%. For each evaluable photo, the participant's score was compared with each independent researcher's score for agreement, calculated using percentage agreement (exact match between scores) and mean level of disagreement. The mean level of disagreement was defined as the mean of the difference in scores, when disagreements occurred. For instance, given a participant score of 2 and independent researcher score of 4, the magnitude and direction of disagreement for that photo is À2.
DISCUSSION In this work, we demonstrate the feasibility of using MLbased image recognition in a clinical application relevant to a very broad population: diaper-wearing children. The assessment of stool consistency is a source of worry for many parents and a topic frequently discussed during healthcare provider consultations. We developed an easy-to-use mobile device-based workflow that facilitates collection of photos and manual annotation, and used it to establish a photo collection of used diapers. The photo dataset was used to generate a novel ML model running on users' smartphones for the assessment of stool consistency in healthy infants/ toddlers. Finally, the model's performance was evaluated alongside that of human raters in predicting infant/toddler stool consistency from photos, a nontrivial task even for clinicians  (4).
The use of ML for image recognition has been explored in a number of specialized areas in medicine including gastroenterology  (21) , for example, for distinguishing pathological versus healthy features in duodenal tissue  (22). Other applications range from classification of skin lesions  (23)  to glaucoma diagnosis based on fundus images  (24). Many applications have relied on specialized or invasive procedures (biopsy, endoscopy), or standardized imaging techniques and acquisition protocols (radiology, microscopy). In contrast, the use of smartphone photos, which are typically captured under highly variable conditions (resolution, lighting exposure, orientation), is less well explored in medical image recognition. Deep learning methods have allowed important and rapid progress in the field of image recognition but have historically required massive datasets and computing power to perform well.