dataesr/scientific-paragraphs-categorization huggingface dataset:
[95][90][91][92][93][94]88]huge development has been done due to a new interest of mine companies and constructors, reporting high potential for elemental or mineral mapping with industrial or academic LIBS set up[89][90][91][92][93][94]. These set ups provide elemental mapping of several cm 2 of surface for numerous elements, revealing rather complex elemental distributions in less than 3 hours. This acquisition is faster than conventional techniques such as EPMA or SEM. Zn and Cu, is risky since this mineral is very rare. Actually, Arsenic is more probably present as trace element in a Zn-sulfur, or with the Cu in substitution. It is hard to compare the element intensities in different minerals, and the relative ratio between two elements are tricky to establish. As Moncayo et al. [95]reminded, the points that limit the development of the LIBS technique are undeniably the expertise and the time required to extract a relevant signal from the dataset.
5 image products. The reference datasets need to have superior spatial resolution, wide global coverage, and spectral compatibi lity with respect to the SEVIRI sensor, which has 12 spectral bands with 1 km and 3 km spatial resolutions. The SEVIRI sensor works with whiskbroom principle at a geostationary orbit and collects data at 5 minutes (rapid scan) and 15-minutes (full scan) intervals. Although preliminary investigations on reference data were performed by using images of different satellite sensors during the study, in-depth investigations were performed with MERIS global image mosaic and Landsat imagery. The progress and different problems observed in the images are reported here.
Another method was proposed for absolute accuracy assessment of SEVIRI HRV Level 1. 5 images based on lake matching over Switzerland and parts of neighbouring countries  (Aksakal, 2013; Aksakal et al. 2013a Aksakal et al. , 2013b)). In the frame of a study initiated by EUMETSAT a new approach for GQA of SEVIRI Level 1. 5 images has been developed collaboratively with EUMETSAT, Telespazio SA, France, and Hacettepe University, Turkey. With the developed approach, the absolute location, interband (band-to-band) registration and multi-temporal stability of MSG SEVIRI images can be assessed using dense image matching techniques with sub-pixel accuracy and their suitability for Low Earth Observation (LEO) satellites were investigated in preliminary tests as well. The developed methodology is being implemented in a standalone software, GQA Tool, which is based on open source libraries and Python programming language.
The developed absolute GQA approach is essentially based on comparing reference and search (or working) images using a least squares matching technique in the GEOS (geostationary) projection system. Although the developed methodology is powerful and allows dense matching results (thousands of points depending on the selected band and cloud coverage), it has the main requirement of provision of reference images with superior spatial resolution and geometric accuracy. In addition, a homogenous, global coverage is also needed to assess the full extent of SEVIRI images. For this purpose, a number of freely available image datasets (i. e. Landsat 5-7-8, MERIS, OLCI, Sentinel-2) have been investigated within the study in terms of geometric accuracy, spectral bandwidth compatibility with SEVIRI, image artefacts, geographical coverage, and availability of global mosaics.
e. HRV, Visible 0. 8, Visible 0. 6 and Infra-red 10. 8). One of the advantages of the developed system is flexibility in terms of reference data, which allows replacement/update of the reference datasets and on-the-fly keypoint generation whenever needed. Thus, the reference data can be updated on a regular or on-demand basis. However, the preparation of such datasets requires detailed investigations and in particular extensive error-analyses. The experiences and key issues for compilation of the reference datasets for the absolute GQA of MSG SEVIRI images are presented in this paper with particular focus on Landsat and MERIS imagery.
Requirements for Absolute GQA of SEVIRI The absolute geometric quality (i. e. georeferencing) of a sensor is assessed using external reference data. The reference data should have superior resolution and accuracy with respect to the data to be assessed, and in practice reference data with at 3-5 times or even higher accuracy should be sought  (Gruen and Kocaman, 2008). The evaluation method (e. g. point-wise, regional, or global) depends on the density, distribution and other characteristics (e. g. 2D and 3D coordinates, availability of earth surface reflectance data in different spectral bands, temporal characteristics, etc. ) of the reference data. In this study, the absolute GQA of MSG SEVIRI images were assessed with respect to orthoimages due to low spatial resolution of the datasets.
REFERENCE DATA PRODUCTION The initial reference data preparation efforts in the project involve Landsat images, including the Global Land Survey (GLS) dataset, Sentinel-2 images, and the MERIS images. The list of all datasets considered for absolute GQA are provided in Table  2. Their advantages and disadvantages are presented in Table  3. The sensor specifications and the issues with each dataset, except for the Sentinel-2 images, are explained in the following sub-sections. Here, MERIS RGB with 260 m x 300 m GSD and Landsat including GLS Dataset with 30 m GSD have been selected for detailed assessment due to their resolution and accessibility to archives for on-demand production. The global coverage (large spatial distribution) of these datasets are high and they provide sufficient resolution and accuracy for the 2D (planimetric) absolute accuracy assessment.
Landsat Data The or better RMSE as reported by USGS (GLS, 2020) and  Gutman et al. (2013). Thus, the geometric accuracies of the used products were found sufficient for the purposes of the study. The initial tests with Landsat mosaic generation were performed over Europe with GLS2010. Band 4 of Landsat 4-5-7 were used to create the visible image mosaic, and Band 6 was employed for creating the infrared image mosaic. However, in some areas, GLS2010 data were not available and therefore these parts were filled with Landsat 8 data (Band 4 for visible and Band 5 for infrared). On the other hand, it was observed that Landsat 8 Band 5 was not included in some products, therefore this band was also replaced either with Band 4 or with Band 6 in those areas (depending on the radiometric similarity in the area).
MERIS Data The Medium Resolution Imaging Spectrometer (MERIS) was one of the primary instruments on-board ESA's Envisat-1, which was launched in March 2002. MERIS acquires images with pushbroom principle in 15 programmable spectral bands. The instrument operates at 260 m x 300 m (full) and 1200 m (reduced) spatial resolutions with a swath width of nearly 1150 km. The main task of MERIS is monitor the ocean colour, but it is widely used for a global environmental monitoring system in the European scientific community (NASA, 2020). The imaging principle of MERIS is provided in Figure  11. The spectral band specifications are given in Table  7. 77 m of total RMSE error was obtained from a number of geographically distributed MERIS GlobCover Orthotectified datasets by  Bicheron et al. (2011). Initial tests were performed with the MERIS RGB mosaic obtained from GAEL consultant, France.
However, several problems were observed with image such as very bright areas and geometric problems at the coastlines. On the other hand, matching with the MERIS RGB (only red band was used) was found successful in terms of the number and the distribution of the keypoints over the globe. Therefore, a second attempt was made to produce own MERIS L3 mosaic generation using the ESA Climate Change Initiative (CCI) archives  (ESA, 2020). The MERIS products (Level 1B, Level 2 and Level 3) are radiometrically corrected and resampled on a path-oriented grid level. Level 3 products are weekly synthesis of more than one MERIS products. Here, all possible MERIS images in the archive (5 months of data per year) were mosaicked by using the median values of each pixel in the overlapping areas. Mosaics of every MERIS band could be produced on demand.
DISCUSSION The produced reference datasets were evaluated for their relative geometric accuracy. A comparison of shifts computed between the produced Landsat Europe mosaic, the MERIS RGB (red band) and the MERIS L3 (cropped for the same area) are provided in Table  8. A number of joint feature points were used for each comparison. The tests were performed at different resolutions (GSDs) during the preliminary investigations. Large shifts were observed with the MERIS red band mosaic as can be seen in the Table. The shifts vary over different regions, which is also reflected in the standard deviations. A visual comparison between the MERIS RGB mosaic (red band only) and the MERIS L3 mosaic over Port Gentil area, Gabon, is shown in Figure  13. Figure  14  shows the radiometric differences between the reference mosaics and the SEVIRI HRV band image over Sicilia, Italy.
CONCLUSIONS AND FUTURE WORK This study was carried out as a part of new Tool development study for EUMETSAT satellite image project. While the tool is developed mainly with the main idea of image matching based on textural features, suitable global imagery to be used as reference for day and night images was required. The investigations with the MSG SEVIRI Level 1. 5 HRV, visible 0. 8 and infrared 10. 8 images have shown that Landsat and MERIS datasets can provide level of geometric accuracy spectral compatibility. However, processing of Landsat images has shown that a global cloudfree mosaic is possible by combined use of Landsat 4-5, 7 and 8 data, which cause radiometric differences between neighbouring images and visible mosaic seamlines. In addition, radiometric artefacts such as striping or missing bands are also occasionally present.
After the initial tests with an MERIS RGB global mosaic, this dataset was also found suitable for the purposes of the study. Recent MERIS L3 weekly synthesis data provided by ESA CCI service were used for reference image generation. The resolution, geometric accuracy, and spectral band availability of the MERIS L3 mosaic is also sufficient for the GQA of MSG SEVIRI. While the GQA Tool also aims at assessing EUMETSAT LEO (Low Earth Observation) imagery, reference data with higher spatial resolution are also required. Sentinel-2 archives have the potential to satisfy this aim.
Landsat Program was established by a partnership of the United States Geological Survey (USGS) and the National Aeronautics and Space Administration (NASA). The Landsat satellites provide essential and continuous information about the Earth's land surface acquiring space-based images since 1972 (USGS, 2015). Landsat imagery has been used in a variety of application areas such as global change studies, agriculture, forestry, geology, geography, mapping, resource management, water quality, and coastal researches (USGS, 2012). On the other hand, the GLS dataset was designed to provide global orthorectified, cloud-free Landsat imagery supporting researches about global land cover, land cover change, and ecosystem dynamics (USGS, 2009). The datasets have different versions centred in different years (e.
Indeed, recent abundant research has demonstrated the strong relationship between these microorganisms and complex and chronic human diseases such as diabetes, cirrhosis, autism, obesity and more (D. Liang et al. 2018 ). The field is maturing rapidly and large public repositories store an increasing number of standardized datasets. Data collection is performed using Next Generation Sequencing (NGS), taking advantage of parallelization of DNA sequencing (shotgun), which generates millions of sequence reads in a short time. Reads are short DNA sequences (generally between 50 to several thousands of nucleotides (A, C, G, T) and are stored in standardized files such as the fastq. The development of metagenomic sequencing came along with the rapid development of bioinformatics workflows, which ultimately yield quantitative measurements of biological objects such as genes, species, genera and other taxonomic levels, functional pathways, etc in the form of count tables  (Nayfach et al.
read2vec: Learning Read Embeddings It has been shown that the algorithms constructing word embeddings give good results for representing short sentences by simple arithmetic operations on word vectors  (Mikolov et al. 2013; Pennington et al. 2014; Joulin et al. 2016 ). Nevertheless, more sophisticated approaches for sentence embeddings have been developed. Sequence-to-sequence models  (Sutskever et al. 2014)  for instance, use a first network called the encoder to encode the sentence information. And a second one, called decoder decodes the sentence information for a specific task such as a translation where the authors have demonstrated great performance. The hidden layers of the encoder represent the sentence embeddings. Skip-thought  (Kiros et al. 2015)  vectorizes sentences with this approach, learning to generate surrounding sentences.
2017)  works with this concept). Since there is no information about textual concepts to learn about DNA sequences, another approach is to train the model by predicting the taxonomy of a read as it is performed with FastDNA (Section 3). In our experiments, three algorithms were integrated in the workflow. fastText (see Appendix C for more details) sentence embeddings as implemented in the package, FastDNA sequence embeddings extracted after the model were trained to classify taxonomy reads at the species level and finally a sequence-to-sequence based transformer applied on language modeling task (determine the probability distribution for the likelihood of a given word to follow a sequence of words). Each of these algorithms respect two properties: (i) being efficient enough to process the millions of sequences per metagenome (a non-blocking point in theory but important for implementation), and (ii) not involving sentence order in the prediction task.
metagenome2vec: Multiple instances learning representation Metagenomic data can be thought as a set that contains millions of reads representing one sample. The size of a set can varies depending on the sequencing technology and there is no specific order between reads within a set. Learning from such bags of reads correspond to a Multiple-Instance Learning (MIL) problem that deals with an objective function that is invariant to permutation and operates on non-finite dimensional vectors. In this study, we implement a deep learning algorithm from the work of Zaheer et al. (  2017 ) named DeepSets and an extension of the multiple instance layer with an attention mechanism from Ilse et al. (  2018 ) (see Appendix D for more details about these algorithms). The attention mechanism assigns a weight for each instance to determine which one in the set helped to predict the label.
Datasets Datasets used to learn embeddings: kmer2vec and read2vec The catalog we used here was composed of 506 different genomes, that were used to build the IGC gene catalog (MetaHIT  Consortium, Li, et al. 2014 ). These genomes correspond to 235 species, 79 genera and 37 families. Datasets to learn Read2Genomes classifier The CAMISIM  (Fritz et al. 2019 ) software was used to simulate 5,11M and 2,19M DNA sequences (for the train and validation set respectively) based on the genome catalog mentioned above. This allowed us to train and test the DNN models in how well their performed in estimating the abundance profiles resulting from read classification onto the genomes. The reference abundance used for the simulation followed a uniform distribution so that it would be balanced with a number of sequences almost equal for each genomes.
This is not the case in real metagenomes where abundance follows exponential distributions. Nevertheless, in the case of read classification modeling, this avoids the classifier to focus and predict the predominant classes, thus learning more robust embedding. Datasets to learn the Disease prediction tasks Our experiments were performed on two datasets with controlled disease liver cirrhosis (Cirrhosis) and colorectal cancer (Colorectal) from UCBI open source studies. In colorectal datast 15 patients had an adenoma, this is a benign tumor so they have been labeled as control cases as in  Pasolli et al. (2016). We also artificially simulated a third dataset (Simulated) with an artificial disease using CAMISIM  (Fritz et al. 2019 ) generator based on known bacterial genomes and abundance profiles which significantly vary in abundance between two groups (namely disease vs not).
Results of the Disease prediction tasks All experiments are done by limiting to one Million the number of reads for each metagenome. Indeed, learning computation time can take several days and there are many parameters to test for both structuring and machine learning. This is a way to cover many experiments and save time before running on the whole data. The datasets are composed by only hundreds of samples so to tune the hyper parameters and avoid over fitting we apply a nested cross validation. In that way, 20% of the data form the test set, the 80% remaining are used to perform a 10-fold cross validation to tune hyper parameters with 20% of the data as validation set. The whole operation is repeated 10 times with different train and test sets. The tuning is driven by accuracy score. AUC, precision, recall and F1 score are also computed.
Details about code implementation are given in appendix E. MetaML  (Pasolli et al. 2016)  and DeepMicro  (Oh et al. 2020 ) are the reference methods. Bag of K-mers (BoK) method is related to Bag of Word (BoW). Thus, BoK represents a metagenome as a vector by counting all the occurrences of its k-mers without using embeddings. is the baseline to confirm embeddings usefulness. M2V-VR is "metagenome2vec vectorial representation" and M2V-MIL is "metagenome2vec multiple instance learning representation". For each method, all the scores are the best of all the experiments. The BoK baseline has been computed with three different values of k equal to 3, 6 and 9. Better results are obtained when k = 6 or k = 9 without a significant difference among them. It cannot be computed with higher value of k due to the number of distinct k-mers in the vocabulary that becomes too large.
Scores are lower than other methods as expected but still leads quite good results for a simple, training free representation. Our results demonstrate that adding embeddings abstraction increases the performance and that the MIL representation yields better results. Our approach M2V-MIL reaches the same level as the state-of-the-art for Colorectal but is still lag behind for the Cirrhosis dataset. On the Simulated dataset, M2V MIL provides better results. This dataset is useful to assess whether the MIL model retrieved the genomes that play a role in the classification or not. We extract, for each well-ranked positive (artificial disease) sample, the five main genomes that had the greatest impact on the outcome, resulting in a total of 12 distinct genomes. Among them, two of the five genomes invoking disease (altered abundance) were predicted at the species level at 18% and 14.
Conclusion Phenotype prediction using metagenomic data is a critical task especially in a clinical setting. In this work, we proposed metagenome2vec, an algorithm which transforms raw metagenomic data onto automatically learned representations allowing us to apply on them multiple instance learning-based approaches. Combined with an attention mechanism, the algorithm offers some interpretability by being able to detect instances / genomes in the MIL set that could play a role in the phenotype prediction. Compared to standard quantitative metagenomics, our model displayed a performance comparable with the state of the art ML approaches directly applied to data processed through bioin-formatics workflows. However, as it is an end-to-end algorithm, the time which would be required to run a costly bioinformatics pipeline is saved in inference mode.
One of the limits of this work lies in the computational complexity of the learning of the different embeddings and the identification of each one's contribution to the prediction power. That is why we have proposed some analyses for each step to facilitate their intrinsic evaluation and optimization. We plan to test a wider range of parameters in the pipeline along with various algorithms so as to read multi-purpose embeddings and test them on all existing public datasets related to other diseases  (Pasolli et al. 2016). A first layer converts k-mers to their embeddings learnt in kmer2vec (Figure  5 ). The encoder creates the read embeddings with two blocks composed by a multi-head attention and a feed forwoard neural network. The decoder tries to predict the next k-mers from the source sequence passing the read embeddings in a fully connected layer before computing the softmax to get a probability for each k-mers.
B. 4 read2genome Figure  7 : A catalog of complete genome is used by the CAMISIM software  (Fritz et al. 2019)  to simulate metagenomic data with a specific taxonomic profile (abundance of species). The resulting dataset is a set of reads associated with the identifier of the genome from which they originate. Reads are embedded by read2vec (Figure  6 ) before being passed into read2genome trained to retrieve their source genome.
E Code implementation The full Pipeline is developed to train k-mers embeddings, reads embeddings, classfier of reads into taxa and classifier of metagenomes into disease. The algorithms and frameworks used for experiments are summarized below. Data processing Given the amount of data we decided to take advantage of the Spark 2. 4. 5 python Framework  (Zaharia et al. 2012)  running on distributed clusters to manage memory and make parallel computing on CPU for all data processing. We use YARN or Torque as cluster managers to change the resources allocated for each of our experiments. The deep learning models are always trained on GPU computing.
H Metagenomes clustering m 1 metagenomes are selected and m 2 < m 1 others are cut in 10 sub parts. Each metagenome or part of the metagenome is represented by one vector. An agglomerative clustering is trained on these embeddings to compute a clustermap and show distances between them (Figure  11 ). Results show logically that embeddings from portions of the same metagenome are closer to each other. It also indicates this relation for metagenomes from the same class even if some of them are in the wrong cluster. Figure  11 : Cluster map computed on the Colorectal Dataset with the metagenome2vec vectorial representation. Blue ids and red ids refer to healthy patients and sick patients respectively. Underscores on ids followed by a digit correspond to partitions of a same metagenome. On the map, the darker the color, the more similar the metagenomes.
Figure4: Numbers indicate the order of each operation block. Raw metagenomic data is the input of metagenome2vec. All DNA sequences are embedded by kmer2vec and read2vec algorithms (Figure5 and 6). Then, read2genome (Figure7) uses these embeddings to assign a cluster, corresponding to a genome id, for all reads. Embeddings of reads in the same cluster are averaged. It results in a multiple instance dataset where a bag of embeddings represents one metagenome. At the end, a multiple instance learning model DeepSets is trained to perform disease prediction.
Datasets have a massive size considering they only have few hundreds of patients. Metagenomes are composed of ∼ 80 million reads, each one composed of ∼ 90 nucleotides. It is a case where D >> N with D the number of features and N the number of examples. This is, in general, a complex task for machine learning algorithms. Information about datasets are described in Table1. Information about the metagenomic datasets.